Date
Publisher
arXiv
This article presents the results and their discussion for the third wave (with n=23 participants) within a multinational longitudinal study that investigates the evolving paradigm of human-AI collaboration in problem-solving contexts. Building upon previous waves, our findings reveal the consolidation of a hybrid problem-solving culture characterized by strategic integration of AI tools within structured cognitive workflows. The data demonstrate near-universal AI adoption (95.7% with prior knowledge, 100% ChatGPT usage) primarily deployed through human-led sequences such as "Think, Internet, ChatGPT, Further Processing" (39.1%). However, this collaboration reveals a critical verification deficit that escalates with problem complexity. We empirically identify and quantify two systematic epistemic gaps: a belief-performance gap (up to +80.8 percentage points discrepancy between perceived and actual correctness) and a proof-belief gap (up to -16.8 percentage points between confidence and verification capability). These findings, derived from behavioral data and problem vignettes across complexity levels, indicate that the fundamental constraint on reliable AI-assisted work is solution validation rather than generation. The study concludes that educational and technological interventions must prioritize verification scaffolds (including assumption documentation protocols, adequacy criteria checklists, and triangulation procedures) to fortify the human role as critical validator in this new cognitive ecosystem.
Study at a glance
- Question: How do individuals integrate AI into problem-solving, and how accurately do they verify AI-assisted solutions across task complexities?
- Setting and data: European Campus Rottal-Inn community, 23 multinational participants (students and non-students).
- Method: Descriptive analysis of longitudinal survey data and problem vignettes.
Why it matters
AI outputs can be persuasive yet logically flawed, making human verification necessary. Understanding how people integrate AI into problem-solving and verify solutions helps address automation bias as tasks become more complex.
Key findings
- A belief-performance gap widened with task complexity, reaching an 80.8 percentage point overconfidence gap where 93.8% trusted AI solutions on a complex problem but only 13.0% solved it correctly.
- A proof-belief gap showed that participants' self-assessed ability to prove solutions lagged behind their belief in correctness across all tasks, peaking at a 16.8 percentage point deficit.
- Participants predominantly used human-first hybrid workflows, with 39.1% following a 'Think, Internet, ChatGPT, Further Processing' sequence and none relying on an internet-only workflow without AI.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
