Date
AI Overview
Why it matters
Higher education policies focus mainly on whether to permit or prohibit AI rather than how interaction shapes learning. Overreliance on AI risks weakening students' reasoning, raising the question of how AI interaction can support critical thinking.
Study at a glance
- Question: How do students interact with generative AI, and how can instructional design encourage higher-order thinking?
- Setting and data: Undergraduate research methods course at a Southern U.S. university; 14 students; 50 randomly sampled AI interaction messages, plus 57 comparison messages.
- Method: Qualitative content analysis of student-AI interaction logs with human and AI coding.
- AI tools: GPT-5
Key findings
- Undergraduate student interactions fell into three paths: 46% Passive Review, 18% Direct Question, and 36% Strategic Dialogue in primary classifications.
- A contrast assignment requiring scaffolded reflection was dominated by Strategic Dialogue messages, suggesting instructional framing may influence how students interact with AI.
- Human coders achieved 68% agreement overall and 80% on Strategic Dialogue, while GPT-5 identified an additional computational check category absent in the human codebook.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
How students interact with artificial intelligence (AI) systems in educational settings may determine whether that interaction supports or displaces critical thinking. This paper introduces two contributions. The first is the Three Paths of Student-AI Interaction, a typological framework identifying three qualitatively distinct modes of student-AI engagement: Passive Review, Direct Question, and Strategic Dialogue. The second is the Next Level Teaching Blueprint (NLTB), a three-stage instructional design system intended to make Strategic Dialogue more likely. Qualitative content analysis of 50 randomly sampled student-AI interaction messages from an undergraduate research methods course was used to examine the typology. Two human coders achieved 68% path-level agreement ($κ$ = .48), with 80% agreement on Strategic Dialogue identification specifically. GPT-5, used as a third coder, produced a similar overall distribution and introduced a coding category absent from the human scheme. Path 1 (Passive Review) accounted for 46% of exchanges in the primary researcher's classifications, Path 2 (Direct Question) for 18%, and Path 3 (Strategic Dialogue) for 36%. A second, descriptively examined dataset contained predominantly Strategic Dialogue content, offering a preliminary indication that instructional framing may influence which path students take. Together, the Three Paths framework and the NLTB contribute a language for describing student-AI interaction and a design approach for supporting higher-order engagement.
Research Categories
What is the application?
Who age?
Why use AI?
Methods
AI tools
Publication Source
Publisher
arXiv
