Search and Filter

Learning Behavior Accounts For Background-Related Advantage In AI-Assisted Education

Date

AI Overview

Why it matters

Average effects of generative AI in education remain inconsistent and may mask variation across student populations. Understanding who benefits from AI assistance and how learning behaviors shape these patterns is central to addressing educational equity.

Study at a glance
  • Question: Which students benefit from AI assistance, and does how they interact with AI explain differences in learning gains?
  • Setting and data: 318 university students in China, tested across Python (81 students) and game theory (237 students) courses.
  • Method: Randomized controlled trial with observational chat-log annotation and mediation analysis.
  • AI tools: GPT-4o, DeepSeek-R1, Claude 3.7 Sonnet, Llama 4 Maverick, GPT-4.1, GPT-5
Key findings
  • Proactive and critical engagement with AI, such as attempting problems first or verifying answers, was associated with significantly higher exam scores than limited engagement.
  • Students from higher-ranking universities and those with stronger prior knowledge were more likely to adopt proactive and critical learning strategies with AI.
  • Accounting for learning behaviors accounted for 13.7% to 31.7% of background-related exam score differences, substantially weakening the link between student background and performance.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Generative AI has been found, and will likely be found increasingly, useful in education. However, existing AI-for-education studies provide inconsistent evidence on its average effects. More broadly, research on prior educational technologies shows that average effects often mask substantial heterogeneity across student populations. Motivated by this evidence, this study examines heterogeneity in students' learning behavior with AI, which students benefit from AI assistance, and how learner profiles and learning behavior shape these patterns. To this end, we recruited 318 university students to participate in structured learning experiments lasting up to 125 minutes. Our findings indicate that students' learning behavior is strongly associated with learning outcomes, with behaviors characterized by proactive and critical engagement, rather than limited engagement, associated with significantly better performance. These behavioral differences are related to learner profiles, with students from higher-ranking universities and those with greater prior knowledge tending to benefit more, consistent with their greater likelihood of adopting proactive interaction strategies. Accounting for learning behavior substantially weakens or eliminates the associations between learner profiles and learning outcomes, suggesting that how students use AI is a key pathway through which background differences are linked to learning gains. Overall, this work provides a deeper understanding of AI assistance in education by showing how differences in learner profiles and learning behavior shape who benefits from AI-supported learning. These insights can help educators and students better navigate and integrate AI into educational practices.