Date
Publisher
arXiv
The rise of Human-AI Collaborative Learning (HAICL) is shifting education toward dialogue-centric paradigms, creating an urgent need for new assessment methods. Evaluating Self-Regulated Learning (SRL) in this context presents new challenges, as the limitations of conventional approaches become more apparent. Questionnaires remain interrupted, while the utility of non-interrupted metrics like clickstream data is diminishing as more learning activity occurs within the dialogue. This study therefore investigates whether the student-AI dialogue can serve as a valid, non-interrupted data source for SRL assessment. We analyzed 421 dialogue logs from 98 university students interacting with a generative AI (GenAI) learning partner. Using large language model embeddings and clustering, we identified 22 dialogue patterns and quantified each student's interaction as a profile of alignment scores, which were analyzed against their Online Self-Regulated Learning Questionnaire (OSLQ) scores. Findings revealed a significant positive association between proactive dialogue patterns (e.g., post-class knowledge integration) and overall SRL. Conversely, reactive patterns (e.g., foundational pre-class questions) were significantly and negatively associated with overall SRL and its sub-processes. A group comparison substantiated these results, with low-SRL students showing significantly higher alignment with reactive patterns than their high-SRL counterparts. This study proposed the Dialogue-Based Human-AI Self-Regulated Learning (DHASRL) framework, a practical methodology for embedding SRL assessment directly within the HAICL dialogue to enable real-time monitoring and scaffolding of student regulation.
Study at a glance
- Question: Can student-AI dialogue patterns reliably indicate university students' self-regulated learning abilities?
- Setting and data: Undergraduate finance students in a mandatory statistics course, four-week period mid-semester, 421 dialogue logs and 98 completed questionnaires.
- Method: Observational study using text embeddings and clustering, Spearman's rank correlations, and Mann-Whitney U group comparisons.
Why it matters
As learning becomes dialogue-centric in generative AI environments, traditional questionnaires interrupt learning and clickstream data misses cognitive intent. Assessing self-regulated learning requires unobtrusive, valid methods embedded directly within student-AI conversations.
Key findings
- Proactive dialogue patterns, such as post-class systematic knowledge integration, were positively associated with students' overall self-regulated learning scores (r = 0.164).
- Reactive patterns, such as asking about foundational conceptual difficulties or eliciting deep methodological AI elaborations, were negatively associated with overall self-regulation (r = -0.230).
- Students with lower self-regulation scores aligned significantly more with reactive questions about basic concepts and triggered more scaffolding-intensive AI responses than high-regulation students.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
