Date
AI Overview
Why it matters
Large language models often talk at students rather than with them. No open dataset labels specific dialogic tutoring behaviors to provide the training signal needed to steer models toward collaborative knowledge-building.
Study at a glance
- Question: Can fine-tuning language models on teacher-authored dialogic conversations improve their tutoring effectiveness?
- Setting and data: United States, 323 certified teachers, 2,134 middle school Earth Science dialogues, and 322 NCTE elementary math classroom transcripts.
- Method: Dataset creation, external validation with a transformer classifier, and fine-tuning evaluation via blind teacher ratings (linear mixed-effects model).
- AI tools: GEMINI-1.5-PRO, GPT-4O, CLAUDE HAIKU, CLAUDE SONNET 4.5, MISTRAL-7B-INSTRUCT-V0.3, GEMINI 2.0 FLASH
Key findings
- Instructional quality scores from a classifier trained on CONVOLEARN correlated significantly with expert ratings of authentic elementary math classrooms across several measures (r = .118 to .258).
- Credentialed teachers rated a fine-tuned Mistral-7B model comparably to Claude Sonnet 4.5 on dialogic tutoring effectiveness (mean 3.49 versus 3.56).
- Gemini 2.0 Flash scored significantly higher overall than the fine-tuned Mistral-7B model (mean 3.82 versus 3.49).
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
In educational applications, LLMs exhibit several fundamental pedagogical limitations, such as their tendency to reveal solutions rather than support dialogic learning. We introduce ConvoLearn (https://huggingface.co/datasets/masharma/convolearn ), a dataset grounded in knowledge building theory that operationalizes six core pedagogical dimensions: cognitive engagement, formative assessment, accountability, cultural responsiveness, metacognition, and power dynamics. We construct a semi-synthetic dataset of 1250 tutor-student dialogues (20 turns each) in middle school Earth Science through controlled interactions between human teachers and a simulated student. Using QLoRA, we demonstrate that training on this dataset meaningfully shifts LLM behavior toward knowledge-building strategies. Human evaluation by 31 teachers shows our fine-tuned Mistral 7B (M = 4.10, SD = 1.03) significantly outperforms both its base version (M = 2.59, SD = 1.11) and Claude Sonnet 4.5 (M = 2.87, SD = 1.29) overall. This work establishes a potential framework to guide future development and evaluation of constructivist AI tutors.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Datasets
Publication Source
Publisher
arXiv
