Date
Publisher
arXiv
Large language models (LLMs) deployed in educational settings often behave as direct answerers: they disclose target concepts in the opening turn instead of guiding students through progressive inquiry, as Socratic pedagogy prescribes. We present HeuristicEdu, a two-phase pipeline that aligns Qwen2.5-7B toward Socratic tutoring via supervised warm-up and Group Relative Policy Optimization (GRPO). Training uses SocraticEdu, 797 multi-turn Chinese children's science dialogues reconstructed from a live platform, with a heuristic reward over cognitive depth (R_cog), curiosity engagement (R_eng), and directness (R_dir), together with a K_query correction for student-introduced terms. We introduce Scaffolding Effectiveness (SE) and Conversation Depth (CD) to evaluate outcomes beyond surface fluency. On 30 held-out questions, the best GRPO variant improves SE from 30.0% to 63.3% and lowers keyword leakage from 30.0% to 13.3%. Notably, this best variant omits the directness penalty during optimization, suggesting that explicit anti-leakage terms can conflict with gradient-based behavioral alignment. An unaligned Qwen-72B baseline reaches 0% SE and 96.7% leakage, showing that scale alone does not induce Socratic behavior.
Study at a glance
- Question: Can reinforcement learning train AI models to guide students through questions instead of directly giving answers?
- Setting and data: 10,512 Chinese science questions from students aged 8-12 across 14 primary schools using the Dr. Curious platform, curated into 797 dialogue trajectories.
- Method: Two-phase AI model training and evaluation using supervised fine-tuning and reinforcement learning (Group Relative Policy Optimization) tested with a simulated student.
Why it matters
AI tutors often give immediate answers rather than guiding students through questions to discover concepts on their own. This direct answering fails to support durable conceptual understanding.
Key findings
- Reinforcement learning without a directness penalty raised scaffolding effectiveness to 63.3% and cut answer keyword leakage to 13.3% on 30 test questions.
- Larger model scale did not induce guiding behavior, as an unaligned 72-billion parameter model achieved 0% scaffolding effectiveness and leaked keywords in 96.7% of dialogues.
- Adding an explicit penalty against leaking keywords during reinforcement learning worsened both scaffolding effectiveness and leakage rates.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
