Date
Publisher
arXiv
Large Language Models (LLMs) are increasingly employed as AI tutors due to
their scalability and potential for personalized instruction. However,
off-the-shelf LLMs often underperform in educational settings: they frequently
reveal answers too readily, fail to adapt their responses to student
uncertainty, and remain vulnerable to emotionally manipulative prompts. To
address these challenges, we introduce CoDAE, a framework that adapts LLMs for
educational use through Chain-of-Thought (CoT) data augmentation. We collect
real-world dialogues between students and a ChatGPT-based tutor and enrich them
using CoT prompting to promote step-by-step reasoning and pedagogically aligned
guidance. Furthermore, we design targeted dialogue cases to explicitly mitigate
three key limitations: over-compliance, low response adaptivity, and threat
vulnerability. We fine-tune four open-source LLMs on different variants of the
augmented datasets and evaluate them in simulated educational scenarios using
both automatic metrics and LLM-as-a-judge assessments. Our results show that
models fine-tuned with CoDAE deliver more pedagogically appropriate guidance,
better support reasoning processes, and effectively resist premature answer
disclosure.
Study at a glance
- Question: Can enriching student conversations help AI tutors guide learners step-by-step without immediately revealing answers?
- Setting and data: Real-world dialogues between students and a ChatGPT-based tutor.
- Method: Tests four open-source AI models in simulated educational scenarios using automatic metrics and AI evaluation (LLM-as-a-judge).
Why it matters
Off-the-shelf large language models often underperform as AI tutors in educational settings. They frequently reveal answers too readily, fail to adapt to student uncertainty, and remain vulnerable to emotionally manipulative prompts.
Key findings
- In simulated educational scenarios, adapted AI tutors delivered more pedagogically appropriate guidance and better supported students' reasoning processes.
- The adapted AI models effectively resisted giving away answers prematurely during learning interactions.
- Standard AI tutors frequently give away answers too quickly, fail to adapt to student uncertainty, and remain vulnerable to emotionally manipulative prompts.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Study design
