Date
AI Overview
Why it matters
Large language models often fail to tailor responses to students' grade levels, producing outputs that are too advanced or vague for younger learners. Standardized benchmarks are lacking to evaluate their adaptability across cognitive and developmental stages.
Study at a glance
- Question: How well do open-source large language models adapt their answers to different K-12 grade levels across science subjects?
- Setting and data: 47,734 grade-labeled question-answer pairs derived from Wikipedia across nine science subjects for Grades 1–12.
- Method: Benchmark dataset creation, expert human evaluation, and language model evaluation using accuracy and an automated model judge.
- AI tools: Phi-4, GPT-4o, Mistral-Small-24B, Gemma-2B-it, Qwen2.5-14B, LLaMA3-8B
Key findings
- Language models consistently struggled to generate grade-appropriate responses for early-grade students in Grades 1–5, achieving lower performance than on higher grades.
- Larger models such as Qwen2.5-14B and Mistral-Small-24B achieved higher multiple-choice accuracy across all grades, whereas smaller models achieved only 50% to 60% accuracy in Grades 1–5.
- Traditional text metrics like BERTScore often exceeded 90% even when responses were wrong or developmentally inappropriate, failing to capture factual correctness and grade alignment compared to automated judges.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Large language models (LLMs) are transforming education by answering
questions, explaining complex concepts, and generating content across a wide
range of subjects. Despite strong performance on academic benchmarks, they
often fail to tailor responses to students' grade levels. This is a critical
need in K-12 education, where age-appropriate vocabulary and explanation are
essential for effective learning. Existing models frequently produce outputs
that are too advanced or vague for younger learners, and there are no
standardized benchmarks to evaluate their ability to adjust across cognitive
and developmental stages. To address this gap, we introduce EduAdapt, a
benchmark of nearly 48k grade-labeled QA pairs across nine science subjects,
spanning Grades 1-12 and grouped into four grade levels. We evaluate a diverse
set of open-source LLMs on EduAdapt and find that while larger models generally
perform better, they still struggle with generating suitable responses for
early-grade students (Grades 1-5). Our work presents the first dataset and
evaluation framework for assessing grade-level adaptability in LLMs, aiming to
foster more developmentally aligned educational AI systems through better
training and prompting strategies. EduAdapt code and datasets are publicly
available at https://github.com/NaumanNaeem/EduAdapt.
Research Categories
Why use AI?
Study design
AI tools
Datasets
Publication Source
Publisher
arXiv
