Search and Filter

Eduadapt: A Question Answer Benchmark Dataset For Evaluating Grade-Level Adaptability In Llms

Date

AI Overview

Why it matters

Large language models often fail to tailor responses to students' grade levels, producing outputs that are too advanced or vague for younger learners. Standardized benchmarks are lacking to evaluate their adaptability across cognitive and developmental stages.

Study at a glance
  • Question: How well do open-source large language models adapt their answers to different K-12 grade levels across science subjects?
  • Setting and data: 47,734 grade-labeled question-answer pairs derived from Wikipedia across nine science subjects for Grades 1–12.
  • Method: Benchmark dataset creation, expert human evaluation, and language model evaluation using accuracy and an automated model judge.
  • AI tools: Phi-4, GPT-4o, Mistral-Small-24B, Gemma-2B-it, Qwen2.5-14B, LLaMA3-8B
Key findings
  • Language models consistently struggled to generate grade-appropriate responses for early-grade students in Grades 1–5, achieving lower performance than on higher grades.
  • Larger models such as Qwen2.5-14B and Mistral-Small-24B achieved higher multiple-choice accuracy across all grades, whereas smaller models achieved only 50% to 60% accuracy in Grades 1–5.
  • Traditional text metrics like BERTScore often exceeded 90% even when responses were wrong or developmentally inappropriate, failing to capture factual correctness and grade alignment compared to automated judges.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Large language models (LLMs) are transforming education by answering questions, explaining complex concepts, and generating content across a wide range of subjects. Despite strong performance on academic benchmarks, they often fail to tailor responses to students' grade levels. This is a critical need in K-12 education, where age-appropriate vocabulary and explanation are essential for effective learning. Existing models frequently produce outputs that are too advanced or vague for younger learners, and there are no standardized benchmarks to evaluate their ability to adjust across cognitive and developmental stages. To address this gap, we introduce EduAdapt, a benchmark of nearly 48k grade-labeled QA pairs across nine science subjects, spanning Grades 1-12 and grouped into four grade levels. We evaluate a diverse set of open-source LLMs on EduAdapt and find that while larger models generally perform better, they still struggle with generating suitable responses for early-grade students (Grades 1-5). Our work presents the first dataset and evaluation framework for assessing grade-level adaptability in LLMs, aiming to foster more developmentally aligned educational AI systems through better training and prompting strategies. EduAdapt code and datasets are publicly available at https://github.com/NaumanNaeem/EduAdapt.