Edueval: A Hierarchical Cognitive Benchmark For Evaluating Large Language Models In Chinese Education

Guoqing Ma, Zhejiang Normal University

Jia Zhu, Zhejiang Normal University

Hanghui Guo, Zhejiang Normal University

Weijie Shi, Hong Kong University of Science and Technology

Yue Cui, Hong Kong University of Science and Technology

Jiawei Shen, Zhejiang Normal University

Zilong Li, Zhejiang Normal University

Yidan Liang, Zhejiang Normal University

Date

11/2025

AI Overview

Why it matters

Unscrutinized deployment of large language models poses risks to educational standards. Existing static benchmarks rely heavily on multiple-choice questions and fail to capture complex classroom reasoning, dynamic tasks, and ethical decision-making.

Study at a glance

Question: How effectively do large language models perform across diverse cognitive dimensions in Chinese primary and secondary education?
Setting and data: Chinese K-12 education contexts, analyzing 11,150 items across 24 tasks from exam archives, classroom transcripts, and expert-designed prompts.
Method: Benchmark evaluation comparing 14 large language models across six cognitive dimensions using zero-shot and few-shot testing.
AI tools: GPT-4o, Spark-X1, Qwen-plus, Qwen-72B, DeepSeek-R1-32B, EduChat-sft-002-13B-Baichuan

Key findings

Models performed well on memory and factual tasks, but accuracy dropped below 30% on authentic tasks like classroom dialogue classification.
Spark-X1 achieved the highest overall zero-shot score (81.1%), while several open-source models matched or outperformed proprietary models like GPT-4o.
Providing few-shot examples reduced average performance for most models, helping with structured dialogue and ethics tasks but constraining creative generation.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Large language models (LLMs) demonstrate significant potential for educational applications. However, their unscrutinized deployment poses risks to educational standards, underscoring the need for rigorous evaluation. We introduce EduEval, a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education. This benchmark makes three key contributions: (1) Cognitive Framework: We propose the EduAbility Taxonomy, which unifies Bloom's Taxonomy and Webb's Depth of Knowledge to organize tasks across six cognitive dimensions including Memorization, Understanding, Application, Reasoning, Creativity, and Ethics. (2) Authenticity: Our benchmark integrates real exam questions, classroom conversation, student essays, and expert-designed prompts to reflect genuine educational challenges; (3) Scale: EduEval comprises 24 distinct task types with over 11,000 questions spanning primary to high school levels. We evaluate 14 leading LLMs under both zero-shot and few-shot settings, revealing that while models perform well on factual tasks, they struggle with classroom dialogue classification and exhibit inconsistent results in creative content generation. Interestingly, several open source models outperform proprietary systems on complex educational reasoning. Few-shot prompting shows varying effectiveness across cognitive dimensions, suggesting that different educational objectives require tailored approaches. These findings provide targeted benchmarking metrics for developing LLMs specifically optimized for diverse Chinese educational tasks.