Date
Publisher
arXiv
User simulation is important for developing and evaluating human-centered AI, yet current student simulation in educational applications has significant limitations. Existing approaches focus on single learning experiences and do not account for students' gradual knowledge construction and evolving skill sets. Moreover, large language models are optimized to produce direct and accurate responses, making it challenging to represent the incomplete understanding and developmental constraints that characterize real learners. In this paper, we introduce a novel framework for memory-based student simulation that incorporates developmental trajectories through a hierarchical memory mechanism with structured knowledge representation. The framework also integrates metacognitive processes and personality traits to enrich the individual learner profiling, through dynamical consolidation of both cognitive development and personal learning characteristics. In practice, we implement a curriculum-aligned simulator grounded on the Next Generation Science Standards. Experimental results show that our approach can effectively reflect the gradual nature of knowledge development and the characteristic difficulties students face, providing a more accurate representation of learning processes.
Study at a glance
- Question: Can a memory-based simulation framework enable language models to realistically mimic elementary students' gradual learning, errors, and personalities?
- Setting and data: Simulated elementary science tutoring interactions across grades 1 to 5, grounded in the Next Generation Science Standards (NGSS).
- Method: System development and model evaluation of simulated tutoring dialogues across four language models.
Why it matters
Testing AI educational tools with real students poses recruitment, cost, and ethical hurdles. Current simulated learners provide direct, mature answers and fail to capture children's gradual learning trajectories, incomplete understanding, and individual differences.
Key findings
- Simulated students using SimLearner consistently performed within curriculum standards (77.2% to 91.6% knowledge coverage), whereas unconstrained baselines performed at 179.1% in grade 1.
- GPT models effectively suppressed knowledge of advanced concepts with an 11% leakage ratio, while open-source models struggled, leaking up to 51% of unlearned knowledge.
- SimLearner maintained stable Big Five personality profiles and reflected how traits influence learning, with high openness and extraversion associated with higher concept mastery.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
