Date
Publisher
arXiv
Personalized programming tutoring, such as exercise recommendation, can
enhance learners' efficiency, motivation, and outcomes, which is increasingly
important in modern digital education. However, the lack of sufficient and
high-quality programming data, combined with the mismatch between offline
evaluation and real-world learning, hinders the practical deployment of such
systems. To address this challenge, many approaches attempt to simulate learner
practice data, yet they often overlook the fine-grained, iterative nature of
programming learning, resulting in a lack of interpretability and granularity.
To fill this gap, we propose a LLM-based agent, CoderAgent, to simulate
students' programming processes in a fine-grained manner without relying on
real data. Specifically, we equip each human learner with an intelligent agent,
the core of which lies in capturing the cognitive states of the human
programming practice process. Inspired by ACT-R, a cognitive architecture
framework, we design the structure of CoderAgent to align with human cognitive
architecture by focusing on the mastery of programming knowledge and the
application of coding ability. Recognizing the inherent patterns in
multi-layered cognitive reasoning, we introduce the Programming Tree of Thought
(PTOT), which breaks down the process into four steps: why, how, where, and
what. This approach enables a detailed analysis of iterative problem-solving
strategies. Finally, experimental evaluations on real-world datasets
demonstrate that CoderAgent provides interpretable insights into learning
trajectories and achieves accurate simulations, paving the way for personalized
programming education.
Study at a glance
- Question: Can an intelligent agent simulate students' iterative coding processes and predict their next modifications without relying on real data?
- Setting and data: Python submissions from CodeNet and college-level Java submissions from CSEDM (506 students).
- Method: System development and empirical evaluation against baseline models on four prediction tasks, plus simulation experiments.
Why it matters
Personalized programming tutoring relies on high-quality practice data, but data scarcity, privacy restrictions, and existing simulators' failure to capture iterative coding processes hinder system deployment.
Key findings
- CoderAgent outperformed baseline models in predicting next modification intentions, edit locations, and subsequent code submissions, achieving CodeBLEU scores up to 0.7698 on CSEDM.
- CoderAgent improved prediction of whether a submission would be accepted over traditional baselines, reaching an AUC of 0.5493 on CSEDM and 0.5522 on CodeNet.
- In a simulated classroom experiment with 20 students, CoderAgent identified a wider range of mistake-prone points and supported the generation of more test cases.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
