Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

CoderAgent: Simulating Student Behavior for Personalized Programming Learning with Large Language Models

Date
Publisher
arXiv
Personalized programming tutoring, such as exercise recommendation, can enhance learners' efficiency, motivation, and outcomes, which is increasingly important in modern digital education. However, the lack of sufficient and high-quality programming data, combined with the mismatch between offline evaluation and real-world learning, hinders the practical deployment of such systems. To address this challenge, many approaches attempt to simulate learner practice data, yet they often overlook the fine-grained, iterative nature of programming learning, resulting in a lack of interpretability and granularity. To fill this gap, we propose a LLM-based agent, CoderAgent, to simulate students' programming processes in a fine-grained manner without relying on real data. Specifically, we equip each human learner with an intelligent agent, the core of which lies in capturing the cognitive states of the human programming practice process. Inspired by ACT-R, a cognitive architecture framework, we design the structure of CoderAgent to align with human cognitive architecture by focusing on the mastery of programming knowledge and the application of coding ability. Recognizing the inherent patterns in multi-layered cognitive reasoning, we introduce the Programming Tree of Thought (PTOT), which breaks down the process into four steps: why, how, where, and what. This approach enables a detailed analysis of iterative problem-solving strategies. Finally, experimental evaluations on real-world datasets demonstrate that CoderAgent provides interpretable insights into learning trajectories and achieves accurate simulations, paving the way for personalized programming education.
Study at a glance
  • Question: Can an intelligent agent simulate students' iterative coding processes and predict their next modifications without relying on real data?
  • Setting and data: Python submissions from CodeNet and college-level Java submissions from CSEDM (506 students).
  • Method: System development and empirical evaluation against baseline models on four prediction tasks, plus simulation experiments.
Why it matters

Personalized programming tutoring relies on high-quality practice data, but data scarcity, privacy restrictions, and existing simulators' failure to capture iterative coding processes hinder system deployment.

Key findings
  • CoderAgent outperformed baseline models in predicting next modification intentions, edit locations, and subsequent code submissions, achieving CodeBLEU scores up to 0.7698 on CSEDM.
  • CoderAgent improved prediction of whether a submission would be accepted over traditional baselines, reaching an AUC of 0.5493 on CSEDM and 0.5522 on CodeNet.
  • In a simulated classroom experiment with 20 students, CoderAgent identified a wider range of mistake-prone points and supported the generation of more test cases.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?
Who age?