Automated Program Repair Of Uncompilable Student Code

Griffin Pitts, North Carolina State University

Aum Pandya, North Carolina State University

Darsh Rank, North Carolina State University

Tirth Bhatt, North Carolina State University

Muntasir Hoq, North Carolina State University

Bita Akram, North Carolina State University

Date

11/2025

AI Overview

Why it matters

Many student programming submissions fail to compile, excluding them from automated analysis and student modeling. Repairing only syntax errors can preserve learners' original reasoning and structure without discarding valuable evidence of their learning.

Study at a glance

Question: How effectively can large language models repair syntax errors in novice code while preserving students' original logic?
Setting and data: A CS1 course at a U.S. university in Spring 2019; 100 randomly sampled uncompilable Java submissions from 368 students on CodeWorkout.
Method: Model evaluation comparing three language models across low- and high-context prompts, evaluated using compilation rates, edit distance, and expert annotations.
AI tools: GPT-5, Claude 3.5 Haiku, Gemini 2.5 Flash

Key findings

All tested models produced compilable code at high rates (95.5% to 98.5%), with prompting context showing no significant effect on compilability.
GPT-5 achieved the lowest average edit distance (11.4) and the highest rates of preserving code logic (86.5%) and original control flow (96.4%).
Models occasionally introduced stylistic or structural alterations beyond minimal syntax corrections, straying from the intended pedagogical scope of minimal repair.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

A significant portion of student programming submissions in CS1 learning environments are uncompilable, limiting their use in student modeling and downstream knowledge tracing. Traditional modeling pipelines often exclude these cases, discarding observations of student learning. This study investigates automated program repair as a strategy to recover uncompilable code while preserving students' structural intent for use in student modeling. Within this framework, we assess large language models (LLMs) as repair agents under high- and low-context prompting conditions. Repairs were evaluated for compilability, edit distance, and preservation of students' original structure and logic. While all models produced compilable repairs, they differed in how well they preserve students' control flow and code structure, affecting their pedagogical utility. By recovering uncompilable submissions, this work enables richer and more comprehensive analyses of learners' coding processes and development over time.