Date
AI Overview
Why it matters
Many AI debugging tools provide complete solutions to semantic errors rather than guiding learners step by step, risking over-reliance and hindering independent problem-solving skills.
Study at a glance
- Question: How usable and effective is an interactive AI debugging assistant designed to guide novice programmers through fixing semantic errors?
- Setting and data: 10 first- and second-year undergraduate students at the Singapore University of Technology and Design.
- Method: Usability testing comparing two debugging tools in a counterbalanced sequence, using surveys, System Usability Scale, and system interaction logs.
- AI tools: CodeHinter, GPT-4o, ChatGPT, SID
Key findings
- Undergraduate participants gave the new tool an average System Usability Scale score of 75/100 and a satisfaction rating of 89/100, outperforming the previous version.
- All participants used the 'End-to-End Test' feature, and 'Locate Lines With Errors' was the most frequently accessed and highly favored helper tool.
- Six out of ten students used the 'Provide Code Solution' button, sometimes multiple times to check and compare their answers.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Debugging is a fundamental skill that novice programmers must develop.
Numerous tools have been created to assist novice programmers in this process.
Recently, large language models (LLMs) have been integrated with automated
program repair techniques to generate fixes for students' buggy code. However,
many of these tools foster an over-reliance on AI and do not actively engage
students in the debugging process. In this work, we aim to design an intuitive
debugging assistant, CodeHinter, that combines traditional debugging tools with
LLM-based techniques to help novice debuggers fix semantic errors while
promoting active engagement in the debugging process. We present findings from
our second design iteration, which we tested with a group of undergraduate
students. Our results indicate that the students found the tool highly
effective in resolving semantic errors and significantly easier to use than the
first version. Consistent with our previous study, error localization was the
most valuable feature. Finally, we conclude that any AI-assisted debugging tool
should be personalized based on user profiles to optimize their interactions
with students.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
Publication Source
Publisher
arXiv
