Date
Publisher
arXiv
The increasing demand for programming language education and growing class
sizes require immediate and personalized feedback. However, traditional code
review methods have limitations in providing this level of feedback. As the
capabilities of Large Language Models (LLMs) like GPT for generating accurate
solutions and timely code reviews are verified, this research proposes a system
that employs GPT-4 to offer learner-friendly code reviews and minimize the risk
of AI-assist cheating.
To provide learner-friendly code reviews, a dataset was collected from an
online judge system, and this dataset was utilized to develop and enhance the
system's prompts. In addition, to minimize AI-assist cheating, the system flow
was designed to provide code reviews only for code submitted by a learner, and
a feature that highlights code lines to fix was added. After the initial system
was deployed on the web, software education experts conducted usability test.
Based on the results, improvement strategies were developed to improve code
review and code correctness check module, thereby enhancing the system.
The improved system underwent evaluation by software education experts based
on four criteria: strict code correctness checks, response time, lower API call
costs, and the quality of code reviews. The results demonstrated a performance
to accurately identify error types, shorten response times, lower API call
costs, and maintain high-quality code reviews without major issues. Feedback
from participants affirmed the tool's suitability for teaching programming to
primary and secondary school students. Given these benefits, the system is
anticipated to be a efficient learning tool in programming language learning
for educational settings.
Study at a glance
- Question: Can an AI tool provide helpful code reviews for students while minimizing AI-assisted cheating?
- Setting and data: Code dataset from an online judge system and software education experts; participant numbers not stated.
- Method: System development, usability testing, and expert evaluation.
Why it matters
Increasing demand for programming language education and growing class sizes require immediate and personalized feedback. However, traditional code review methods have limitations in providing this feedback, while new tools must minimize the risk of AI-assist cheating.
Key findings
- For primary and secondary school programming, software education experts affirmed that a GPT-based code review tool is suitable for teaching students.
- The AI tool accurately identified coding error types and maintained high-quality code reviews without major issues.
- The system successfully shortened response times and reduced operating costs while strictly checking student code correctness.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Why use AI?
Study design
