Date
Publisher
arXiv
The ability to comprehend code has long been recognized as an essential skill in software engineering. As programmers lean more heavily on generative artificial intelligence (GenAI) assistants to develop code solutions, it is becoming increasingly important for programmers to comprehend GenAI solutions so that they can verify their appropriateness and properly integrate them into existing code. At the same time, GenAI tools are increasingly being enlisted to provide programmers with tailored explanations of code written both by GenAI and humans. Thus, in computing education, GenAI presents new challenges and opportunities for learners who are trying to comprehend computer programs. To provide computing educators with evidence-based guidance on the use of GenAI to facilitate code comprehension and to identify directions for future research, we present a systematic literature review (SLR) of state-of-the-art approaches and tools that leverage GenAI to enhance code comprehension. Our SLR focuses on 31 studies published between 2022 and 2024. Despite their potential, GenAI assistants often yield inaccurate or unclear explanations, and novice programmers frequently struggle to craft effective prompts, thereby impeding their ability to leverage GenAI to aid code comprehension. Our review classifies GenAI-based approaches and tools, identifies methods used to study them, and summarizes the empirical evaluations of their effectiveness. We consider the implications of our findings for computing education research and practice, and identify directions for future research.
Study at a glance
- Question: How can generative AI assistants facilitate code comprehension, and how effective are they?
- Setting and data: 31 peer-reviewed papers published between 2022 and 2024 from ACM Digital Library, IEEE Xplore, and ScienceDirect.
- Method: Systematic literature review (SLR)
Why it matters
Programmers must increasingly comprehend generative AI code solutions, yet no comprehensive reviews examine how AI assistants facilitate code comprehension to provide evidence-based guidance for computing educators.
Key findings
- Generative AI tools consistently improve code comprehension accuracy, task speed, and perceived usefulness, though benefits are larger for experienced professionals than students.
- AI explanations are consistently rated as understandable and helpful by students, but frequent inaccuracies, omissions, and hallucinations remain persistent risks.
- Adding code context significantly improves generative AI assistance, raising the perfect-fix rate for error messages from 11% without context to 78% with context.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
Study design
