Date
Publisher
arXiv
As AI tools such as ChatGPT enter programming classrooms, students encounter differing rules across courses and instructors, which shape how they use AI and leave them with unequal capabilities for leveraging it. We investigate how students engaged with AI in an introductory Python assignment, analyzing student-LLM chat histories and final code submissions from 163 students. We examined prompt-level strategies, traced trajectories of interaction, and compared AI-generated code with student submissions. We identified trajectories ranging from full delegation to iterative refinement, with hybrid forms in between. Although most students directly copied AI-generated code in their submission, many students scaffolded the code generation through iterative refinement. We also contrasted interaction patterns with assignment outcomes and course performance. Our findings show that prompting trajectories serve as promising windows into students' self-regulation and learning orientation. We draw design implications for educational AI systems that promote personalized and productive student-AI collaborative learning.
Study at a glance
- Question: How do students interact with large language models during a timed introductory programming assignment, and how do patterns relate to performance?
- Setting and data: Introductory undergraduate Python course at a U.S. university, Spring 2025, 146 analyzed student submissions and 662 prompts.
- Method: Mixed methods combining qualitative transcript coding, k-means clustering of code features, and non-parametric group comparisons.
Why it matters
Instant access to AI solutions risks displacing the productive struggle essential for learning to program, yet little is known about how students navigate problem solving with AI under authentic classroom time pressure.
Key findings
- Over 80% of students directly reused AI-generated code in their final submissions, though researchers identified eight distinct interaction trajectories ranging from full delegation to iterative refinement.
- Students who engaged in minimal AI use scored significantly lower on the assignment than those who simply delegated the entire problem to the AI in one prompt.
- Overall final course letter grades did not significantly differ across the different student-AI interaction trajectory types.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
