Date
Publisher
arXiv
In recent years, autonomous AI agents such as GitHub Copilot and Claude Code have been rapidly gaining popularity. This study reports on the practical implementation of Spec-Driven Development, a software development methodology premised on AI agents, within a Software Development Project-Based Learning (SDPBL) course for third-year undergraduate students. We defined a workflow consisting of four phases, namely investigation, planning, implementation, and review. We also established an environment tailored for the SDPBL course where AI agents generate documentation and code during each phase. We analyzed the results from three perspectives, namely students' subjective AI usage, implementation throughput, and code comprehension. The analysis reveals that AI usage patterns varied across development phases and teams. Moreover, while AI agent utilization increased implementation throughput, it also tended to encourage students to proceed with development without fully understanding the code. This study demonstrates that regular verification of code comprehension by instructors and appropriate feedback are essential for maintaining educational effectiveness when introducing SDD into SDPBL.
Study at a glance
- Question: How does using AI agents in project-based learning affect students' development activities, coding throughput, and code comprehension?
- Setting and data: Osaka Institute of Technology, third-year undergraduates in a project-based learning course, 2022 to 2025 cohorts, including 14 students across 4 teams in 2025.
- Method: Analysis of weekly surveys, code commit data (Steel-Dwass tests), and weekly one-on-one instructor assessment interviews.
Why it matters
Determining how to effectively incorporate autonomous AI agents into project-based learning is a critical challenge, as little is known about how they affect student collaboration, coding output, and understanding in practice.
Key findings
- Students added significantly more lines of code per lecture in 2024 and 2025 compared to earlier cohorts in 2022 and 2023.
- AI agent usage varied substantially across teams during investigation and planning phases, while all teams used AI for implementation and debugging.
- Although students with lower comprehension tended to rely more on AI, the relationship between AI usage levels and code comprehension was not statistically significant.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
