Date
Publisher
arXiv
The emergence of large language models has enabled vibe coding, a natural language approach to programming in which users describe intent and AI generates or revises code, potentially broadening access to programming while preserving meaningful learning outcomes. We investigate its educational value through a month-long online hackathon that welcomed participants from multiple countries, ranging from complete beginners to experienced developers. The hackathon offered three tracks with increasing technical demands. Spark emphasized basic frontend functionality and dynamic features such as buttons, forms, and API calls. Build required backend or database integration. Launch targeted production ready web applications, including deployment. Participants were required to develop projects using only LLM generated code without manual edits and submitted complete chat histories, source code, demo videos, and functionality reports. We assessed educational effectiveness with a mixed methods design that combined standardized project evaluations across functionality, user interface and user experience design, impact, prompt quality, and code readability, along with post-hackathon surveys of perceived learning outcomes and thematic analysis of open-ended feedback. Our findings describe how participants with different backgrounds engage with vibe coding as task complexity increases, how the no manual editing constraint shapes prompting and debugging practices, and what these patterns imply for integrating AI assisted development into programming education and competitive learning environments.
Study at a glance
- Question: How can natural language coding support learners with different programming backgrounds during an online hackathon?
- Setting and data: Month-long online hackathon with 229 participants across 184 teams from eight countries, including university students, interns, and professionals.
- Method: Mixed methods combining standardized rubric scoring of 40 projects with post-hackathon surveys and thematic analysis.
Why it matters
Despite rising enthusiasm for natural-language-driven vibe coding, empirical research on its educational value and how it supports learners across different programming skill levels remains limited.
Key findings
- Participants rated the event's overall helpfulness very high, with an average score of 4.68 out of 5, viewing vibe coding as highly beneficial for their learning.
- Across 40 evaluated projects, the mean final score was 81.48 out of 100, though achieving high-quality, production-ready applications remained challenging for many teams.
- Higher-difficulty tracks had higher average scores but also greater score variability, with Launch track projects averaging 84.52 points compared to 77.60 for Spark.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Study design
