Date
Publisher
arXiv
We present Owlgorithm, an educational platform that supports Self-Regulated Learning (SRL) in competitive programming (CP) through AI-generated reflective questions. Leveraging GPT-4o, Owlgorithm produces context-aware, metacognitive prompts tailored to individual student submissions. Integrated into a second- and third-year CP course, the system-provided reflective prompts adapted to student outcomes: guiding deeper conceptual insight for correct solutions and structured debugging for partial or failed ones.
Our exploratory assessment of student ratings and TA feedback revealed both promising benefits and notable limitations. While many found the generated questions useful for reflection and debugging, concerns were raised about feedback accuracy and classroom usability. These results suggest advantages of LLM-supported reflection for novice programmers, though refinements are needed to ensure reliability and pedagogical value for advanced learners.
From our experience, several key insights emerged: GenAI can effectively support structured reflection, but careful prompt design, dynamic adaptation, and usability improvements are critical to realizing their potential in education. We offer specific recommendations for educators using similar tools and outline next steps to enhance Owlgorithm's educational impact. The underlying framework may also generalize to other reflective learning contexts.
Study at a glance
- Question: Can an automated AI tool generate effective reflective and diagnostic questions to support competitive programming students?
- Setting and data: Competitive programming courses at a university, Spring 2025, involving six undergraduate teaching assistants and one instructor, plus student pilot users.
- Method: System development and an exploratory perception survey with in-app ratings and open-ended feedback.
Why it matters
Competitive programming students rarely reflect on their code due to time pressure and lack of feedback. Existing automated tools seldom provide structured, self-regulated reflection tailored to students' code outcomes.
Key findings
- Teaching assistants viewed Owlgorithm most favorably for introductory programming students, but perceived its usefulness and learning impact to drop sharply for higher-level courses.
- Generated questions achieved an average correctness rating of 62.1 percent, whereas feedback quality averaged 49.9 percent correctness, with roughly 40 percent feedback accuracy reported.
- Students valued many high-quality prompts for clarifying reasoning, but nearly half reported low satisfaction due to repetitive scenarios, broad edge-case questions, and interface delays.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
