Date
Publisher
arXiv
Generative AI is reshaping education, but it also raises concerns about instability and overreliance. In programming classrooms, we aim to leverage its feedback capabilities while reinforcing the educator's role in guiding student-AI interactions. We developed ClassAid, a real-time orchestration system that integrates TA Agents to provide personalized support and an AI-driven dashboard that visualizes student-AI interactions, enabling instructors to dynamically adjust TA Agent modes. Instructors can configure the Agent to provide technical feedback (direct coding solutions), heuristic feedback (hint-based guidance), automatic feedback (autonomously selecting technical or heuristic support), or silent operation (no AI support). We evaluated ClassAid through three aspects: (1) the TA Agents' performance, (2) feedback from 54 students and one instructor during a classroom deployment, and (3) interviews with eight educators. Results demonstrate that dynamic instructor control over AI supports effective real-time personalized feedback and provides design implications for integrating AI into authentic educational settings.
Study at a glance
- Question: How effectively can an instructor-controlled AI orchestration system support real-time, personalized feedback in classroom programming activities?
- Setting and data: A 75-minute graduate data visualization course at a research university, including 54 students, one instructor, two teaching assistants, and eight programming educators.
- Method: Mixed methods design combining system development, expert evaluations of AI outputs, classroom user study, student surveys, and semi-structured educator interviews.
Why it matters
Generative AI provides programming feedback but raises concerns regarding instability and student overreliance. Current tools remain passive and lack mechanisms for real-time instructor monitoring and adaptive pedagogical control during class activities.
Key findings
- Evaluators showed 94.71% agreement on feedback correctness and 95.99% agreement on cognitive-level classification, confirming the AI teaching assistant reliably modeled student needs.
- During the classroom deployment, the teaching team used real-time dashboard alerts to make eight class-wide feedback adjustments and 22 individual student mode changes.
- Students gave general feedback high scores above 5.0 out of 7, with 80% reporting that ClassAid was better suited for classroom learning than general-purpose chatbots.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
