Date
Publisher
arXiv
Feedback is one of the most powerful influences on student learning, with
extensive research examining how best to implement it in educational settings.
Increasingly, feedback is being generated by artificial intelligence (AI),
offering scalable and adaptive responses. Two widely studied approaches are
directive feedback, which gives explicit explanations and reduces cognitive
load to speed up learning, and metacognitive feedback which prompts learners to
reflect, track their progress, and develop self-regulated learning (SRL)
skills. While both approaches have clear theoretical advantages, their
comparative effects on engagement, confidence, and quality of work remain
underexplored. This study presents a semester-long randomised controlled trial
with 329 students in an introductory design and programming course using an
adaptive educational platform. Participants were assigned to receive directive,
metacognitive, or hybrid AI-generated feedback that blended elements of both
directive and metacognitive feedback. Results showed that revision behaviour
differed across feedback conditions, with Hybrid prompting the most revisions
compared to Directive and Metacognitive. Confidence ratings were uniformly
high, and resource quality outcomes were comparable across conditions. These
findings highlight the promise of AI in delivering feedback that balances
clarity with reflection. Hybrid approaches, in particular, show potential to
combine actionable guidance for immediate improvement with opportunities for
self-reflection and metacognitive growth.
Study at a glance
- Question: How do directive, metacognitive, and hybrid AI feedback affect student engagement, confidence, and resource quality?
- Setting and data: University of Queensland, introductory design and programming course, 12-week semester, 329 undergraduate and postgraduate students.
- Method: Randomized controlled trial with between-subjects design.
Why it matters
Providing timely, personalized feedback is challenging in large courses. While AI can generate directive or metacognitive feedback, strong empirical evidence directly comparing their effects on engagement, confidence, and work quality is lacking.
Key findings
- Students receiving hybrid AI feedback were most likely to revise their work (27.5%), compared to 21.1% for directive and 12.1% for metacognitive feedback.
- Engagement time following feedback did not differ significantly across the directive, metacognitive, and hybrid feedback conditions.
- Student confidence ratings and final resource quality scores were comparable across all three feedback groups, showing no statistically significant differences.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
