Search and Filter

Leveraging Peer, Self, And Teacher Assessments For Generative Ai-Enhanced Feedback

Date

AI Overview

Why it matters

AI-enhanced feedback systems often treat teacher and peer reviews as equally reliable, while ignoring evaluator bias and underutilizing student self-assessments that support self-regulated learning.

Study at a glance
  • Question: How do teacher, peer, and self-assessments compare and align when evaluating engineering students' oral presentations?
  • Setting and data: 46 undergraduate engineering students at Universidad Autónoma de Madrid, evaluated across 46 oral presentation sessions.
  • Method: Correlational and inter-rater agreement analysis comparing pairwise and overall ratings.
  • AI tools: AICoFe, GePeTo
Key findings
  • Evaluators showed moderate to substantial agreement, with Gwet's AC2 values ranging between 0.58 and 0.63 across pairwise comparisons.
  • Teacher rubric ratings strongly correlated with the overall teacher grade (r = 0.88), while peer ratings (r = 0.69) and self-ratings (r = 0.59) had lower correlations.
  • Teachers assigned significantly lower rubric scores than peers and gave significantly lower rubric scores compared to self-assessments, indicating slight leniency from students.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Providing timely and meaningful feedback remains a persistent challenge in higher education, especially in large courses where teachers must balance formative depth with scalability. Recent advances in Generative Artificial Intelligence (GenAI) offer new opportunities to support feedback processes while maintaining human oversight. This paper presents an study conducted within the AICoFe (AI-based Collaborative Feedback) system, which integrates teacher, peer, and self-assessments of engineering students' oral presentations. Using a validated rubric, 46 evaluation sets were analyzed to examine agreement, correlation, and bias across evaluators. The analyses revealed consistent overall alignment among sources but also systematic variations in scoring behavior, reflecting distinct evaluative perspectives. These findings informed the proposal of an enhanced GenAI model within AICoFe system, designed to integrate human assessments through weighted input aggregation, bias detection, and context-aware feedback generation. The study contributes empirical evidence and design principles for developing GenAI-based feedback systems that combine data-based efficiency with pedagogical validity and transparency.