Date
Publisher
arXiv
Interactive feedback, where feedback flows in both directions between teacher
and student, is more effective than traditional one-way feedback. However, it
is often too time-consuming for widespread use in educational practice. While
Large Language Models (LLMs) have potential for automating feedback, they
struggle with reasoning and interaction in an interactive setting. This paper
introduces CAELF, a Contestable AI Empowered LLM Framework for automating
interactive feedback. CAELF allows students to query, challenge, and clarify
their feedback by integrating a multi-agent system with computational
argumentation. Essays are first assessed by multiple Teaching-Assistant Agents
(TA Agents), and then a Teacher Agent aggregates the evaluations through formal
reasoning to generate feedback and grades. Students can further engage with the
feedback to refine their understanding. A case study on 500 critical thinking
essays with user studies demonstrates that CAELF significantly improves
interactive feedback, enhancing the reasoning and interaction capabilities of
LLMs. This approach offers a promising solution to overcoming the time and
resource barriers that have limited the adoption of interactive feedback in
educational settings.
Study at a glance
- Question: Can an AI system allow students to question, challenge, and discuss feedback on their essays?
- Setting and data: 500 critical thinking essays and user studies.
- Method: Case study and user studies evaluating a newly developed multi-agent AI system.
Why it matters
Interactive feedback is more effective than traditional one-way feedback, but it is often too time-consuming for widespread educational use. While large language models have potential to automate feedback, they struggle with reasoning and interaction in interactive settings.
Key findings
- In a case study of 500 critical thinking essays, an interactive AI framework was linked to improved two-way feedback and stronger reasoning.
- The system enabled students to query, challenge, and clarify their evaluations to refine their understanding.
- Standard large language models struggle with the reasoning and interaction required for effective two-way feedback.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
