Date
Publisher
arXiv
Providing timely, targeted, and multimodal feedback helps students quickly correct errors, build deep understanding and stay motivated, yet making it at scale remains a challenge. This study introduces a real-time AI-facilitated multimodal feedback system that integrates structured textual explanations with dynamic multimedia resources, including the retrieved most relevant slide page references and streaming AI audio narration. In an online crowdsourcing experiment, we compared this system against fixed business-as-usual feedback by educators across three dimensions: (1) learning effectiveness, (2) learner engagement, (3) perceived feedback quality and value. Results showed that AI multimodal feedback achieved learning gains equivalent to original educator feedback while significantly outperforming it on perceived clarity, specificity, conciseness, motivation, satisfaction, and reducing cognitive load, with comparable correctness, trust, and acceptance. Process logs revealed distinct engagement patterns: for multiple-choice questions, educator feedback encouraged more submissions; for open-ended questions, AI-facilitated targeted suggestions lowered revision barriers and promoted iterative improvement. These findings highlight the potential of AI multimodal feedback to provide scalable, real-time, and context-aware support that both reduces instructor workload and enhances student experience.
Study at a glance
- Question: How does real-time AI multimodal feedback compare to educator feedback in learning gains, engagement, and student perceptions?
- Setting and data: Online experiment via Prolific, 197 US college students, learning science course materials.
- Method: Randomized controlled trial (online crowdsourcing experiment).
Why it matters
Delivering personalized, multimodal feedback at scale is difficult due to instructor workload. The educational effectiveness of real-time multimodal models combining text, visuals, and audio narration in authentic learning tasks remains under-documented.
Key findings
- AI multimodal feedback produced learning gains equivalent to human educator feedback from pre-test to post-test.
- Students receiving AI feedback reported significantly lower mental effort and higher satisfaction, clarity, specificity, conciseness, and motivation, with comparable trust and perceived correctness.
- Overall engagement time was similar, but students in the AI condition submitted fewer multiple-choice attempts and showed a trend toward more open-ended revisions.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
