Breadcrumb
- Home
- AI Hub For Education
- Research Study Repository
- Evidence Explorer
- Methods
- User Study / Usability Testing
The Repository
Search and Filter
User study / usability testing
Studies tagged with AI from their full text or abstract; see each study for details.
Showing 151 - 180 of 283 studies
Code2Video: A Code-Centric Paradigm For Educational Video Generation
Yanzhe Chen, Kevin Qinghong Lin, Mike Zheng Shou. (10/2025). arXiv.
Code2Video achieved a 40% improvement in teaching effectiveness over direct code generation and substantially outperformed pixel-based video models.
Towards An Ai-Augmented Textbook
LearnLM Team. (09/2025). arXiv.
Students using the AI-augmented textbook scored significantly higher on both immediate (77% vs. 68%) and three-day retention assessments (78% vs. 67%) compared to digital reader users.
From Frustration to Fun: An Adaptive Problem-Solving Puzzle Game Powered by Genetic Algorithm
Matthew McConnell, Richard Zhao. (09/2025). arXiv.
All three game versions moderately reduced puzzle-solving frustration, but differences among the models were not statistically significant.
DraftMarks: Enhancing Transparency in Human-AI Co-Writing Through Interactive Skeuomorphic Process Traces
Momin N. Siddiqui, Nikki Nasseri, Adam Coscia, Roy Pea, Hari Subramonyam. (09/2025). arXiv.
Readers using DraftMarks achieved 50% higher comprehension scores regarding the writing and revision process compared to a split-screen chat log baseline.
Student Engagement with GenAI's Tutoring Feedback: A Mixed Methods Study
Sven Jacobs, Jan Haas, Natalie Kiesler. (09/2025). arXiv.
Overall, 82.0% of AI feedback requests resulted in at least one code improvement, with students acting on 52.9% of the read feedback components.
Leveraging Large Language Models for Robot-Assisted Learning of Morphological Structures in Preschool Children with Language Vulnerabilities
Stina Sundstedt, Mattias Wingren, Susanne Hagglund, Daniel Ventus. (09/2025). arXiv.
Tests showed the TalBot prototype successfully managed three-player turn-taking and used contextual cues to interpret children's mispronunciations or speech recognition errors.
Designing for Novice Debuggers: A Pilot Study on an Al-Assisted Debugging Tool
Oka Kurniawan, Erick Chandra, Christopher Michael POSKITT, Yannic Noller, Kenny Tsu Wei Choo, Cyrille Jegourel. (09/2025). arXiv.
Undergraduate participants gave the new tool an average System Usability Scale score of 75/100 and a satisfaction rating of 89/100, outperforming the previous version.
Enhancing Python Programming Education with an AI-Powered Code Helper: Design, Implementation, and Impact
Sayed Mahbub Hasan Amiri, Md Mainul Islam. (09/2025). arXiv.
The AI chatbot resolved 85% of errors across 1,500 student submissions, outperforming standalone GPT-4 (73%) and pylint (62%).
ClassMind: Scaling Classroom Observation and Instructional Feedback with Multimodal AI
Ao Qu, Yuxi Wen, Jiayi Zhang, Yunge Wen, Yibo Zhao, Alok Prakash, Andres Salazar-Gomez, Paul Pu Liang, Jinhua Zhao. (09/2025). arXiv.
The AVA-Align video agent achieved 100% factuality and higher significance and feasibility scores than Gemini-2.5-Pro and Deep Video Discovery when evaluated on ten classroom videos.
OnlineMate: An LLM-Based Multi-Agent Companion System for Cognitive Support in Online Learning
Xian Gao, Zongyun Zhang, Ting Liu, Yuzhuo Fu. (09/2025). arXiv.
OnlineMate raised students' average cognitive level to 5.20 compared to 4.10 for standard multi-agent setups in simulations, while also boosting emotional engagement scores.
DeliverC: Teaching Pointers through GenAI-Powered Game-Based Learning
Wyatt Petula, Anushcka Joshi, Peggy Tu, Amrutha Somasundar, Suman Saha. (09/2025). arXiv.
Over 75% of surveyed students agreed that DeliverC supported confidence building, application, and metacognition, and 72% agreed it supported motivation.
Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice
Si Chen, Isabel R. Molnar, Peiyu Li, Adam Acunin, Ting Hua, Alex Ambrose, Nitesh V Chawla, Ronald Metoyer. (09/2025). arXiv.
Instructors engaged more actively with the Socratic chatbot, contributing significantly more turns and words, while the Narrative chatbot generated much longer responses.
MusicScaffold: Bridging Machine Efficiency and Human Growth in Adolescent Creative Education through Generative AI
Zhejing Hu, Yan Liu, Zhi Zhang, Gong Chen, Bruce X.B. Yu, Junxian Li, Jiannong Cao. (09/2025). arXiv.
Students using MusicScaffold achieved higher prompt specificity (average score 3.00 versus 1.53 and 1.80) and incorporated significantly broader musical dimensions than control groups.
Skill-based Explanations for Serendipitous Course Recommendation
Hung Chau, Run Yu, Zachary Pardos, Peter Brusilovsky. (08/2025). arXiv.
Skill-based explanations did not significantly alter overall interest in course recommendations, but for courses with high unexpectedness, interest increased by 0.22 points.
MathBuddy: A Multimodal System for Affective Math Tutoring
Debanjana Kar, Leopold Böss, Dacia Braca, Sebastian Maximilian Dennerlein, Nina Christine Hubig, Philipp Wintersberger, Yufang Hou. (08/2025). arXiv.
Integrating student emotion features into the LearnLM math tutor increased its pedagogical win rate against ground truth responses by 23 percentage points.
GOLDMIND: A Teacher-Centered Knowledge Management System For Higher Education AI Lessons From Iterative Design
Gloria Fernández-Nieto, Lele Sha, Yuheng Li, Yi-Shan Tsai, Guanliang Chen, Yinwei Wei, Weiqing Wang, Jinchun Wen, Shaveen Singh, Ivan Silva, Yuanfang Li, Dragan Gašević, Zachari Swiecki. (08/2025). arXiv.
In the final evaluation, university teachers using GoldMind had a 60% lower task failure rate and 47% higher task completion quality than those using an LLM baseline.
RoboBuddy in the Classroom: Exploring LLM-Powered Social Robots for Storytelling in Learning and Integration Activities
Daniel Tozadore, Nur Ertug, Yasmine Chaker, Mortadha Abderrahim. (08/2025). arXiv.
Students reported significantly higher enjoyment when the social robot delivered content through storytelling rather than expository lecture explanations.
Enabling Multi-Agent Systems as Learning Designers: Applying Learning Sciences to AI Instructional Design
Jiayi Wang, Ruiwei Xiao, Xinying Hou, John Stamper. (08/2025). arXiv.
Teachers strongly preferred activities from the collaborative multi-agent discussion system, rating them as significantly more creative, contextually relevant, and classroom-ready.
PAPPL: Personalized AI-Powered Progressive Learning Platform
Shayan Bafandkar, Sungyong Chung, Homa Khosravian, Alireza Talebpour. (08/2025). arXiv.
Students receiving AI-generated hints solved engineering problems with fewer attempts, reaching a 55% success rate on their second attempt compared to 40% in the baseline group.
CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM Integration
Zixin Chen, Jiachen Wang, Yumeng Li, Haobo Li, Chuhan Shi, Rong Zhang, Huamin Qu. (08/2025). arXiv.
Participant grades using CoGrader strongly correlated with ground truth course grades, yielding a 0.900 Pearson correlation and a 0.899 Spearman's rank correlation.
LEARN: A Story-Driven Layout-to-Image Generation Framework for STEM Instruction
Maoquan Zhang, Bisser Raytchev, Xiujuan Sun. (08/2025). arXiv.
In a user study, LEARN's outputs scored 23% higher in clarity and reduced perceived cognitive load by 31% compared to baseline visuals.
From Misunderstandings To Learning Opportunities: Leveraging Generative AI In Discussion Forums To Support Student Learning
Stanislav Pozdniakov, Jonathan Brazil, Oleksandra Poquet, Stephan Krusche, Santiago Berrezueta-Guzman, Shazia Sadiq, Hassan Khosravi. (08/2025). arXiv.
All five instructors supported the M2M approach, viewing AI-identified student misunderstandings and generated learning opportunities as insightful, actionable, and valuable for weekly teaching.
Listening with Language Models: Using LLMs to Collect and Interpret Classroom Feedback
Sai Siddartha Maram, Ulia Zaman, Magy Seif El-Nasr. (08/2025). arXiv.
The instructor reported that the chatbot system yielded richer, more actionable, and timely feedback than standard end-of-quarter surveys, gathering 23 responses in five minutes.
From Self-Crafted to Engineered Prompts: Student Evaluations of AI-Generated Feedback in Introductory Physics
Amogh Sirnoorkar, N. Sanjay Rebello. (08/2025). arXiv.
Most students (61.4%) ranked AI feedback generated from prompts combining prompt engineering and effective feedback principles as most useful, while 91.1% preferred structured prompts overall.
Al Conversational Tutors in Foreign Language Learning: A Mixed-Methods Evaluation Study
Nikolaos Avouris. (08/2025). arXiv.
Talkio and Langua achieved the highest overall scores across most evaluated dimensions, especially on learning value, coherence, and interaction quality.
Evaluating the Impact of LLM-guided Reflection on Learning Outcomes with Interactive AI-Generated Educational Podcasts
Vishnu Menon, Andy Cherney, Elizabeth B. Cloude, Li Zhang, Tiffany D. Do. (08/2025). arXiv.
Learning outcomes did not differ significantly between undergraduates in the reflection condition and those in the standard condition.
A Mixed User-Centered Approach to Enable Augmented Intelligence in Intelligent Tutoring Systems: The Case of MathAlde app
Guilherme Guerino, Luiz Rodrigues, Luana Bianchini, Mariana Alves, Marcelo Marinho, Thomaz Veloso, Valmir Macario, Diego Dermeval, Thales Vieira, Ig Bittencourt, Seiji Isotani. (08/2025). arXiv.
In the classroom study, teachers used the correction feature on 139 of 784 student answers (17.6%) to overturn incorrect AI assessments.
WIP: Enhancing Game-Based Learning with AI-Driven Peer Agents
Chengzhang Zhu, Cecile H. Sam, Yanlai Wu, Ying Tang. (08/2025). arXiv.
Students completing agent-supported modules with language model inquiry improved test scores from 39.2% to 45.7%, whereas passive video instruction module scores dropped from 50.0% to 28.3%.
Automated Feedback on Student-Generated UML and ER Diagrams Using Large Language Models
Sebastian Gürtl, Gloria Schimetta, David Kerschbaumer, Michael Liut, Alexander Steinmaurer. (07/2025). arXiv.
All six participants highlighted the tool's ability to provide instant, iterative, and scalable feedback to support self-directed student learning without increasing tutor workload.
A ChatGPT-Based Approach For Questions Generation In Higher Education
Sinh Trong Vu, Huong Thu Truong, Oanh Tien Do, Tu Anh Le, Tai Tan Mai. (07/2025). arXiv.
University lecturers distinguished ChatGPT-generated questions from human-written ones with high accuracy, averaging 13.5 out of 15 correct, while students averaged around 7 to 8.
