Breadcrumb
- Home
- AI Hub For Education
- Research Study Repository
- Evidence Explorer
- Methods
- User Study / Usability Testing
The Repository
Search and Filter
User study / usability testing
Studies tagged with AI from their full text or abstract; see each study for details.
Showing 121 - 150 of 283 studies
Supporting Students In Navigating Llm-Generated Insecure Code
Jaehwan Park, Kyungchan Lim, Seonhye Park, Doowon Kim. (11/2025). arXiv.
Over 95% of students accepted insecure AI-generated code across two programming tasks when not explicitly prompted to look for vulnerabilities, despite having prior security coursework.
Presentcoach: Dual-Agent Presentation Coaching Through Exemplars And Interactive Feedback
Sirui Chen, Jinsong Zhou, Xinli Xu, Xiaoyu YANG, Litao Guo, Ying-Cong Chen. (11/2025). arXiv.
Rehearsal with PresentCoach significantly increased speaking confidence on the PRCS scale compared with self-directed practice, with scores rising by 36.3% versus 16.1%.
Collaclassroom: An Al-Augmented Collaborative Learning Platform With Llm Support In The Context Of Bangladeshi University Students
Salman Sayeed, Bijoy Ahmed Saiem, Al-Amin Sany, Sadia Sharmin, A. B. M. Alim Al Islam. (11/2025). arXiv.
Most Bangladeshi university students (91.66%) agreed or strongly agreed that integrating large language models into study sessions would be helpful.
Bridging Lms And Generative Ai: Dynamic Course Content Integration (Dcci) For Enhancing Student Satisfaction And Engagement Via The Ask Me Assistant
Kovan Mzwri, Márta Turcsányi-Szabo. (11/2025). arXiv.
Students gave the assistant an average satisfaction rating of 4.65 out of 5, with 90.4% viewing responses as consistent with course materials.
Owlgorithm: Supporting Self-Regulated Learning In Competitive Programming Through Llm-Driven Reflection
Juliana Nieto-Cardenas, Erin Joy Kramer, Peter Kurto, Ethan Dickey, Andres Bejarano. (11/2025). arXiv.
Teaching assistants viewed Owlgorithm most favorably for introductory programming students, but perceived its usefulness and learning impact to drop sharply for higher-level courses.
Slidebot: A Multi-Agent Framework For Generating Informative, Reliable, Multi-Modal Presentations
Eric Xie, Danielle Waterfield, Michael Kennedy, Aidong Zhang. (11/2025). arXiv.
SlideBot outperformed Microsoft Copilot across all evaluated metrics, earning higher student ratings for explanation style (3.96 vs. 2.24) and credibility (4.36 vs. 1.93).
Understanding Student Interaction With Ai-Powered Next-Step Hints: Strategies And Challenges
Anastasiia Birillo, Aleksei Rostovskii, Yaroslav Golubev, Hieke Keuning. (11/2025). arXiv.
Across hint interactions, 28.48% were positive explicit acceptances, 40.00% were neutral, and 27.05% were negative, with most negative sessions caused by internal system errors.
Eduagentqg: A Multi-Agent Workflow Framework For Personalized Question Generation
Rui Jia, Min Zhang, Fengrui Liu, Bo Jiang, Kun Kuang, Zhongxiang Dai. (11/2025). arXiv.
EduAgentQG outperformed baseline approaches across diversity, objective consistency, and win rates, raising win rates by up to 4.75 percentage points with Gemini-2.5-Flash.
AI Literacy For Community Colleges: Instructors' Perspectives On Scenario-Based And Interactive Approaches To Teaching AI
Aparna Maya Warrier, Arav Agarwal, Jaromir Savelka, Christopher A Bogart, Heather Burte. (11/2025). arXiv.
Instructors ranked interactive demonstrations highest among instructional support materials (mean rank 3.20 out of 4), followed by lecture slides, conceptual guides, and reflection prompts.
How Teachers Can Use Large Language Models And Bloom'S Taxonomy To Create Educational Quizzes
Sabina Elkins, Ekaterina Kochmar, Jackie Chi Kit Cheung, Iulian Serban. (11/2025). arXiv.
Teachers strongly preferred creating quizzes using controlled AI generations aligned with Bloom's taxonomy and directly copied more of those candidate questions into their quizzes.
Comprehension-Performance Gap In Genai-Assisted Brownfield Programming: A Replication And Extension
Yunhan Qiao, Christopher Hundhausen, Summit Haque, Md Istiak Hossain Shihab. (11/2025). arXiv.
Although Copilot cut completion time by 50.7 percent and increased passed tests by 71.4 percent, participants showed no corresponding improvement in overall code comprehension.
Exploring Student-Ai Interactions In Vibe Coding
Francis Geng, Anshul Shah, Haolin Li, Nawab Mulla, Steven Swanson, Gerald Soosai Raj, Daniel Zingaro, Leo Porter. (11/2025). arXiv.
Most student interactions involved testing common cases in the live prototype (91.67%) and writing prompts to debug issues (61.01%), with very little code reading or editing.
Consistently Simulating Human Personas With Multi-Turn Reinforcement Learning
Marwa Abdulhai, Ryan Cheng, Donovan Clay, Tim Althoff, Sergey Levine, Natasha Jaques. (10/2025). arXiv.
Fine-tuning with multi-turn Proximal Policy Optimization reduced persona inconsistency by over 55% across simulated tasks compared to the baseline model.
Transforming Higher Education With Ai-Powered Video Lectures
Dengsheng Zhang. (10/2025). arXiv.
Student quiz performance across four tests showed no statistically significant differences between classes using AI-generated instructional videos and those using human-recorded instructional videos.
Improving Human Verification Of Llm Reasoning Through Interactive Explanation Interfaces
Runtao Zhou, Anh Totti Nguyen, Giang Nguyen, Nikita Kharya, Chirag Agarwal. (10/2025). arXiv.
Participants using the interactive graph interface achieved the highest verification accuracy at 85.6%, followed by interactive program-of-thought (82.5%) and interactive chain-of-thought (80.6%), all significantly outperforming standard chain-of-thought (73.5%).
"Learning Together": Ai-Mediated Support For Parental Involvement In Everyday Learning
Yao Li, Jingyi Xie, Ya-Fang Lin, He Zhang, Ge Wang, Gaojian Huang, Rui Yu, Si Chen. (10/2025). arXiv.
During the field deployment, 11 families completed 29 major tasks and 160 subtasks, with caregiver participation reaching up to 77% across evenly distributed tasks.
Plural Voices, Single Agent: Towards Inclusive Ai In Multi-User Domestic Spaces
Joydeep Chandra*, Satyam Kumar Navneet*. (10/2025). arXiv.
The single-agent Agora-4B outperformed multi-agent baselines in compliance (76% versus 70%), fairness (90% versus 85%), and safety-violation rates (0% versus 7%).
A Survey On Feedback Types In Automated Programming Assessment Systems
Eduard Frankford, Tobias Antensteiner, Michael Vierhauser, Clemens Sauerwein, Vivien Wallner, Iris Groher, Reinhold Plösch, Ruth Breu. (10/2025). arXiv.
Students receiving AI feedback achieved the highest average correctness score of 65.44, significantly outperforming unit test feedback (57.52) and compiler feedback (41.41).
Human Or Ai? Comparing Design Thinking Assessments By Teaching Assistants And Bots
Sumbul Khan, Wei Ting Liow, Lay Kee Ang. (10/2025). arXiv.
AI scores showed low agreement with instructor benchmarks for empathy (20%, Kappa = -0.026) and pain point identification (20%, Kappa = -0.039), but fair agreement for visual communication (50%, Kappa = 0.242).
Learnlens: Llm-Enabled Personalised, Curriculum-Grounded Feedback With Educators In The Loop
Runcong Zhao, Artem Bobrov, Jiazheng Li, Cesare Aloisi, Yulan He. (10/2025). arXiv.
LearnLens achieved higher scoring accuracy and a correlation of 0.388 on 100 student answers, outperforming baseline models while cutting latency compared to QwQ-32B.
Globalizeed: A Multimodal Translation System That Preserves Speaker Identity In Academic Lectures
Hoang-Son Vo, Karina Kolmogortseva, Ngumimi Karen Iyortsuun, Hong-Duyen Vo, Soo-Hyung Kim. (10/2025). arXiv.
Most students (77.7%) preferred dubbed formats over subtitled ones, reporting higher concentration (mean 4.06 out of 5) and reduced cognitive effort.
Nim: Neuro-Symbolic Ideographic Metalanguage For Inclusive Communication
Prawaal Sharma, Poonam Goyal, Navneet Goyal, Vidisha Sharma. (10/2025). arXiv.
Semi-literate participants reached over 80 percent semantic comprehensibility by day five, outperforming benchmarked pictographic methods by 1.9 times in comprehensibility and 8.5 times in learnability.
Edumath: Generating Standards-Aligned Educational Math Word Problems
Bryan R. Christ, Penelope Molitz, Jonathan Kropko, Thomas Hartvigsen. (10/2025). arXiv.
The EDUMATH 30B model met all educational criteria on 94.6% of generated problems, outperforming closed models including GPT-4o, GPT-4.1, and GPT-4.5.
Expertagent: Enhancing Personalized Education Through Dynamic Planning And Retrieval-Augmented Long-Chain Reasoning
Binrong Zhu, Guiran Liu, Nina Jiang. (10/2025). arXiv.
Volunteer participants rated performance expectancy highest at 4.33 on a Likert scale, indicating the system could improve learning efficiency.
Chrysalis: A Unified System For Comparing Active Teaching And Passive Learning With Ai Agents In Education
Prashanth Arun, Vinita Vader, Erya Xu, Brent McCready-Branch, Sarah Seabrook, Kyle Scholz, Ana Crisan, Igor Grossmann, Pascal Poupart. (10/2025). arXiv.
Students showed no significant overall preference or quiz score difference between learning from an AI tutor and teaching an AI agent.
Exploring Student Choice And The Use Of Multimodal Generative Ai In Programming Learning
Xinying Hou, Ruiwei Xiao, Runlong Ye, Michael Liut, John Stamper. (10/2025). arXiv.
Students most frequently chose screen-sharing as their context input modality (18 cases), pairing it with speech input in 83% of those cases and audio output in 84%.
Ai-Driven Grading And Moderation For Collaborative Projects In Computer Science Education
Songmei Yu, Andrew Zagula. (10/2025). arXiv.
AI-generated grades showed high alignment with instructor-assigned grades, achieving a Pearson correlation coefficient of r = 0.91.
Llm-Driven Rubric-Based Assessment Of Algebraic Competence In Multi-Stage Block Coding Tasks With Design And Field Evaluation
Yong Oh Lee, Byeonghun Bang, Sejun Oh. (10/2025). arXiv.
Language-model rubric scores strongly agreed with expert evaluations across five dimensions (r = 0.79, rho = 0.76), showing an average upward bias of 5.1 points.
Can an AI-Powered Presentation Platform Based On The Game "Just a Minute" Be Used To Improve Students' Public Speaking Skills?
Frederic Higham, Tommy Yuan. (10/2025). arXiv.
Most playtest participants found the game enjoyable and 80% believed playing it more in the future could improve their speaking skills and reduce anxiety.
Enhancing Personalized Multi-Turn Dialogue With Curiosity Reward
Yanming Wan, Jiaxing Wu, Marwa Abdulhai, Lior Shani, Natasha Jaques. (10/2025). arXiv.
Incorporating curiosity rewards based on differential log accuracy led human evaluators to prefer the model's personalization 75.75% of the time over standard multi-turn RLHF in education dialogues.
