Breadcrumb
- Home
- AI Hub For Education
- Research Study Repository
- Impact – Randomized Controlled Trial
Search and Filter
The Repository
Impact – Randomized Controlled Trial
Research synthesis is AI-generated, human reviewed. Updated 11/2026.
Displaying 1 - 30 of 166
Calibrating LLM Tutor Initiation to Learner Expertise: Proactive and Reactive Support for Computational Thinking
Arun Rai, Liwei Chen, Balasubramaniam Ramesh, Cynthia Breazeal, Yuan Long, Benjamin M. Ampel, Andrea Aria. (11/2026). SSRN.
Overall, there was no average treatment effect on computational thinking across all 90 students.
Prompt-Based Emotional Scaffolding In Generative AI: Effects On Self-Regulated Learning And Socio-Emotional Outcomes
Elena-Alexandra Diordiev, Valdemar Aksel Stenberdt, Guido Makransky. (10/2026). British Journal of Educational Technology.
Students across all conditions improved in conceptual knowledge and transfer from pretest to posttest, with no significant differences between conditions in post-to-follow-up changes.
Not Faster, But Better: Generative AI And Teacher Skill In Peru
Carla Z. Glave, Carolina Lopez, Ezequiel Molina, Roman Andres Zarate. (10/2026). World Bank.
Providing AI licenses, training, and follow-up sharply increased AI adoption but saved teachers no time on any of their core tasks.
The Effects Of Course-Integrated AI Tutoring On Student Performance And Engagement: A Randomized University Trial
Jing Liu, Tracy Sweet, Marty Haoyuan Chen, Jonathan Engelberg, Megan C. Masters, Martyn Clark, Axel Persaud, Alia Lancaster, Jeffrey Hollingsworth, Jennifer King Rice. (10/2026). EdWorking Papers.
Among sections of the same course, access to the AI tutor reduced final grades by 0.37 standard deviations and learning management system participation by 0.90 standard deviations.
Experimental Evidence on the Learning Impact of Generative AI
Zara Contractor, German Reyes. (10/2026). arXiv.
AI access increased immediate test scores by 7.2 percentage points (0.28 standard deviations), and a smaller gain of 3.9 percentage points persisted one week later.
Studentbench: AI And Human Tutoring Yield Equivalent Gre Learning Gains
Curtis Northcutt, Inaara Hasmani, Kevin Feng, Trevor Khangi, Andreas Plesner, Jonas Mueller. (09/2026). arXiv.
AI tutoring produced learning gains statistically equivalent to expert human tutoring on the combined GRE assessment (p=.015).
When AI Tutors Speak: Evidence From A Randomized Field Experiment
Shihao Yang, Marshall Van Alstyne, Chrysanthos Dellarocas. (09/2026). arXiv.
Students assigned to the structured AI tutor gained 6.63 more points on a 55-point assessment than peers in the holdout group.
A Brief AI Literacy Intervention Does Not Significantly Reduce Over-Reliance And Increases Under-Reliance On ChatGPT: A Randomized Study
Brett Puppart, Jaan Aru. (09/2026). arXiv.
A brief AI literacy text did not significantly reduce over-reliance on incorrect ChatGPT advice, with incorrect recommendations accepted in 48.7% of intervention trials versus 55.9% in control trials.
Biased AI Improves Human Performance But Reduces Perceived Helpfulness
Shiyang Lai, Jiwoong Choi, Junsol Kim, Nadav Kunievsky, Yujin Potter, James Evans. (09/2026). arXiv.
Biased AI assistants improved or preserved human performance relative to default and neutral AI across fact-checking, investment, and graduate coding tasks.
Your Programming Students' Cognition With ChatGPT: Higher Performance, Lower Retention, And Reduced Ownership
Christian Bergh, Benjamin Tag, Alexandra Vassar, Jake Renzella. (09/2026). arXiv.
ChatGPT-assisted students achieved higher coding scores (89% vs. 69%), but scored significantly lower on recall quizzes immediately (41% vs. 53%) and 48 hours later (39% vs. 52%).
Enhancing Procedural Writing Through Personalized Example Retrieval: A Case Study On Cooking Recipes
Paola Mejia-Domenzain, Jibril Frej, Seyed Parsa Neshaei, Luca Mouchel, Tanya Nazaretsky, Thiemo Wambsganss, Antoine Bosselut, Tanja Kaser. (09/2026). arXiv.
Participants receiving adaptive feedback improved their recipe writing quality and completeness significantly more than those without adaptive feedback.
Evaluating AI Tutoring At The Speed Of Innovation: Practitioner-Led Micro-Randomised Trials Of An AI Tutoring Platform In Gcse Science
Wayne Harrison, Rahil Khowaja, Emma Dobson, Germaine Uwimpuhwe, Steve Higgins. (09/2026). arXiv.
Students allocated to the AI tutoring platform scored significantly higher on science post-tests than those doing business-as-usual revision, with an effect size of 0.33.
Generative AI Assisted Workflows In Architectural Conceptual Design: Performance, Creative Self-Efficacy, And Cognitive Load
Yao Xiao, Han Jiang, Rachel Hurley, Shichao Liu. (09/2026). arXiv.
Generative AI and precedent search showed no significant overall differences in conceptual design performance, cognitive workload, or task-specific creative self-efficacy.
Learning To Prompt: Improving Student Engagement With Adaptive LLM-Based High-School Tutoring
Po-Chin Chang, Nicholas Hogan, Aske Plaat, Michiel T. van der Meer. (08/2026). arXiv.
In simulations, the adaptive prompt router outperformed static baselines (0.694 versus 0.647 and 0.64) and matched optimal strategies in 8 of 13 evaluated subjects.
AI, Expert Or Peer? Provider Biases And Feedback Uptake Among Pre-Service Teachers
Lucas Jasper Jacobsen, Ute Mertens, Thorben Jansen, Kira Elena Weber. (08/2026). arXiv.
Pre-service teachers rated large language model feedback more positively on fairness, usefulness, and acceptance when they mistakenly believed it came from a human.
Does Generative AI Narrow Education-Based Productivity Gaps? Evidence From A Randomized Experiment
Guillermo Cruces, Diego Fernandez Meijide, Sebastian Galiani, Ramiro Galvez, Maria Lombardi. (08/2026). arXiv.
AI assistance closed about three-quarters of the baseline performance gap between higher- and lower-education participants, raising scores for both groups but with substantially larger gains for lower-education adults.
One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment
Philip Oreopoulos, Nina Low. (08/2026). NBER.
Assignment to Khanmigo raised math achievement by 1.3 national percentile ranks per term, or about 0.06 to 0.08 standard deviations over a school year.
Reflection-Satisfaction Tradeoff: Investigating Impact Of Reflection On Student Engagement With AI-Generated Programming Hints
Heeryung Choia, Tung Phungb, Mengyan Wu, Adish Singlab, Christopher Brooksa. (08/2026). arXiv.
Students prompted before receiving hints, with planning prompts, or with directed prompts produced higher-quality reflections but reported lower hint satisfaction.
Virtual Tutoring with Computer-Assisted Learning: An Experiment in Take-Up and Learning
Philip Oreopoulos, Ruochong Dong, Nina Low. (08/2026). NBER.
Offering the tutoring raised Khan Academy practice by about 10 minutes weekly and led to an average math assessment score gain of 0.055 to 0.06 standard deviations.
Making AI Tutoring Productive: Evidence from a Mastery-Based Math Practice Experiment
Philip Oreopoulos, Michael Liut, Alp Sungu, Nina Low. (08/2026). NBER.
AI tutoring students progressed more slowly and attempted fewer questions, but they answered more accurately once reaching an attempt.
Is Solving Better Than Evaluating Genai Solutions?
Ethan Dickey, Marios Mertzanidis, Alexandros Psomas. (07/2026). arXiv.
Evaluating AI solutions caused no statistically significant differences in midterm scores, final exam scores, overall course grades, or performance on aligned exam transfer problems.
The Agency Gap In AI-Supported Writing: How Reactive And Proactive Agent Designs Shape Multimodal Reasoning
Yueqiao Jin, Kaixun Yang, Roberto Martinez-Maldonado, Dragan Gašević, Lixiang Yan. (07/2026). arXiv.
Proactive AI dialogues fostered stronger connections among conceptual reasoning, evidence use, and constructive engagement, whereas reactive interactions were more factual and procedural.
Emergent Learner Agency In Implicit Human-AI Collaboration: How Supportive And Contrarian AI Personas Reshape Interaction
Yueqiao Jin, Roberto Martinez-Maldonado, Dragan Gašević, Xibin Han, Lixiang Yan. (07/2026). arXiv.
Contrarian AI teammates significantly lowered participants' teamwork satisfaction and psychological safety compared to human-only groups, while supportive AI maintained higher satisfaction.
Teachy Mini: Development And Preliminary Evaluation Of A Knowledge-Based Generative Social Robot For Higher Education
Stephan Vonschallen, Karim Kaufmann, Dominique Oberle, Friederike Eyssel, Theresa Schmiedel. (07/2026). arXiv.
Students rated the knowledge-based robot significantly higher on perceived alignment with responsible tutoring behavior than the control robot (d = 0.930).
AI For Proactive Mental Health: A Longitudinal, Multi-Institutional Trial
Julie Youko Anne Cachia, Xuan Zhao, John Hunter, Delancey Wu, Eta Lin, Julian De Freitas. (07/2026). New England Journal of Medicine Group.
Students instructed to use the app reported significantly greater resilience and social well-being, including higher belonging, closer community ties, and lower loneliness.
Generative AI Can Harm Teaching
Alp Sungu, Benjamin Lira, Angela Lee Duckworth. (07/2026). SSRN.
Providing teachers with an AI support tool reduced student intrinsic motivation by 0.11 standard deviations, with students rating courses as less enjoyable, interesting, and important.
Learning In Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework For Language Learning
Nicy Scaria, Silvester John Joseph Kennedy, Deepak Subramani. (07/2026). arXiv.
Combining multi-agent debate scoring and recommendations with mastery-based progression and spaced review yielded significantly higher gains across grammar, vocabulary, and communication than feedback alone.
Learning Behavior Accounts For Background-Related Advantage In AI-Assisted Education
Jingwei Yi, Yueqi Xie, Jiyan He, Rui Ye, Junming Huang, Bin Zhu, Sean Rintel, Yu Xie, Xing Xie, Fangzhao Wu. (07/2026). arXiv.
Proactive and critical engagement with AI, such as attempting problems first or verifying answers, was associated with significantly higher exam scores than limited engagement.
Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming
Jérôme Brender, Laila El-Hamamsy, Kim Uittenhove, Aitor Perez, Patrick Jermann, Francesco Mondada, Engin Bumbacher. (07/2026). arXiv.
Students using the Socratic tutor achieved significantly greater learning gains by the final lab session than those using the prompt-refinement tutor, despite lower initial task performance.
Reflective Dialogue Or Prompt Refinement? Effects Of Tutor Scaffolding On Students' ndependent LLM Use For Programming
Jérôme Brender, Laila El-Hamamsy, Kim Uittenhove, Aitor Perez, Patrick Jermann, Francesco Mondada, Engin Bumbacher. (07/2026). arXiv.
Students using the Socratic tutor achieved significantly greater learning gains by the final lab session than those using the prompt-refinement tutor, despite having lower pre-test scores.
