Search and Filter

Effective Personalized AI Tutors Via Llm-Guided Reinforcement Learning

Date

AI Overview

Why it matters

Emerging generative AI platforms largely focus on chatbot tutors that reactively answer student questions rather than proactively guiding student learning.

Study at a glance
  • Question: Does adaptively sequencing practice problems using student-chatbot interactions improve high school students' exam performance?
  • Setting and data: Ten high schools in Taipei during a five-month Python course, student sample size not stated.
  • Method: Randomized controlled trial
Key findings
  • Adaptive problem sequencing increased unassisted final exam performance by 0.15 standard deviations, equivalent to 6 to 9 months of schooling by some estimates.
  • Mediation analysis suggests that the learning gains from adaptive sequencing were driven by increased student engagement.
  • Student-chatbot interactions provide valuable signals for proactively optimizing and personalizing student learning.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

Generative AI (GenAI) is rapidly reshaping education by unlocking the potential for personalized tutoring. Yet, emerging platforms largely focus on GenAI chatbot tutors that reactively answer student questions. We hypothesize that the efficacy of GenAI chatbot tutors can be substantially improved by proactively guiding student learning. To test this, we design a novel tutoring platform that tightly integrates a carefully-designed GenAI chatbot with a reinforcement learning algorithm for sequencing practice problems. Critically, this algorithm leverages rich signals from student-chatbot interactions to adaptively select practice problems of an appropriate difficulty level. In partnership with the Taipei City Government and American Institute in Taiwan, we deployed our tutoring platform in conjunction with a five-month course to teach Python to students across ten high schools. We randomized students between a fixed practice problem sequence and our adaptive sequencing algorithm. We find that adaptive sequencing increased unassisted final exam performance by 0.15 standard deviations (equivalent to 6-9 months of schooling by some estimates); mediation analysis suggests that gains were driven by increased engagement. Our work provides large-scale field evidence that student-chatbot interactions provide valuable signals for proactively optimizing and personalizing student learning.