Date
AI Overview
Why it matters
One-to-one tutoring is the gold standard for personalized education, but it is prohibitively expensive to scale. Generative AI may help expand access to individualized learning support.
Study at a glance
- Question: Can a pedagogically fine-tuned AI model safely and effectively support students in chat-based tutoring?
- Setting and data: Five UK secondary schools, 165 students, using the Eedi mathematics platform.
- Method: Randomized controlled trial and tutor interviews
- AI tools: LearnLM
Key findings
- Students supported by the AI were 5.5 percentage points more likely to solve novel problems on later topics (66.2% success) than those with human tutors alone (60.7%).
- Students guided by the AI performed at least as well as students chatting with human tutors on every measured learning outcome.
- Supervising expert tutors approved 76.4% of the AI's drafted messages with zero or minimal edits of only one or two characters.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
One-to-one tutoring is widely considered the gold standard for personalized education, yet it remains
prohibitively expensive to scale. To evaluate whether generative AI might help expand access to this
resource, we conducted an exploratory randomized controlled trial (RCT) with __ = 165 students across
fiveUKsecondaryschools. WeintegratedLearnLM—agenerativeAImodelfine-tunedforpedagogy—into
chat-based tutoring sessions on the Eedi mathematics platform. In the RCT, expert tutors directly
supervised LearnLM, with the remit to revise each message it drafted until they would be satisfied
sendingitthemselves. LearnLMprovedtobeareliablesourceofpedagogicalinstruction,withsupervising
tutors approving 76.4% of its drafted messages making zero or minimal edits (i.e., changing only one or
two characters). This translated into effective tutoring support: students guided by LearnLM performed
at least as well as students chatting with human tutors on each learning outcome we measured. In
fact, students who received support from LearnLM were 5.5 percentage points more likely to solve
novel problems on subsequent topics (with a success rate of 66.2%) than those who received tutoring
from human tutors alone (rate of 60.7%). In interviews, tutors highlighted LearnLM’s strength at
drafting Socratic questions that encouraged deeper reflection from students, with multiple tutors even
reporting that they learned new pedagogical practices from the model. Overall, our results suggest
that pedagogically fine-tuned AI tutoring systems may play a promising role in delivering effective,
individualized learning support at scale.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
Google
