Date
AI Overview
Why it matters
One-on-one tutoring is highly effective but expensive. If artificial intelligence can deliver comparable learning at lower cost, individualized instruction could become widely affordable and expand access to educational opportunities.
Study at a glance
- Question: Can artificial intelligence tutors produce learning gains equivalent to human tutors on the GRE?
- Setting and data: Handshake platform, 2,383 adult participants (mostly aged 18-24), 2,469 tutoring sessions across Quantitative and Verbal GRE sections, summer 2026.
- Method: Randomized controlled trial with pre- and post-tests, equivalence testing (TOST), and pairwise expert rubric evaluations.
- AI tools: StudentBench, Gemini, GPT-5.5, Opus, Gemma, Kimi
Key findings
- AI tutoring produced learning gains statistically equivalent to expert human tutoring on the combined GRE assessment (p=.015).
- Gemma 4 31B achieved learning gains equivalent to human tutoring at 918 times lower cost per percentage point gained.
- In Quantitative tutoring, faster AI response times were associated with increased student messages, more correct practice, and larger learning gains.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). To support future research, we open-source the de-identified data collected in our studies.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
Publication Source
Publisher
arXiv
