Search and Filter

Learning To Prompt: Improving Student Engagement With Adaptive LLM-Based High-School Tutoring

Date

AI Overview

Why it matters

Static-prompt tutoring systems struggle to adapt across diverse academic disciplines, and systems tested primarily in simulations often face performance gaps when deployed with real students.

Study at a glance
  • Question: Can an adaptive prompt routing system improve student engagement and instructional quality across different high school subjects?
  • Setting and data: Online tutoring platform in the Netherlands, March–April 2026, 359 Dutch high school students across 656 conversation sessions.
  • Method: Online A/B experiment with user-level randomization, contextual bandit policy optimization, and simulated benchmark evaluations.
  • AI tools: GPT-5-mini, E5-base-V2
Key findings
  • In simulations, the adaptive prompt router outperformed static baselines (0.694 versus 0.647 and 0.64) and matched optimal strategies in 8 of 13 evaluated subjects.
  • In longer student interactions of five or more turns, the adaptive router reduced conversation length by about three turns while maintaining instructional quality.
  • A stochastic prompt router achieved a 28.1% exercise conversion rate compared to 19.1% for a greedy router and 19.6% for the static baseline, though overall differences were not statistically significant.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

LLMs can personalize education, although current static-prompt tutoring systems struggle to adapt to diverse academic disciplines. We develop and test a system with subject-aware prompting, based on 14 pedagogical features (e.g., tutor scaffolding, student understanding) extracted from raw transcripts. We first train a prompt routing model in a simulation environment, and then deploy it for online adaptation with actual high-school students. The simulation benchmark shows the router outperforming two static baselines ($0.694$ vs. $0.647$ and $0.64$, $p<0.001$). A/B testing ($N=656$ conversations from 359 students) shows sim-to-real transfer where the model switches from analytical to scaffolding learning strategies. Our adaptive prompt selection mechanism improves instructional efficiency, maintains pedagogical quality and reduces interactions by around 3 turns ($p=0.007$). While a greedy router achieves a comparable exercise conversion rate with the baseline ($19.1\%$ vs. $19.6\%$), a stochastic router that samples strategies leads to a higher conversion rate ($28.1\%$).