Date
Publisher
arXiv
The capabilities of generative AI in mathematics education are rapidly evolving, posing significant challenges for research to keep pace. Research syntheses remain scarce and risk being outdated by the time of publication. To address this issue, we present a Living Meta-Analysis (LIMA) on the effects of generative AI-based interventions for learning mathematics. Following PRISMA-LSR guidelines, we continuously update the literature base, apply a Bayesian multilevel meta-regression model to account for nested and cumulative data, and publish updated versions on a preprint server at regular intervals. This paper reports results from the second version, including 21 studies, 6 of which were newly included since the first version. The analyses indicate a positive effect (g = 0.42) with a wide credible interval [0.13, 0.72], reflecting the still limited evidence base. Results indicate no publication bias.
Study at a glance
- Question: How do generative AI interventions affect mathematics learning outcomes, and what factors influence their effectiveness?
- Setting and data: 34 experimental and quasi-experimental studies, 69 effect sizes, and 4,349 participants across primary, secondary, post-secondary, and adult education settings.
- Method: Living meta-analysis using a cumulative Bayesian multilevel meta-regression model
Why it matters
Generative AI capabilities in mathematics education are rapidly evolving, leaving empirical evidence fragmented and causing conventional research syntheses to quickly become outdated.
Key findings
- Instructional interventions using generative AI show an overall positive effect on mathematics learning outcomes (Hedges’ g = 0.57).
- Learning gains were substantially larger when generative AI supplemented regular teacher instruction (g = 0.88) than when it replaced teachers (g = 0.08) or functioned standalone (g = 0.17).
- Stronger learning effects were observed in longer interventions, with moderate evidence for larger effects among younger learners and more recent technology.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Why use AI?
Study design
