Search and Filter

Llama Lima: A Living Meta-Analysis On The Effects Of Generative AI On Learning Mathematics

Date

AI Overview

Why it matters

Generative AI capabilities in mathematics education are rapidly evolving, making conventional research syntheses outdated quickly. A living synthesis is needed to understand whether and under what conditions these interventions effectively support mathematics learning.

Study at a glance
  • Question: How does generative AI affect mathematics learning, and which factors moderate its effectiveness?
  • Setting and data: 34 experimental and quasi-experimental studies, 69 effect sizes, and 4,349 participants across primary, secondary, post-secondary, and adult education through August 2026.
  • Method: Living meta-analysis (Bayesian multilevel meta-regression).
  • AI tools: ChatGPT, ChatGPT-3.5, ChatGPT-4, Google Bard, MatGPT, LuzIA
Key findings
  • Generative AI interventions showed an overall positive effect on mathematics learning outcomes (Hedges' g = 0.57), with no evidence of publication bias.
  • Effects were substantially larger when generative AI supplemented regular teacher instruction (g = 0.88) compared to replacing teachers (g = 0.08) or standalone use (g = 0.17).
  • Stronger learning effects were associated with longer intervention durations, and there was moderate evidence for larger gains among younger learners.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

The capabilities of generative AI in mathematics education are rapidly evolving, posing significant challenges for research to keep pace. Research syntheses remain scarce and risk being outdated by the time of publication. We present a Living Meta-Analysis (LIMA) on the effects of generative AI-based interventions for learning mathematics. We update the literature base every two months and apply a cumulative Bayesian multilevel meta-regression model. This paper reports results from the fifth version, including 34 studies, 7 of which were newly included since the fourth version. Results indicate an increasingly positive effect (g = 0.57, CrI [0.36, 0.79]) and no evidence in favor of a publication bias. Moderator analyses indicate that effects vary by integration mode and intervention duration: Larger effects were observed when generative AI supplemented teacher instruction and in longer interventions. Moderate evidence for stronger effects was observed in studies involving younger learners and more recent technology. For other moderators, evidence is still inconclusive.