Date
Publisher
arXiv
In this paper, we present a novel approach for distilling math word problem
solving capabilities from large language models (LLMs) into smaller, more
efficient student models. Our approach is designed to consider the student
model's weaknesses and foster a tailored learning experience by generating
targeted exercises aligned with educational science principles, such as
knowledge tracing and personalized learning. Concretely, we let GPT-3 be a math
tutor and run two steps iteratively: 1) assessing the student model's current
learning status on a GPT-generated exercise book, and 2) improving the student
model by training it with tailored exercise samples generated by GPT-3.
Experimental results reveal that our approach outperforms LLMs (e.g., GPT-3 and
PaLM) in accuracy across three distinct benchmarks while employing
significantly fewer parameters. Furthermore, we provide a comprehensive
analysis of the various components within our methodology to substantiate their
efficacy.
Study at a glance
- Question: Can a large AI model teach a smaller AI model to solve math word problems using customized practice exercises?
- Setting and data: Not stated in the abstract
- Method: Computational experiment testing an approach where a large AI model generates customized exercises to train a smaller model
Why it matters
Distilling math word problem solving from large language models into smaller, more efficient student models involves considering model weaknesses. Tailored learning can be fostered by generating targeted exercises aligned with educational science principles like knowledge tracing and personalized learning.
Key findings
- Across three math word problem benchmarks, smaller AI models trained with customized exercises solved problems more accurately than large models like GPT-3.
- The specialized math-solving models achieved superior accuracy while remaining significantly smaller and more efficient than large language models like PaLM.
- Analysis confirmed that both evaluating a model's learning status and generating tailored exercises effectively contributed to math problem-solving gains.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Why use AI?
Study design
