Date
Publisher
arXiv
Multiple choice questions (MCQs) are a popular method for evaluating
students' knowledge due to their efficiency in administration and grading.
Crafting high-quality math MCQs is a labor-intensive process that requires
educators to formulate precise stems and plausible distractors. Recent advances
in large language models (LLMs) have sparked interest in automating MCQ
creation, but challenges persist in ensuring mathematical accuracy and
addressing student errors. This paper introduces a prototype tool designed to
facilitate collaboration between LLMs and educators for streamlining the math
MCQ generation process. We conduct a pilot study involving math educators to
investigate how the tool can help them simplify the process of crafting
high-quality math MCQs. We found that while LLMs can generate well-formulated
question stems, their ability to generate distractors that capture common
student errors and misconceptions is limited. Nevertheless, a human-AI
collaboration has the potential to enhance the efficiency and effectiveness of
MCQ generation.
Study at a glance
- Question: How can an AI tool help math educators create high-quality multiple-choice questions?
- Setting and data: Math educators (location, sample size, and timing not stated in the abstract).
- Method: Pilot study testing a prototype tool.
Why it matters
Crafting high-quality math multiple choice questions is a labor-intensive process that requires precise stems and plausible distractors. While large language models could automate question creation, challenges persist in ensuring mathematical accuracy and addressing student errors.
Key findings
- In a pilot study with math educators, large language models successfully generated well-formulated math question stems.
- Large language models showed limited ability to create wrong-answer choices that reflect common student errors and misconceptions.
- Human-AI collaboration has the potential to improve both the efficiency and effectiveness of generating math multiple-choice questions.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
