Search and Filter

Hallucination-Free Automatic Question & Answer Generation For Intuitive Learning

Date

AI Overview

Why it matters

AI-generated practice questions often contain factual, logical, or mathematical errors that undermine student trust and reduce learning efficiency.

Study at a glance
  • Question: Can an iterative multi-agent verification framework reduce errors in AI-generated multiple-choice questions?
  • Setting and data: A sample of Advanced Placement-aligned STEM multiple-choice questions.
  • Method: Iterative multi-agent framework evaluation and cost analysis.
  • AI tools: GPT-4.1-nano, GPT-o3-mini
Key findings
  • The iterative multi-agent framework reduced question hallucination rates by over 90% compared to baseline generation after seven iterations.
  • Hallucination errors dropped by approximately 50% after just a single verification iteration compared to the reference solution.
  • Using the lightweight GPT-4.1-nano model achieved these reductions at $0.10 per million tokens, compared to $1.10 for GPT-o3-mini.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Hallucinations in large language models (LLMs), defined as fluent yet incorrect or incoherent outputs, pose a significant challenge to the automatic generation of educational multiple-choice questions (MCQs). We identified four key hallucination types in MCQ generation: reasoning inconsistencies, insolvability, factual errors, and mathematical errors. To address this, we propose a hallucination-free multi-agent generation framework that breaks down MCQ generation into discrete, verifiable stages. Our framework utilizes both rule-based and LLM-based detection agents, as well as hallucination scoring metrics to optimize question quality. We redefined MCQ generation as an optimization task minimizing hallucination risk while maximizing validity, answerability, and cost-efficiency. We also introduce an agent-led refinement process that uses counterfactual reasoning and chain-of-thought (CoT) to iteratively improve hallucination in question generation. We evaluated a sample of AP- aligned STEM questions, where our system reduced hallucination rates by over 90% compared to baseline generation while preserving the educational value and style of questions. Our results demonstrate that structured multi-agent collaboration can mitigate hallucinations in educational content creation at scale, paving the way for more reliable LLM-powered learning tools.