Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Generating Al Literacy MCQs: A Multi-Agent LLM Approach

Authors
Jiayi Wang,
Ruiwei Xiao,
Ying-Jui Tseng
Date
Publisher
arXiv
Link
http://arxiv.org/pdf/2412.00970v1
Artificial intelligence (AI) is transforming society, making it crucial to prepare the next generation through AI literacy in K-12 education. However, scalable and reliable AI literacy materials and assessment resources are lacking. To address this gap, our study presents a novel approach to generating multiple-choice questions (MCQs) for AI literacy assessments. Our method utilizes large language models (LLMs) to automatically generate scalable, high-quality assessment questions. These questions align with user-provided learning objectives, grade levels, and Bloom's Taxonomy levels. We introduce an iterative workflow incorporating LLM-powered critique agents to ensure the generated questions meet pedagogical standards. In the preliminary evaluation, experts expressed strong interest in using the LLM-generated MCQs, indicating that this system could enrich existing AI literacy materials and provide a valuable addition to the toolkit of K-12 educators.
Study at a glance
  • Question: How can artificial intelligence be used to generate high-quality multiple-choice questions for K-12 AI literacy assessments?
  • Setting and data: An unspecified number of experts.
  • Method: Development of an automated AI question-generation workflow using critique agents, followed by a preliminary expert evaluation.
Why it matters

As artificial intelligence transforms society, preparing the next generation through AI literacy in K-12 education is crucial. However, scalable and reliable AI literacy materials and assessment resources are lacking.

Key findings
  • In K-12 artificial intelligence education, experts expressed strong interest in using the system's automatically generated multiple-choice assessment questions.
  • The AI tool produced assessment questions aligned with designated learning objectives, grade levels, and cognitive thinking levels.
  • Automated critique agents within the tool ensured that the generated questions met pedagogical standards.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

What is the application?
Teaching – Assessment and Feedback
Who is the user?
Educator
Who age?
Middle School (6-8),
High School (9-12)
Why use AI?
Efficiency
Study design
Technical – Computational
Download PDF