Date
Publisher
arXiv
Preparing high-quality instructional materials remains a labor-intensive
process that often requires extensive coordination among teaching faculty,
instructional designers, and teaching assistants. In this work, we present
Instructional Agents, a multi-agent large language model (LLM) framework
designed to automate end-to-end course material generation, including syllabus
creation, lecture scripts, LaTeX-based slides, and assessments. Unlike existing
AI-assisted educational tools that focus on isolated tasks, Instructional
Agents simulates role-based collaboration among educational agents to produce
cohesive and pedagogically aligned content. The system operates in four modes:
Autonomous, Catalog-Guided, Feedback-Guided, and Full Co-Pilot mode, enabling
flexible control over the degree of human involvement. We evaluate
Instructional Agents across five university-level computer science courses and
show that it produces high-quality instructional materials while significantly
reducing development time and human workload. By supporting institutions with
limited instructional design capacity, Instructional Agents provides a scalable
and cost-effective framework to democratize access to high-quality education,
particularly in underserved or resource-constrained settings.
Study at a glance
- Question: Can a multi-agent AI system effectively automate coherent, university-level course material design and reduce instructor workload?
- Setting and data: Five university-level computer science and AI courses evaluated by five expert instructors and automated AI reviewers.
- Method: System development and multi-method evaluation using an adapted Quality Matters rubric, ablation studies, and cost-runtime analysis.
Why it matters
Preparing high-quality instructional materials is labor-intensive and manual, often leading to fragmented pedagogical alignment. Without instructional design support, educators face high course preparation costs that limit scalable development.
Key findings
- Full Co-Pilot mode consistently achieved the highest instructional quality, outperforming the autonomous mode by 0.5 to 0.9 points on a 5-point scale across all five courses.
- The gpt-4o-mini backend matched the quality scores of gpt-4o and o1-preview (averaging 3.54 out of 5) while offering the lowest runtime and compute cost.
- Automated AI evaluators gave tightly clustered, mediocre scores between 2.9 and 3.1, failing to show the discriminative sensitivity of human expert reviewers.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
