Date
Publisher
arXiv
Large language models (LLMs) have advanced virtual educators and learners,
bridging NLP with AI4Education. Existing work often lacks scalability and fails
to leverage diverse, large-scale course content, with limited frameworks for
assessing pedagogic quality. To this end, we propose WikiHowAgent, a
multi-agent workflow leveraging LLMs to simulate interactive teaching-learning
conversations. It integrates teacher and learner agents, an interaction
manager, and an evaluator to facilitate procedural learning and assess
pedagogic quality. We introduce a dataset of 114,296 teacher-learner
conversations grounded in 14,287 tutorials across 17 domains and 727 topics.
Our evaluation protocol combines computational and rubric-based metrics with
human judgment alignment. Results demonstrate the workflow's effectiveness in
diverse setups, offering insights into LLM capabilities across domains. Our
datasets and implementations are fully open-sourced.
Study at a glance
- Question: Can AI agents simulate interactive teacher-student conversations and accurately evaluate the quality of teaching?
- Setting and data: 114,296 teacher-learner conversations based on 14,287 tutorials across 17 domains and 727 topics.
- Method: System development evaluated using computational and rubric-based measures compared with human judgments.
Why it matters
Large language models have advanced virtual educators and learners, but existing work often lacks scalability and fails to use diverse, large-scale course content. There are also limited frameworks for assessing pedagogic quality.
Key findings
- Across 14,287 tutorials in 17 subject areas, an interactive AI teacher-and-learner system effectively supported step-by-step procedural learning.
- Automated quality ratings combining computational metrics and scoring rubrics aligned with human judgments of AI teaching dialogues.
- The multi-agent AI system successfully generated 114,296 interactive teacher-learner conversations across 727 distinct educational topics.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
