Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Learning In Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework For Language Learning

Date
Publisher
arXiv
Most digital language learning curricula rely on discrete-item quizzes that test recall rather than applied conversational proficiency. When progression is driven by quiz performance, learners can advance despite persistent gaps in using grammar and vocabulary during interaction. Recent work on LLM-based judging suggests a path toward scoring open-ended conversations, but using interaction evidence to drive progression and review requires scoring protocols that are reliable and validated. We introduce Learning in Blocks, a framework that grounds progression in demonstrated conversational competence evaluated using CEFR-aligned rubrics. The framework employs heterogeneous multi-agent debate (HeteroMAD) in two stages: a scoring stage where role-specialized agents independently evaluate Grammar, Vocabulary, and Interactive Communication, engage in debate to address conflicting judgments, and a judge synthesizes consensus scores; and a recommendation stage that identifies specific grammar skills and vocabulary topics for targeted review. Progression requires demonstrating 70% mastery, and spaced review targets identified weaknesses to counter skill decay. We benchmark four scoring and recommendation methods on CEFR A2 conversations annotated by ESL experts. HeteroMAD achieves a superior score agreement with a 0.23 degree of variation and recommendation acceptability of 90.91%. An 8-week study with 180 CEFR A2 learners demonstrates that combining rubric-aligned scoring and recommendation with spaced review and mastery-based progression produces better learning outcomes than feedback alone.
Study at a glance
  • Question: Can multi-agent AI debate improve conversational scoring and, combined with mastery-based review, enhance language learning outcomes?
  • Setting and data: 180 CEFR A2 English language learners across an 8-week study, plus 150 transcribed learner conversations scored by experts.
  • Method: Benchmarking evaluation and an 8-week randomized controlled trial comparing three learner cohorts.
Why it matters

Digital language curricula often rely on discrete-item recall quizzes, allowing learners to advance despite persistent gaps in conversational proficiency. Grounding progression in open-ended conversations requires scoring protocols validated against expert judgment.

Key findings
  • Combining multi-agent debate scoring and recommendations with mastery-based progression and spaced review yielded significantly higher gains across grammar, vocabulary, and communication than feedback alone.
  • Heterogeneous multi-agent debate achieved the closest score agreement to human experts (0.23 degree of variation) and 90.91% recommendation acceptability.
  • Enabling explicit thinking mechanisms substantially degraded rubric-aligned scoring performance across single-model and self-refinement approaches, more than doubling score variation.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?