Search and Filter

Evidence-Decision-Feedback: Theory-Driven Adaptive Scaffolding For LLM Agents

Date

AI Overview

Why it matters

Many large language model pedagogical agents provide one-size-fits-all support, risk student overreliance, and lack transparent, learning theory-grounded decision-making in open-ended learning environments.

Study at a glance
  • Question: Can a theory-driven framework adapt AI peer scaffolding to student mastery, avoid overreliance, and maintain interpretable guidance?
  • Setting and data: High school science classroom in Nashville, USA; 33 sophomore student dyads over six weeks across three computational modeling tasks.
  • Method: Correlational analysis of interaction logs and dialogue, permutation tests of semantic trace similarity, surveys, and educator discussions.
  • AI tools: Copa, GPT-5, all-MiniLM-L6-v2
Key findings
  • As student task mastery increased, the AI agent significantly reduced probing questions and increased concrete suggestions and limit-pushing prompts.
  • Student success in verbally explaining their code increased alongside task mastery, while their requests for agent support dropped significantly as mastery improved.
  • The AI agent's internal reasoning links showed statistically significant grounding in student data, alignment with pedagogical decisions, and faithful feedback generation.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

LLMs offer tremendous opportunities for pedagogical agents to help students construct knowledge and develop problem-solving skills, yet many of these agents operate on a "one-size-fits-all" basis, limiting their ability to personalize support. To address this, we introduce Evidence-Decision-Feedback (EDF), a theoretical framework for adaptive scaffolding with LLM agents. EDF integrates elements of intelligent tutoring systems (ITS) and agentic behavior by organizing interactions around evidentiary inference, pedagogical decision-making, and adaptive feedback. We instantiate EDF through Copa, a Collaborative Peer Agent for STEM+C problem-solving. In an authentic high school classroom study, we show that EDF-guided interactions align feedback with students' demonstrated understanding and task mastery; promote scaffold fading; and support interpretable, evidence-grounded explanations without fostering overreliance.