Date
Publisher
arXiv
Tutoring dialogues have gained significant attention in recent years, given
the prominence of online learning and the emerging tutoring abilities of
artificial intelligence (AI) agents powered by large language models (LLMs).
Recent studies have shown that the strategies used by tutors can have
significant effects on student outcomes, necessitating methods to predict how
tutors will behave and how their actions impact students. However, few works
have studied predicting tutor strategy in dialogues. Therefore, in this work we
investigate the ability of modern LLMs, particularly Llama 3 and GPT-4o, to
predict both future tutor moves and student outcomes in dialogues, using two
math tutoring dialogue datasets. We find that even state-of-the-art LLMs
struggle to predict future tutor strategy while tutor strategy is highly
indicative of student outcomes, outlining a need for more powerful methods to
approach this task.
Study at a glance
- Question: Can AI predict what tutors will do next and how students will perform in tutoring conversations?
- Setting and data: Two math tutoring dialogue datasets.
- Method: Tests how well two AI models predict tutor moves and student outcomes from dialogue data.
Why it matters
Tutor strategies can have significant effects on student outcomes, creating a need to predict how tutors will behave and impact students. However, few works have studied predicting tutor strategy in dialogues.
Key findings
- In math tutoring dialogues, even state-of-the-art language models struggle to predict what strategies a tutor will use next.
- A tutor's instructional strategy is highly indicative of student outcomes during tutoring conversations.
- The specific strategies used by tutors can have significant effects on student learning outcomes.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
