Date
Publisher
arXiv
Today's generative AI systems are tuned to present information by default rather than engage users in service of learning as a human tutor would. To address the wide range of potential education use cases for these systems, we reframe the challenge of injecting pedagogical behavior as one of \textit{pedagogical instruction following}, where training and evaluation examples include system-level instructions describing the specific pedagogy attributes present or desired in subsequent model turns. This framing avoids committing our models to any particular definition of pedagogy, and instead allows teachers or developers to specify desired model behavior. It also clears a path to improving Gemini models for learning -- by enabling the addition of our pedagogical data to post-training mixtures -- alongside their rapidly expanding set of capabilities. Both represent important changes from our initial tech report. We show how training with pedagogical instruction following produces a LearnLM model (available on Google AI Studio) that is preferred substantially by expert raters across a diverse set of learning scenarios, with average preference strengths of 31\% over GPT-4o, 11\% over Claude 3.5, and 13\% over the Gemini 1.5 Pro model LearnLM was based on.
Study at a glance
- Question: How can generative AI models be improved to act more like human tutors to support learning?
- Setting and data: Diverse learning scenarios evaluated by expert raters.
- Method: Compares expert ratings of an AI model trained to follow educational instructions (pedagogical instruction following) against other AI models.
Why it matters
Today's generative AI systems are tuned to present information by default rather than engage users in service of learning as a human tutor would. Instead, education requires models that let teachers or developers specify desired pedagogical behavior.
Key findings
- Across diverse learning scenarios, expert raters preferred the education-adapted model by 11% to 31% over other leading commercial AI models.
- Expert raters preferred the education-adapted model 13% more than the original AI model it was built upon.
- The model allows teachers and developers to specify desired instructional behaviors without committing the system to a single definition of teaching.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who age?
