Date
Publisher
arXiv
Coaching, which involves classroom observation and expert feedback, is a
widespread and fundamental part of teacher training. However, the majority of
teachers do not have access to consistent, high quality coaching due to limited
resources and access to expertise. We explore whether generative AI could
become a cost-effective complement to expert feedback by serving as an
automated teacher coach. In doing so, we propose three teacher coaching tasks
for generative AI: (A) scoring transcript segments based on classroom
observation instruments, (B) identifying highlights and missed opportunities
for good instructional strategies, and (C) providing actionable suggestions for
eliciting more student reasoning. We recruit expert math teachers to evaluate
the zero-shot performance of ChatGPT on each of these tasks for elementary math
classroom transcripts. Our results reveal that ChatGPT generates responses that
are relevant to improving instruction, but they are often not novel or
insightful. For example, 82% of the model's suggestions point to places in the
transcript where the teacher is already implementing that suggestion. Our work
highlights the challenges of producing insightful, novel and truthful feedback
for teachers while paving the way for future research to address these
obstacles and improve the capacity of generative AI to coach teachers.
Study at a glance
- Question: Can ChatGPT effectively coach teachers by scoring classroom instruction and providing helpful feedback?
- Setting and data: Elementary math classroom transcripts and expert math teachers.
- Method: Expert teachers evaluate ChatGPT's performance on scoring and providing feedback across three coaching tasks.
Why it matters
Coaching is fundamental to teacher training, but most teachers lack consistent, high-quality coaching due to limited resources and expertise. Generative AI could serve as a cost-effective complement to expert feedback by providing automated coaching.
Key findings
- In elementary math transcripts, 82% of ChatGPT's coaching suggestions pointed to places where teachers were already implementing the suggested instructional strategy.
- Expert teachers found that ChatGPT generated feedback relevant to improving instruction, but its suggestions were often not novel or insightful.
- The AI tool struggled to produce coaching feedback that was truthful, novel, and insightful for teachers.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
