A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education

Michael A. Hedderich, Cornell University

Natalie N. Bazarova, Cornell University

Wenting Zou, Pennsylvania State University

Ryun Shim, Cornell University

Xinda Ma, Cornell University

Qian Yang, Cornell University

Date

02/2024

AI Overview

Why it matters

Cyberbullying harms teenagers' mental health, but teacher shortages make personal instruction on upstanding difficult. Teacher-built chatbots could scale interactive guidance, but educators' needs for designing these tools remain unexplored.

Study at a glance

Question: How do middle school teachers design and evaluate language model chatbots to teach students cyberbullying intervention skills?
Setting and data: Thirteen middle school teachers from the United States, Canada, and Southeast Asia.
Method: Design probe study using think-aloud protocols, contextual inquiry, and semi-structured interviews analyzed with affinity diagrams.
AI tools: Co-Pilot, Text-Davinci-003, GPT-3.5-Turbo

Key findings

Teachers approached chatbot design as playwrights, creating multi-participant role-play scenarios that allowed students to rehearse and explore prosocial bystander behaviors in safe environments.
Educators valued prompt chaining for customizing chatbots to reflect local school contexts and teaching styles while automatically rephrasing responses during practice.
Participants struggled to anticipate diverse adolescent behaviors and requested artificial intelligence writing assistance, collaborative design features, and controls to guide chatbot improvisation.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Cyberbullying harms teenagers' mental health, and teaching them upstanding intervention is crucial. Wizard-of-Oz studies show chatbots can scale up personalized and interactive cyberbullying education, but implementing such chatbots is a challenging and delicate task. We created a no-code chatbot design tool for K-12 teachers. Using large language models and prompt chaining, our tool allows teachers to prototype bespoke dialogue flows and chatbot utterances. In offering this tool, we explore teachers' distinctive needs when designing chatbots to assist their teaching, and how chatbot design tools might better support them. Our findings reveal that teachers welcome the tool enthusiastically. Moreover, they see themselves as playwrights guiding both the students' and the chatbot's behaviors, while allowing for some improvisation. Their goal is to enable students to rehearse both desirable and undesirable reactions to cyberbullying in a safe environment. We discuss the design opportunities LLM-Chains offer for empowering teachers and the research opportunities this work opens up.