Search and Filter

Kahaani: A Multimodal Co-Creative Storytelling System

Date

AI Overview

Why it matters

Children often struggle to sustain attention and build narrative competence with traditional storytelling delivery, while existing automated storytelling systems rarely combine co-creative text, audio, animation, and music for educational engagement.

Study at a glance
  • Question: How effectively can an AI system co-create multimodal stories with children to support engagement and learning?
  • Setting and data: Six undergraduate computer science evaluators and three parent–child pairs (children ages 6–9); stories sampled from Project Gutenberg and LLM-generated prompts.
  • Method: Multi-agent system development, component-level model evaluations using human ratings, and a user study with questionnaires and interviews.
  • AI tools: Kahaani, Gemma-2-9b, GPT-4o, XTTS-v2, CogVideoX-5b, MusicGen-Large
Key findings
  • In story generation evaluations, Gemma-2-9b achieved the highest Bradley-Terry strength score (1.224), outperforming larger models in creativity and structural consistency.
  • For content moderation, GPT-4o attained the lowest false positive rate of 9% on Project Gutenberg stories and 0% on synthetic stories.
  • Participating children rated overall storytelling experience and comprehension at 4.0 out of 5, while parents rated content age-appropriateness at 5.0 out of 5.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

This paper introduces Kahaani, a multimodal, co-creative storytelling system that leverages Generative Artificial Intelligence, designed for children to address the challenge of sustaining engagement to foster educational narrative experiences. Here we define co-creative as a collaborative creative process in which both the child and Kahaani contribute to the generation of the story. The system combines Large Language Model (LLM), Text-to-Speech (TTS), Text-to-Music (TTM), and Text-to-Video (TTV) generation to produce a rich, immersive, and accessible storytelling experience. The system grounds the co-creation process in two classical storytelling framework, Freytag's Pyramid and Propp's Narrative Functions. The main goals of Kahaani are: (1) to help children improve their English skills, (2) to teach important life lessons through story morals, and (3) to help them understand how stories are structured, all in a fun and engaging way. We present evaluations for each AI component used, along with a user study involving three parent-child pairs to assess the overall experience and educational value of the system.