Metabook: A System to Automatically Generate Interactive AR Storybooks to Improve Children's Reading

Reading is important for children to acquire knowledge, enhance cognitive abilities, and improve language skills. However, current reading methods either offer limited visual presentation, making them less interesting to children, or lack channels for children to share insights and ask questions during reading. AR/VR books provide rich visual cues that address the issue of children's lack of interest in reading, but the high production costs and need for professional expertise limit the volume of AR/VR books and children's choices. We propose Metabook, a system to automatically generate interactive AR storybooks to improve children's reading. Metabook introduces a story-to-3D-book generation scheme and a 3D avatar that combines multiple AI models as a reading companion. We invited six primary and secondary school teachers to conduct a formative study to explore the design considerations for an ideal children's AR reading tool. In the user study, we invited relevant professionals (art, computer science professionals, and a semanticist), 44 children, and six teachers to evaluate Metabook. Our user study shows that Metabook can significantly increase children's interest in reading and deepen their impression of reading materials and vocabulary in books. Teachers acknowledged Metabook's effectiveness in facilitating reading communication and enhancing reading enthusiasm by connecting verbal and visual thinking, expressing high expectations for its future potential in education.