Date
Publisher
arXiv
Personalization is crucial for effective learning, yet online learning, designed for widespread availability and open access, lacks personalized guidance. Recent advancements in large language models (LLMs) offer opportunities to bridge this gap. We explore how LLM-driven tools may be designed to support personalized and adaptive learning and examine how they shape user experience and learning outcomes. We iteratively designed \tool{} to support online learning by providing personalized study plans, real-time contextual assistance, and adaptive learning activities. A preliminary study ($n=24$) assessed the effectiveness and usability of \tool{} and informed refinements in our system, which we then evaluated ($n = 16$) against a combination of a state-of-the-art online learning platform and an LLM for learning support. Results indicate that \tool{} advances AI pedagogy by improving both learning outcomes and user experience compared to existing online learning and support tools. This work advances our understanding of the design space of personalized, AI-driven educational tools and their potential impact on user experience.
Study at a glance
- Question: How does an integrated, AI-powered system for study planning, real-time assistance, and plan adaptation affect online learning and user experience?
- Setting and data: United States adults recruited via university mailing lists; preliminary study with 24 participants and final study with 16 participants studying Khan Academy history courses.
- Method: Within-subjects lab experiments comparing the system to a baseline of an online learning platform paired with an AI assistant, plus interviews.
Why it matters
Online learning often lacks personalized guidance, leading to low completion rates. Emerging AI systems offer promise, but how to integrate these capabilities across a complete learning workflow remains underexplored.
Key findings
- Participants scored significantly higher on both comprehension quizzes when using LearnMate2 compared to the baseline condition using an online course platform with Gemini-2.5-pro.
- Learners reported significantly higher usefulness, ease of use, satisfaction, and usability for LearnMate2's real-time assistance and adaptive plan features compared to the baseline.
- Qualitative feedback showed learners preferred mixed-initiative planning where the AI provides structured recommendations while allowing them to maintain control and make manual adjustments.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
