Date
Publisher
arXiv
Massive Open Online Courses (MOOCs) lack direct interaction between learners and instructors, making it challenging for learners to understand new knowledge concepts. Recently, learners have increasingly used Large Language Models (LLMs) to support them in acquiring new knowledge. However, LLMs are prone to hallucinations which limits their reliability. Retrieval-Augmented Generation (RAG) addresses this issue by retrieving relevant documents before generating a response. However, the application of RAG across different MOOCs is limited by unstructured learning material. Furthermore, current RAG systems do not actively guide learners toward their learning needs. To address these challenges, we propose a Graph RAG pipeline that leverages Educational Knowledge Graphs (EduKGs) and Personal Knowledge Graphs (PKGs) to guide learners to understand knowledge concepts in the MOOC platform CourseMapper. Specifically, we implement (1) a PKG-based Question Generation method to recommend personalized questions for learners in context, and (2) an EduKG-based Question Answering method that leverages the relationships between knowledge concepts in the EduKG to answer learner selected questions. To evaluate both methods, we conducted a study with 3 expert instructors on 3 different MOOCs in the MOOC platform CourseMapper. The results of the evaluation show the potential of Graph RAG to empower learners to understand new knowledge concepts in a personalized learning experience.
Study at a glance
- Question: Can a knowledge graph system effectively generate relevant questions and accurate answers to help online learners understand difficult concepts?
- Setting and data: Three online courses (Learning Analytics, Human-Computer Interaction, Web Technologies) at a German university, evaluated by three expert instructors on 100 question-answer pairs.
- Method: System development and prototype evaluation using expert reviews of generated questions and answers.
Why it matters
Online courses lack direct instructor interaction, making complex concepts difficult to master. While large language models can help, they frequently hallucinate, and existing retrieval tools neither organize unstructured materials well nor guide learners toward what to ask.
Key findings
- Instructors rated generated personalized questions highly for fluency (2.98 out of 3), clarity (2.84), and relevance to slides (2.85) and misunderstood concepts (2.67).
- Generated answers achieved only 45 percent accuracy across 100 evaluated pairs, with instructors noting that many extracted responses were too abstract or imprecise.
- Question-answering accuracy varied widely by course topic, reaching 56.67 percent in Learning Analytics, 45.00 percent in Human-Computer Interaction, and 33.33 percent in Web Technologies.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
