Date
AI Overview
Why it matters
Understanding prerequisite relationships is critical for identifying knowledge gaps, but instructors in large courses cannot feasibly diagnose individual misunderstandings. Existing knowledge graph methods miss the fine-grained, instructor-aligned pedagogical signals embedded in course materials.
Study at a glance
- Question: Can course knowledge graphs automatically capture learning progressions using teaching order and concept roles from lecture materials?
- Setting and data: Five university courses from four institutions and five Coursera MOOCs in machine learning and data structures and algorithms.
- Method: System development and model evaluation using language models, clustering, human expert review, and benchmark comparisons.
- AI tools: INSTRUCTKG, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, Qwen2.5-14B, gpt-oss-120b, GPT-4o-mini
Key findings
- INSTRUCTKG achieved 95% to 98% concept significance and 52% to 85% relationship accuracy across courses, outperforming baseline models by 11 to 41 percentage points on relationships.
- Human experts judged 65% to 98% of extracted concept nodes and 58% to 85% of relationship triplets as pedagogically correct across five university courses.
- On a zero-shot MOOC prerequisite benchmark, the system achieved F1 scores of 0.35 in machine learning and 0.29 in data structures and algorithms, surpassing training-free baselines.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Mastering educational concepts requires understanding both their prerequisites (e.g., recursion before merge sort) and sub-concepts (e.g., merge sort as part of sorting algorithms). Capturing these dependencies is critical for identifying students' knowledge gaps and enabling targeted intervention for personalized learning. This is especially challenging in large-scale courses, where instructors cannot feasibly diagnose individual misunderstanding or determine which concepts need reinforcement. While knowledge graphs offer a natural representation for capturing these conceptual relationships at scale, existing approaches are either surface-level (focusing on course-level concepts like "Algorithms" or logistical relationships such as course enrollment), or disregard the rich pedagogical signals embedded in instructional materials. We propose InstructKG, a framework for automatically constructing instructor-aligned knowledge graphs that capture a course's intended learning progression. Given a course's lecture materials (slides, notes, etc.), InstructKG extracts significant concepts as nodes and infers learning dependencies as directed edges (e.g., "part-of" or "depends-on" relationships). The framework synergizes the rich temporal and semantic signals unique to educational materials (e.g., "recursion" is taught before "mergesort"; "recursion" is mentioned in the definition of "merge sort") with the generalizability of large language models. Through experiments on real-world, diverse lecture materials across multiple courses and human-based evaluation, we demonstrate that InstructKG captures rich, instructor-aligned learning progressions.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
Publication Source
Publisher
arXiv
