Date
Publisher
arXiv
Knowledge Tracing (KT) aims to mine students' evolving knowledge states and predict their future question-answering performance. Existing methods based on heterogeneous information networks (HINs) are prone to introducing noises due to manual or random selection of meta-paths and lack necessary quality assessment of meta-path instances. Conversely, recent large language models (LLMs)-based methods ignore the rich information across students, and both paradigms struggle to deliver consistently accurate and evidence-based explanations. To address these issues, we propose an innovative framework, HIN-LLM Synergistic Enhanced Knowledge Tracing (HISE-KT), which seamlessly integrates HINs with LLMs. HISE-KT first builds a multi-relationship HIN containing diverse node types to capture the structural relations through multiple meta-paths. The LLM is then employed to intelligently score and filter meta-path instances and retain high-quality paths, pioneering automated meta-path quality assessment. Inspired by educational psychology principles, a similar student retrieval mechanism based on meta-paths is designed to provide a more valuable context for prediction. Finally, HISE-KT uses a structured prompt to integrate the target student's history with the retrieved similar trajectories, enabling the LLM to generate not only accurate predictions but also evidence-backed, explainable analysis reports. Experiments on four public datasets show that HISE-KT outperforms existing KT baselines in both prediction performance and interpretability.
Study at a glance
- Question: Can combining relational learning networks and language models improve student performance prediction and generate explainable analysis reports?
- Setting and data: Four datasets: Assistment09, Slepemapy (5,000 students), Statics2011 (CMU, 331 students), and Frcsub (middle school fraction subtraction, 536 students).
- Method: Model development and benchmark evaluation comparing a network-enhanced language model against deep learning, graph, and language model baselines.
Why it matters
Accurate knowledge tracing is critical for personalized learning and instructional interventions, but existing network methods introduce noise and lack human-aligned explanations, while language model methods ignore cross-student information and risk hallucinated explanations.
Key findings
- HISE-KT outperformed all baselines across four datasets, improving accuracy by up to 24.31% and AUC by up to 29.06% compared to existing language model methods.
- Removing similar-student information led to the largest performance drops across datasets, demonstrating that cross-student interaction context is critical for student performance prediction.
- Ablation tests confirmed that filtering relational paths with a language model and providing item response theory parameters were necessary to maximize predictive accuracy.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
