Date
Publisher
arXiv
Cognitive diagnostics in the Web-based Intelligent Education System (WIES)
aims to assess students' mastery of knowledge concepts from heterogeneous,
noisy interactions. Recent work has tried to utilize Large Language Models
(LLMs) for cognitive diagnosis, yet LLMs struggle with structured data and are
prone to noise-induced misjudgments. Specially, WIES's open environment
continuously attracts new students and produces vast amounts of response logs,
exacerbating the data imbalance and noise issues inherent in traditional
educational systems. To address these challenges, we propose DLLM, a
Diffusion-based LLM framework for noise-robust cognitive diagnosis. DLLM first
constructs independent subgraphs based on response correctness, then applies
relation augmentation alignment module to mitigate data imbalance. The two
subgraph representations are then fused and aligned with LLM-derived,
semantically augmented representations. Importantly, before each alignment
step, DLLM employs a two-stage denoising diffusion module to eliminate
intrinsic noise while assisting structural representation alignment.
Specifically, unconditional denoising diffusion first removes erroneous
information, followed by conditional denoising diffusion based on graph-guided
to eliminate misleading information. Finally, the noise-robust representation
that integrates semantic knowledge and structural information is fed into
existing cognitive diagnosis models for prediction. Experimental results on
three publicly available web-based educational platform datasets demonstrate
that our DLLM achieves optimal predictive performance across varying noise
levels, which demonstrates that DLLM achieves noise robustness while
effectively leveraging semantic knowledge from LLM.
Study at a glance
- Question: Can combining language models with two-stage diffusion denoising improve student cognitive diagnosis when learning data contain noise?
- Setting and data: ASSIST0910, Junyi1808, and Eedi50 educational datasets, spanning 2009 to 2020, covering up to 4,918 students and over 1.3 million response logs.
- Method: Machine learning model evaluation comparing a diffusion-based framework (DLLM) against baseline diagnostic models across varied simulated noise levels.
Why it matters
Online learning systems produce imbalanced data and noisy student response logs. Using large language models to diagnose student mastery can introduce external errors and hallucinations, making noise removal critical during model integration.
Key findings
- DLLM consistently outperformed baseline cognitive diagnosis frameworks across all three datasets, achieving higher accuracy, AUC, and interpretability agreement scores.
- Across tests with 5%, 10%, and 15% added noise, the proposed framework maintained the highest diagnostic accuracy compared to all baseline models.
- Removing the unconditional denoising diffusion module caused greater accuracy drops under high noise levels, while removing conditional denoising degraded performance more at lower noise levels.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Why use AI?
Study design
