Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Harnessing Llm For Noise-Robust Cognitive Diagnosis In Web-Based Intelligent Education Systems

Date
Publisher
arXiv
Cognitive diagnostics in the Web-based Intelligent Education System (WIES) aims to assess students' mastery of knowledge concepts from heterogeneous, noisy interactions. Recent work has tried to utilize Large Language Models (LLMs) for cognitive diagnosis, yet LLMs struggle with structured data and are prone to noise-induced misjudgments. Specially, WIES's open environment continuously attracts new students and produces vast amounts of response logs, exacerbating the data imbalance and noise issues inherent in traditional educational systems. To address these challenges, we propose DLLM, a Diffusion-based LLM framework for noise-robust cognitive diagnosis. DLLM first constructs independent subgraphs based on response correctness, then applies relation augmentation alignment module to mitigate data imbalance. The two subgraph representations are then fused and aligned with LLM-derived, semantically augmented representations. Importantly, before each alignment step, DLLM employs a two-stage denoising diffusion module to eliminate intrinsic noise while assisting structural representation alignment. Specifically, unconditional denoising diffusion first removes erroneous information, followed by conditional denoising diffusion based on graph-guided to eliminate misleading information. Finally, the noise-robust representation that integrates semantic knowledge and structural information is fed into existing cognitive diagnosis models for prediction. Experimental results on three publicly available web-based educational platform datasets demonstrate that our DLLM achieves optimal predictive performance across varying noise levels, which demonstrates that DLLM achieves noise robustness while effectively leveraging semantic knowledge from LLM.
Study at a glance
  • Question: Can combining language models with two-stage diffusion denoising improve student cognitive diagnosis when learning data contain noise?
  • Setting and data: ASSIST0910, Junyi1808, and Eedi50 educational datasets, spanning 2009 to 2020, covering up to 4,918 students and over 1.3 million response logs.
  • Method: Machine learning model evaluation comparing a diffusion-based framework (DLLM) against baseline diagnostic models across varied simulated noise levels.
Why it matters

Online learning systems produce imbalanced data and noisy student response logs. Using large language models to diagnose student mastery can introduce external errors and hallucinations, making noise removal critical during model integration.

Key findings
  • DLLM consistently outperformed baseline cognitive diagnosis frameworks across all three datasets, achieving higher accuracy, AUC, and interpretability agreement scores.
  • Across tests with 5%, 10%, and 15% added noise, the proposed framework maintained the highest diagnostic accuracy compared to all baseline models.
  • Removing the unconditional denoising diffusion module caused greater accuracy drops under high noise levels, while removing conditional denoising degraded performance more at lower noise levels.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?