Date
AI Overview
Why it matters
While large language models are transforming education, concerns about their risks to students and learning systems are rising. However, empirical evidence on these risks remains fragmented across disciplines.
Study at a glance
- Question: What educational applications of large language models have been empirically studied, how, and what risks stem from them?
- Setting and data: 70 empirical studies published between 2023 and 2025 across computer science, education, and psychology, identified through five academic databases.
- Method: Systematic literature review (PRISMA guidelines)
- AI tools: ChatGPT, GPT-3.5, GPT-4, LLaMA 2, Gemini, Bard
Key findings
- Most evaluated applications rely on GPT models (predominantly GPT-3.5 and GPT-4) and cluster around operational effectiveness, personalized applications, and interactive learning tools.
- System-level model risks include superficial understanding, bias, limited robustness, anthropomorphism, and hallucinations, with bias appearing across evaluation, personalized content, and recommendations.
- Student interaction with large language models is associated with reduced neural activity, over-reliance, diminished critical engagement, metacognitive laziness, and potential erosion of memory retention.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Large Language Models (LLMs) are transforming education by enabling personalization, feedback, and knowledge access, while also raising concerns about risks to students and learning systems. Yet empirical evidence on these risks remains fragmented. This paper presents a systematic review of 70 empirical studies across computer science, education, and psychology. Guided by four research questions, we examine: (i) which applications of LLMs in education have been most frequently explored; (ii) how researchers have measured their impact; (iii) which risks stem from such applications; and (iv) what mitigation strategies have been proposed. We find that research on LLMs clusters around three domains: operational effectiveness, personalized applications, and interactive learning tools. Across these, model-level risks include superficial understanding, bias, limited robustness, anthropomorphism, hallucinations, privacy concerns, and knowledge constraints. When learners interact with LLMs, these risks extend to cognitive and behavioural outcomes, including reduced neural activity, over-reliance, diminished independent learning skills, and a loss of student agency. To capture this progression, we propose an LLM-Risk Adapted Learning Model that illustrates how technical risks cascade through interaction and interpretation to shape educational outcomes. As the first synthesis of empirically assessed risks, this review provides a foundation for responsible, human-centred integration of LLMs in education.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
Methods
AI tools
Publication Source
Publisher
arXiv
