Date
Publisher
arXiv
Due to perceptions of efficiency and significant productivity gains, various
organisations, including in education, are adopting Large Language Models
(LLMs) into their workflows. Educator-facing, learner-facing, and
institution-facing LLMs, collectively, Educational Large Language Models
(eLLMs), complement and enhance the effectiveness of teaching, learning, and
academic operations. However, their integration into an educational setting
raises significant cybersecurity concerns. A comprehensive landscape of
contemporary attacks on LLMs and their impact on the educational environment is
missing. This study presents a generalised taxonomy of fifty attacks on LLMs,
which are categorized as attacks targeting either models or their
infrastructure. The severity of these attacks is evaluated in the educational
sector using the DREAD risk assessment framework. Our risk assessment indicates
that token smuggling, adversarial prompts, direct injection, and multi-step
jailbreak are critical attacks on eLLMs. The proposed taxonomy, its application
in the educational environment, and our risk assessment will help academic and
industrial practitioners to build resilient solutions that protect learners and
institutions.
Study at a glance
- Question: What cyberattacks threaten AI models used in education, and how severe are these risks?
- Setting and data: Fifty cyberattacks on large language models evaluated for the education sector
- Method: Classification of attacks into categories and risk evaluation (DREAD risk assessment framework)
Why it matters
Integrating educational large language models raises significant cybersecurity concerns. However, a comprehensive landscape of contemporary attacks on these models and their impact on the educational environment is missing.
Key findings
- In educational settings, analysis indicates that token smuggling, adversarial prompts, direct injection, and multi-step jailbreaks are critical attacks on language models.
- Language models face 50 distinct types of cyberattacks, which target either the AI models themselves or their underlying technical infrastructure.
- Integrating language models into teaching, learning, and academic operations introduces significant cybersecurity concerns for educational institutions.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Why use AI?
Study design
