Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessment

Authors
Farzana Zahid,
Anjalika Sewwandi,
Lee Brandon,
Vimal Kumar,
Roopak Sinha
Date
Publisher
arXiv
Due to perceptions of efficiency and significant productivity gains, various organisations, including in education, are adopting Large Language Models (LLMs) into their workflows. Educator-facing, learner-facing, and institution-facing LLMs, collectively, Educational Large Language Models (eLLMs), complement and enhance the effectiveness of teaching, learning, and academic operations. However, their integration into an educational setting raises significant cybersecurity concerns. A comprehensive landscape of contemporary attacks on LLMs and their impact on the educational environment is missing. This study presents a generalised taxonomy of fifty attacks on LLMs, which are categorized as attacks targeting either models or their infrastructure. The severity of these attacks is evaluated in the educational sector using the DREAD risk assessment framework. Our risk assessment indicates that token smuggling, adversarial prompts, direct injection, and multi-step jailbreak are critical attacks on eLLMs. The proposed taxonomy, its application in the educational environment, and our risk assessment will help academic and industrial practitioners to build resilient solutions that protect learners and institutions.
Study at a glance
  • Question: What cyberattacks threaten AI models used in education, and how severe are these risks?
  • Setting and data: Fifty cyberattacks on large language models evaluated for the education sector
  • Method: Classification of attacks into categories and risk evaluation (DREAD risk assessment framework)
Why it matters

Integrating educational large language models raises significant cybersecurity concerns. However, a comprehensive landscape of contemporary attacks on these models and their impact on the educational environment is missing.

Key findings
  • In educational settings, analysis indicates that token smuggling, adversarial prompts, direct injection, and multi-step jailbreaks are critical attacks on language models.
  • Language models face 50 distinct types of cyberattacks, which target either the AI models themselves or their underlying technical infrastructure.
  • Integrating language models into teaching, learning, and academic operations introduces significant cybersecurity concerns for educational institutions.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

Who is the user?
Study design