Date
AI Overview
Why it matters
AI-assisted programming tools boost productivity but can generate insecure code. Conventional computer science education overlooks these risks, leaving students unprepared to identify and mitigate vulnerabilities in AI-assisted workflows.
Study at a glance
- Question: How do students perceive and respond to insecure AI-generated code, and how does targeted feedback affect their attitudes?
- Setting and data: 61 undergraduate students in a university cybersecurity course; 21 completed a matched post-feedback survey.
- Method: Classroom deployment with pre- and post-surveys, coding tasks using a poisoned model, and automated vulnerability feedback.
- AI tools: Bifröst, CodeGen 6.1B
Key findings
- Over 95% of students accepted insecure AI-generated code across two programming tasks when not explicitly prompted to look for vulnerabilities, despite having prior security coursework.
- Most students expressed skepticism or critical attitudes toward AI-generated code functionality and security in initial surveys, yet this skepticism rarely translated into secure coding behavior.
- Individualized feedback through the Bifröst framework was associated with a statistically significant shift toward greater skepticism and more frequent security-specific reasoning among matched survey respondents.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
The advent of Artificial Intelligence (AI), particularly large language models (LLMs), has revolutionized software development by enabling developers to specify tasks in natural language and receive corresponding code, boosting productivity. However, this shift also introduces security risks, as LLMs may generate insecure code that can be exploited by adversaries. Current educational approaches emphasize efficiency while overlooking these risks, leaving students underprepared to identify and mitigate security issues in AI-assisted workflows.
To address this gap, we present Bifst, an educational framework that cultivates security awareness in AI-augmented development. Bifröst integrates (1) a Visual Studio Code extension simulating realistic environments, (2) adversarially configured LLMs that generate insecure code, and (3) a feedback system highlighting vulnerabilities. By immersing students in tasks with compromised LLMs and providing targeted security analysis, Bifröst cultivates critical evaluation skills; classroom deployments (n=61) show vulnerability to insecure code, while a post-intervention survey (n=21) indicates increased skepticism toward LLM outputs.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
