Date
Publisher
arXiv
The advent of Artificial Intelligence (AI), particularly large language models (LLMs), has revolutionized software development by enabling developers to specify tasks in natural language and receive corresponding code, boosting productivity. However, this shift also introduces security risks, as LLMs may generate insecure code that can be exploited by adversaries. Conventional educational approaches emphasize efficiency while overlooking these risks, leaving students unprepared to identify and mitigate security issues in AI-assisted workflows. To surface this gap, we present Bifröst, a classroom measurement and feedback framework that pairs an adversarially configured code-generation model with a VS Code extension and automated vulnerability reports that instructors can use to guide follow-up discussions. Through classroom deployments with undergraduate students ($n=61$), we observe that students frequently accepted insecure LLM-generated code despite prior security coursework and stated skepticism. A post-feedback survey ($n=21$) provides preliminary evidence that students' stated trust shifted toward greater skepticism after receiving Bifröst feedback, and that some students articulated more security-specific concerns about AI-generated code.
Study at a glance
- Question: How do students perceive and handle insecure AI-generated code, and how does targeted feedback change their security trust?
- Setting and data: Undergraduate cybersecurity course, 61 students with prior AI programming experience (21 matched in post-survey).
- Method: Classroom deployment using a custom framework with pre- and post-surveys, coding tasks, and static analysis (Wilcoxon signed-rank test).
Why it matters
AI-assisted programming tools can generate insecure or poisoned code. Although students must learn to critically assess these outputs, educational approaches often emphasize efficiency, leaving learners unprepared to identify and mitigate security vulnerabilities in practice.
Key findings
- Over 95% of undergraduate students accepted insecure AI-generated code during programming tasks, despite 83.6% having completed security coursework.
- Prior to the tasks, most students exhibited skepticism rather than blind trust, with 50.8% distrusting the security of AI-generated code.
- Receiving individualized vulnerability feedback was associated with a statistically significant shift toward greater skepticism and more frequent security-specific reasoning.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
