Date
Publisher
arXiv
Plagiarism in programming assignments is a persistent issue in computer
science education, increasingly complicated by the emergence of automated
obfuscation attacks. While software plagiarism detectors are widely used to
identify suspicious similarities at scale and are resilient to simple
obfuscation techniques, they are vulnerable to advanced obfuscation based on
structural modification of program code that preserves the original program
behavior. While different defense mechanisms have been proposed to increase
resilience against these attacks, their current evaluation is limited to the
scope of attacks used and lacks a comprehensive investigation regarding
AI-based obfuscation. In this paper, we investigate the resilience of these
defense mechanisms against a broad range of automated obfuscation attacks,
including both algorithmic and AI-generated methods, and for a wide variety of
real-world datasets. We evaluate the improvements of two defense mechanisms
over the plagiarism detector JPlag across over four million pairwise program
comparisons. Our results show significant improvements in detecting obfuscated
plagiarism instances, and we observe an improved detection of AI-generated
programs, even though the defense mechanisms are not designed for this use
case. Based on our findings, we provide an in-depth discussion of their broader
implications for academic integrity and the role of AI in education.
Study at a glance
- Question: How well can defense tools help software plagiarism detectors identify code disguised by AI?
- Setting and data: Real-world code datasets involving over four million pairwise program comparisons.
- Method: Computational evaluation testing two defense mechanisms added to a plagiarism detector against automated and AI-generated code alterations.
Why it matters
Plagiarism in programming assignments is increasingly complicated by automated obfuscation attacks that alter code while preserving behavior. Existing defenses help detectors, but their evaluations are limited and lack a comprehensive investigation regarding AI-based obfuscation.
Key findings
- In computer science programming assignments, adding defense mechanisms to standard plagiarism detectors significantly improved the identification of disguised, copied code.
- These defenses also improved the detection of AI-generated computer code, even though the tools were not originally designed for that purpose.
- Standard software plagiarism detectors remain vulnerable to advanced techniques that alter code structure while preserving the original program behavior.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
