Date
Publisher
arXiv
As artificial intelligence tools become ubiquitous in education, maintaining
academic integrity while accommodating pedagogically beneficial AI assistance
presents unprecedented challenges. Current AI detection systems fail to control
false positive rates (FPR) and suffer from bias against minority student
groups, prompting institutional suspensions of these technologies. Watermarking
techniques offer statistical rigor through precise $p$-values but remain
untested in educational contexts where students may use varying levels of
permitted AI edits. We present the first adaptation of watermarking-based
detection methods for classroom settings, introducing conformal methods that
effectively control FPR across diverse classroom settings. Using essays from
native and non-native English speakers, we simulate seven levels of AI editing
interventions--from grammar correction to content expansion--across multiple
language models and watermarking schemes, and evaluate our proposal under these
different setups. Our findings provide educators with quantitative frameworks
to enforce academic integrity standards while embracing AI integration in the
classroom.
Study at a glance
- Question: How can watermarking reliably detect different levels of student AI writing assistance without falsely accusing students?
- Setting and data: Essays written by native and non-native English speakers.
- Method: Computational simulation testing an AI watermarking detection framework across seven levels of AI editing and multiple language models.
Why it matters
Current AI detection systems fail to control false positive rates and suffer from bias against minority student groups. Additionally, watermarking techniques remain untested in educational contexts where students use varying levels of permitted AI edits.
Key findings
- Across essays from native and non-native English speakers, adapted AI watermarking methods effectively controlled false accusation rates across diverse classroom settings.
- Current AI detection systems fail to control false positive rates and suffer from bias against minority student groups.
- Watermarking-based detection reliably manages false positives across multiple language models and seven levels of AI editing, from grammar correction to content expansion.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
