Date
Publisher
arXiv
In high-stakes English Language Assessments, low-skill test takers may employ
memorized materials called ``templates'' on essay questions to ``game'' or fool
the automated scoring system. In this study, we introduce the automated
detection of inauthentic, templated responses (AuDITR) task, describe a machine
learning-based approach to this task and illustrate the importance of regularly
updating these models in production.
Study at a glance
- Question: How can computer systems automatically detect memorized essay templates designed to fool automated scoring?
- Setting and data: Not stated in the abstract
- Method: Machine learning-based approach
Why it matters
In high-stakes English language assessments, low-skill test takers may use memorized materials called templates on essay questions to game or fool automated scoring systems.
Key findings
- In high-stakes English language assessments, lower-skilled test takers may use memorized templates on essay questions to fool automated scoring systems.
- Machine learning models can automatically detect inauthentic, templated essay responses.
- Template-detection tools require regular updates in active production to remain effective.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
