Date
Publisher
arXiv
Generative artificial intelligence (GenAI) has challenged the validity of unsupervised online assessment, especially in technical subjects where plausible answers can be produced with little effort. This paper reports lessons from designing and implementing an AI-aware, AI-testing assessment in a large second-year undergraduate database systems module. The design combined two linked elements: (1) a structured three-part response format (X1-X2-X3) in which students documented a sourced answer, produced their own answer, and evaluated the sourced output; and (2) an AI-aware question-design process in which draft tasks were stress-tested against contemporary GenAI tools and revised when generic prompting produced superficially adequate answers. The account draws on archived assessment materials, rubrics, planning records, design-time GenAI trials, practice-response data, attainment records, and external review comments. Its main contribution is a reusable assessment-design method rather than a claim of measured learning gains. We show how the pattern developed across iterations and how it can support authentic assessment, visible AI literacy, student judgement, and more transparent marking. The paper offers practical guidance for lecturers adapting assessment to routine GenAI use, focusing on testing AI literacy rather than penalising students for misconduct.
Study at a glance
- Question: How can assessments be redesigned to make student AI use, reasoning, and reflection visible and gradable?
- Setting and data: Second-year undergraduate database systems module at a UK university, 2025/26, 275 enrolled students with coursework and exam records.
- Method: Practice-based assessment design research with descriptive analysis across four iterative deployments.
Why it matters
Generative AI challenges unmonitored online assessment validity because tasks often fail to reveal whether students exercised individual understanding, while blanket bans and AI detectors are unreliable.
Key findings
- Integrating mandatory AI use through a three-part format made student sourcing, revision, and critical evaluation visible to markers without causing scores to artificially hit a maximum ceiling.
- Stress-testing draft questions with AI showed that generic prompts were too easily solved, requiring redesign toward context-rich, scenario-based tasks with specific constraints to elicit student reasoning.
- Among 22 complete practice responses, 20 included an explicit technical critique or correction of the AI-sourced answer, and 14 reintroduced course concepts or terminology.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
