The Conversational Exam: A Scalable Assessment Design For The Ai Era

Lorena A. Barba, George Washington University

Laura Stegner, George Washington University

Date

01/2026

AI Overview

Why it matters

Generative AI can cause an illusion of competence when students offload tasks without engagement. Simply banning AI or adding rules is ineffective, leaving instructors unsure how to maintain valid assessment.

Study at a glance

Question: Can small-group oral coding exams provide a scalable and valid assessment format in the era of generative AI?
Setting and data: Undergraduate engineering computations course, George Washington University, 58 students across two days in 10 half-hour sessions.
Method: Design and implementation case study evaluating a structured, small-group oral examination format using structured observation rubrics.
AI tools: ChatGPT, Google Gemini

Key findings

Examining 58 students in small groups across 10 half-hour sessions over two days demonstrated that oral coding exams can scale to typical class sizes.
The class average on the two conversational exam administrations was around 80 percent, effectively differentiating genuine understanding from surface performance.
While the structured format enabled successful assessment, some students felt frustrated and viewed the randomized question format as unfair, and coordinating extra-time accommodations proved challenging.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Traditional assessment methods collapse when students use generative AI to complete work without genuine engagement, creating an illusion of competence where they believe they're learning but aren't. This paper presents the conversational exam -- a scalable oral examination format that restores assessment validity by having students code live while explaining their reasoning. Drawing on human-computer interaction principles, we examined 58 students in small groups across just two days, demonstrating that oral exams can scale to typical class sizes. The format combines authentic practice (students work with documentation and supervised AI access) with inherent validity (real-time performance cannot be faked). We provide detailed implementation guidance to help instructors adapt this approach, offering a practical path forward when many educators feel paralyzed between banning AI entirely or accepting that valid assessment is impossible.