Search and Filter

Generative AI Usage and Exam Performance

Date

AI Overview

Why it matters

Higher education faces extensive debate over generative AI's role and potential bans, yet empirical evidence on how student AI usage affects tangible exam performance remains unexamined.

Study at a glance
  • Question: What is the effect of students' generative AI usage on their exam scores?
  • Setting and data: Introductory accounting course at a German university, winter term 2023/2024, 193 university students.
  • Method: Fixed effects ordinary least squares regression (multivariate OLS) combined with AI detection on student essays.
  • AI tools: ChatGPT, ChatGPT 4, ChatGPT 3, ZeroGPT
Key findings
  • Students identified as using generative AI scored on average 6.71 points lower out of 100 on their final exam.
  • The negative effect was particularly large for students with higher prior achievement and high tutorial attendance, while insignificant for students with lower grades or attendance.
  • Among students repeating the exam, those using generative AI improved significantly less than non-users compared to their pre-AI attempt.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

This study evaluates the impact of students' usage of generative artificial intelligence (GenAI) tools such as ChatGPT on their exam performance. We analyse student essays using GenAI detection systems to identify GenAI users among the cohort. Employing multivariate regression analysis, we find that students using GenAI tools score on average 6.71 (out of 100) points lower than non-users. While GenAI may offer benefits for learning and engagement, the way students actually use it correlates with diminished exam outcomes. Exploring the underlying mechanism, additional analyses show that the effect is particularly detrimental to students with high learning potential, suggesting an effect whereby GenAI tool usage hinders learning. Our findings provide important empirical evidence for the ongoing debate on the integration of GenAI in higher education and underscores the necessity for educators, institutions, and policymakers to carefully consider its implications for student performance.