Date
AI Overview
Why it matters
Concerns exist that students substitute generative AI for cognitive effort to earn high grades without learning, potentially eroding the signaling value of grades and undermining student satisfaction and engagement.
Study at a glance
- Question: Did generative AI availability increase student grades or alter course satisfaction in courses with susceptible assessments?
- Setting and data: Large public Midwest U.S. university, 2016–2025, 138,386 students across 72,730 course offerings and administrative records.
- Method: Difference-in-differences design comparing outcomes across courses differing in AI susceptibility before and after ChatGPT.
- AI tools: ChatGPT, GPT-5.1
Key findings
- Effects of ChatGPT's release on student grades were small and non-robust, ranging from 0.010 to 0.071 grade points depending on how pandemic trends were modeled.
- Student withdrawal and failure rates showed no significant changes after ChatGPT's release, and grade effects did not differ across students' prior academic preparation.
- Student course evaluations revealed no robust evidence that generative AI availability changed self-reported subject understanding, interest, or perceived relative workload.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
The spread of generative AI (GenAI) in higher education has raised concerns that students offload cognitive effort to AI, earning high grades without learning. If this "GenAI substitution hypothesis" is true, grades should rise disproportionately in GenAI-susceptible courses--those relying more on assessments like take-home problem sets and essays rather than in-class exams. Substitution could also affect student satisfaction, measured here as self-reported understanding and interest in the subject, which prior research links to assessments. We test the substitution hypothesis using syllabus and administrative data from a large U.S. university (2016-2025; 138,386 students; 72,730 course offerings). We measure courses' GenAI susceptibility using a human-validated LLM pipeline to extract assessment types from syllabi, and use a differences-in-differences design comparing outcomes across courses before and after ChatGPT's release, while modeling COVID-19 pandemic effects as either persistent or transient. We find no significant differential effect of GenAI availability on grades overall or among previously lower-performing students. Effects on self-reported understanding are likewise insignificant; effects on interest are significant only assuming transient pandemic effects. Our findings temper concerns that GenAI inflates grades and reduces students' satisfaction.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
Methods
AI tools
Publication Source
Publisher
arXiv
