Date
AI Overview
Why it matters
Generative AI can execute many procedural tasks, raising questions about whether homework still reflects student understanding and leaving limited empirical evidence on how AI availability reshapes the alignment between homework and proctored exams.
Study at a glance
- Question: How does AI availability affect student performance and alignment between unproctored homework and proctored exams in mathematics?
- Setting and data: Wenzhou-Kean University, China; undergraduate Business Calculus, Linear Algebra, and Calculus III students across Spring and Fall 2025; sections sized 25 to 50 students.
- Method: Descriptive analysis comparing unproctored homework and proctored exam score differences, Spearman rank correlations, and distributional morphology (Wasserstein distance).
Key findings
- Performance gaps between homework and final exams reached 20.9 to 25.1 points in Business Calculus, 20 to over 48 points in Linear Algebra, and 41.8 to 44.4 points in Calculus III.
- Homework-to-final-exam rank correlations collapsed in Calculus III (r = 0.273 to 0.391) and showed extreme instability in Linear Algebra, while remaining moderate in Business Calculus (r = 0.410 to 0.546).
- Hand-written, TA-graded homework did not produce substantively different performance patterns or smaller integrity gaps compared to automated online WebWorK homework.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
The rapid adoption of generative artificial intelligence (AI) tools in higher education is transforming how students engage with undergraduate mathematics, raising concerns about learning and assessment validity. This study examines the impact of AI accessibility across a two-semester, multi-course dataset including Business Calculus, Linear Algebra, and Calculus III. By comparing unproctored homework and proctored exam performance, we analyze how student learning behaviors shift in AI-accessible environments, particularly through epistemic off-loading of mathematical work. Guided by a sociocognitive framework, we employ complementary measures -- performance gaps, homework-exam correlations, and Wasserstein distance -- to characterize divergence between practice and mastery. Results reveal a growing integrity gap as course content shifts from procedural to conceptual and spatially intensive mathematics. In both Business Calculus and Linear Algebra, differences in homework format (online versus hand-written, TA-graded) do not yield substantively different performance patterns, indicating that paper-based homework is not inherently more resistant to AI-mediated offloading. While homework retains partial predictive validity in procedural courses, upper-division courses exhibit a collapse in alignment between homework and exams, indicating that unproctored assessments increasingly reflect synthetic fluency rather than internalized understanding. These findings highlight the need to rethink assessment practices in the AI era.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
Methods
Publication Source
Publisher
arXiv
