Date
Publisher
arXiv
Multimodal AI can read handwritten physics solutions, but high-stakes grading requires agreement with official scores and outcomes. This study evaluated GPT-5.5-based grading on 10364 scanned pages from 520 handwritten submissions by 416 unique candidates or students across three assessments: a national Physics Olympiad theory examination, the final Olympiad selection camp with theory and experiment components, and a university quantum-mechanics examination. Each submission was graded twice by AI using the official rubrics. The second round used revised page-by-page and evidence-location instructions developed after first-round disagreement analysis. During grading, AI did not see official human marks or AI--human comparisons. Total-score correlations with official marks were high (0.91--0.97). For the final Olympiad selection, AI recovered the same five-student team as official grading. The second round improved aggregate question-part agreement, especially where first-round disagreements were larger. The main difficulty remained exact partial-credit grading, especially in experimental work. Reliable AI grading therefore depends on detailed rubrics and should be used as a second reader or audit tool under examiner control.
Why it matters
Multimodal AI can read handwritten physics solutions, but high-stakes grading requires agreement with official scores and outcomes.
Key findings
- Across 416 high school and college physics students, AI grading of handwritten exams closely matched official total scores, with correlations between 0.91 and 0.97.
- AI grading successfully selected the exact same five-student team for the final Physics Olympiad as official human grading.
- The AI struggled most with assigning exact partial credit, particularly on tasks involving experimental work.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
