Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

When LLM Essays Outscore Student Essays: What A Korean Writing Rubric Rewards And Where Readers Disagree

Authors
Shinwoo Park,
Yo-Sub Han
Date
Publisher
arXiv
LLMs now help students plan, draft, and revise essays. Educational assessment therefore faces a basic question: how should student and LLM writing be compared? Rubrics assign points to content, organization, and expression. Their total can still hide which criteria drive the comparison, where ratings approach the maximum, and where readers disagree. We therefore conducted a secondary, post hoc audit of a Korean writing study with source-informed scoring. Three Korean language and literature majors used a 16-criterion, 100-point rubric to score six student essays and 24 essays from four LLMs prompted for three school levels. After source disclosure, they helped develop the rubric and, according to the protocol, scored the Phase 2 essays without gold labels while recording source judgments. The pooled LLM mean exceeds the student mean by 18.35 points. Orthographic norms and genre-appropriate register contribute 8.19 points, or 44.7% of the gap. All 72 LLM ratings reach the orthographic maximum, and 70 reach the register maximum. The student source mean is highest or tied highest on both creativity criteria and natural Korean phrasing, where reader agreement is weak. Criterion-level auditing therefore offers a deeper account of human and LLM writing than the total alone.
What is the application?
Who is the user?
Why use AI?