Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

RATAS: A Generative AI Framework for Explainable and Scalable Automated Answer Grading

Authors
Masoud Safilian,
Amin Beheshti,
Stephen Elbourn
Date
Publisher
arXiv
Automated answer grading is a critical challenge in educational technology, with the potential to streamline assessment processes, ensure grading consistency, and provide timely feedback to students. However, existing approaches are often constrained to specific exam formats, lack interpretability in score assignment, and struggle with real-world applicability across diverse subjects and assessment types. To address these limitations, we introduce RATAS (Rubric Automated Tree-based Answer Scoring), a novel framework that leverages state-of-the-art generative AI models for rubric-based grading of textual responses. RATAS is designed to support a wide range of grading rubrics, enable subject-agnostic evaluation, and generate structured, explainable rationales for assigned scores. We formalize the automatic grading task through a mathematical framework tailored to rubric-based assessment and present an architecture capable of handling complex, real-world exam structures. To rigorously evaluate our approach, we construct a unique, contextualized dataset derived from real-world project-based courses, encompassing diverse response formats and varying levels of complexity. Empirical results demonstrate that RATAS achieves high reliability and accuracy in automated grading while providing interpretable feedback that enhances transparency for both students and nstructors.
Study at a glance
  • Question: Can generative AI accurately grade student written answers using rubrics while clearly explaining its scores?
  • Setting and data: Text responses of varying complexity from real-world project-based courses.
  • Method: Tests an automated AI grading framework on a dataset of course responses.
Why it matters

Automated answer grading can streamline assessments, ensure consistency, and provide timely student feedback. However, existing approaches are often constrained to specific exam formats, lack interpretability in score assignment, and struggle across diverse subjects and assessment types.

Key findings
  • In real-world project-based courses, a generative AI grading framework evaluated complex student text responses with high reliability and accuracy.
  • The AI tool provided structured, explainable rationales for its scores, enhancing grading transparency for both students and instructors.
  • The grading system adapted to a wide range of rubrics and successfully evaluated diverse response formats across multiple subjects.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

What is the application?
Who is the user?
Who age?