Search and Filter

Rubiscot: A Framework For Ai-Supported Academic Assessment

Date

AI Overview

Why it matters

Traditional academic thesis evaluation is time-consuming and prone to evaluator variability. Automated systems often focus narrowly on essays or short answers and lack transparency, limiting educator trust.

Study at a glance
  • Question: How can an AI framework systematically and transparently support academic thesis assessment using rubrics and structured reasoning?
  • Setting and data: IU International University of Applied Sciences, Germany; pilot implementations with thesis documents across academic disciplines.
  • Method: Design science research developing and piloting an AI framework (RubiSCoT)
  • AI tools: RubiSCoT, ChatGPT, GPT-4o
Key findings
  • Preliminary institutional testing showed promising improvements in thesis evaluation consistency and time efficiency.
  • A multi-step evaluation sequence combining structured chain-of-thought prompting with human-created expectation documents provided clear, contestable grading criteria across different thesis sections.
  • Generating visual flow diagrams of extracted thesis components highlighted structural weaknesses and misalignments between research objectives and conclusions.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

The evaluation of academic theses is a cornerstone of higher education, ensuring rigor and integrity. Traditional methods, though effective, are time-consuming and subject to evaluator variability. This paper presents RubiSCoT, an AI-supported framework designed to enhance thesis evaluation from proposal to final submission. Using advanced natural language processing techniques, including large language models, retrieval-augmented generation, and structured chain-of-thought prompting, RubiSCoT offers a consistent, scalable solution. The framework includes preliminary assessments, multidimensional assessments, content extraction, rubric-based scoring, and detailed reporting. We present the design and implementation of RubiSCoT, discussing its potential to optimize academic assessment processes through consistent, scalable, and transparent evaluation.