Date
AI Overview
Why it matters
Traditional academic thesis evaluation is time-consuming and prone to evaluator variability. Automated systems often focus narrowly on essays or short answers and lack transparency, limiting educator trust.
Study at a glance
- Question: How can an AI framework systematically and transparently support academic thesis assessment using rubrics and structured reasoning?
- Setting and data: IU International University of Applied Sciences, Germany; pilot implementations with thesis documents across academic disciplines.
- Method: Design science research developing and piloting an AI framework (RubiSCoT)
- AI tools: RubiSCoT, ChatGPT, GPT-4o
Key findings
- Preliminary institutional testing showed promising improvements in thesis evaluation consistency and time efficiency.
- A multi-step evaluation sequence combining structured chain-of-thought prompting with human-created expectation documents provided clear, contestable grading criteria across different thesis sections.
- Generating visual flow diagrams of extracted thesis components highlighted structural weaknesses and misalignments between research objectives and conclusions.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
The evaluation of academic theses is a cornerstone of higher education,
ensuring rigor and integrity. Traditional methods, though effective, are
time-consuming and subject to evaluator variability. This paper presents
RubiSCoT, an AI-supported framework designed to enhance thesis evaluation from
proposal to final submission. Using advanced natural language processing
techniques, including large language models, retrieval-augmented generation,
and structured chain-of-thought prompting, RubiSCoT offers a consistent,
scalable solution. The framework includes preliminary assessments,
multidimensional assessments, content extraction, rubric-based scoring, and
detailed reporting. We present the design and implementation of RubiSCoT,
discussing its potential to optimize academic assessment processes through
consistent, scalable, and transparent evaluation.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
Publication Source
Publisher
arXiv
