Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

AGACCI : Affiliated Grading Agents for Criteria-Centric Interface in Educational Coding Contexts

Date
Publisher
arXiv
Recent advances in AI-assisted education have encouraged the integration of vision-language models (VLMs) into academic assessment, particularly for tasks that require both quantitative and qualitative evaluation. However, existing VLM based approaches struggle with complex educational artifacts, such as programming tasks with executable components and measurable outputs, that require structured reasoning and alignment with clearly defined evaluation criteria. We introduce AGACCI, a multi-agent system that distributes specialized evaluation roles across collaborative agents to improve accuracy, interpretability, and consistency in code-oriented assessment. To evaluate the framework, we collected 360 graduate-level code-based assignments from 60 participants, each annotated by domain experts with binary rubric scores and qualitative feedback. Experimental results demonstrate that AGACCI outperforms a single GPT-based baseline in terms of rubric and feedback accuracy, relevance, consistency, and coherence, while preserving the instructional intent and evaluative depth of expert assessments. Although performance varies across task types, AGACCI highlights the potential of multi-agent systems for scalable and context-aware educational evaluation.
Study at a glance
  • Question: Can a multi-agent AI system improve grading accuracy and feedback quality for complex student programming assignments?
  • Setting and data: University-level course setting; 360 code-based assignments from 60 participants completing six graduate-level programming tasks written in Korean.
  • Method: System development and benchmark evaluation comparing a multi-agent system against a single-model baseline across six rounds.
Why it matters

Single large language models often struggle to evaluate complex programming tasks accurately, frequently producing vague advice, misaligning with fine-grained instructional rubrics, or demonstrating inconsistent grading across similar student submissions.

Key findings
  • AGACCI achieved 12 percentage points higher average rubric-level accuracy than the single-model baseline across 360 graduate-level programming submissions.
  • The multi-agent system outperformed the baseline in qualitative feedback consistency and coherence while achieving comparable performance in rubric relevance.
  • Performance gains were largest in natural language processing tasks requiring layered reasoning, but the system performed similarly to the baseline on tasks requiring external submission verification.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?
Who age?