Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Sketchmind: A Multi-Agent Cognitive Framework For Assessing Student-Drawn Scientific Sketches

Date
Publisher
arXiv
Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by \textsc{SketchMind} with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.
Study at a glance
  • Question: Can a multi-agent AI framework improve the accuracy and pedagogical quality of evaluating student-drawn science models?
  • Setting and data: 3,575 student-generated sketches across six science assessment items aligned with NGSS standards; four expert science educators evaluated feedback on 890 sketches.
  • Method: Multi-agent system development, model evaluation against human-expert scores, and double-blind expert rating of model feedback.
Why it matters

Assessing student-drawn scientific sketches remains challenging because automated systems often lack interpretability and pedagogical alignment across different cognitive levels.

Key findings
  • Integrating Sketch Reasoning Graphs improved GPT-4o's sketch evaluation accuracy from 55.6% to 77.1%, while GPT-4.1 achieved the highest accuracy at 90.2%.
  • Modular multi-agent orchestration consistently outperformed single-agent pipelines across all test items, boosting GPT-4.1 accuracy from 82.8% to 90.2% with graph guidance.
  • Expert educators rated SKETCHMIND feedback and modified sketches generated with GPT-4.1 at 4.1 out of 5, compared to 2.3 out of 5 for baseline GPT-4o.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?