Date
Publisher
arXiv
As generative artificial intelligence (AI) continues to transform education, most existing AI evaluations rely primarily on technical performance metrics such as accuracy or task efficiency while overlooking human identity, learner agency, contextual learning processes, and ethical considerations. In this paper, we present TEACH-AI (Trustworthy and Effective AI Classroom Heuristics), a domain-independent, pedagogically grounded, and stakeholder-aligned framework with measurable indicators and a practical toolkit for guiding the design, development, and evaluation of generative AI systems in educational contexts. Built on an extensive literature review and synthesis, the ten-component assessment framework and toolkit checklist provide a foundation for scalable, value-aligned AI evaluation in education. TEACH-AI rethinks "evaluation" through sociotechnical, educational, theoretical, and applied lenses, engaging designers, developers, researchers, and policymakers across AI and education. Our work invites the community to reconsider what constructs "effective" AI in education and to design model evaluation approaches that promote co-creation, inclusivity, and long-term human, social, and educational impact.
Study at a glance
- Question: What criteria define effective, value-aligned human-AI collaboration in education, and how can they guide evaluation tools?
- Setting and data: Literature review across major academic venues and Google Scholar, analyzing 126 publications spanning pre-2017 to the present.
- Method: Scoping review and iterative thematic synthesis.
Why it matters
Most educational AI evaluations focus primarily on technical metrics like accuracy, overlooking human identity, learner agency, contextual learning, and ethics. Value-aligned benchmarks are needed to assess how generative AI supports learning in authentic contexts.
Key findings
- A scoping review of 126 publications across three eras identified ten recurring components essential for human-centered educational AI evaluation.
- The TEACH-AI framework outlines core dimensions including explainability, helpfulness, adaptivity, consistency, learning exploration, usability, ethics, accessibility, workflow coordination, and refinement.
- A practical toolkit checklist of reflective questions was developed to translate abstract values into actionable, domain-independent criteria for classroom and research use.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Why use AI?
Study design
