Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Teaching at Scale: Leveraging AI to Evaluate and Elevate Engineering Education

Authors
J.-F. Chamberland,
M. Carlisle,
A. Jayaraman,
K. R. Narayanan,
S. Palsole,
K. Watson
Date
Publisher
arXiv
Evaluating teaching effectiveness at scale remains a persistent challenge for large universities, particularly within engineering programs that enroll tens of thousands of students. Traditional methods, such as manual review of student evaluations, are often impractical, leading to overlooked insights and inconsistent data use. This article presents a scalable, AI-supported framework for synthesizing qualitative student feedback using large language models. The system employs hierarchical summarization, anonymization, and exception handling to extract actionable themes from open-ended comments while upholding ethical safeguards. Visual analytics contextualize numeric scores through percentile-based comparisons, historical trends, and instructional load. The approach supports meaningful evaluation and aligns with best practices in qualitative analysis and educational assessment, incorporating student, peer, and self-reflective inputs without automating personnel decisions. We report on its successful deployment across a large college of engineering. Preliminary validation through comparisons with human reviewers, faculty feedback, and longitudinal analysis suggests that LLM-generated summaries can reliably support formative evaluation and professional development. This work demonstrates how AI systems, when designed with transparency and shared governance, can promote teaching excellence and continuous improvement at scale within academic institutions.
Study at a glance
  • Question: Can AI reliably summarize student feedback to help evaluate and improve university teaching at scale?
  • Setting and data: A large college of engineering, using open-ended student evaluation comments and numeric scores.
  • Method: System deployment and validation through comparisons with human reviewers, faculty feedback, and longitudinal analysis.
Why it matters

Evaluating teaching effectiveness at scale remains a persistent challenge for large universities. Traditional manual reviews of student evaluations are often impractical, leading to overlooked insights and inconsistent data use.

Key findings
  • In a large college of engineering, comparisons with human reviewers suggest AI-generated summaries of student comments reliably support formative evaluation and professional development.
  • The AI framework successfully pulled actionable themes from open-ended student feedback while maintaining data anonymization and ethical safeguards.
  • The evaluation system supported meaningful assessment by combining student, peer, and self-reflective inputs without automating high-stakes personnel decisions.

Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.

Who is the user?
Who age?