Date
AI Overview
Why it matters
Discussions of classroom AI focus mostly on student chatbots, leaving instructor-facing support tools largely unexplored. Local open-weight models could offer cost-effective, private, and customizable assistance for both students and instructors.
Study at a glance
- Question: How can open-source language models running locally support both undergraduate engineering students and instructors from lecture transcripts?
- Setting and data: University of Delaware, 39 undergraduate thermodynamics lecture recordings (35.7 hours) from Fall 2025 and a course textbook.
- Method: System development and descriptive technical evaluation of speech-to-text and retrieval-augmented generation pipelines.
- AI tools: Stan, Whisper large-v3, Llama 3.1 8B
Key findings
- A three-layer mitigation strategy reduced transcription hallucination loops from 6.4% of baseline segments down to 0.02% across the full semester of lectures.
- Whisper speech recognition with domain-specific prompting correctly identified technical terminology that general models consistently misrecognized, such as transcribing fugacity instead of GASB.
- A two-pass architecture for question extraction eliminated an eight-question bimodal output artifact, producing a natural distribution averaging 8.8 questions per lecture.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Discussions of AI in education focus predominantly on student-facing tools -- chatbots, tutors, and problem generators -- while the potential for the same infrastructure to support instructors remains largely unexplored. We describe Stan, a suite of tools for an undergraduate chemical engineering thermodynamics course built on a data pipeline that we develop and deploy in dual roles: serving students and supporting instructors from a shared foundation of lecture transcripts and a structured textbook index. On the student side, a retrieval-augmented generation (RAG) pipeline answers natural-language queries by extracting technical terms, matching them against the textbook index, and synthesizing grounded responses with specific chapter and page references. On the instructor side, the same transcript corpus is processed through structured analysis pipelines that produce per-lecture summaries, identify student questions and moments of confusion, and catalog the anecdotes and analogies used to motivate difficult material -- providing a searchable, semester-scale record of teaching that supports course reflection, reminders, and improvement. All components, including speech-to-text transcription, structured content extraction, and interactive query answering, run entirely on locally controlled hardware using open-weight models (Whisper large-v3, Llama~3.1 8B) with no dependence on cloud APIs, ensuring predictable costs, full data privacy, and reproducibility independent of third-party services. We describe the design, implementation, and practical failure modes encountered when deploying 7--8 billion parameter models for structured extraction over long lecture transcripts, including context truncation, bimodal output distributions, and schema drift, along with the mitigations that resolved them.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
