Date
Publisher
arXiv
Generative AI tutors hold significant promise for higher education, yet designing systems that scaffold learning without undermining academic integrity remains an open design challenge. This paper presents PeteChat, a course-aligned AI tutor developed and deployed at Purdue University, documented through the lens of design-based research (DBR). Drawing on literature-informed design inputs, a pre-deployment baseline analysis of authentic student-system interactions, and formative expert evaluation with teaching assistants and UX/developer stakeholders, we report eight transferable design principles for assessment-aware AI tutors: from homework guardrails and debugging scaffolds to self-regulated learning support and instructor-facing customization tools. The system is built on a locally hosted Llama-3 model enhanced with retrieval-augmented generation (RAG) grounded in course-specific materials. Rather than reporting controlled experimental outcomes, this design case foregrounds the situated design reasoning, iterative refinement, and principled decision-making that shaped PeteChat across multiple development phases. The resulting principles and methodological approach offer actionable guidance for institutions seeking to deploy responsible, integrity-preserving AI tutors at scale.
Study at a glance
- Question: How can an AI tutor be designed to scaffold learning without undermining academic integrity?
- Setting and data: Undergraduate Python programming course at a large U.S. research university, Fall 2025; 284 baseline chat messages and 4 expert evaluation sessions.
- Method: Design-based research including directed content analysis of baseline interaction logs and expert think-aloud interviews.
Why it matters
Generative AI tutors hold promise for higher education, yet designing systems that scaffold learning without undermining academic integrity or over-relying on direct answers remains an open design challenge.
Key findings
- Baseline logs showed students probed system boundaries in 13.4% of messages, reaching 22.0% during homework debugging, while the unguardrailed tutor sometimes provided direct solutions.
- Student self-reflection and hint uptake were virtually absent in baseline interactions, appearing in only 1.4% of messages and 0% of mini-project queries.
- Formative expert evaluations and baseline failures motivated eight design decisions, including homework guardrails, error-explanation debugging modes, and instructor-controlled retrieval freshness disclaimers.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
Why use AI?
Study design
