Date
Publisher
arXiv
How do students develop AI literacy through everyday practice rather than formal instruction? While normative AI literacy frameworks proliferate, empirical understanding of how students actually learn to work with generative AI remains limited. This study analyzes 10,536 ChatGPT messages from 36 undergraduates over one academic year, revealing five use genres -- academic workhorse, emotional companion, metacognitive partner, repair and negotiation, and trust calibration -- that constitute distinct configurations of student-AI learning. Drawing on domestication theory and emerging frameworks for AI literacy, we demonstrate that functional AI competence emerges through ongoing relational negotiation rather than one-time adoption. Students develop sophisticated genre portfolios, strategically matching interaction patterns to learning needs while exercising critical judgment about AI limitations. Notably, repair work during AI breakdowns produces substantial learning about AI capabilities, developing what we term "repair literacy" -- a crucial but underexplored dimension of AI competence. Our findings offer educators empirically grounded insights into how students actually learn to work with generative AI, with implications for AI literacy pedagogy, responsible AI integration, and the design of AI-enabled learning environments that support student agency.
Study at a glance
- Question: How do undergraduate students develop AI literacy through naturalistic, everyday interaction with generative AI?
- Setting and data: 36 undergraduates at a northeastern U.S. research university, December 2022 to January 2024, analyzing 10,536 ChatGPT messages across 1,631 chats.
- Method: Mixed methods combining manual thematic qualitative coding with GPT-4o-based annotation scaling and validation.
Why it matters
While normative AI literacy frameworks proliferate, empirical understanding remains limited regarding how students actually develop competence to work with generative AI through naturalistic, everyday practice rather than formal instruction.
Key findings
- Undergraduate interactions with ChatGPT fell into five distinct use genres: the Workhorse retrieval engine, Algorithmic Auditing and Repair, Anthropomorphic Sensemaking, Metacognitive Processing, and Trust Calibration.
- The Workhorse genre was the most prevalent but involved shallow exchanges, while the remaining four relational and system-side genres unfolded over sustained, multi-turn interactions.
- System apologies were the single strongest positive predictor of sustained student engagement, whereas stated system limitations and wrong or contradictory outputs were negative predictors.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
