Search and Filter

Repository Feedback

Your feedback helps us improve the repository's content relevance and usability. Please share your thoughts to help us better serve researchers and practitioners.

Submit feedback

Submit a research study

Contribute to the repository:

Add a paper

Tracing Genai Literacy: Uncovering Student-AI Interaction Patterns In Academic Writing Through Epistemic Network Analysis

Date
Publisher
arXiv
As Generative AI (GenAI) becomes integral to education, fostering GenAI literacy is critical. However, current assessments largely rely on self-reported scales, lacking insights into how literacy manifests in actual learning processes. This study leverages Learning Analytics (LA) to bridge this gap. We collected interaction logs from 162 university students engaged in a GenAI-assisted abstract writing task. Using Epistemic Network Analysis (ENA), we modeled and compared the questioning strategies of students with varying GenAI literacy levels. Preliminary results reveal distinct interaction signatures: high-literacy students engage in iterative refinement and strategic questioning, while low-literacy students rely on direct generation commands. This work contributes to the workshop by demonstrating how process data can characterize GenAI literacy, paving the way for data-driven literacy assessment and real-time interventions.
Study at a glance
  • Question: How do student interaction patterns with generative AI during academic writing differ across literacy levels?
  • Setting and data: 162 university students (average age 20.1) completing an abstract writing task.
  • Method: Network analysis of prompt interaction logs (Epistemic Network Analysis)
Why it matters

Generative AI literacy assessments largely rely on self-reported questionnaires, missing how literacy actually manifests during real-time human-AI collaboration.

Key findings
  • Students with high and low generative AI literacy showed significantly different interaction patterns on the primary dimension of the network space (U = 683.50, p < .001).
  • High-literacy students frequently linked text refinement requests with conceptual clarification and meta-commands, reflecting an iterative, collaborative approach.
  • Low-literacy students showed a transactional approach dominated by broad content generation commands paired with factual questions or reactive challenges.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

What is the application?
Who is the user?
Who age?
Why use AI?