Search and Filter

The Agency Gap: How Generative Ai Literacy Shapes Independent Writing After Ai Support

Date

AI Overview

Why it matters

The educational value of generative AI depends on how control is shared between learner and system. Little empirical evidence shows whether reactive or proactive agent designs cultivate independent writing or create dependency.

Study at a glance
  • Question: How do reactive versus proactive AI agents and AI literacy shape students' reasoning and independent writing performance?
  • Setting and data: 79 medical and nursing students or recent graduates recruited via Prolific Academic.
  • Method: Randomized online experiment with discourse network analysis, ordinal regression, mediation modeling, and qualitative thematic analysis (mixed methods).
  • AI tools: GPT-4o, VizChat
Key findings
  • Proactive AI dialogues fostered stronger connections among conceptual reasoning, evidence use, and constructive engagement, whereas reactive interactions focused more on factual and procedural information.
  • Higher generative AI literacy predicted an 11% to 15% increase in the odds of higher independent writing scores for visual data integration, critical thinking, and overall quality.
  • AI literacy was directly associated with independent writing performance under the reactive agent, but this relationship was not observed under the proactive agent.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Generative AI (GenAI) tools are rapidly transforming higher education, yet little is known about how students' GenAI literacy shapes their ability to perform independently once such support is removed. This study investigates what we term the agency gap, introduced as the extent to which GenAI literacy predicts student writing performance in contexts that require self-initiation and regulation. Seventy-nine medical and nursing students completed multimodal academic writing tasks based on visual data, supported either by a reactive or proactive GenAI chatbot, followed by a parallel task without AI support. Writing was evaluated across insightfulness, visual data integration, organisation, linguistic quality, and critical thinking. Results showed that GenAI literacy predicted independent writing performance only in the reactive condition, where students had to actively mobilise their own strategies. Mediation analyses revealed no indirect effect via in-task performance, indicating that GenAI literacy acts as a direct, task-general competence rather than a proxy for domain knowledge or other literacies. By contrast, proactive scaffolding equalised outcomes across literacy levels, reducing reliance on learners' GenAI literacy. The agency gap highlights when GenAI literacy matters most, with implications for designing equitable AI-supported learning environments that both leverage and mitigate differences in students' GenAI literacy.