Date
AI Overview
Why it matters
The educational value of generative AI depends on how control is shared between learner and system. Little empirical evidence shows whether reactive or proactive agent designs cultivate independent writing or create dependency.
Study at a glance
- Question: How do reactive versus proactive AI agents and AI literacy shape students' reasoning and independent writing performance?
- Setting and data: 79 medical and nursing students or recent graduates recruited via Prolific Academic.
- Method: Randomized online experiment with discourse network analysis, ordinal regression, mediation modeling, and qualitative thematic analysis (mixed methods).
- AI tools: GPT-4o, VizChat
Key findings
- Proactive AI dialogues fostered stronger connections among conceptual reasoning, evidence use, and constructive engagement, whereas reactive interactions focused more on factual and procedural information.
- Higher generative AI literacy predicted an 11% to 15% increase in the odds of higher independent writing scores for visual data integration, critical thinking, and overall quality.
- AI literacy was directly associated with independent writing performance under the reactive agent, but this relationship was not observed under the proactive agent.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Generative AI (GenAI) tools are rapidly transforming higher education, yet
little is known about how students' GenAI literacy shapes their ability to
perform independently once such support is removed. This study investigates
what we term the agency gap, introduced as the extent to which GenAI literacy
predicts student writing performance in contexts that require self-initiation
and regulation. Seventy-nine medical and nursing students completed multimodal
academic writing tasks based on visual data, supported either by a reactive or
proactive GenAI chatbot, followed by a parallel task without AI support.
Writing was evaluated across insightfulness, visual data integration,
organisation, linguistic quality, and critical thinking. Results showed that
GenAI literacy predicted independent writing performance only in the reactive
condition, where students had to actively mobilise their own strategies.
Mediation analyses revealed no indirect effect via in-task performance,
indicating that GenAI literacy acts as a direct, task-general competence rather
than a proxy for domain knowledge or other literacies. By contrast, proactive
scaffolding equalised outcomes across literacy levels, reducing reliance on
learners' GenAI literacy. The agency gap highlights when GenAI literacy matters
most, with implications for designing equitable AI-supported learning
environments that both leverage and mitigate differences in students' GenAI
literacy.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
Publication Source
Publisher
arXiv
