Date
Publisher
arXiv
The dynamic nature of interactions between students and GenAI, as well as their relationship to writing quality, remains underexplored. While most research has examined how general-purpose GenAI can support writing, fewer studies have investigated how students interact with pedagogically designed systems across different phases of the writing process. To address this gap, we evaluated a GenAI-driven essay-writing assistant (EWA) designed to support higher education students in argumentative writing. Drawing on 1,282 interaction logs from 32 undergraduates during a two-hour writing session, Sequential Pattern Mining and K-Means clustering were used to identify behavioral patterns. Two clusters emerged: Cluster 1 emphasized outline planning and essay structure, while Cluster 2 focused on content development. A Mann-Whitney U test revealed a moderate effect size (r = 0.36) in the essay Organization dimension, with Cluster 1 showing higher scores. Qualitative analysis indicated that students with better performance actively wrote and shared essay sections with EWA for feedback, rather than interacted passively by asking questions. These findings suggest implications for teaching and system design. Teachers can encourage active engagement, while future EWAs may integrate automatic labeling and monitoring to prompt students to move from questioning to writing, enabling fuller benefits from GenAI-supported learning.
Study at a glance
- Question: How do student interactions with an AI essay-writing assistant affect the quality of their writing?
- Setting and data: 1,282 interaction logs from 32 undergraduate students during a two-hour writing session.
- Method: Pattern clustering of interaction logs (Sequential Pattern Mining, K-Means clustering), statistical comparison, and qualitative analysis.
Why it matters
The dynamic nature of student interactions with GenAI and their relationship to writing quality remain underexplored. Few studies investigate how students interact with pedagogically designed systems across different phases of the writing process rather than general-purpose GenAI.
Key findings
- Among 32 undergraduates writing argumentative essays, prioritizing outline planning and structure with an AI assistant was linked to higher essay organization scores.
- Higher-performing students actively wrote and shared essay sections with the AI assistant for feedback instead of passively asking it questions.
- Students showed two distinct AI interaction patterns during writing, either emphasizing outline planning and structure or focusing on content development.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
