Date
AI Overview
Why it matters
Students increasingly turn to large language models for writing tasks, raising concerns about reduced critical engagement. However, how students functionally use ChatGPT and how this usage shapes their writing and perceptions remain underexplored.
Study at a glance
- Question: How do college students use ChatGPT during essay writing, and how does it relate to their writing and perceptions?
- Setting and data: 77 college students in the United States completing an online argumentative essay task.
- Method: Mixed methods user study combining qualitative query coding, keystroke and paste trace tracking, surveys, and Poisson regression.
- AI tools: ChatGPT, OpenAI API (model 3.5-turbo)
Key findings
- Over half of participants used queries delegating the entire writing process, with 19.5% submitting essays that were entirely written by ChatGPT through iterative prompting or direct generation.
- Lower writing self-efficacy significantly predicted more frequent ChatGPT queries overall, specifically for translating and reviewing tasks rather than planning.
- Students delegating the entire essay to ChatGPT felt significantly less perceived ownership, whereas students using it only for planning or reviewing felt ownership comparable to non-users.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
As large language models (LLMs) advance and become widespread, students increasingly turn to systems like ChatGPT for assistance with writing tasks. Educators are concerned with students' usage of ChatGPT beyond cheating; using ChatGPT may reduce their critical engagement with writing, hindering students' learning processes. The negative or positive impact of using LLM-powered tools for writing will depend on how students use them; however, how students use ChatGPT remains largely unknown, resulting in a limited understanding of its impact on learning. To better understand how students use these tools, we conducted an online study $(n=70)$ where students were given an essay-writing task using a custom platform we developed to capture the queries they made to ChatGPT. To characterize their ChatGPT usage, we categorized each of the queries students made to ChatGPT. We then analyzed the relationship between ChatGPT usage and a variety of other metrics, including students' self-perception, attitudes towards AI, and the resulting essay itself. We found that factors such as gender, race, and perceived self-efficacy can help predict different AI usage patterns. Additionally, we found that different usage patterns were associated with varying levels of enjoyment and perceived ownership over the essay. The results of this study contribute to discussions about how writing education should incorporate generative AI-powered tools in the classroom.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
