Date
Publisher
arXiv
As large language models (LLMs) become more common in educational tools and
programming environments, questions arise about how these systems should
interact with users. This study investigates how different interaction styles
with ChatGPT-4o (passive, proactive, and collaborative) affect user performance
on simple programming tasks. I conducted a within-subjects experiment where
fifteen high school students participated, completing three problems under
three distinct versions of the model. Each version was designed to represent a
specific style of AI support: responding only when asked, offering suggestions
automatically, or engaging the user in back-and-forth dialogue.Quantitative
analysis revealed that the collaborative interaction style significantly
improved task completion time compared to the passive and proactive conditions.
Participants also reported higher satisfaction and perceived helpfulness when
working with the collaborative version. These findings suggest that the way an
LLM communicates, how it guides, prompts, and responds, can meaningfully impact
learning and performance. This research highlights the importance of designing
LLMs that go beyond functional correctness to support more interactive,
adaptive, and user-centered experiences, especially for novice programmers.
Study at a glance
- Question: How do passive, proactive, and collaborative AI interaction styles affect high school students' programming speed and experience?
- Setting and data: Remote experiment, 15 high school students with basic programming experience.
- Method: Within-subjects experiment comparing three AI interaction styles using repeated-measures ANOVA and post-task surveys.
Why it matters
As large language models become common in education, research often treats them as fixed entities. Understanding how different interaction styles shape student learning and user experience remains an essential but largely overlooked question.
Key findings
- High school students solved programming tasks significantly faster using the collaborative interaction style (2.63 minutes) compared to the passive style (3.60 minutes).
- Students completing tasks with the proactive style (3.69 minutes) also worked significantly faster than those using the passive style, though they did not differ significantly from the collaborative style.
- Participants reported higher satisfaction, clarity, and perceived helpfulness with the collaborative AI model, though confidence effects were mixed across individuals.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
