Date
Publisher
arXiv
The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students' learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students' interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master's level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students' interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT's responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system's effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz's capacity to enhance educators' insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.
Study at a glance
- Question: How can a visual tool help instructors track and understand how students interact with ChatGPT?
- Setting and data: Chat data from 48 students in a one-semester master's data visualization course, plus interviews with six instructors.
- Method: Tool development evaluated through conversation coding, expert interviews, and three case studies.
Why it matters
To empower students to fully leverage ChatGPT, understanding their interaction patterns is crucial for instructors. However, analyzing these evolving interactions is challenging due to a lack of datasets focused on student-ChatGPT conversations.
Key findings
- In a master's data visualization course with 48 students, an analytics tool effectively tracked student prompt patterns and ChatGPT response quality over a semester.
- Six data visualization instructors confirmed that the system enhanced their insights into the pedagogical value of ChatGPT.
- Comparing student prompt patterns with AI response quality at multiple scales revealed significant pedagogical insights for educators.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Who age?
Study design
