AI Overview
Information problem solving is essential for academic and workplace success, but empirical evidence on how students engage with generative AI during complex problem solving remains limited.
- Question: How do students interact with an embedded generative AI chatbot during an educational information problem-solving task?
- Setting and data: A public Australian university; 275 postgraduate data science students; Semester 2, 2024; system logs, dialogues, essays, and surveys.
- Method: Descriptive analysis, qualitative coding of dialogues, and cohort stability testing (Mann–Whitney U test).
- AI tools: FLoRA, GPT-4o, text-embedding-3-small
- Students averaged 12.9 chatbot utterances during the task, with idea exploration focused on judgement and writing advice being their most frequent query types.
- The embedded chatbot predominantly produced suggestion responses (3,174 instances), followed by critiques (712 instances) and factual answers (286 instances).
- Trace log distributions per student did not differ significantly between cohorts across two consecutive semesters, demonstrating temporal stability in platform engagement.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Information Problem Solving (IPS) is a critical competency for academic and professional success in education, work, and life. The advent of Generative Artificial Intelligence (GenAI), particularly tools like ChatGPT, has introduced new possibilities for supporting students in complex IPS tasks. However, empirical insights into how students engage with GenAI during IPS and how these tools can be effectively leveraged for learning remain limited. Moreover, differences in background--shaped by cultural and socioeconomic factors--pose additional challenges to the equitable integration of GenAI in educational contexts. To address this gap, we present an open-source dataset collected from 279 students at a public Australian university. The dataset was generated through students' use of FLoRA, a GenAI-powered educational platform that is widely adopted in the field of learning analytics. Within FLoRA, students interacted with an embedded GenAI chatbot to gather information and synthesize it into data science project proposals. The dataset captures fine-grained, multi-dimensional records of GenAI-assisted IPS processes, including: (i) student-GenAI dialogue transcripts; (ii) writing process log traces; (iii) final project proposals with human-assigned assessment scores; (iv) two surveys assessing students demographic background and their prior knowledge and experience in data science and AI; and (v) surveys capturing students' perceptions of GenAI's effectiveness in supporting IPS and platform use experience. This dataset provides a valuable resource for advancing our understanding of GenAI's role in educational IPS and informing the design of adaptive, inclusive AI-powered learning tools.
