Date
AI Overview
Why it matters
Existing learning analytics methods struggle to capture the complex, multi-typed interactions between learners, digital tools, and educational content in modern digital environments.
Study at a glance
- Question: How do students interact with peers versus an AI chatbot during small-group collaborative learning tasks?
- Setting and data: 27 undergraduate students across six groups in a two-week project management course in Hong Kong.
- Method: Network analysis (Heterogeneous Interaction Network Analysis)
- AI tools: GPT-4
Key findings
- Most students engaged flexibly with both peers and the AI chatbot, while a subset interacted exclusively with either peers (five students) or the AI (four students).
- Students split into two distinct engagement patterns: a peer-dominant group directing 12 percent of messages to AI, and a hybrid group directing 57 percent of messages to AI.
- The hybrid cluster primarily relied on the AI chatbot for resolving factual questions, while reserving idea generation and consensus building for their human peers.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
Existing learning analytics approaches, which often model learning processes as sequences of learner actions or homogeneous relationships, are limited in capturing the distributed, multi-faceted nature of interactions in contemporary learning environments. To address this, we propose Heterogeneous Interaction Network Analysis (HINA), a novel multi-level learning analytics framework for modeling complex learning processes across diverse entities (e.g., learners, behaviours, AI agents, and task designs). HINA integrates a set of original methods, including summative measures and a new non-parametric clustering technique, with established practices for statistical testing and interactive visualization to provide a flexible and powerful analytical toolkit. In this paper, we first detail the theoretical and mathematical foundations of HINA for individual, dyadic, and meso-level analysis. We then demonstrate HINA's utility through a case study on AI-mediated small-group collaborative learning, revealing students' interaction profiles with peers versus AI; distinct engagement patterns that emerge from these interactions; and specific types of learning behaviors (e.g., asking questions, planning) directed to AI versus peers. By transforming process data into Heterogeneous Interaction Networks (HINs), HINA introduces a new paradigm for modeling learning processes and provides the dedicated, multi-level analytical methods required to extract meaning from them. It thereby moves beyond a single process data type to quantify and visualize how different elements in a learning environment interact and co-influence each other, opening new avenues for understanding complex educational dynamics.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
Methods
AI tools
Publication Source
Publisher
arXiv
