Measuring Inclusion In Interaction: Inclusion Analytics For Human-AI Collaborative Learning

Jaeyoon Choi, University of California, Irvine

Nia Nixon, University of California, Irvine

Date

02/2026

AI Overview

Why it matters

Inclusion is widely valued in AI education but is usually measured through coarse descriptors or retrospective surveys. These miss how inclusion unfolds moment-by-moment in collaborative learning, especially as AI teammates reshape group dynamics.

Study at a glance

Question: Can automated discourse metrics capture moment-to-moment inclusion across participation, affective tone, and idea uptake in collaborative problem solving?
Setting and data: Simulated student chats and empirical data from 74 teams (37 human-only, 37 human-AI) completing the NASA Moon Survival Task.
Method: Analysis of simulated conversations and human-AI teaming experiment data using automated natural language processing metrics and Mann-Whitney U tests.
AI tools: GPT-5.2-Thinking, GPT-5.2, GPT-4o, sentence-BERT

Key findings

Teams working with an AI teammate showed significantly greater word-count participation imbalance (0.64) than all-human teams (0.39), with no significant difference in turn counts.
Human participants received significantly higher semantic idea uptake and endorsement uptake scores than AI teammates in collaborative teams.
Politeness uptake did not differ significantly between human and AI teammates in human-AI collaboration data.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Inclusion, equity, and access are widely valued in AI and education, yet are often assessed through coarse sample descriptors or post-hoc self-reports that miss how inclusion is shaped moment by moment in collaborative problem solving (CPS). In this proof-of-concept paper, we introduce inclusion analytics, a discourse-based framework for examining inclusion as a dynamic, interactional process in CPS. We conceptualize inclusion along three complementary dimensions -- participation equity, affective climate, and epistemic equity -- and demonstrate how these constructs can be made analytically visible using scalable, interaction-level measures. Using both simulated conversations and empirical data from human-AI teaming experiments, we illustrate how inclusion analytics can surface patterns of participation, relational dynamics, and idea uptake that remain invisible to aggregate or post-hoc evaluations. This work represents an initial step toward process-oriented approaches to measuring inclusion in human-AI collaborative learning environments.