Date
Publisher
arXiv
Generative Artificial Intelligence (GenAI) is transforming how firms create, process, and apply knowledge, yet little is known about the heterogeneity of its productivity effects across users. We report results from a randomized controlled experiment in which participants-analogs of early-career knowledge workers-were assigned to self-study a technical domain using either traditional resources or large-language-model (LLM) assistance. On average, GenAI access significantly increased task performance, but the distribution of gains was highly uneven. Improvements were not predicted by GPA or prior knowledge, but by \textit{AI Interaction Competence (AIC)} -- the ability to elicit, filter, and verify model outputs. High-AIC participants realized outsized gains; low-AIC participants saw limited or even negative marginal returns. A scaffolding intervention (conceptual maps) reduced outcome variance, indicating that standardized workflows can mitigate inequality in AI-mediated performance. We interpret these findings through the lens of human-AI complementarities: GenAI raises mean productivity while introducing a new axis of capability inequality. Managerially, firms should pair GenAI access with short AIC micro-training and simple standard operating procedures to capture value consistently and avoid uneven adoption outcomes.
Study at a glance
- Question: How does generative AI affect learning performance and inequality across users with different skill levels?
- Setting and data: Texas A&M University, 179 undergraduate and graduate students primarily across engineering majors.
- Method: Randomized controlled experiment
Why it matters
Generative AI is transforming knowledge work, yet little is known about how its productivity effects vary across users. Understanding whether it lifts performance uniformly or widens disparities is central to digital transformation.
Key findings
- Generative AI access significantly raised post-study exam scores from an average of 0.48 to 0.56, while also reducing participant dropout.
- Performance improvements were moderated by AI Interaction Competence rather than GPA or prior knowledge, with low-knowledge learners benefiting more from AI only if their competence was high.
- Providing novice learners with a conceptual roadmap scaffold narrowed performance variance and had its greatest impact on participants with low AI Interaction Competence.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
