Date
Publisher
arXiv
Large Language Models (LLMs) are rapidly being adopted by STEM-focused educational institutions and students worldwide. They generate personalized instructions, explanations, and provide feedback on demand. However, these systems tailor instruction to demographic signals rather than demonstrated ability. In such cases, personalization becomes a mechanism of inequality. We conduct one of the first large-scale intersectional audits of LLM-generated STEM educational content, constructing synthetic student profiles. We combine dimensions specific to Indian education (caste, medium of instruction, college tier) and American education (race, HBCU attendance, school type), alongside shared dimensions of income, gender, and disability. We audit four LLMs (Qwen 2.5-32B-Instruct, GPT-4o, GPT-4o-mini, GPT-OSS 20B) across ranking and generation tasks on two STEM datasets, evaluating outputs with FDR-corrected significance testing and SHAP feature attribution. Across both cultural contexts, marginalized profiles receive lower-quality outputs. Income is the most pervasive bias, producing significant effects across every model and context. Disability triggers simpler explanations. Intersectional analysis reveals non-additive compounding: the gap between the most privileged and most marginalized profiles reaches 2.55 grade levels. These biases persist even when marginalized students attend elite institutions. All four models converge on similar patterns. These findings carry direct design and policy implications for incorporating AI into global STEM education.
Study at a glance
- Question: Do language models vary educational explanation complexity based on students' intersecting demographic and institutional backgrounds?
- Setting and data: 100 Indian and 100 American synthetic student profiles tested on math and science problems from MATH-50 and JEEBench.
- Method: Algorithmic audit using ranking and text generation experiments with readability indices and SHAP feature attribution
Why it matters
Personalized educational AI promises to adapt instruction to student needs, but relying on demographic signals rather than demonstrated ability can withhold complex instruction from marginalized students. Most AI fairness research also overlooks institutional structures outside Western contexts.
Key findings
- Language models generate simpler STEM explanations for marginalized student profiles, with the gap between the most privileged and most marginalized reaching 2.55 grade levels.
- Student income produced significant complexity differences across every model and context, while instruction medium created the largest single effect in Indian education.
- Marginalized identities compound non-additively across intersecting traits, causing bias penalties to persist even within elite institutions like IIT and Ivy League universities.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who age?
Study design
