Date
Publisher
arXiv
In the field of educational assessment, automated scoring systems increasingly rely on deep learning and large language models (LLMs). However, these systems face significant risks of bias amplification, where model prediction gaps between student groups become larger than those observed in training data. This issue is especially severe for underrepresented groups such as English Language Learners (ELLs), as models may inherit and further magnify existing disparities in the data. We identify that this issue is closely tied to representation bias: the scarcity of minority (high-scoring ELL) samples makes models trained with empirical risk minimization favor majority (non-ELL) linguistic patterns. Consequently, models tend to under-predict ELL students who even demonstrate comparable domain knowledge but use different linguistic patterns, thereby undermining the fairness of automated scoring outcomes. To mitigate this, we propose BRIDGE, a Bias-Reducing Inter-group Data GEneration framework designed for low-resource assessment settings. Instead of relying on the limited minority samples, BRIDGE synthesizes high-scoring ELL samples by "pasting" construct-relevant (i.e., rubric-aligned knowledge and evidence) content from abundant high-scoring non-ELL samples into authentic ELL linguistic patterns. We further introduce a discriminator model to ensure the quality of synthetic samples. Experiments on California Science Test (CAST) datasets demonstrate that BRIDGE effectively reduces prediction bias for high-scoring ELL students while maintaining overall scoring performance. Notably, our method achieves fairness gains comparable to using additional real human data, offering a cost-effective solution for ensuring equitable scoring in large-scale assessments.
Study at a glance
- Question: Can combining non-ELL content with ELL writing patterns reduce scoring bias against high-performing English learners?
- Setting and data: Five Grade 8 California Science Test items, each with roughly 200,000 student constructed responses (about 30,000 from English language learners).
- Method: Machine learning model evaluation testing synthetic data augmentation against oversampling, paraphrasing, and real data baselines.
Why it matters
Automated scoring systems can amplify training data biases, unfairly predicting lower scores for underrepresented English language learners who demonstrate comparable knowledge using different linguistic patterns.
Key findings
- Baseline scoring models amplified scoring bias against high-scoring English learners across all five test items, widening score disparities by 3.4% to 9.0%.
- The BRIDGE framework reduced both group score gaps and high-score bias amplification across most items, achieving fairness gains comparable to using additional real student data.
- The data augmentation method maintained overall scoring accuracy without exhibiting typical trade-offs between fairness and predictive performance across all five datasets.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
