Date
Publisher
ACL Anthology
Warning: This paper contains content that is stereotypical and may be upsetting. This paper addresses the issue of demographic stereotypes present in Transformer-based pre-trained language models (PLMs) and aims to deepen our understanding of how these biases are encoded in these models. To accomplish this, we introduce an easy-to-use framework for examining the stereotype-encoding behavior of PLMs through a combination of model probing and textual analyses. Our findings reveal that a small subset of attention heads within PLMs are primarily responsible for encoding stereotypes and that stereotypes toward specific minority groups can be identified using attention maps on these attention heads. Leveraging these insights, we propose an attention-head pruning method as a viable approach for debiasing PLMs, without compromising their language modeling capabilities or adversely affecting their performance on downstream tasks.
Study at a glance
- Question: How do artificial intelligence language models store demographic stereotypes?
- Setting and data: Not stated in the abstract
- Method: Analysis of language model components and text (model probing and textual analyses)
Why it matters
Demographic stereotypes are present in Transformer-based pre-trained language models. Addressing this issue requires deepening our understanding of how these biases are encoded in these models.
Key findings
- A small subset of internal components in pre-trained language models is primarily responsible for encoding demographic stereotypes.
- Stereotypes toward specific minority groups can be identified by analyzing the focus patterns of these specific internal components.
- Removing these bias-encoding components reduces stereotypes without compromising the models' language capabilities or performance on other tasks.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
What is the application?
Why use AI?
Study design
