Date
Publisher
arXiv
Due to their similarity-based learning objectives, pretrained sentence encoders often internalize stereotypical assumptions that reflect the social biases that exist within their training corpora. In this paper, we describe several kinds of stereotypes concerning different communities that are present in popular sentence representation models, including pretrained next sentence prediction and contrastive sentence representation models. We compare such models to textual entailment models that learn language logic for a variety of downstream language understanding tasks. By comparing strong pretrained models based on text similarity with textual entailment learning, we conclude that the explicit logic learning with textual entailment can significantly reduce bias and improve the recognition of social communities, without an explicit de-biasing process
Study at a glance
- Question: Can teaching AI language models logical reasoning help reduce their social biases and stereotypes?
- Setting and data: Not stated in the abstract
- Method: Compares AI language models trained on text similarity with models trained on language logic (textual entailment)
Why it matters
Pretrained sentence models often internalize stereotypical assumptions that reflect social biases in their training data. Because of similarity-based learning objectives, popular models carry stereotypes concerning different communities.
Key findings
- Training AI models to learn language logic can significantly reduce social bias without requiring an explicit de-biasing process.
- Learning language logic also improves an AI model's recognition of different social communities.
- Popular language models trained on text similarity often absorb and reflect stereotypes about different communities from their training data.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
Why use AI?
Study design
