Date
AI Overview
Why it matters
Personalized AI models risk widening educational and informational inequities if they amplify societal biases, provide inaccurate information, or withhold helpful answers depending on a user's language background, nationality, or education level.
Study at a glance
- Question: Does AI response quality and accuracy decline for users with less education, non-native English, or non-US backgrounds?
- Setting and data: Benchmark questions from TruthfulQA (817 questions) and SciQ (1,000 questions), paired with synthetic and human-written user personas representing different backgrounds.
- Method: Model or benchmark evaluation across user personas, with statistical chi-square tests and qualitative text analysis of model refusals.
- AI tools: GPT-4 (gpt-4-0125-preview), Claude 3 Opus (claude-3-opus-20240229), Llama 3-8B
Key findings
- Language models exhibited significant accuracy drops for users depicted as less educated, non-native English speakers, or originating from outside the United States.
- Claude 3 Opus refused to answer roughly 11% of questions for low-educated non-native personas, compared to 3.61% for the neutral control.
- When refusing to answer less educated personas, Claude 3 Opus used condescending, patronizing, or mocking language in 43.74% of instances compared to under 1% for other groups.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
While state-of-the-art large language models (LLMs) have shown impressive performance on many tasks, there has been extensive research on undesirable model behavior such as hallucinations and bias. In this work, we investigate how the quality of LLM responses changes in terms of information accuracy, truthfulness, and refusals depending on three user traits: English proficiency, education level, and country of origin. We present extensive experimentation on three state-of-the-art LLMs and two different datasets targeting truthfulness and factuality. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US, rendering these models unreliable sources of information towards their most vulnerable users.
Research Categories
What is the application?
Who age?
Why use AI?
Study design
AI tools
Datasets
Publication Source
Publisher
arXiv
