Date
Publisher
arXiv
Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by humans who use these systems, however, remains underexplored. We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students. We first verify that a gender-biased prompt induces gender-differentiated language in LLM-generated essays, while a neutral prompt does not. We then recruited participants (N = 123) in a controlled environment to write career plan essays for paired biographical profiles differing only in gender under three conditions: no AI assistance, neutral LLM assistance, or gender-biased LLM assistance. Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions. Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected. Our findings highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools.
Study at a glance
- Question: Does gender bias in an AI writing assistant transfer into student-written career plan essays?
- Setting and data: 123 undergraduate students at an English-medium university in Bangladesh, writing career essays for hypothetical peer profiles.
- Method: Between-subjects lab experiment comparing writing under no AI, neutral AI, and biased AI conditions.
Why it matters
Students increasingly use AI writing assistants, but whether prompt-induced gender bias within large language models propagates into human-authored text remains underexplored.
Key findings
- Students assisted by a gender-biased LLM wrote essays with a significantly larger agentic language gap and more gender-stereotypic occupation recommendations than students in control or neutral conditions.
- Bias transfer operated through an asymmetric suppression mechanism, significantly lowering agency language in female-target essays while male-target essays remained largely unaffected across conditions.
- Participants in the biased condition showed higher stereotype alignment without realizing it, reporting lower perceived AI influence and noticing no unusual suggestions during the task.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
