Where does bias in GPT-3 come from?

GPT-3 has no opinions of its own. It is a language model: a program trained to guess the next piece of text, again and again, until it has written a reply. What it guesses depends entirely on the text it studied during training. If that text links nurses with women and bankers with men more often than not, the model learns that link as a pattern worth repeating. Bias, in this sense, is a statistical habit picked up from people's writing.

The researchers who built GPT-3 say so plainly in their own paper. Their summary is that models trained on the internet carry biases on the scale of the internet. This matters now because GPT-3 is reaching real products through OpenAI's private API beta, and OpenAI's own FAQ for the API acknowledges that its models show biases that will appear from time to time in generated text.

What GPT-3 learned from

According to the paper, GPT-3 was trained on a mix of five text collections, measured in tokens (a token is a word or a piece of a word):

Common Crawl is a huge public archive of web pages. OpenAI filtered it, keeping pages that looked similar to higher-quality reference text, and removed near-duplicates. WebText2 is a larger version of the set of pages linked from Reddit that was used for GPT-2. Filtering improves quality, but it does not remove stereotypes. A well-written article can still assume that a doctor is a "he."

There is a second, quieter source of bias: who writes on the internet in the first place. Some groups, languages and points of view appear far more than others. The model can only reflect the voices it saw.

What the GPT-3 paper found

The authors ran a first round of tests on three areas, gender, race and religion, while stressing that the analysis is preliminary and far from complete.

• Gender and jobs. They gave the model sentences such as "The detective was a" and compared how likely it was to continue with a male word or a female word. Of 388 occupations tested, 83% leaned male. Jobs needing more education, like legislator or banker, leaned male strongly, while midwife, nurse and receptionist leaned female. Adding "competent" before the job made the male lean stronger.
• Describing words. From prompts like "She was very" and "He was very," the model described women far more often with looks-related words such as "beautiful" and "gorgeous."
• Race. Prompts about people of different races produced text whose words carried different average sentiment. "Black" scored lowest in five of the seven model sizes tested.
• Religion. For Islam, words such as "violent" and "terrorism" appeared with the religion more often than with others.

None of this is new to GPT-3. A 2019 study by Emily Sheng and colleagues fed GPT-2 paired openings such as "The man worked as" and "The woman worked as" and found the endings treated groups differently. They proposed measuring "regard": whether the generated text makes people think better or worse of a group.

A worked example: rewriting for neutral output

Bias tends to surface when the prompt leaves gaps the model fills with its most common association. Compare these two prompts for a job ad.

Write a short job ad for a nurse at a family clinic.

The model may write "she" throughout, because that is the pattern in much of its data. Now close the gap:

Write a short job ad for a nurse at a family clinic. Address the reader directly as "you" and never use he, she, him or her. Describe the role only by its duties and the skills it needs.

Job ad:

This does not change what GPT-3 believes; it has no beliefs. It changes what counts as the likely continuation. You have told it the form the text should take, so the usual association has less room to appear.

What can you do about it as a user?

Start by assuming bias is present, even when an answer looks polished. Read anything about people, jobs, nationalities or religions with extra care before using it, and never let the model make decisions about real individuals on its own. The GPT-3 paper itself notes that the generated text tends to mirror the stereotypes in its training data, so the burden of checking falls on you.

Next, write prompts that leave less to chance. Specify the audience, use names or "you" instead of letting the model choose pronouns, and give examples that show the balance you want. Since GPT-3 copies the pattern in your examples, a few varied ones can steer it more reliably than an instruction alone.

Then test before you rely on something. Swap one detail at a time, such as a name, a gender or a country, and see whether the answer shifts in ways it shouldn't. This is the same method the researchers used, and it works with a handful of prompts.

Finally, if you are building with the API, follow OpenAI's usage guidelines and keep a person reviewing output that reaches the public. OpenAI says it is developing tools to label and intervene on harmful bias, but for now careful prompts and human review are the practical safeguards.