Why do language models hallucinate, according to OpenAI's research?

A hallucination is a statement from an AI model that sounds plausible but is false: an invented book title, a wrong date said with confidence, a quote nobody said. Users have known about the problem since the first chatbots. What has been missing is a clear explanation of why it persists even as models get better at almost everything else.

On September 5, OpenAI published a research paper titled Why Language Models Hallucinate, with a plain-language summary on its blog. Its authors are Adam Tauman Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang. Its answer has two parts: where the errors come from, and why training does not remove them.

Part one: some facts can't be learned from patterns

A language model first learns by predicting the next word across a huge amount of text. That text has no labels saying "this sentence is true" or "this one is false"; the model sees only examples of fluent writing.

Some things follow patterns that can be learned this way. Spelling and matching brackets are consistent, so large models rarely get them wrong. Other things are arbitrary. A particular person's birthday has no pattern; you either saw it or you didn't. The paper shows that if a fact appears only once in the training data, the model has very little to go on, and it estimates that the rate of errors on such facts will be at least as high as the share of facts that appear only once. If a fifth of birthdays in the data are mentioned a single time, expect the base model to get at least a fifth of birthday questions wrong.

The authors illustrate this with the lead author's own birthday. Asked for it, with an instruction to answer only if known, an open-source model gave three different wrong dates in three tries.

Part two: tests reward bluffing

After that first stage, models are refined and then measured on benchmarks, standard sets of test questions. Most benchmarks score an answer as right or wrong, one point or zero. An "I don't know" also scores zero.

OpenAI compares this to a multiple-choice exam. If you leave a question blank, you certainly get nothing; if you guess, you might get lucky. A student who always guesses ends up with a better score than one who admits uncertainty, and a model tuned to score well learns the same habit. The paper calls this an epidemic of penalizing uncertainty.

OpenAI's blog gives an example from its own testing on SimpleQA, a set of short factual questions:

The older model scores slightly higher on accuracy, so it would rank higher on a leaderboard that only counts right answers. But it gives a wrong answer nearly three times as often.

The proposed fix is to change the scoring: penalize confident errors more than abstentions, and state the rule in the question. The paper suggests instructions like "answer only if you are more than 75% confident; a wrong answer costs 2 points, a correct one earns 1, and 'I don't know' earns 0."

A worked prompt that allows "I don't know"

You can't retrain ChatGPT, but you can apply the same idea in your own prompts by making uncertainty an acceptable answer.

I need accurate facts for a printed handout, so a wrong answer is worse than no answer.

For each question below:
- Answer only if you are confident. If you are not, write "Not sure" and say what I should check.
- Mark each answer as High, Medium or Low confidence.
- If you use web search, name the page you relied on.

1. In what year did our town's public library open?
2. Who designed the covered bridge on Mill Road?
3. What is the current fee for a library card for non-residents?

This prompt does three things. It explains the cost of a mistake, it gives the model a scoring rule that rewards honesty, and it asks for a confidence level for each item. It will not make the model perfectly calibrated, but it removes the pressure to fill every blank.

What does it mean for how you check answers?

First, expect the riskiest answers to be about rare details: a small company's founding year, a minor author's book list, a local bylaw, a specific statistic. The research suggests these are exactly the facts a model has seen least, and therefore the ones it is most likely to invent. Famous facts are safer; obscure ones need checking every time.

Second, treat confidence as style, not evidence. A model trained to score well has learned that a definite answer usually pays. A precise date or a neat quotation tells you nothing about whether it's right. Confirm anything you will repeat, print or act on against a source you can open and read.

Third, reward honesty in your own use. When a model says "I'm not sure," that is useful information, not a failure. Ask follow-up questions, turn on web search for current facts, and ask for the source so you can check it. OpenAI says GPT-5 hallucinates less than earlier models, especially when it reasons, but it still does; the habit of verifying stays the same.