Prompting

Why does AI still invent things in 2026?

· By Robert Scales · 3 min read

Short answer

Language models are trained to produce a likely next word, and most tests still reward a confident guess over "I don't know," so they fill gaps with plausible inventions, especially for rare facts, names, numbers and citations. Newer models make fewer errors, but even the best still get a meaningful share of hard questions wrong, which is why checking remains part of using AI.

A hallucination is an answer that sounds right but is false: a study that does not exist, a wrong date stated calmly, a quote nobody said. Models in October 2026 are far better than the chatbots of a few years ago, and they invent less. But they have not stopped, and the reasons are now well enough understood to explain in plain words.

The numbers give a sense of scale. When OpenAI released GPT-6.1 Sol in September 2026, it reported a test on deliberately difficult prompts, drawn from chats in which people had spotted a mistake by an older model. At low reasoning effort, answers with one or more factual errors dropped from 11.4% to 7.7%. OpenAI stresses that these prompts are harder than typical use. Still, roughly one hard answer in thirteen with an error is a good reason to keep checking.

Reason 1: some facts cannot be learned from patterns

A model learns by predicting the next word across enormous amounts of text. Frequent facts, such as the capital of France, appear thousands of times and are learned well. A fact that appears only once, such as an obscure person's birthday, offers no pattern to learn. The 2025 research paper Why Language Models Hallucinate, by researchers at OpenAI and Georgia Tech, argues that for such one-off facts, the error rate will be at least as high as their share of the training data. No amount of polish removes that floor; the model simply does not know.

Reason 2: tests reward bluffing

The same paper points to how models are graded. Most benchmarks give one point for a right answer and zero for both a wrong answer and "I don't know." Under that rule, guessing always scores at least as well as admitting uncertainty, so training pushes models toward confident guesses, like a student filling in every blank on a multiple-choice exam. The authors propose scoring that penalizes wrong answers more than abstentions, and they suggest stating a confidence target directly in instructions.

Reason 3: a "don't know" switch that misfires

Anthropic's interpretability research, which looks inside a model at the internal features it uses, found something that matches everyday experience. In Claude, declining to answer was the default: a circuit saying "I can't answer this" was active unless features for a "known entity" switched it off. Hallucinations appeared when that switch flipped by mistake, for example when the model recognized a name but knew nothing else about the person. Familiar-sounding questions about little-known people, papers or products are therefore the riskiest kind.

How do you verify an answer in five steps?

Step 1: Mark the checkable claims. Read the answer and underline names, numbers, dates, quotes, prices and references. Explanations and opinions are judged differently; it is these specific claims that can be invented.

Step 2: Ask the model to separate what it knows from what it guesses. A short follow-up prompt, based on the confidence-target idea above, changes the incentive:

Prompt
Go back over your previous answer. For each factual claim (names, numbers, dates, quotes, sources), list it on its own line and mark it:
- SURE if you would bet on it being correct,
- UNSURE if you are guessing or reconstructing it,
- UNKNOWN if you do not know.
Answer only when you are more than 75% confident; a wrong answer counts as worse than "UNKNOWN". For each SURE claim, name where someone could check it.

Step 3: Open the source yourself. For every claim that matters, follow the link or search the reference. Confirm three things: the source exists, it is the kind of source that would know (an official page, the original paper, a government site), and it actually says what the answer claims. Invented citations often have real-sounding authors and journals.

Step 4: Cross-check once more. Find a second independent source, or ask the same question in a fresh conversation without your earlier wording. If the model gives different specifics each time, treat them all as guesses. In the 2025 paper, an open model asked for a researcher's birthday, even with an instruction to answer only if it knew, produced three different wrong dates.

Step 5: Match the checking to the stakes. A wrong detail in a birthday card costs nothing; a wrong dose, deadline or legal rule can cost a great deal. For health, money, law and anything you will publish, the primary source or a qualified person has the final word, not the model.

Try it: Ask your usual assistant about a real but little-known person or company in your region, then run steps 2 to 4. Count how many of its specific claims survive, and note which kind of claim failed.

These steps take a few minutes, and with practice you learn where a given model tends to slip. The goal is not to distrust everything, but to know which parts of an answer need a second look.

Sources

  1. Why Language Models Hallucinate — arXiv, September 4, 2025
  2. Introducing GPT-6.1 Sol — OpenAI, September 29, 2026
  3. Tracing the thoughts of a large language model — Anthropic, March 27, 2025

Get the free Guide to Artificial Intelligence

Sign up for the weekly newsletter and receive the free guide: what changed in AI this week, one lesson, one prompt to try. Unsubscribe in one click.

How we handle your address

Back to top

Display

Your choices are saved in this browser only. No data is sent.

Text size
100%
Colours
Reading
More settings
Accessibility statement