Why can't you trust everything ChatGPT says?

ChatGPT is very good at sounding right. Its grammar is clean, its tone is calm, and it rarely hedges. That polish is exactly what makes its mistakes easy to miss. When it gets a date, a statistic or a book title wrong, it says so in the same confident voice it uses for everything else.

A Beginner's Guide to ChatGPT, Robert Scales's book published in April, opens with a disclaimer on this point before teaching a single prompt: GPT models produce errors and can invent information, so check sources, data, contact details and numbers, and keep a human in charge of the final version. This last article in our series on the book's method covers the final step: verify.

Why ChatGPT gets things wrong

A language model is trained to predict likely text, not to look up facts. When you ask a question, it writes the answer that best fits the patterns it learned. Most of the time the likely answer and the true answer are the same. When they are not, you get what people call a hallucination: a fluent statement with nothing real behind it.

OpenAI is open about this. Its ChatGPT help page says the tool is not connected to the internet, can occasionally give incorrect answers, has limited knowledge of events after 2021, and will sometimes make up facts. The same page recommends checking whether the model's responses are accurate. It also explains why the answers feel so human: the models learned from huge amounts of human writing, which makes them sound natural even when they are wrong.

GPT-4, the newer model available to ChatGPT Plus subscribers, is better but not cured. OpenAI reports it scored 40% higher than GPT-3.5 on its internal tests of factual accuracy, and in the same announcement says GPT-4 "still is not fully reliable." It can be confidently wrong and can accept a false statement from the user too easily.

Which claims to check first

Not every sentence carries the same risk. A brainstorm of party themes cannot really be wrong. A summary of tax rules can cost you money. Spend your checking time where errors hurt:

A useful rule: the more specific and checkable a claim looks, the more important it is to check it. Specific details make an answer persuasive, and they are also where invented content tends to hide.

Ask for sources, then check them

The book suggests asking ChatGPT to show where its claims come from. That helps, as long as you treat the sources it lists as leads rather than proof. Because the model can make up facts, it can also produce titles, authors or links that look real and are not. Ask in a way that makes checking easy:

For each factual claim in your last answer, list it on its own line. Next to each one, say how confident you are (high, medium or low) and name the kind of source where I could confirm it, such as an official website, a government page or a published study. Do not invent titles or links; if you are not sure a specific source exists, say so.

This does not make the model truthful, but it turns one long answer into a checklist, and it flags the weakest claims for you.

How do you check an AI answer quickly?

Use a short routine that fits into a few minutes. First, skim the answer and underline the claims that matter: numbers, names, dates, quotes and instructions you would act on. Ignore the rest for now.

Second, check each underlined claim against one reliable source, ideally the original: the official website, the government page, the company's own announcement, the published study. A search engine is the right tool here, because it shows you where information comes from. If you cannot find a claim anywhere outside the chat, treat it as unconfirmed and leave it out.

Third, test the shaky parts with the model itself. Ask the same question in a new chat or in different words. If two answers disagree on a fact, at least one is wrong. You can also challenge it directly ("Are you sure about that date?"), but remember it may simply agree with you either way.

Finally, keep the human in charge. Use ChatGPT for what it does best, such as drafting, explaining, organizing and suggesting, and do the confirming yourself. The book's method ends here for a reason: role, context, task and format get you a better answer, and verifying is what makes it safe to use.