Why do Google's AI Overviews sometimes get things wrong?

On May 14, at its I/O conference, Google began showing AI Overviews to everyone in the United States: a short, AI-written answer at the top of the search results, with links to the pages it drew on. Within days, screenshots of strange answers were everywhere. One treated the question of how many rocks a person should eat as if it were serious. Another recommended glue as a way to keep cheese from sliding off pizza.

On May 30, Liz Reid, who leads Google Search, published an explanation of what went wrong. Her post is useful for anyone who uses AI search, because the mistakes follow patterns, and once you know the patterns you can spot them.

How AI Overviews work

According to Google, AI Overviews are not a chatbot answering from memory. A custom version of Google's Gemini model works together with the company's ranking systems, the same machinery that decides which pages appear first in ordinary results. The model is meant to write only what the top results support and to link to them, so you can check.

Google says this means AI Overviews generally don't "hallucinate," the term for AI inventing facts. Instead, Google points to three different causes: misunderstanding what the person asked, misreading a nuance on a web page, and lack of good information on the topic. In other words, the summary is only as good as the pages it reads and the way it reads them.

The worked example: rocks and glue

The two famous mistakes show two of those causes at work.

Eating rocks: a data void. Almost nobody had ever searched for how many rocks to eat, and almost nobody had written seriously about it. Google calls this a data void: a question with little reliable content behind it. One of the few pages that did address it was a satirical article, later republished on the website of a geology software company. The system treated it as a real answer and linked to it.

Glue on pizza: a forum joke. Discussion forums often contain genuine first-hand advice, which is why search engines value them. They also contain sarcasm and trolling. In this case, a joking suggestion from a forum ended up presented as a cooking tip.

Google also says many of the screenshots shared online were fake, including some that showed dangerous advice about depression or leaving dogs in cars. Its suggestion: run the search yourself before believing a screenshot.

What Google changed

Google says it made more than a dozen technical changes. They include better detection of nonsensical questions that should not get an AI Overview, less use of satire and humor, less reliance on user-written content where it could mislead, and fewer AI Overviews for queries where they were not helping. For health topics, it added extra restrictions, and it aims not to show AI Overviews at all for hard news. Google reports that content-policy violations turned up in fewer than one out of 7 million distinct searches that displayed an AI Overview.

Scale is part of the story. TechCrunch reported at launch that AI Overviews were reaching hundreds of millions of US users that week, and that Google's goal is to pass one billion people before 2024 ends. Even a very small error rate means a lot of odd answers when the numbers are that large.

How do you use them safely?

Treat an AI Overview as a starting point, not a verdict. It is a quick summary of what some pages say, and its value depends on those pages. The links are the most useful part: open at least one before you act on anything that matters.

Look at who is speaking. If the answer leans on a forum, a humor site or a company selling something, weigh it accordingly. For health, money, legal or safety questions, go to an authoritative source directly, such as a government health agency, your bank or a licensed professional, even if the overview looks reasonable.

Be most careful with unusual questions. The rock example shows that the strangest queries are the ones most likely to land in a data void. If you are asking something few people have asked before, the summary may rest on one odd page.

Finally, check screenshots before you share them. Some of the most alarming examples circulating in recent days never appeared in Google at all. Searching the same question yourself takes a few seconds and keeps a fake from spreading.