# How is an answer engine like Perplexity different from ChatGPT?

Date: 2022-12-16 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/answer-engine-like-perplexity-different-chatgpt/

**Answer:** ChatGPT writes from what it absorbed during training and shows no sources, while an answer engine like Perplexity Ask first runs a web search, then has a language model summarize the results and cite the pages it used. Citations make the answer checkable, but they do not make it correct, so you still have to open them.

On December 7, a week after ChatGPT appeared, a startup called Perplexity announced Perplexity Ask, a tool that answers questions in plain sentences. At first glance it looks like ChatGPT: you type a question, you get a paragraph back. Underneath, it works differently, and the difference is the most important thing to understand about it.

Perplexity describes Ask as a search interface that combines OpenAI's GPT-3.5 language model with Microsoft's Bing search engine. Instead of returning a list of links, it summarizes what the search found and includes citations so you can verify the information. A handy name for tools like this is "answer engine": a search engine that gives you an answer rather than a page of results.

## Two ways to answer a question

ChatGPT answers from memory. It is a language model trained on a large collection of text, and when you ask a question it writes the reply that its training makes most likely. It does not look anything up, and it cannot point to where an idea came from. OpenAI's own launch post warns that it sometimes produces answers that sound plausible but are wrong.

An answer engine adds a step before the writing. First it sends your question, or a version of it, to a search engine. Then it hands the model some of the pages that come back and asks it to write an answer based on them, citing which page supports which part. Researchers call this pattern retrieval: fetch relevant documents first, then generate.

Perplexity says the idea was inspired by WebGPT, a research project OpenAI described a year ago. In WebGPT, a version of GPT-3 learned to use a text-only web browser: it searched through Bing, opened pages, collected passages and wrote an answer from them, citing its sources. OpenAI's reason for the citations was practical: they let a person check whether each claim is backed by a reliable page.

| | ChatGPT | Answer engine (Perplexity Ask) |
|---|---|---|
| Where the answer comes from | Patterns learned in training | Web pages found by a search, then summarized |
| Shows sources | No | Yes, as citations |
| Recent events | Limited; its knowledge thins out after 2021 | Can draw on pages found by a live search |
| Best at | Explaining, drafting, rewriting, brainstorming | Fact-finding questions you want to verify |
| Main risk | Confident invention you cannot trace | A wrong summary of a source, or a weak source |

## Why citations are not a guarantee

Citations are a big step forward, but OpenAI's WebGPT post lists the catches clearly. Its model sometimes quoted unreliable sources. It still made basic errors. And OpenAI warned that answers with citations tend to look authoritative, which can hide those errors. A capable enough system could even pick sources that sound convincing rather than ones that give a fair picture.

There are three ways a cited answer can go wrong:

1. **The source is weak.** The search found a forum post, an outdated page or a site with an agenda.
2. **The summary is wrong.** The page is fine, but the model misread it or stretched what it says.
3. **The citation does not match.** The sentence and the link next to it are about different things.

## A worked example: checking a cited answer

Suppose you ask an answer engine a question where the facts matter.

```prompt
How much vitamin D should an adult take per day?
```

Say the reply gives a number and cites two pages. Here is how to check it in about three minutes.

1. **Open every citation.** Do not judge the answer by its tone or by the number of links.
2. **Find the exact sentence.** Use your browser's find command (Ctrl+F or Cmd+F) to search the page for the number or key phrase in the answer.
3. **Compare carefully.** Does the page say the same thing, with the same conditions? Medical guidance often depends on age, country and health, and summaries drop those details.
4. **Judge the source.** Is it a health agency, a university or a medical body, or an anonymous blog? When was it last updated?
5. **Look for a second source** that the tool did not choose, such as your national health authority's site.

If the cited page does not contain the claim, treat that part of the answer as unsupported, however confident it sounds.

> **Try it:** Ask Perplexity Ask and ChatGPT the same factual question about something in the news this month. Compare the two replies, then open every citation Perplexity gives and note whether each one actually supports the sentence it is attached to.

## How do you check the sources it cites?

Start by reading the citation, not just noticing it. A link beside a sentence is a claim that the page supports that sentence; your job is to confirm it. Open each one, locate the passage, and check that the details match, including numbers, dates, names and any conditions or exceptions.

Next, weigh the source itself. Prefer primary sources: the organization, study or official document the fact comes from. Be wary of pages that only repeat other pages, and of sources with something to sell. Check the date, because a correct page from several years ago can still be wrong today.

Finally, use the tool for what it does best. An answer engine is an excellent way to find where to start reading. ChatGPT is an excellent way to have a concept explained or a draft written. Neither replaces looking at the evidence yourself when the answer matters.

## Sources

- Announcing Perplexity Ask (launch thread): https://twitter.com/perplexity_ai/status/1600551871554338816
- WebGPT: Improving the Factual Accuracy of Language Models through Web Browsing: https://openai.com/blog/webgpt/
- ChatGPT: Optimizing Language Models for Dialogue: https://openai.com/blog/chatgpt/