What is Poe, and why use several chatbots in one app?

Poe opened to everyone on February 3, after an invite-only start in December. The name stands for Platform for Open Exploration, and the idea is simple: Quora does not build its own language model. Instead, it gives you one place to talk to models made by others. The app looks like a messaging app, and each bot is a separate chat thread, the way each friend gets a thread in a texting app.

That design changes how you can use AI. With a single chatbot, you get one answer and must decide on your own whether to trust it. With several, you can put the same question to each and compare. When they agree, you have a little more confidence. When they disagree, you know exactly where to start checking.

What you get in the app today

Right now Poe runs on iPhone and iPad only, and Quora says other platforms should follow in a few months. To sign up, you verify your account with both an email address and a phone number. The app itself is free.

Three general-purpose bots are available:

• Sage, powered by OpenAI, the company behind ChatGPT.
• Dragonfly, also powered by OpenAI. TechCrunch notes it may decline to answer some questions.
• A third bot built on technology from Anthropic, another AI research company.

Two limits apply to all of them. Sage and the Anthropic bot know nothing about events after 2021, because their training text stops there. And all three have been known to state wrong facts with confidence, which is one reason Quora keeps Poe separate from its main question-and-answer site for now.

Poe also has a social side. You can create a profile, follow other people and publish a bot's answer to a public feed, where others can like or repost it. Quora's CEO, Adam D'Angelo, has said he expects many more models to arrive, each tuned for different tasks or points of view, and the company plans to let outside developers plug their own models into the app.

Why several bots beat one

Every language model is shaped by its training text and by the choices of the people who tuned it. That gives each one a recognizable personality: one may write long, careful answers, another short ones; one may refuse a borderline question, another may answer it.

Comparing bots gives you three practical benefits:

1. A quick error check. If two bots give different dates or numbers, at least one of them is wrong.
2. Better drafts. For writing tasks, two versions give you more phrasing to choose from.
3. A sense of each tool. After a week of side-by-side questions, you learn which bot suits which kind of task, the same way you learn which colleague to ask about what.

A worked comparison

Here is a simple test you can run in a few minutes. Send exactly the same prompt to each bot, in a fresh thread, so earlier messages do not influence the reply.

In three sentences, explain why the sky is blue to a 10-year-old. Then give one simple experiment a child could do at home to see the same effect.

Then fill in a small table like this one. The entries below are illustrative, to show the kind of differences to note, not real results from the app.

A question like this works well because you can check the science in any encyclopedia. Once you trust your scoring method, try questions from your own work.

How do you compare answers from different bots?

Start by keeping the test fair. Use the same wording for every bot, start a new conversation each time, and ask only one question per prompt. If you change the wording between bots, you are no longer comparing the bots; you are comparing your prompts.

Next, judge the answers against a short list of criteria decided in advance: is it accurate, did it follow the instructions (length, format, audience), is it complete, and is it clear? Writing the criteria down first stops you from simply preferring the answer that sounds most confident. Confidence is not accuracy; all three bots can sound sure of a wrong fact.

Finally, treat agreement and disagreement differently. When the bots agree on a fact, it is still worth a quick check in a trusted source, because they may have learned the same mistake. When they disagree, check before you use anything. And for anything that happened after 2021, do not rely on these bots at all: look it up in a current source instead.