What are ChatGPT plugins?

Until this week, everything ChatGPT told you came from one place: the text it was trained on, which mostly stops around September 2021. It could not check today's news, look up a restaurant's opening hours or do anything except write text back to you. Yesterday OpenAI announced plugins, a way to connect ChatGPT to outside tools so it can fetch current information, calculate and act on your behalf.

OpenAI describes plugins as giving the model "eyes and ears": a way to reach facts that are newer, more private or more specialized than anything in its training data. Access is limited for now. OpenAI is admitting a limited group of developers and Plus subscribers from a waitlist, and says it will widen access as it learns.

How a plugin works

A plugin is a connection between ChatGPT and a web service. The company that builds it writes two things: an API (a set of addresses a program can call to get data or take an action) and a short description of what that API does, written for the model to read. That description is packed into a small "manifest" file along with a standard technical outline called an OpenAPI specification.

When you start a chat, you pick which plugins to turn on. ChatGPT then sees the descriptions of those plugins as part of the conversation. If your request needs one, the model decides to call it, reads what comes back and writes its answer from that. You do not type commands; you ask in plain language, and the model chooses the tool.

Here is the flow for a dinner booking:

You ask: "Find a quiet Italian restaurant near me for four people on Saturday at 7."
ChatGPT sees the OpenTable plugin is on and that it handles reservations.
It calls the plugin with the details it pulled from your sentence.
The plugin returns options, and ChatGPT presents them with a link to book.

The first plugins

OpenAI is launching with plugins from twelve companies. For travel and dining: Expedia, KAYAK and OpenTable. For shopping: Instacart, Klarna and Shopify. For work and knowledge: Slack, Zapier, Wolfram and FiscalNote. The last two are Speak, a language tutor, and Milo, a family assistant. It also runs two of its own and has published the code for a third:

Plugin	What it adds
Browsing (OpenAI)	Searches with Bing, reads pages, shows which sites it used and cites them
code interpreter (OpenAI)	Runs Python in a sealed-off sandbox; accepts file uploads and gives files back
Retrieval (open source)	Lets a developer connect ChatGPT to their own notes, files or documents
Wolfram	Exact math, curated data and computation
Zapier	Links to thousands of apps, such as Google Sheets, Trello and Gmail
OpenTable, Instacart, KAYAK, Expedia	Restaurant bookings, groceries, flights, stays and trips

The browsing plugin is the one most people have wanted. OpenAI's demo shows it answering a question about this month's Oscar winners, something the model alone could not do. Because it shows its sources, you can click through and check them.

What to be careful about

Connecting a language model to tools that act in the world raises new risks, and OpenAI says so plainly. Its red teams, testers whose job is to try to break the system, found ways a plugin without safeguards could be tricked by hidden instructions on a web page (known as prompt injection), send spam or fraudulent email, or misuse data. OpenAI says it has added limits in response: the browser can only read pages, not submit forms, and the code sandbox has no internet access.

TechCrunch points out a related issue. The live web is messier than a curated training set, and an earlier OpenAI research system with web access, WebGPT, sometimes quoted unreliable sources. A cited answer is easier to check, but it still needs checking.

Try it: Even without plugin access, write down three questions you asked ChatGPT that failed because it lacked current or personal information. For each, name the kind of plugin that would have fixed it: browsing, a calculator, a booking service or your own documents.

Which everyday tasks will plugins help with first?

The first wins are likely to be questions about the present. With browsing, you can ask about recent news, a product released this month or a policy change, and get an answer with links instead of a polite apology about its training data. That alone removes one of the most common frustrations beginners have with ChatGPT.

Next come calculations and data. Language models can make simple calculation and reasoning errors; Wolfram and the code interpreter hand the math to tools built for it. OpenAI suggests the code interpreter for solving math problems, analyzing data and drawing charts, and converting files from one format to another. For a small business owner with a spreadsheet of sales, that could mean uploading the file and asking for a monthly chart in plain English.

Then there is planning and errands: comparing flights and hotels, finding a restaurant table, building a grocery list and sending it to Instacart. These save time, but they involve your money and your personal details, so read every step before confirming anything.

Finally, Zapier and the retrieval plugin point toward work tasks: drafting an email from notes in another app, or asking questions of your own documents. These are early and will take time to roll out widely. Many expect plugins to become a main way people use ChatGPT over the coming months, but for now they are an alpha feature open to only a few people.