Short answer
On March 1, 2023, OpenAI let developers build ChatGPT and its Whisper speech-to-text model into their own apps through its API, a way for programs to send requests and get answers back. The ChatGPT model, gpt-3.5-turbo, costs $0.002 per 1,000 tokens, ten times less than OpenAI's earlier GPT-3.5 models, and Whisper costs $0.006 per minute of audio.
Follow-up question How can a beginner start building with the ChatGPT API?
Yesterday, March 1, 2023, OpenAI opened two of its best-known models to software developers. The model behind ChatGPT is now available through OpenAI's API under the name gpt-3.5-turbo, and so is Whisper, the speech recognition model OpenAI made open source last September.
An API (application programming interface) is a doorway for programs rather than people. Instead of typing into the ChatGPT website, a developer's app sends a request over the internet and receives the model's answer, which it can show inside its own screens. That means ChatGPT-style help can now appear in shopping apps, study tools and messaging apps, not only on OpenAI's site.
What was released
OpenAI says that system-wide optimizations have cut the cost of running ChatGPT by 90% since December, and that it is passing the savings on.
| Item | What it does | Price |
|---|---|---|
| gpt-3.5-turbo | The model that powers ChatGPT itself, for chat and many non-chat tasks | $0.002 per 1,000 tokens |
| Whisper (large-v2) | Turns speech into text, or into English text from other languages | $0.006 per minute of audio |
| Dedicated instances | Computing capacity reserved for one customer, on Microsoft's Azure | Paid by time period |
A token is a chunk of text; TechCrunch notes that 1,000 tokens are roughly 750 words. Here is what those prices mean in practice:
- A chat exchange of about 750 words, question and answer together, costs about $0.002, so 1,000 such exchanges cost about $2.
- Transcribing a one-hour recording with Whisper costs 60 × $0.006 = $0.36.
- OpenAI says gpt-3.5-turbo is ten times cheaper than its existing GPT-3.5 models, and early testers moved from text-davinci-003 to it with small prompt changes.
How the chat API works
Earlier GPT models took one block of plain text and continued it. The new chat endpoint instead takes a list of messages, each labeled with a role: for example, a "system" message from the developer that sets the rules, then "user" messages from the person, and "assistant" messages with the model's earlier replies. Underneath, OpenAI converts this into a format it calls Chat Markup Language, or ChatML.
A simple request might contain:
system: You are a patient cooking tutor. Answer in three short numbered steps.
user: How do I keep rice from sticking to the pot?The reply comes back as an "assistant" message, along with a count of the tokens used for the prompt and the answer. To continue the conversation, the app sends the whole list again with the new question added; the model itself does not remember earlier calls.
Greg Brockman, OpenAI's president, told TechCrunch that separating developer instructions from user text should make apps harder to trick with "prompt attacks," messages that try to make a model ignore its rules. He gave the example of an AI tutor that should always explain an answer rather than simply give it away.
OpenAI also introduced model snapshots. Developers who ask for gpt-3.5-turbo always get the recommended stable version, which will be updated in April, but they can pin a specific one, gpt-3.5-turbo-0301, which OpenAI will support until at least June 1.
Who is already using it
OpenAI names five early users:
- Snap runs My AI, a chatbot for Snapchat+ subscribers, on the ChatGPT API.
- Quizlet is launching Q-Chat, an adaptive tutor that quizzes students on their study material.
- Instacart plans "Ask Instacart," which answers food questions with suggestions customers can buy.
- Shopify's Shop app uses it for a shopping assistant that recommends products.
- Speak, a language-learning app, uses the Whisper API for a speaking companion.
Why it matters
Price and access are the story. A tenfold drop in cost makes it realistic for small teams and individual developers to build on the same model millions of people already use through ChatGPT. OpenAI also announced changes developers have asked for: data sent through the API is no longer used to train models unless a company opts in, data is kept for 30 days by default, and a pre-launch review step has been removed. It also admits that its uptime over the past two months has not been good enough and says reliability is now its engineering team's top priority.
Try it: Take a task you repeat at work, such as replying to common customer questions. Write the "system" message you would give a chatbot for it in two or three sentences, then write two sample "user" messages it should handle well and one it should politely refuse.
How can a beginner start building with the ChatGPT API?
Begin by planning on paper. Decide what your app should do, write the system message that sets its tone and limits, and collect a handful of realistic user questions. Clear instructions matter more than code at this stage, and the same prompting skills you use in ChatGPT apply directly.
Next, create an account on OpenAI's developer platform and follow its Chat guide, which shows a request in a few lines of Python or a single command-line call. Start with small tests, and watch the token counts that come back with each reply so you know what each experiment costs.
Finally, design for the model's weaknesses. It can still state wrong facts with confidence, so keep a person in the loop for anything important, show users that answers come from AI, and do not send private data unless you understand OpenAI's data policy. If you need speech, add Whisper later: record audio, send it for transcription, and pass the text to the chat model.
Sources
- Introducing ChatGPT and Whisper APIs — OpenAI, March 1, 2023
- OpenAI launches an API for ChatGPT, plus dedicated capacity for enterprise customers — TechCrunch (Kyle Wiggers), March 1, 2023