GPT-4o is OpenAI's new flagship model, announced on May 13, 2024, where the "o" stands for "omni": one model that handles text, audio, images and video instead of chaining separate systems. It offers GPT-4-level intelligence, runs faster and costs half as much in the API, and it is rolling out to free ChatGPT users as well as paying ones.

What is GPT-4o?

Yesterday OpenAI introduced GPT-4o, its new flagship model, in a live online event. The "o" stands for "omni." Where earlier versions of ChatGPT handled voice by passing your speech through several separate programs, GPT-4o is a single model trained end to end on text, images and audio. It can take any mix of text, audio, images and video as input and produce text, audio and images as output.

For most people, two things matter right away. First, it is coming to ChatGPT's free tier, so many more people will get GPT-4-level answers without paying. Second, its voice abilities point toward spoken conversations that feel close to talking with a person, although that part is not switched on yet.

What is new, in plain terms

Until now, ChatGPT's Voice Mode worked like a relay team. One model turned your speech into text, GPT-3.5 or GPT-4 wrote a reply in text, and a third model read that reply aloud. OpenAI says this took 2.8 seconds on average with GPT-3.5 and 5.4 seconds with GPT-4. Worse, the main model never heard you: it could not pick up your tone, tell several speakers apart or notice background sounds, and it could not laugh or sing.

GPT-4o removes the relay. Because one network handles everything, OpenAI says its fastest replies to speech arrive after 232 milliseconds and the typical reply after about 320, which is close to the pace of human conversation.

	Earlier Voice Mode	GPT-4o
How it works	Three separate models in a chain	One model for text, audio and vision
Average response to speech	2.8 s (GPT-3.5) or 5.4 s (GPT-4)	About 0.32 s
Hears tone and background	No	Yes, according to OpenAI

On text, OpenAI says GPT-4o matches GPT-4 Turbo in English and in code, does better in other languages, and is notably better at understanding images and audio than earlier models. For developers, it is twice as fast as GPT-4 Turbo in the API, half the price, with five times higher rate limits.

What you can use today, and what is coming

OpenAI is rolling features out in stages:

Now: GPT-4o's text and image abilities are starting to appear in ChatGPT for Plus and Team users and, with usage limits, for free users. Plus subscribers get up to five times more messages than free users. When free users hit their limit, ChatGPT switches back to GPT-3.5.
For free users, over the coming weeks: web answers, data analysis and charts, chatting about photos, file uploads, access to GPTs and the GPT Store, and Memory.
Coming weeks: a new Voice Mode built on GPT-4o, first as an alpha test for Plus users. At first, audio output will be limited to a set of preset voices.
Also new: a ChatGPT desktop app for macOS, starting with Plus users, which uses the existing Voice Mode for now.

OpenAI is clear that audio brings new risks, which is why it is releasing text and image first and holding back the new voice and video features while it works on safety.

A worked example with images

You can try GPT-4o's image understanding as soon as it reaches your account. OpenAI's own example is photographing a menu in another language. A prompt in the same spirit:

Prompt

Here is a photo of a menu in Portuguese.
1. Translate each dish name into English.
2. Mark which dishes are vegetarian, and say if you are unsure.
3. Suggest how to order the soup politely in Portuguese, with a pronunciation guide.

Asking it to flag uncertainty matters. Reading a photo is harder than reading typed text, and blurry or handwritten items are where mistakes creep in.

Try it: Photograph a page of handwritten notes or a printed chart, ask GPT-4o to explain it back to you in simpler words, and check each point against the original.

What does real-time voice change for learners?

The biggest change is the rhythm of a lesson. With pauses of several seconds, talking to ChatGPT felt like leaving voicemails. At around a third of a second, it can feel like a conversation, where you interrupt, ask "wait, why?" and get an answer before you lose your train of thought. That back-and-forth is how much good teaching works.

OpenAI's demonstrations hint at what that could look like. One is a math session with Sal and Imran Khan. Others show pointing at objects to learn Spanish words and translating a conversation in real time. For language learners in particular, a partner that hears your pronunciation and tone, and answers at conversational speed, is a meaningful step beyond typing.

A few cautions keep expectations realistic. The new voice features are demonstrations today, not something most people can use yet; OpenAI plans an alpha for Plus users in the coming weeks. A fast, natural voice also sounds more confident, which can make errors harder to notice, so it helps to ask for the reasoning, or to check key facts in writing afterward. And speaking aloud to an AI raises fresh privacy questions at home, at school or at work.

Until the new voice arrives, you can practice the habits that will matter. Ask the model to tutor rather than tell, for example: "Don't give me the answer. Ask me one question at a time until I solve it." That instruction works by text today and will carry over to voice.