# What are Structured Outputs in the OpenAI API, and why do developers care?

Date: 2024-08-07 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/what-are-structured-outputs-openai-api/

**Answer:** Structured Outputs is a feature OpenAI added to its API on August 6, 2024 that makes a model's reply match a JSON Schema the developer supplies, every time, instead of usually. It works by limiting which tokens the model is allowed to write next, so programs can read AI answers without checking and retrying them.

Yesterday, August 6, 2024, OpenAI released **Structured Outputs** in its API. The promise is simple: when a developer describes exactly what shape an answer must have, the model's reply will have that shape. Not most of the time, but reliably. OpenAI says it is the feature developers have asked for most, and Sam Altman posted that it arrived "by very popular demand."

If you only chat with ChatGPT, this sounds technical. It still matters to you, because many apps you use pass AI answers to other software behind the scenes, and that only works when the answer arrives in a predictable format.

## The problem it solves

Software does not read prose. A booking app that asks a model to pull details from an email needs the answer in a fixed format, such as:

```json
{"name": "Ana Silva", "date": "2024-08-15", "guests": 4}
```

That format is **JSON**, a common way to write data as labeled fields that both people and programs can read. A **JSON Schema** is the rulebook for one particular piece of JSON: which fields must appear, what type each one is (text, number, true or false), and which values are allowed.

Language models are good at producing JSON, but not perfect. They sometimes leave out a required field, add an extra one, or invent a value that is not on the allowed list. Developers have worked around this with extra tools, careful prompts and repeated requests until the output passed their checks.

Last year at DevDay, OpenAI added **JSON mode**, which guarantees valid JSON but not that it follows your schema. Structured Outputs closes that gap. OpenAI's own summary: "While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence."

## What OpenAI released

The feature comes in two forms:

| Form | How you turn it on | Which models |
|---|---|---|
| Function calling | Add `strict: true` to a function definition | Every model that supports tools, from gpt-3.5-turbo-0613 and gpt-4-0613 onward, including GPT-4o and GPT-4o mini |
| Response format | Pass your schema as `json_schema` in `response_format`, with `strict: true` | The new gpt-4o-2024-08-06 and gpt-4o-mini |

**Function calling** is when the model asks your app to run a task, such as "look up order 1234," by filling in the inputs your function expects. The **response format** option is for when the model answers the user directly but in a set structure.

OpenAI also released that new GPT-4o snapshot, gpt-4o-2024-08-06. It costs $2.50 per million input tokens and $10.00 per million output tokens, half the input price and a third less on output compared with the May version. OpenAI's Python and Node.js libraries can now take a schema written as a Pydantic or Zod object, two popular ways of describing data in code, and convert it for you.

## How it works

OpenAI took two steps. First, it trained the new model to understand complicated schemas better; on its test of complex schemas, the model alone scored 93%. That is good, but a program that fails seven times in a hundred is not good enough.

So the second step is engineering. A model writes its answer one **token** (a word or piece of a word) at a time, and at each step it could in principle choose any token in its vocabulary, including one that breaks the format. With Structured Outputs, the system first turns your schema into a **context-free grammar**, a precise set of rules about what may come next. Then, after every token, it works out which next tokens would still fit the rules and blocks all the others. This approach is called **constrained decoding**.

Think of filling in a paper form with boxes. A free-form letter can say anything; a form only accepts a date in the date box. Constrained decoding gives the model the form. With it, OpenAI reports a perfect 100% on its schema test, compared with under 40% for the older gpt-4-0613.

There are limits worth knowing:

- The first request with a new schema takes longer, usually under 10 seconds and up to a minute for complex ones, while the grammar is built. After that it is cached.
- Only part of the JSON Schema standard is supported.
- The model can still refuse an unsafe request. When it does, the reply includes a `refusal` field so your app can tell.
- The structure is guaranteed, not the facts. A field can be filled with a wrong value.
- It does not work with parallel function calls.

## Why it matters this week

For builders, this removes a whole category of fragile code: the checks, retries and repair steps around every AI call. It makes it more practical to use models for data entry, extracting details from documents, filling in forms, or generating interface elements from a request. OpenAI credits open-source projects such as Outlines, Jsonformer, Instructor, Guidance and Lark for paving the way.

> **Try it:** Write down, on paper, the three to five fields you would want from a typical email in your inbox (for example sender, request, deadline). That list is the start of a JSON Schema, and the first step to automating that task.

## How do you start using Structured Outputs in your own project?

Begin with a small schema. Decide the fields, their types and which are required, then pass it as `json_schema` in `response_format` with `strict: true`, using gpt-4o-2024-08-06 or gpt-4o-mini. If you already use function calling, adding `strict: true` to your existing definitions is the quickest win.

Keep your instructions clear even though the format is enforced. A prompt like this pairs well with a schema that has `sender`, `request` and `deadline` fields:

```prompt
Extract the sender, the main request and the deadline from the email below. If no deadline is given, use null. Do not guess.

[paste email here]
```

Then test with real, messy examples. Check the values, not just the shape, since the schema guarantees structure but not accuracy. Handle the `refusal` case in your code, and expect a short delay the first time each new schema is used.

## Sources

- Introducing Structured Outputs in the API: https://openai.com/index/introducing-structured-outputs-in-the-api/
- OpenAI has finally released the No. 1 feature developers have been desperate for: https://venturebeat.com/ai/openai-has-finally-released-the-no-1-feature-developers-have-been-desperate-for/