# How do you get reliable JSON from a model with Structured Outputs?

Date: 2024-08-19 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/get-reliable-json-model-structured-outputs/

**Answer:** Describe the exact shape you want as a schema, send it with strict mode turned on, and the reply will match that shape every time unless the model refuses or runs out of tokens. Your code still checks those two cases, then checks the values themselves, because a perfect structure can hold a wrong number.

On August 6, OpenAI added Structured Outputs to its API, and the practical question has moved from "what is it?" to "how do I use it without getting burned?" The short version: you write down the fields you need, their types and which ones are required, and the API holds the model to that blueprint. What you get back is JSON (JavaScript Object Notation, the plain-text format most programs use to exchange data) that your code can load without repair steps.

The blueprint is called a **JSON Schema**, a standard way of describing what a valid piece of JSON looks like. The rest of this guide walks through one real task, pulling the key details out of a supplier invoice, from schema to safety checks.

## Step 1: Decide the fields before you write any code

Start on paper. For an invoice, a small bookkeeping tool might need:

| Field | Type | Why it matters |
|---|---|---|
| vendor_name | text | Who to pay |
| invoice_number | text | Avoids paying twice |
| invoice_date | text, written as YYYY-MM-DD | Sorting and due dates |
| currency | text, a three-letter code such as CAD | Amounts mean nothing without it |
| line_items | list of description, quantity, unit_price | Checking the total |
| total | number | What the vendor claims you owe |

Keep it small. Every extra field is one more thing the model has to fill and one more thing you have to check. You can add fields once the first version works.

## Step 2: Turn the schema on in strict mode

There are two ways to switch the feature on. If your app already uses function calling, add `strict: true` to each function definition. If you simply want the answer itself in a fixed shape, pass your schema through `response_format` with the type `json_schema`. That second form works with the newest snapshots, gpt-4o-2024-08-06 and gpt-4o-mini, and is what we use here.

OpenAI's updated Python library can build the schema for you from a Pydantic class (Pydantic is a popular Python package for describing data with type hints). This sketch uses the library's `parse` helper, which sits under `client.beta`:

```python
import openai
from openai import OpenAI
from pydantic import BaseModel

class LineItem(BaseModel):
    description: str
    quantity: float
    unit_price: float

class Invoice(BaseModel):
    vendor_name: str
    invoice_number: str
    invoice_date: str
    currency: str
    line_items: list[LineItem]
    total: float

client = OpenAI()   # reads OPENAI_API_KEY from your environment

invoice_text = open("invoice.txt").read()

try:
    completion = client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {"role": "system", "content": (
                "Extract the invoice details. Copy names and numbers exactly "
                "as written. Write dates as YYYY-MM-DD. If a value is missing, "
                "use an empty string for text and 0 for numbers. Never guess.")},
            {"role": "user", "content": invoice_text},
        ],
        response_format=Invoice,
        max_tokens=1000,
    )
except openai.LengthFinishReasonError:
    raise SystemExit("Reply was cut off: raise max_tokens or shorten the input.")

message = completion.choices[0].message
if message.refusal:
    raise SystemExit("Model refused: " + message.refusal)

invoice = message.parsed   # an Invoice object, not raw text
print(invoice.vendor_name, invoice.total)
```

If you prefer to send raw JSON instead of using Pydantic, the published examples follow two habits worth copying: every property is listed under `required`, and each object sets `"additionalProperties": false`, so the model cannot invent extra keys. Note that only part of the JSON Schema standard is supported, so very fancy rules may be rejected.

Expect the first call with a new schema to be slow. OpenAI prepares and caches each schema on first use, which usually takes under ten seconds and can take up to a minute for complex ones. Later calls run at normal speed.

> **Try it:** Paste the text of one real receipt or invoice into a file, run the script above, then edit the receipt to remove the date and run it again. Check whether you get an empty string, as instructed, or a made-up date.

## Step 3: Treat the two exceptions as normal events

OpenAI describes the format guarantee with two conditions, and the code above checks both. First, the model may decline a request it considers unsafe; the reply then carries a `refusal` message instead of your data. Second, if the reply hits the `max_tokens` limit or another stop condition, it ends mid-object and will not match the schema. The API then reports a `finish_reason` of `length` rather than `stop`, and the `parse` helper raises an error instead of handing you half an invoice. Neither is rare enough to ignore in a program that runs every day.

Two more practical limits: Structured Outputs does not work with parallel function calls, so set `parallel_tool_calls` to false if you use tools, and the feature is available in the Chat Completions, Assistants and Batch APIs, including requests with images. A photo of an invoice can go through the same schema.

## What still needs validation in your code?

The values. Strict mode guarantees that `total` is a number; it says nothing about whether it is the right number. OpenAI states plainly that the model can still make mistakes inside the JSON, so the checks that matter now are about meaning, not syntax. For the invoice, a short set of rules catches most problems:

- **Arithmetic.** Multiply quantity by unit price for each line, add them up and compare with `total`. Allow for taxes and rounding, but flag a large gap.
- **Formats.** Try to read `invoice_date` as a real calendar date, and check `currency` against the short list of codes you actually deal with.
- **Empty values.** An empty `invoice_number` is legal under the schema but useless to you. Decide which empty fields send the invoice to a person.
- **Duplicates and context.** Look up the vendor and invoice number in your own records before anything is paid.

When a check fails, don't silently retry until the numbers look right. Send the document to a human review queue with the extracted values beside the original, and keep a log of which fields fail most often. If one field keeps going wrong, OpenAI's own suggestion is to add worked examples to the instructions or split the task into smaller steps, for instance extracting line items in a separate call. The schema gives you clean data to check; the checking is still your job.

## Sources

- Introducing Structured Outputs in the API: https://openai.com/index/introducing-structured-outputs-in-the-api/
- OpenAI has finally released the No. 1 feature developers have been desperate for: https://venturebeat.com/ai/openai-has-finally-released-the-no-1-feature-developers-have-been-desperate-for/