How do you get reliable JSON from a model with Structured Outputs?

On August 6, OpenAI added Structured Outputs to its API, and the practical question has moved from "what is it?" to "how do I use it without getting burned?" The short version: you write down the fields you need, their types and which ones are required, and the API holds the model to that blueprint. What you get back is JSON (JavaScript Object Notation, the plain-text format most programs use to exchange data) that your code can load without repair steps.

The blueprint is called a JSON Schema, a standard way of describing what a valid piece of JSON looks like. The rest of this guide walks through one real task, pulling the key details out of a supplier invoice, from schema to safety checks.

Step 1: Decide the fields before you write any code

Start on paper. For an invoice, a small bookkeeping tool might need:

Keep it small. Every extra field is one more thing the model has to fill and one more thing you have to check. You can add fields once the first version works.

Step 2: Turn the schema on in strict mode

There are two ways to switch the feature on. If your app already uses function calling, add strict: true to each function definition. If you simply want the answer itself in a fixed shape, pass your schema through response_format with the type json_schema. That second form works with the newest snapshots, gpt-4o-2024-08-06 and gpt-4o-mini, and is what we use here.

OpenAI's updated Python library can build the schema for you from a Pydantic class (Pydantic is a popular Python package for describing data with type hints). This sketch uses the library's parse helper, which sits under client.beta:

import openai
from openai import OpenAI
from pydantic import BaseModel

class LineItem(BaseModel):
    description: str
    quantity: float
    unit_price: float

class Invoice(BaseModel):
    vendor_name: str
    invoice_number: str
    invoice_date: str
    currency: str
    line_items: list[LineItem]
    total: float

client = OpenAI()   # reads OPENAI_API_KEY from your environment

invoice_text = open("invoice.txt").read()

try:
    completion = client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {"role": "system", "content": (
                "Extract the invoice details. Copy names and numbers exactly "
                "as written. Write dates as YYYY-MM-DD. If a value is missing, "
                "use an empty string for text and 0 for numbers. Never guess.")},
            {"role": "user", "content": invoice_text},
        ],
        response_format=Invoice,
        max_tokens=1000,
    )
except openai.LengthFinishReasonError:
    raise SystemExit("Reply was cut off: raise max_tokens or shorten the input.")

message = completion.choices[0].message
if message.refusal:
    raise SystemExit("Model refused: " + message.refusal)

invoice = message.parsed   # an Invoice object, not raw text
print(invoice.vendor_name, invoice.total)

If you prefer to send raw JSON instead of using Pydantic, the published examples follow two habits worth copying: every property is listed under required, and each object sets "additionalProperties": false, so the model cannot invent extra keys. Note that only part of the JSON Schema standard is supported, so very fancy rules may be rejected.

Expect the first call with a new schema to be slow. OpenAI prepares and caches each schema on first use, which usually takes under ten seconds and can take up to a minute for complex ones. Later calls run at normal speed.

Step 3: Treat the two exceptions as normal events

OpenAI describes the format guarantee with two conditions, and the code above checks both. First, the model may decline a request it considers unsafe; the reply then carries a refusal message instead of your data. Second, if the reply hits the max_tokens limit or another stop condition, it ends mid-object and will not match the schema. The API then reports a finish_reason of length rather than stop, and the parse helper raises an error instead of handing you half an invoice. Neither is rare enough to ignore in a program that runs every day.

Two more practical limits: Structured Outputs does not work with parallel function calls, so set parallel_tool_calls to false if you use tools, and the feature is available in the Chat Completions, Assistants and Batch APIs, including requests with images. A photo of an invoice can go through the same schema.

What still needs validation in your code?

The values. Strict mode guarantees that total is a number; it says nothing about whether it is the right number. OpenAI states plainly that the model can still make mistakes inside the JSON, so the checks that matter now are about meaning, not syntax. For the invoice, a short set of rules catches most problems:

• Arithmetic. Multiply quantity by unit price for each line, add them up and compare with total. Allow for taxes and rounding, but flag a large gap.
• Formats. Try to read invoice_date as a real calendar date, and check currency against the short list of codes you actually deal with.
• Empty values. An empty invoice_number is legal under the schema but useless to you. Decide which empty fields send the invoice to a person.
• Duplicates and context. Look up the vendor and invoice number in your own records before anything is paid.

When a check fails, don't silently retry until the numbers look right. Send the document to a human review queue with the extracted values beside the original, and keep a log of which fields fail most often. If one field keeps going wrong, OpenAI's own suggestion is to add worked examples to the instructions or split the task into smaller steps, for instance extracting line items in a separate call. The schema gives you clean data to check; the checking is still your job.