How do reasoning models decide how long to think?

A reasoning model writes out a private chain of thought before it gives its answer: it breaks the problem into steps, tries an approach, checks it and sometimes starts over. That thinking is made of tokens, the same small pieces of text the visible answer is made of. The model produces one token at a time, and at each point it effectively chooses whether to keep reasoning or to start the answer. That is why there is no fixed thinking time. A simple arithmetic question may take a handful of reasoning tokens; a tricky logic puzzle may take thousands.

So the length of the thinking is decided by the model, in response to the problem. What you, the user or developer, control is the pressure on that decision and a hard ceiling on it.

The controls you have

OpenAI and Anthropic expose different knobs for this.

Two details matter. First, these are limits and hints, not targets. Anthropic's documentation notes that Claude may not use its whole budget, especially with budgets above 32,000 tokens. Second, the thinking costs money even though you don't see it. OpenAI counts reasoning tokens in the completion total for billing and for the context-window limit, and Anthropic bills thinking tokens as output tokens.

A worked test with o3-mini

The best way to understand effort is to measure it on your own questions. This short script uses the official openai Python library to send the same question at all three levels and print how many reasoning tokens each used. It needs an API key in the OPENAI_API_KEY environment variable.

from openai import OpenAI

client = OpenAI()

QUESTION = (
    "A bakery sells muffins in boxes of 4 and 6. "
    "List every way to buy exactly 38 muffins, and explain how you know the list is complete."
)

for effort in ["low", "medium", "high"]:
    response = client.chat.completions.create(
        model="o3-mini",
        reasoning_effort=effort,
        max_completion_tokens=20000,
        messages=[{"role": "user", "content": QUESTION}],
    )
    usage = response.usage
    reasoning = usage.completion_tokens_details.reasoning_tokens
    print(f"{effort:>6}: {reasoning} reasoning tokens, "
          f"{usage.completion_tokens} completion tokens in total")
    print(response.choices[0].message.content[:300], "\n")

The usage object in each response reports the completion tokens, with reasoning_tokens as the part spent thinking. To turn that into money, multiply by the output price: o3-mini costs $4.40 per million output tokens on OpenAI's price list this month.

Here is the kind of result to expect. These numbers are illustrative, not measured; run the script to get yours.

The pattern the table illustrates is common: low is cheapest and fastest, medium adds care, and high costs the most with little gain on a question this easy. On a genuinely hard problem, the gap in quality between low and high can be much larger.

Setting a sensible ceiling

Always set a cap, but leave headroom. Because max_completion_tokens covers the thinking and the visible answer together, a tight cap on a hard problem can be used up by reasoning and cut the answer short. With Claude, Anthropic suggests starting at the minimum budget and raising it in steps until quality stops improving, and recommends batch processing for budgets above 32,000 tokens to avoid timeouts.

What does 'reasoning effort' change?

It changes how much thinking the model is encouraged to do before answering, and therefore three things you can feel: speed, cost and, on hard problems, accuracy. OpenAI's specification says it plainly: lowering effort can produce faster responses and spend fewer tokens on reasoning.

It does not change the model. o3-mini at high effort is still o3-mini, with the same knowledge and the same limits. Effort can't supply facts the model doesn't have; for that you need to provide context in the prompt or use a tool such as search.

Use low effort for routine work: classifying messages, simple calculations, short code fixes, anything where you will check the result anyway. Keep medium, the default, for most questions that need some care. Save high effort for problems where a wrong answer is expensive, such as complex math, multi-step logic or tricky code, and confirm with your own tests that it actually helps. Because the setting is a single parameter, you can choose it per request, sending most traffic at low or medium and reserving high for the questions that earn it.