Is fine-tuning GPT-4o worth it for your project?

On August 20, OpenAI opened fine-tuning for GPT-4o to every developer on a paid usage tier, and it is offering one million free training tokens per organization per day until September 23. That deadline makes the question urgent for many teams: should you train your own version now, while it's cheap to try?

This guide assumes you know what fine-tuning is (training a copy of a model on your own example conversations) and how to prepare the data. The question here is narrower. Will it pay off for your project, and how will you know?

What GPT-4o fine-tuning offers, in numbers

So a fine-tuned model costs 1.5 times as much per token as the base model. OpenAI says fine-tuning can set the structure and tone of replies and teach the model complex instructions specific to a field, and that a few dozen examples can already give strong results. Its two partner stories are both specialist tasks: a coding assistant trained on how real engineers work, and a system that turns questions into database queries.

A worked decision

Imagine a regional insurance broker that wants replies to customer emails in its house style, following twelve internal rules (never promise coverage, always give the claim number format, and so on). Run through these questions in order.

1. Have you pushed prompting as far as it goes? Write the best instructions you can, add three or four example replies, and test them. Many "the model can't do it" problems are really "the model was never told" problems.

2. Is the problem format? If the failures are malformed or missing fields, use Structured Outputs instead, which makes the reply match a schema you supply. That fixes structure without any training.

3. Is the task narrow and repeated? Thousands of similar emails a month, yes. A different kind of request every day, no. Fine-tuning teaches patterns, so it needs a pattern.

4. Do you have good examples? You need real, approved replies, not drafts. A few dozen can start the test; more will help if the first results are promising.

5. Does the per-call cost work? Here an illustrative calculation helps. Say the long prompt with rules and examples uses 2,000 input tokens per email and the reply is 200 tokens. A fine-tuned model already knows the rules, so the prompt might shrink to 800 tokens.

The savings only appear if the prompt really gets shorter. If it doesn't, the fine-tuned model costs more on every call, and the case must rest on quality alone. Training is cheap by comparison: 300,000 tokens of examples trained over three passes is 900,000 tokens, about $22.50, and under the free daily allowance until September 23.

If the answer to any of the first four questions is no, stop and fix that first.

How do you measure whether it helped?

With an eval: a fixed set of test cases you run against both versions and score the same way. OpenAI's open-source Evals framework describes the essentials. Each test case has an input and, where possible, an ideal answer or a clear grading rubric; the set should be consistent in theme, challenging, and spot-checked by a person.

Build yours before you train:

1. Hold out test cases. Set aside 50 to 100 real emails that are not in the training data. Testing on training examples only shows that the model memorized them.
2. Write the scoring rule. For the broker: does the reply follow each of the twelve rules (yes or no per rule), and would a staff member send it unchanged, with small edits or not at all?
3. Run the baseline. Score base GPT-4o with your best prompt. This is the number to beat.
4. Train, then run the same test. Same cases, same scoring, ideally graded without knowing which model wrote which reply.
5. Compare cost and quality together.

An illustrative result table might look like this:

A gain like that is worth keeping. A gain of two or three cases out of 100 is within the noise of a small test; run more cases before you switch.

Keep the eval after launch. Rerun it whenever you change the prompt, retrain with new examples, or OpenAI releases a new base model, since a fine-tuned model is tied to the snapshot it was trained on. OpenAI says your training data and outputs stay yours and aren't used to train other models, but you remain responsible for checking what your model says to customers.