What does fine-tuning GPT-3.5 Turbo mean, and why would you do it?

Yesterday afternoon OpenAI opened a feature developers have requested ever since the model came out: fine-tuning for GPT-3.5 Turbo. GPT-3.5 Turbo is the lighter, cheaper sibling of GPT-4, and many apps reach it through OpenAI's API, the paid connection programmers use to send text to OpenAI's models and get answers back.

Until now, the only way to shape its behavior was the prompt: the instructions you send with every request. Fine-tuning lets a company bake some of those instructions into a private copy of the model. OpenAI also says fine-tuning for GPT-4, its most capable model, is planned for this fall.

What fine-tuning is

A model like GPT-3.5 Turbo learned its general skills from a vast amount of text. Fine-tuning is a short second round of training on a much smaller, specific set of examples. You show the model many conversations that look exactly the way you want, and it adjusts its parameters, the internal numbers it learned, to lean toward that pattern.

An analogy: a new employee arrives already knowing how to write. Training them on fifty of your company's best customer emails does not teach them English; it teaches them your house style. Fine-tuning does the same for a model.

OpenAI says customers in a private test used it for three kinds of improvement:

• Steerability: following instructions more reliably, such as keeping answers short or always replying in German when asked.
• Reliable formatting: producing the same structure every time, for example turning a user's request into JSON, a strict data format other software can read.
• Custom tone: sounding consistently like a particular brand.

There is a cost benefit too. If the model already knows your instructions, you no longer need to send them with each request. OpenAI says some early testers cut their prompts by as much as 90%, which makes each call faster and cheaper. Fine-tuned GPT-3.5 Turbo can work with 4,000 tokens of text at a time, twice the limit of OpenAI's earlier fine-tunable models. (A token is a piece of a word; 100,000 tokens is roughly 75,000 English words.)

How it works, step by step

OpenAI's announcement describes four steps, all done through the API:

1. Prepare your data. Write example conversations in the same format ChatGPT uses: a system message that sets the role, a user message and the ideal assistant reply.
2. Upload the file to OpenAI.
3. Create a fine-tuning job that names the training file and the base model, gpt-3.5-turbo-0613.
4. Use your new model. When training finishes, you get a model name starting with ft: and call it like any other model.

Here is one training example for a bakery that wants friendly, brief answers that always end with its opening hours:

System: You are the assistant for Rosa's Bakery. Answer in two sentences or fewer, warmly, and end with our hours.
User: Do you have gluten-free bread?
Assistant: Yes, we bake a gluten-free loaf every Tuesday and Friday. We're open 7 a.m. to 6 p.m., Tuesday to Sunday.

A real training file would hold dozens or hundreds of examples like this one, covering the questions customers actually ask. Once the fine-tuned model has learned the pattern, the app can send just the customer's question.

All training data passes through OpenAI's moderation tools, including a system powered by GPT-4, to screen out examples that break its safety rules. The aim is to stop people from fine-tuning away the model's safeguards. OpenAI also states that data sent through the fine-tuning service belongs to the customer and is not used to train other models.

What it costs

Fine-tuning has two prices: a one-time training charge and a higher per-use rate for the custom model.

OpenAI's own example: a training file of 100,000 tokens, run through 3 times (three epochs), costs about $2.40 to train. Because you pay for training and then for every use, the savings have to come from shorter prompts, or from handling a task well enough that you no longer need GPT-4 for it.

The same announcement brings two replacement models, babbage-002 and davinci-002, for the original GPT-3 base models that will be switched off on January 4, 2024. They can be fine-tuned through a new endpoint (an API address), which replaces the old one.

Why it matters

For businesses, this changes the trade-off between quality and price. A small, cheap model trained on your examples may do one job as well as a large general model. OpenAI says early tests show exactly that on certain narrow tasks, though the company gives no detailed numbers, and results will depend on the quality of the examples.

It also lowers the barrier. Fine-tuning was possible before, but only on older GPT-3 models. Now it applies to the conversational model most developers already use, and OpenAI promises a visual dashboard later so people can follow their training jobs without code.

Should you fine-tune a model or just write better prompts?

Start with prompts. OpenAI itself says fine-tuning works best combined with careful prompting, retrieval (fetching relevant documents and adding them to the prompt) and function calling. A clear role, an example or two and a description of the format often solve the problem with no training at all, and you can change a prompt in seconds.

Fine-tuning earns its place when you have a stable, repeated task, enough good examples, and either a format the model keeps getting wrong or long instructions you send thousands of times a day. It teaches style and structure well. It is not a reliable way to teach new facts; for that, retrieval is usually the better tool.

A sensible path is to write the best prompt you can, collect the cases where it fails, and turn those into training examples. That way, if you do fine-tune, you already know what success looks like and can measure whether the custom model is truly better.