# How does fine-tuning GPT-3 work?

Date: 2021-12-23 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/fine-tuning-gpt-3-work/

**Answer:** Fine-tuning trains a private copy of a GPT-3 model on your own examples, each one a prompt paired with the completion you want, so the model learns your task and style instead of reading them from a long prompt every time. Since December 14, 2021 every OpenAI API customer can do it with a JSONL file and a single command.

Until now, most people taught GPT-3 a task the way the original research paper described: by writing a few examples into the prompt and letting the model continue the pattern. On December 14, OpenAI opened another route to all API customers. You can now fine-tune GPT-3, meaning you train a custom version of one of its models on a file of your own examples, and then call that version through the API like any other.

The idea is old in machine learning. A model first learns general patterns from a huge amount of text, then gets extra training on a narrow task. What is new is how easy OpenAI has made it: a data file, one command in its command-line tool, and a wait that can be measured in minutes for small jobs.

## What fine-tuning changes

With a prompt, the model's weights (the billions of numbers learned in training) stay fixed. Your examples sit in the input, use up the 2,048-token window and must be sent and paid for with every request. With fine-tuning, the examples are used to adjust a copy of the weights. The knowledge moves from the prompt into the model.

OpenAI lists three practical benefits. You can supply many more examples than fit in a prompt. Results become more consistent: one customer cut unreliable outputs from 17% to 5%. And because the custom model already knows the task, prompts can be much shorter, which lowers cost and speeds up responses. OpenAI also reports that gains appear with fewer than 100 examples and that each doubling of the dataset tends to bring a steady further improvement.

## The data: prompt-completion pairs in JSONL

Training data goes in a JSONL file, short for JSON Lines: a plain text file where each line is one example written as a small JSON object with two fields, `prompt` and `completion`. OpenAI's example notebooks add a few conventions that matter:

- End every prompt with the same separator, such as a line containing `###`, so the model learns where the input stops.
- Start every completion with a space, because most words are stored as tokens that begin with one.
- End every completion with the same marker, so you can tell the API to stop there.

OpenAI's command-line tool includes a checker, `openai tools fine_tunes.prepare_data`, that reads your file, points out problems and can split it into training and validation sets, so you can measure results on examples the model never saw.

## A worked example: support replies

Imagine a small software company that wants short, on-brand replies to common support emails. It exports past tickets that staff answered well, removes names and account numbers, and writes them as pairs:

```prompt
{"prompt": "Customer: I was charged twice for my November subscription.\n\n###\n\n", "completion": " Sorry about that! I can see the duplicate charge and have requested a refund. It should appear on your card within a few business days. END"}
{"prompt": "Customer: How do I export my data to a spreadsheet?\n\n###\n\n", "completion": " Go to Settings, then Data, and choose Export as CSV. The file will download right away. END"}
{"prompt": "Customer: The app logs me out every few minutes.\n\n###\n\n", "completion": " Thanks for letting us know. Please update to the latest version and sign in again; if it keeps happening, reply here and we'll look into your account. END"}
```

A real file would hold a few hundred of these. Training is then one command, `openai api fine_tunes.create -t replies.jsonl -m ada`, where `-m` picks the base model; OpenAI's own demo uses Ada, the smallest and cheapest, and a larger model can be chosen instead. When the job finishes, the tool prints the custom model's name. To use it, send a new customer message ending with the same separator, set the stop marker to END, and the model writes a reply in the learned style. A human should still review replies before they go out.

The expensive part is rarely the training itself; OpenAI's demo job on its smallest model cost one cent. The real cost is the data: finding good past answers, removing personal information, fixing inconsistent examples and keeping a test set aside. That is skilled work, and it decides the quality of the result.

> **Try it:** Collect ten questions you answer often, in email or at work, and write each with your best answer as a prompt-completion pair using the separator and END marker above. Even if you never train a model, you will see how consistent, or inconsistent, your own examples are.

## When is fine-tuning better than prompting?

Start with a prompt. If a clear instruction and a few examples give good results, you are done, and you can change your approach in seconds. Prompting also suits tasks that shift often, because updating a fine-tuned model means preparing new data and training again.

Fine-tuning earns its keep when you have a stable, repeated task and plenty of good examples. Classification is a classic case: OpenAI's notebook trains the small Ada model to sort sports emails into baseball or hockey and reaches about 99% accuracy on held-out examples. It also helps when you need a consistent format or voice across thousands of requests, when your best prompt has grown so long that it is costly, or when your examples simply do not fit in 2,048 tokens. OpenAI's announcement reports a customer that adds about 500 new examples a week and has raised accuracy from 85% to 93%.

A useful rule of thumb: if you find yourself pasting the same long block of examples into every request, and you have more good examples waiting, that is the moment to try fine-tuning. Keep a validation set so you can show that the custom model really beats your best prompt.

## Sources

- Customizing GPT-3 for Your Application: https://web.archive.org/web/20220131122443/https://openai.com/blog/customized-gpt-3/
- Fine tuning classification example (openai-python examples): https://github.com/openai/openai-python/blob/v0.11.5/examples/finetuning/finetuning-classification.ipynb
- Language Models are Few-Shot Learners: https://arxiv.org/abs/2005.14165