Should you pretrain a model, fine-tune one or use RAG, once you count the real costs?

Our guide to prompts, documents and fine-tuning helps you diagnose a disappointing assistant. This article answers the question that comes next for builders: if you could also train a model of your own from nothing, when would that beat the others?

Three quick definitions. RAG (retrieval-augmented generation) searches your documents and hands the relevant passages to an existing model at question time. Fine-tuning continues training an existing model on your examples. Pretraining builds a new model from random numbers using billions of tokens of text.

The three, side by side

Dollar figures are US dollars, based on list prices read on November 2, 2026 (details).

The middle columns hold the surprise. Spending 40 times more than a fine-tune gets you a far weaker model, because a fine-tune starts from millions of dollars of someone else's training. You are not paying for capability when you pretrain at small scale. You are paying for understanding and control.

A worked example

A Quebec municipality wants a French assistant for citizens' questions about bylaws, permits and collection days.

Facts change every year, and citizens need the current rule with a reference to it. That is a job for RAG, over the municipality's own documents, on a model with good French. A prompt requiring the answer to quote the bylaw handles most of the behavior.

Say the staff also want each answer in a fixed format, and tests show the prompt alone drifts after thousands of questions. That justifies a fine-tune of a small open model, on a few hundred reviewed examples. RAG continues to supply the facts, since the two fixes stack.

Now someone proposes "our own model", for sovereignty. Pretraining at GPT-2 grade for $64 would produce something that invents bylaws with confidence, and a capable one would likely dwarf any municipal project budget. What the request is really after is usually control: where the data goes and who can change the terms. Running an open-weight model on hardware or a cloud that the municipality chooses answers that for far less, and it needs no training at all.

Budget the middle options too

Between these columns sit paths the table hides:

• Distillation: have a strong model label your data, then fine-tune a small one on it. See our explainer, including the terms-of-service limits.
• Hosted fine-tuning: several API providers offer it, with no GPU to manage.
• A model with a smaller and cleaner base: fully documented open models let you check what you are building on, which removes part of the reason for pretraining.

When is pretraining from scratch actually justified?

To learn. A $64 run, or a free laptop one, teaches what no article can. This is the most common good reason, and the rented-GPU walkthrough is written for it. Do not expect a product at the end.

For research. Studying how data, architecture or scale change results needs models you control completely.

When the data's provenance is the point. Fine-tuning inherits a base model's unknowns. If you must state exactly what a model read, pretraining on something like the Common Pile, as the Comma v0.1 models did, can answer that in a way a base model usually cannot.

For languages that models serve badly. If your language has little presence in today's models and enough text exists, and fine-tuning on an open multilingual base has been tried and measured, then a purpose-built dataset may be the way forward. The training bill is rarely the first obstacle; gathering the text with the community's consent often is.

As public infrastructure. The Swiss Apertus project is the clearest example: EPFL, ETH Zurich and CSCS published weights, data and recipes, trained on 15 trillion tokens in more than 1,000 languages, using more than 10 million GPU-hours on a national supercomputer. At a commercial $3.99 per GPU-hour, that compute would be about $40 million. It is a decision for institutions with a public mission rather than for a single project's budget. Canada is having the same debate about its own sovereign capacity.

If none of these describes you, the cheaper options will serve you better. OpenAI's accuracy guide gives advice that applies well beyond its own products: measure a baseline first, then try the smallest change that could fix it.