Should a small business fine-tune a model or build a RAG system?

Both approaches make a general AI model more useful for your business, but they change different things. Fine-tuning means training an existing model further on your own examples, so it picks up a style, a format or a way of handling certain requests. RAG, short for retrieval-augmented generation, leaves the model unchanged: before each answer, your system searches your own documents and passes the most relevant passages to the model along with the question.

A simple way to remember the difference: fine-tuning changes how the model answers, and RAG changes what it knows at the moment it answers. The researchers who named RAG in 2020 pointed out the core weakness it fixes: models store facts inside their parameters, but they cannot easily update those facts or show where an answer came from.

What each one is good at

OpenAI describes fine-tuning as a way to shape the structure and tone of responses, or to teach a model to follow complex, domain-specific instructions. It does not recommend fine-tuning as the way to load a product catalog into a model. Its help pages describe RAG for exactly that purpose, with the example of a support assistant that answers better when it can look up past tickets about similar problems.

A worked decision: a small outdoor-gear shop

The shop has two staff members answering email. Most questions are about order status, return rules, sizing, and repairs. It has a 40-page help center, a returns policy that changes each season, and about two years of answered emails. The goal is an assistant that drafts replies for staff to approve.

Step 1. Sort a sample of real questions. Take 50 recent emails and label each one. Illustrative result:

Step 2. Ask what fails today. Paste a few questions into a general assistant with your policy text included, and look at the drafts. If the facts are wrong or missing, you have a knowledge problem: that points to RAG. If the facts are right but the tone is off, you have a style problem: try better instructions first.

Step 3. Build the smallest RAG version. OpenAI's suggested pipeline is short: split each help article into chunks, turn each chunk into an embedding (a list of numbers that captures its meaning), store them, and at question time send the closest chunks to the model with the question. Many no-code tools, including custom GPTs with uploaded files, do this for you.

Step 4. Write the instructions.

You draft email replies for an outdoor-gear shop. Use only the policy
and product passages provided. If they don't answer the question, say
that a staff member will follow up, and don't guess. Keep replies under
120 words, friendly and plain. End with the name of the page you used.

Step 5. Decide on fine-tuning later, with evidence. After a month, if staff still rewrite most drafts for tone, the two years of answered emails could become training examples. OpenAI says useful fine-tuning results can come from a few dozen well-chosen examples, but they must be clean and consistent.

For this shop, the answer is RAG now, a tool connection for order status, and fine-tuning only if a measured tone problem remains.

When fine-tuning is the right call

Fine-tuning earns its place when the task is narrow and repetitive, the right output is clear, and you have many examples: tagging tickets, filling a fixed form from messy text, or matching a house style that resists instructions. Keep in mind that your training data needs the same privacy care as any customer data.

What does each cost to maintain?

Prices change often, so check the current pricing page of whichever provider you use. What differs most between the two approaches is the work involved after launch.

RAG costs you document upkeep and small usage fees. Each time a policy or product changes, you update the source text and the index refreshes, often automatically. Each question costs a little more than a plain prompt, because the retrieved passages are sent along with it. The ongoing task is reviewing wrong answers and fixing the document behind them.

Fine-tuning costs you a training run each time the examples change, plus the work of preparing and cleaning those examples. Using the fine-tuned model is then billed per use at its own rate, listed separately on the provider's pricing page. And when a provider releases a better base model, you may have to train again to benefit from it.

For a small team, the hidden cost is attention. A RAG system needs someone to own the documents; a fine-tuned model needs someone to own the training set and to retest after every change. Pick the one you can realistically keep up to date.