Short answer
Choose a hosted model such as ChatGPT or Claude when you want the strongest answers with no setup, and an open-weight model such as Llama 2 or Mistral 7B when your data must stay on your own machines, you need full control, or you run the same task at high volume. Most people decide by weighing four things in order: privacy, quality, effort and cost.
Follow-up question What do you give up in each case?
This year has given ordinary teams a real choice. On one side are hosted assistants: ChatGPT, Claude and similar services that run on the provider's servers and that you reach through a website, an app or an API. On the other side are open-weight models such as Meta's Llama 2 and Mistral AI's Mistral 7B, which you can download and run yourself.
The weights of a model are the billions of numbers it learned during training. They are the model. "Open-weight" means those numbers are published so anyone can copy them, though the licence still sets rules. "Hosted" means the company keeps the weights and rents you access by the message or by the month. Neither is better in general. Each fits a different job.
The four questions to ask
1. Where may your data go? With a hosted model, your text travels to someone else's servers. That can be acceptable: OpenAI says ChatGPT Enterprise, launched in August, does not train on business conversations and encrypts data both stored and in transit. But some data, such as patient notes or unreleased financial results, may not be allowed off your premises at all. An open-weight model running on your own computer or your own cloud account keeps everything inside walls you control.
2. How good do the answers need to be? The largest hosted models, such as GPT-4, generally write, reason and follow complex instructions better than the small models most people can run at home. Small open models are improving fast. Mistral AI reports that its 7.3-billion-parameter model beats the larger Llama 2 13B on every benchmark it tested. A parameter is one of those learned numbers, so a "7B" model has about seven billion. Yet a small model still has less general knowledge, and you should test it on your own task before trusting it.
3. How much effort can you spend? A hosted chatbot works the moment you sign in. An open model asks you to pick a version, find hardware with enough memory, install software, and keep it all running and updated. Tools like llama.cpp and Ollama have made this far easier, but someone on the team must own it.
4. What will it cost at your volume? A hosted service charges per user or per word processed, so the bill grows with use. An open model is free to download, but you pay for the machine and the time to run it. For occasional use, hosted is almost always cheaper. For a single repetitive task run thousands of times a day, owning the setup can win.
A worked decision table
Here are three made-up but typical cases, scored against those questions.
| Case | Data sensitivity | Quality needed | Team skills | Volume | Likely choice |
|---|---|---|---|---|---|
| A dental clinic wants short summaries of patient notes | High: health data | Moderate: summaries are checked by staff | Low | Low | Open-weight model on a local machine, set up by a contractor, or no AI until a compliant service is confirmed |
| A five-person marketing team drafts blog posts and emails | Low | High: polished writing | Low | Moderate | Hosted assistant such as ChatGPT or Claude |
| A software company tags 50,000 support tickets a day by topic (illustrative figure) | Medium | Moderate: a narrow, repetitive task | High | Very high | Open-weight model such as Mistral 7B on its own servers, tested against a hosted model first |
Read the table from left to right. A single "high" in the first column can settle the matter on its own. If not, quality usually comes next, then effort, and cost last.
Try it: Write down one task you would like AI to help with. Score it on the four questions as low, medium or high, using the table above as a model. If privacy scores high, look at open-weight options first; if quality scores high and privacy low, start with a hosted assistant.
Read the licence before you build
"Open" does not always mean "no strings". Mistral 7B is released under Apache 2.0, a permissive licence that lets you use, change and sell it with few conditions. Llama 2 comes with Meta's own licence. It allows commercial use, but anyone who redistributes it must pass the agreement on, all use must follow Meta's acceptable use policy, Llama 2's outputs may not be used to improve other large language models, and companies whose products had more than 700 million monthly users need a separate licence. For a small business those terms are rarely a problem, but a lawyer or an advisor should read them before you ship a product.
What do you give up in each case?
With a hosted model, you give up control. The provider can change the model, its price or its rules, and an answer that worked last month may come out differently today. Your data leaves your building, so you rely on the provider's promises and your own contract. You also cannot inspect or retrain the model beyond whatever options the provider offers.
With an open-weight model, you give up convenience and, often, some quality. You become the operator: you choose the hardware, apply updates, and handle safety yourself. That last point matters. Mistral AI notes that its first chat version, Mistral 7B Instruct, comes with no moderation mechanism, so filtering harmful requests is up to you. The licence also comes "as is": if the model makes a mistake, there is no vendor to call.
Many teams end up with both. They use a hosted assistant for everyday writing and research, and keep a small open model for the one sensitive or high-volume job where control matters more than polish. Starting with the four questions makes that split a deliberate decision rather than an accident.
Sources
- Mistral 7B — Mistral AI, September 27, 2023
- Llama 2 Community License Agreement — Meta, July 18, 2023
- Introducing ChatGPT Enterprise — OpenAI, August 28, 2023