How do you use Mistral Large 3 as an open-weight model?

Mistral Large 3, released by Mistral AI on December 2 as the largest member of its Mistral 3 family, is an open-weight model. The weights, the billions of numbers the model learned during training, are published for anyone to download. With a closed model such as those behind ChatGPT or Gemini, you can only send requests to the company's servers. With open weights, you choose where the model runs, who sees your data, and whether to adapt it.

That freedom comes with work. This guide explains what the license lets you do, what the model needs to run, and how to serve it yourself, step by step.

What the license allows

Mistral publishes every Mistral 3 model under Apache 2.0, one of the most permissive software licenses. In plain terms, you may use the model for commercial or personal purposes, change it, fine-tune it (train it further on your own data) and ship it inside a product. You must keep the license and copyright notices, and you get no warranty. The model card adds the usual reminder that you may not use the model to violate other people's rights, including intellectual property.

For a business, this matters because there is no per-request fee to Mistral when you run the model yourself and no risk that a provider changes its terms on a model you depend on. Your costs move to hardware and staff time instead.

What the model needs

Mistral Large 3 is a mixture-of-experts model: 675 billion parameters in total, but only about 41 billion are active for each token, because a router sends each piece of text to a few specialized sub-networks ("experts"). That makes it faster than its size suggests, but all the weights still have to sit in GPU memory.

The model card lists three versions of the instruction-tuned model:

A rough rule of thumb: at 8 bits, each parameter takes one byte, so the FP8 weights alone come to roughly 675 gigabytes. That is why this is a server model, not a laptop model. Mistral also lists known limits: dedicated reasoning models can beat it at strict reasoning, vision-first models can beat it on image tasks, and it is hard to deploy on limited hardware.

A worked example: serving it with vLLM

Mistral recommends vLLM, an open-source program that serves language models efficiently and exposes an API in the same format as OpenAI's. On a rented or in-house server with eight suitable GPUs, the model card gives these steps.

Install or upgrade vLLM (it brings Mistral's tokenizer library with it):

pip install vllm --upgrade

Start the server. --tensor-parallel-size 8 splits the model across the eight GPUs; the last two flags turn on tool calling (letting the model ask your code to run functions):

vllm serve mistralai/Mistral-Large-3-675B-Instruct-2512 \
  --tensor-parallel-size 8 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral \
  --enable-auto-tool-choice --tool-call-parser mistral

By default it accepts up to 262,144 tokens of context. Adding --max-model-len 32768 saves memory if your documents are shorter.

Then call it from Python with the standard openai library, pointed at your own server:

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

reply = client.chat.completions.create(
    model="mistralai/Mistral-Large-3-675B-Instruct-2512",
    messages=[
        {"role": "system", "content": "You answer in plain English for a small business owner."},
        {"role": "user", "content": "Summarize the main obligations in this lease clause: ..."},
    ],
    temperature=0.05,
)
print(reply.choices[0].message.content)

Mistral recommends a temperature below 0.1 for production work, a clear system prompt, and as few tools as the job needs. Because the API format matches OpenAI's, code you already wrote for another model often needs only a new base_url and model name.

Where can you host it?

There are three broad options. The simplest is a hosted API: Mistral listed its own AI Studio, Amazon Bedrock, Azure Foundry, IBM watsonx, OpenRouter, Fireworks, Together AI and Modal among the places offering Mistral 3 at launch, with NVIDIA NIM and AWS SageMaker coming soon. You pay per token, and the provider handles the hardware. Read each provider's data-handling terms, since your prompts go to its servers.

The second is a rented GPU server from a cloud provider, where you run vLLM yourself as shown above. Your data stays on a machine you control, and you pay by the hour whether or not it is busy, so it makes sense when usage is steady.

The third is your own hardware, in your office or data center. This gives the most control, which can matter in regulated fields such as health or law, but an eight-GPU node of this class is a major purchase. Guillaume Lample, Mistral's co-founder and chief scientist, told TechCrunch that most business uses can be handled by small models once they are fine-tuned, and the Ministral 3 models from the same release run on a single GPU. Test one of those before committing to the large one.