What are Llama 4 Scout and Maverick, and what is a mixture-of-experts model?

Yesterday, Saturday, April 5, 2025, Meta released the first models of its Llama 4 family, which it calls "the Llama 4 herd." Two of them can be downloaded now: Llama 4 Scout and Llama 4 Maverick. A third, Llama 4 Behemoth, was described but not released, because it is still training.

Like earlier Llama models, Scout and Maverick are open-weight: Meta publishes the trained model files, the "weights", so developers can run the models on their own hardware, study them and adapt them, under the terms of Meta's license. That sets them apart from models such as GPT-4o or Gemini, which you can only reach through their makers' apps and services.

What Meta released

Here are the three models side by side, using Meta's figures:

Parameters are the numbers inside a model that are adjusted during training; very roughly, more parameters means more capacity to learn. The "active" and "expert" columns need a little explanation, which comes next.

Both available models can be downloaded from llama.com and Hugging Face, a popular site for sharing AI models, and Meta says cloud partners will offer them in the coming days. Meta AI, the assistant built into WhatsApp, Messenger, Instagram Direct and the meta.ai website, now runs on Llama 4. TechCrunch reports this applies in 40 countries, with image features limited to English in the United States for now.

How a mixture of experts works

Older Llama models were dense: every part of the model worked on every word. Llama 4 is Meta's first mixture-of-experts (MoE) design.

Think of a large hospital. A patient does not see every doctor on staff. A front desk sends them to the one or two specialists who fit their problem. The hospital has huge combined expertise, but each visit only uses a small part of it.

In Llama 4 Maverick, each token (a piece of a word) is sent to a shared expert that sees everything, plus one of 128 specialist experts chosen by a routing step. So although Maverick has 400 billion parameters stored in memory, only about 17 billion are used for any given token. Meta says this makes the models cheaper and faster to train and to run than a dense model of similar quality.

Images and a very long memory

Llama 4 is natively multimodal. Rather than attaching an image reader to a finished text model, Meta used what it calls early fusion: text and image pieces go into the same model from the start of training. It trained on more than 30 trillion tokens of text, images and video, covering 200 languages.

Scout's standout feature is its context window, the amount of material it can consider at once: 10 million tokens, up from 128,000 in Llama 3. That is enough, in principle, for many long documents or a large codebase in a single request. Meta suggests uses such as summarizing several documents together, or reasoning over a large codebase.

Behemoth serves as a teacher. Meta trained Maverick partly by having it learn from Behemoth's outputs, a technique called distillation, and reports that on some science and math tests Behemoth outscores Gemini 2.0 Pro, Claude 3.7 Sonnet and GPT-4.5.

How they compare, and the fine print

Meta says Maverick beats GPT-4o and Gemini 2.0 Flash on many common tests and matches DeepSeek V3 on reasoning and coding with fewer active parameters. TechCrunch notes that it does not reach newer models such as Gemini 2.5 Pro, Claude 3.7 Sonnet or GPT-4.5, and that no member of the Llama 4 family is a reasoning model like o1 that thinks before answering.

The license has limits. Companies with more than 700 million monthly users need Meta's permission, and TechCrunch reports that people and companies based in the European Union are barred from using or distributing the models. Meta also says it tuned Llama 4 to refuse fewer questions on debated political and social topics, from 7% of such prompts with Llama 3.3 to under 2%.

Why it matters

The open-model field has become crowded and fast. TechCrunch reports that the success of DeepSeek's open models pushed Meta to speed up its work. Llama 4 is Meta's reply: more capability per unit of computing, image understanding built in, and a context window far beyond most rivals. For developers who want to run models on their own servers, keep data in-house or customize a model, that is a significant new option.

Can you run Llama 4 on your own computer?

For most people, not yet. Scout is the smaller model, and Meta says it fits on a single NVIDIA H100, a data-center graphics card, only when compressed with Int4 quantization, a technique that stores the model's numbers with less precision to save memory. Maverick needs a full H100 server. These are professional machines, far beyond a typical laptop or home desktop.

There are easier routes. The quickest is Meta AI in apps you may already use. Developers can also try the models through Hugging Face and, Meta says, through cloud providers in the coming days, paying only for what they use instead of buying hardware.

If you do download the weights, read the license first, especially the rules on large companies and on use in the European Union. Here is a prompt for a first test of Scout's long memory:

Below are three reports from different years. Read all of them, then list the five
most important changes between the first and the last, and quote the sentence
that supports each change.