What are Llama 3.2's small models good for?

When Meta released Llama 3.2 on September 25, most of the attention went to its new vision models. The two smallest members of the family deserve a closer look, though, because they change where a language model can run. The 1B and 3B models (1 and 3 billion parameters, the numbers a model adjusts during training) are small enough for a laptop, a browser tab and, with some effort, a phone.

Small doesn't mean general-purpose. These models were made by taking the larger Llama 3.1 8B model, pruning it (cutting away parts that contribute least) and then training the smaller result to imitate the outputs of the 8B and 70B models, a technique called distillation. The result is a model that keeps a surprising amount of language skill in a compact package, but with far less stored knowledge about the world.

What they do well

Meta positions the small models for on-device work: summarizing, following instructions and rewriting text locally. Hugging Face adds prompt rewriting, multilingual retrieval, tool use and local assistants to the list. Both small models can call tools, meaning they can output a structured request for your app to act on, such as "create a calendar event." On IFEval, a test of how well a model follows instructions, Hugging Face reports that the 3B model matches Llama 3.1 8B.

Meta's own example gives the flavor: an app that reads your last ten messages, summarizes them, picks out the action items and sends calendar invitations for follow-up meetings, all on the device. Two advantages come from staying local. Replies can feel almost instant because nothing travels over the network, and private data such as messages and calendars never leaves your hands.

Both models accept up to 128,000 tokens of context (tokens are the word pieces a model reads), so a long email thread or a report fits in one request. In practice, a small model handles a long input less reliably than a large one, so short, focused requests get the best results.

A worked example: which jobs to give a small model

Here is how a 3B model running locally might fit into a small office's day. The ratings are illustrative guidance, not benchmark results.

The pattern: give a small model the material and ask it to transform it. Don't ask it to supply facts from memory.

If you have a Mac or Linux computer, you can try this today with llama.cpp, an open-source program for running models locally. Hugging Face's launch post shows how to install it with Homebrew and download a compressed 3B model in one command:

brew install llama.cpp
llama-cli --hf-repo hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF \
  --hf-file llama-3.2-3b-instruct-q8_0.gguf \
  -p "Summarize these meeting notes as a to-do list with owners: ..."

Licence and where you can use them

The models are free to download from llama.com and Hugging Face under Meta's Llama licence. Services with more than 700 million monthly users need a separate licence from Meta. There is also a regional wrinkle: the licence does not grant rights to the multimodal vision models to people or companies based in the European Union, but the 1B and 3B text models are available there as well as everywhere else.

Can you run one on a phone?

Yes, and that is what they were designed for, but in early October it is mainly a developer's project rather than a download from the app store. Meta worked with Qualcomm and MediaTek, the two largest makers of phone chips, so the models run on their hardware from day one, and optimized them for Arm processors, which power almost all mobile devices. Its on-device toolkit is PyTorch ExecuTorch, and Meta showed a demo running on iOS.

Memory is the main constraint. According to Hugging Face, the 3B model needs about 6.5 GB at full 16-bit precision, but only about 1.75 GB when compressed to 4-bit numbers, a process called quantization. The 1B model drops to roughly 0.75 GB. Those smaller sizes fit comfortably in many recent phones and laptops. Meta's released weights use 16-bit precision, and the company says it is working on official quantized versions that will run faster; community-made compressed versions are already available for llama.cpp.

If you'd like to see a small model run on a device without building an app, Hugging Face links to a demo of the 3B model running inside a web browser using WebGPU, a way for web pages to use your graphics chip. On a computer with a recent browser that supports WebGPU, that is the quickest way to judge whether the speed and quality suit what you have in mind.