What is Llama 3, and what changes for Meta AI?

Yesterday, Thursday, April 18, 2024, Meta released Llama 3, the third generation of its openly available large language models. A large language model is a program trained on huge amounts of text to predict and write language; "openly available" means anyone can download the trained model and run it on their own computers, within the terms of Meta's license.

Meta made two announcements at once. On its AI blog, it published the first two Llama 3 models for developers and researchers. On its newsroom, it explained that Meta AI, the free assistant built into its apps, now runs on Llama 3 and is reaching more people. One is for builders, the other for everyday users, and both share the same engine.

What Meta released

The first Llama 3 models come in two sizes, each in two versions:

A parameter is one of the adjustable numbers inside a model that is set during training. A pretrained model has learned to continue text; an instruction-tuned model has had extra training to follow requests and hold a conversation, so it is the version most people want for a chat assistant.

Meta calls these the first models in the Llama 3 collection. It says larger models, with more than 400 billion parameters, are still training, and that in the coming months it plans to add the ability to handle images as well as text, more languages, a longer memory for each conversation and a research paper. The released models are text only and, TechCrunch notes, fine-tuned for English output.

How Llama 3 was built

Meta says it kept the overall design simple and put most of its effort into data and training. Four changes stand out.

More data. Llama 3 learned from more than 15 trillion tokens, the small chunks of text a model reads (often a word or part of a word). That is about seven times the data used for Llama 2, with four times as much computer code. TechCrunch puts it at roughly 750 billion words. Meta says the data comes from publicly available sources but does not list them. A little over 5% is in languages other than English, more than 30 of them.

A bigger vocabulary. The tokenizer, the tool that cuts text into tokens, now knows 128,000 different pieces. A larger vocabulary means common words and phrases fit into fewer tokens. Meta says the same text takes up to 15% fewer tokens than with Llama 2, so the model can process it more efficiently.

Training far past the usual point. Researchers often estimate how much data a model of a given size "needs." Meta reports that its 8B model kept improving well beyond that estimate, all the way to 15 trillion tokens. The training ran on two custom-built clusters of 24,000 GPUs each, the graphics chips used for AI.

Better tuning. To turn the pretrained models into assistants, Meta combined several methods. In plain terms: show the model good example answers, have it produce several answers and keep the best, and teach it from human rankings of which answer is better. Meta says this cut down on false refusals, cases where a model declines a harmless request, and improved reasoning and coding.

How it compares

Meta compared Llama 3 with other models on standard tests. According to TechCrunch's reading of Meta's figures, Llama 3 8B beats two other small open models, Mistral 7B and Google's Gemma 7B, on at least nine benchmarks, sometimes by only a few points. Meta says Llama 3 70B beats Google's Gemini 1.5 Pro on three tests and Anthropic's Claude 3 Sonnet on five, though not Anthropic's top model, Claude 3 Opus.

Meta also built its own set of 1,800 test prompts across 12 everyday tasks, such as brainstorming, coding and summarizing, and says its modeling teams cannot see it. Human raters preferred Llama 3 70B's answers over those of GPT-3.5, Mistral Medium and Claude Sonnet. TechCrunch points out, fairly, that a company's own test deserves some caution.

What changes for Meta AI

Meta AI, first announced last year at Meta's Connect conference, now runs on Llama 3. Meta says it is rolling out in English to more than a dozen countries outside the US, including Australia, Canada, New Zealand, Nigeria, Pakistan, Singapore and South Africa. It appears in the search bar of Facebook, Instagram, WhatsApp and Messenger, where it can pull in current information from the web, and in the Facebook feed, where you can ask about a post. A new website, meta.ai, offers the assistant on a computer, and logging in saves your conversations.

Image generation also gets faster. With the Imagine feature, rolling out in beta on WhatsApp and meta.ai in the US, a picture appears as you type and changes as you add words.

Why it matters

For builders, Llama 3 raises the bar for models you can download and run yourself. A strong model in the 8-billion-parameter class is far cheaper to run than a giant one, which matters to anyone who wants to keep data on their own machines. Meta says the models will soon be on major cloud platforms, including AWS, Google Cloud, Microsoft Azure and Hugging Face.

There are conditions. TechCrunch notes that Meta's license bars using Llama to train other AI models, and services with more than 700 million monthly users need Meta's permission, which is why many call Llama "open" rather than "open source." Meta also ships safety tools alongside it: Llama Guard 2 to screen prompts and answers, CyberSec Eval 2 to test security risks, and Code Shield to catch insecure code a model writes.

For everyone else, the change is quieter: many people will meet Llama 3 without knowing it, through the search bar of an app they already use.

I have chicken, rice, a lemon and spinach. Suggest one dinner I can make in 30 minutes, with steps numbered and quantities for two people.

How can you try Llama 3 today?

The easiest route is Meta AI. If you live in the US or one of the newly added countries, open the search bar in Facebook, Instagram, WhatsApp or Messenger, or go to meta.ai in a browser. Ask a question in plain language; log in on meta.ai if you want your conversations saved. Remember that it can still be wrong, so check facts that matter.

If you build software, you can download the models from the Llama 3 website, under the terms of Meta's license, and Meta's getting-started guide lists the platforms that host them. Start with Llama 3 8B Instruct: it is the easiest to run and a good test of whether the model suits your task before you pay for larger hardware.