What is Code Llama, and can a free model really help you write code?

Yesterday Meta released Code Llama, a set of large language models specialized for programming. Ask it in plain English for "a function that returns the Fibonacci sequence" and it writes the code; show it a block of code and it can explain what it does or help find a bug.

The important word is released. As with Llama 2 last month, Meta is offering the model weights for download under the same community license, which allows research and commercial use. The best-known coding assistants, such as OpenAI's Codex and GitHub Copilot, are services you reach over the internet. Code Llama can run on your own machines.

What Meta released

Code Llama starts from Llama 2, Meta's general-purpose model, and continues training it on a large collection of code. Meta describes it as Llama 2 with sharper coding skills. It supports many popular languages, including Python, C++, Java, PHP, TypeScript and JavaScript, C# and Bash, the language of command-line scripts.

There are three sizes and three flavors.

Each flavor comes in 7, 13 and 34 billion parameters, the internal numbers a model learns. Every size was trained on 500 billion tokens (word pieces, or in code, fragments of keywords and symbols) of code and code-related text.

How it works

Three features stand out.

Fill-in-the-middle. Most language models only continue text from the end. The 7B and 13B models were also trained to insert code into the middle of an existing file, given what comes before and after. That is exactly what an editor's autocomplete needs: you are often typing inside a function, not at the bottom of the file.

Long context. Training used chunks of 16,000 tokens, yet Meta says the models stay stable with inputs of up to 100,000 tokens. The context is how much text a model can consider at once. A long context means you can paste in several files of a project so the suggestions fit your actual code.

Size choices. The sizes trade quality against speed. Meta says the 7B model can be served from a single graphics card (GPU) and is quick enough for real-time completion, while the 34B model gives the best results but needs more hardware.

A worked example shows how you might use the Instruct version:

Write a Python function called is_palindrome that takes a string and returns True if it reads the same forwards and backwards, ignoring spaces, punctuation and capital letters. Add three short test cases.

A good answer will clean the string (keep only letters and digits, make them lower-case), compare it with its reverse, and show tests such as "A man, a plan, a canal: Panama" returning True. Whatever the model writes, run the tests yourself before trusting it.

How well does it score?

Meta tested the models on two common benchmarks. HumanEval asks a model to complete a Python function from its description; MBPP (Mostly Basic Python Programming) asks it to write short programs from a sentence describing the task. Scores are the share of problems solved correctly on the first try.

Meta says these are the highest scores among openly available models and on par with ChatGPT, and that Code Llama beats Llama 2 and other open code models. As always with self-reported benchmarks, independent testing over the coming weeks will tell how well this holds up on real projects.

Meta also ran a safety test, asking the models for clearly malicious code and comparing the answers with those of GPT-3.5 Turbo. It reports that Code Llama's responses were safer. More detail, including known limitations, is in the accompanying research paper.

Why it matters

For developers, the news is a coding model that, by Meta's measurements, leads the openly available field and is free to download, inspect and adapt. A company can run it inside its own network so its source code never leaves, or fine-tune it on its own codebase. VentureBeat calls it an immediate rival to Codex, Copilot and other coding assistants.

For learners, Meta presents Code Llama as a way to lower the barrier for people starting to program, and as a productivity and teaching tool for writing better-documented software. A model that explains code line by line can be a patient tutor, as long as you keep checking its work.

There are limits. Meta does not recommend the base or Python models for general conversation, because they are not designed to follow everyday instructions. And like all language models, Code Llama can produce code that looks right but is wrong or insecure.

Which version of Code Llama should you use?

If you want to ask for code in plain language, as most beginners do, use Code Llama - Instruct. Meta recommends it for code generation because it was tuned to follow instructions and give helpful, safer answers.

If you are building autocomplete into an editor, the base 7B or 13B model makes more sense, since those support fill-in-the-middle and respond fast. If your work is almost entirely in Python, the Python version is specialized for it. Pick the 34B size when quality matters more than speed and you have the hardware.

Whichever you choose, read Meta's license and acceptable use policy before building a product, and treat every suggestion as a draft. Run the code, read the tests, and keep a human in charge of what gets shipped.