# How do you run Llama 2 locally with Ollama?

Date: 2023-08-03 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/run-llama-2-locally-ollama/

**Answer:** On an Apple Silicon Mac, download Ollama, open the Terminal and type "ollama run llama2": Ollama fetches the 3.8 GB model and opens a chat, all on your own computer. Plan on 16 GB of memory for the 7B model and 32 GB for the 13B one.

About two weeks after Meta released Llama 2, the quickest way to try it on your own machine is Ollama, a free tool that downloads open models and runs them with a single command. Llama 2's licence allows both research and commercial use, and running it locally means your prompts never leave your computer. That matters for drafts, notes or client material you would rather not paste into an online service.

Ollama is still an early preview. Its download page currently offers a version for Macs with Apple Silicon (the M1 and M2 chips); Intel Macs, Windows and Linux are listed as coming soon. Developers on other systems can build it from its source code.

## Which Llama 2 you get

Meta trained Llama 2 in three sizes: 7, 13 and 70 billion parameters, the adjustable numbers a model learns during training. Each size also comes as Llama 2-Chat, a version tuned for conversation with human feedback. The models read up to about 4,000 tokens (word pieces) at a time, twice as much as the first LLaMA.

Ollama's library packages Llama 2 in a compressed form, so the files take far less space than Meta's originals:

| Name in Ollama | Parameters | Download size | Command |
|---|---|---|---|
| llama2 | 7B | 3.8 GB | `ollama pull llama2` |
| llama2:13b | 13B | 7.3 GB | `ollama pull llama2:13b` |

The 70B model is not in the library yet. The library also lists other open models, but `llama2` is the one to start with.

## The steps

1. **Install Ollama.** Download the Mac app from ollama.ai/download, install it and open it once.
2. **Open the Terminal.** You will type every command here.
3. **Download the model.** Type `ollama pull llama2` and wait for the 3.8 GB file. The project's quickstart goes straight to step 4; pulling first just separates the download from the chat.
4. **Start chatting.** Type `ollama run llama2`. A `>>>` prompt appears. Type a message and press Return.
5. **See what you have.** `ollama list` shows the models on your computer. Running `ollama pull llama2` again later updates the model if a newer version exists.

A first session might look like this (the reply is illustrative):

```
ollama run llama2
>>> In two sentences, what is a local language model?
A local language model runs entirely on your own computer instead of a remote server.
Your prompts and its answers stay on your machine.
```

Your Mac does all the work, so replies appear more slowly than in ChatGPT, and the first answer takes a moment while the model loads into memory.

## Make your own version

Ollama lets you wrap a model with settings and a standing instruction in a small text file called a Modelfile. Save this as `Modelfile`:

```
FROM llama2

# lower temperature = steadier, less inventive answers
PARAMETER temperature 0.5

SYSTEM """
You are a patient writing tutor for adult learners. Explain one idea at a time, use plain words, and end each answer with one short question that checks understanding.
"""
```

Then build and run it:

```
ollama create tutor -f ./Modelfile
ollama run tutor
```

The `SYSTEM` block works like a system message: the model reads it before each conversation. Programs can use the same models through a local web address, `http://localhost:11434/api/generate`, which is how the first add-ons for code editors and chat apps connect to Ollama.

> **Try it:** Ask `llama2` and your `tutor` model the same question, such as "What is a semicolon for?", and compare the tone and length. Then change the temperature to 1 in the Modelfile, run `ollama create` again and see how the answers shift.

## How much memory do you need?

Ollama's documentation gives a simple rule: at least 8 GB of RAM for 3B models, 16 GB for 7B models and 32 GB for 13B models. RAM is the computer's working memory, not its storage. The whole model is loaded into it while you chat, and your other apps need room too, which is why the advice is well above the download sizes.

| Model | Download | RAM to have |
|---|---|---|
| llama2 (7B) | 3.8 GB | 16 GB |
| llama2:13b (13B) | 7.3 GB | 32 GB |

To check your Mac, open the Apple menu and choose About This Mac; the memory figure is listed there. With 8 GB, the guidance points to a 3B model instead, such as Orca Mini (`ollama pull orca`, 1.9 GB). With 16 GB, the 7B Llama 2 is the natural choice. The 13B model is the larger and more demanding option, so keep it for a Mac with 32 GB.

Disk space matters less but adds up. Each model you pull stays on your drive until you remove it, so keep an eye on the total if you try several.

## Sources

- Ollama README (commit e4a970e, August 1, 2023): https://github.com/ollama/ollama/blob/e4a970ece120ed03ea41a0bad5df91dde3c5a089/README.md
- Meta and Microsoft Introduce the Next Generation of Llama: https://about.fb.com/news/2023/07/llama-2/
- Llama 2: Open Foundation and Fine-Tuned Chat Models: https://arxiv.org/abs/2307.09288v1