# How do you run Muse Glimmer locally?

Date: 2026-08-14 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/run-muse-glimmer-locally/

**Answer:** Install a recent build of llama.cpp (b10353 or newer), download Meta's 16.8 GB Q4_K_M file of Muse Glimmer from Hugging Face, and start llama-server with the --jinja option. It fits a computer or graphics card with about 24 GB of memory, or 32 GB for the higher-quality build.

Meta released **Muse Glimmer** this week, an open-weight model from Meta Superintelligence Lab, the group behind Muse Spark. It is a dense model of about 30 billion parameters (the adjustable numbers a model learns), trained by distillation from Muse Spark, which means it learned partly by imitating the larger model's outputs. Meta built it for agents that run on your own machine: it reads text and images, calls tools, reasons before it answers, and was trained on more than 100 languages. Everything is published under the Apache 2.0 licence.

"Dense" matters for planning. Unlike mixture-of-experts models, which use only part of their parameters for each word, Glimmer uses all of them every time. That makes it simpler to run but slower per word than a sparse model of the same size, which is why Meta also ships a small helper model to speed it up.

## What you need

Meta publishes ready-made files in **GGUF**, the format read by **llama.cpp**, the free engine that runs language models on ordinary hardware. Both main files are **quantized**: each parameter is stored in about 4 bits instead of 16, which shrinks the model a lot at a small cost in quality.

| File | Size | Use it for |
|---|---|---|
| KQuant-17GB-Q4_K_M | 16.8 GB | The starting point; fits 24 GB of graphics memory |
| KQuant-Dynamic-Q4_K_XL | 19.7 GB | Slightly better quality; aim for 32 GB |
| mmproj (perception encoder) | 1.4 GB | Needed only for image input |
| dflash (drafter) | 1.6 GB | Optional, makes replies faster |

Meta reports a 1.0 percent average drop on 15 benchmarks for the smaller build and 0.2 percent for the larger one, compared with full precision. On a Mac, the unified memory counts; on a PC, the graphics card's memory is what matters. Early users report the smaller file running on a single 24 GB card with long documents in view, because the model's attention design keeps its working memory small.

One requirement trips people up: you need llama.cpp **build b10353 or later**. Support for the new architecture was added on August 10, and older builds refuse to load the files.

## A worked install

**1. Get llama.cpp and check the build.**

```bash
llama-cli --version    # the build number must be 10353 or higher
```

If it is older, update it from the llama.cpp releases page or rebuild from source.

**2. Download the model and the image encoder.**

```bash
pip install huggingface_hub
hf download meta-models/Muse-Glimmer-30B-GGUF \
    --local-dir Muse-Glimmer-30B-GGUF \
    --include "Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf" \
    --include "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf"
```

**3. Start a local server.**

```bash
llama-server \
    -m Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
    --mmproj Muse-Glimmer-30B-GGUF/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
    -ngl 99 -c 32768 --jinja \
    --temp 1.0 --top-p 0.95 --top-k 64
```

`-ngl 99` puts all layers on the graphics processor, `-c 32768` sets how many tokens of context the model keeps in view, and the last three values are Meta's recommended sampling settings. `--jinja` is required: it applies the chat template stored inside the file, and without it replies come out garbled or the program stops.

**4. Open `http://127.0.0.1:8080`** in your browser and ask something that needs a few steps:

```prompt
I have 3 meetings on Tuesday: 9:00-10:15, 10:30-12:00 and 14:00-15:30. I need a free block of 90 minutes for focused work. Where does it fit, and what is the latest start time that still works?
```

The server keeps the model's reasoning apart from its final answer, so you can check how it got there.

> **Try it:** Add `--chat-template-kwargs '{"reasoning_strength":"low"}'` to the server command, ask the same question, and compare the speed and the answer with the default setting.

## Settings worth knowing

Glimmer always thinks before it answers; you cannot switch that off. You can change how much it thinks with a **reasoning strength** of low, medium, high or xhigh, high being the default; Meta suggests high or xhigh for coding and agent work. To speed things up, download the drafter file and add `-md` with its path. This is **speculative decoding**: the small drafter proposes several words at once and the main model checks them, so the output is the same, only faster.

Two cautions. Independent testers rate Glimmer as strong for its size but weaker than some peers at knowing when it does not know something, so check facts it gives you. And Meta itself recommends extra safeguards, such as human confirmation before irreversible actions, whenever you let it operate tools. Its card also says it is not meant for users under 18.

## What does its Apache 2.0 licence allow?

Apache 2.0 is one of the most permissive licences in software. It gives you a free, worldwide, permanent right to use, copy, modify and redistribute the model, and it sets no limit on commercial use or on how many people use what you build. You can run Glimmer inside a paid product, fine-tune it, or ship it on devices without paying Meta or asking permission. Meta's card says all of the released parts, from the full-precision weights to the drafter, carry that licence.

The licence asks for a few things in return. If you redistribute the model or a modified version, include a copy of the licence, keep the existing copyright and attribution notices, pass along any NOTICE file, and mark the files you changed. You may put your own terms on your changes, as long as your use of the original still follows Apache 2.0.

It does not give you everything. It grants no right to use Meta's trademarks beyond saying where the model came from, so you cannot market your product as an official Meta model. It comes with no warranty: if the model makes a costly mistake, that responsibility is yours. And it does not override the law: Meta's card excludes uses that break applicable laws, including trade rules, and points to a separate usage policy. Read that policy before you build on the model, alongside the licence itself.

## Sources

- Muse Glimmer Model Card (meta-models/Muse-Glimmer-30B-GGUF): https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/blob/43c7eadd4135/README.md
- Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0
- [AINews] Muse Glimmer and Spark open weights: https://latent.space/p/ainews-muse-glimmer-and-spark-open