How do you write a negative prompt in Stable Diffusion?

When you write a prompt for Stable Diffusion, you describe what you want: subject, style, lighting. A negative prompt is the opposite list, typed in its own box or passed as its own setting: the things you want kept out. Many people's first use is clean-up work, removing blur, stray text or a watermark-like smudge in a corner.

The idea grew out of the Stable Diffusion user community, and as of last week it is also part of Hugging Face's free diffusers library too. Version 0.4.0, released on October 6, lets the Stable Diffusion pipeline take a positive prompt and a negative prompt side by side.

How a negative prompt works

Stable Diffusion is a diffusion model: it starts from random noise and removes a little of it at each step until an image appears. To keep the image on topic it uses a trick called classifier-free guidance. At every step the model makes two predictions, one guided by your prompt and one guided by an empty prompt, and then pushes the result away from the empty version and toward your prompt. The CFG scale (in code, guidance_scale) sets how hard it pushes.

A negative prompt simply takes the place of that empty prompt. Instead of moving away from "nothing in particular," each step moves away from "blurry, watermark, extra fingers." That is why the negative prompt only works when guidance is switched on: in diffusers, it is ignored if the guidance scale is below 1.

This also explains a common surprise. If you write "a street with no cars" in the main prompt, the word "cars" is still sitting in the text the model reads. Putting "cars" in the negative prompt is the clearer instruction.

A worked example: the same prompt, with and without

Here is a portrait prompt, run twice with the same seed so that only the negative prompt changes.

Prompt: portrait photo of an elderly fisherman in a yellow raincoat, harbor at dawn, soft light, 85mm, detailed skin
Negative prompt: (none)

Prompt: portrait photo of an elderly fisherman in a yellow raincoat, harbor at dawn, soft light, 85mm, detailed skin
Negative prompt: blurry, low resolution, cartoon, painting, text, watermark, extra fingers, deformed hands

The table below describes the kind of difference people typically see. The results are illustrative, not measured; your own seed will give different pictures.

Three habits make negative prompts useful:

1. Use plain words for things you can see. "Watermark," "text," "frame," "cartoon" give the model something concrete to move away from. Vague words like "bad" do little.
2. Keep the list short. Prompts are cut into tokens, and the text encoder Stable Diffusion uses reads a fixed window of 77 of them; anything longer is cut off. A tight list of five to ten terms is easier to test.
3. Test one change at a time. Fix the seed, run the image, add one negative term, run again. That is the only way to know which word did the work.

In Python with diffusers 0.4.0, the call looks like this:

generator = torch.Generator("cuda").manual_seed(1024)
image = pipe(prompt, negative_prompt="blurry, text, watermark",
             num_inference_steps=50, guidance_scale=7.5,
             generator=generator).images[0]

Which settings matter most for quality?

Four settings do most of the work, and the diffusers documentation gives sensible starting points for Stable Diffusion 1.x:

Steps add detail up to a point. More steps usually mean a cleaner image but slower generation, and past the default the gains are small for everyday use.

CFG scale is a balance. Raise it and the model sticks more closely to your words, at some cost to variety and sometimes to image quality. Very high values can look striking but give similar-looking pictures. If an image looks overcooked, lower the scale before rewriting the prompt.

The seed is what makes careful testing possible. Change the prompt and keep the seed, and you see the effect of your words. Change the seed and keep everything else, and you see the range of what the prompt can produce.

Size is easy to get wrong. Width and height must be multiples of 8. Going below 512 tends to lower quality, and going above 512 in both directions can make the model repeat parts of the scene. For a landscape or portrait shape, keep one side at 512 and stretch the other.

A good routine: start at 50 steps, CFG 7.5 and 512 × 512, find a seed you like, then refine the prompt and the negative prompt with that seed fixed.