What is Stable Diffusion, and why does an open model matter?

Yesterday, August 22, Stability AI announced the public release of Stable Diffusion, an image generator that had been available only to researchers for the past few weeks. You describe a picture in words, for example "a photograph of an astronaut riding a horse", and the model produces an image to match. Tools that do this already exist, but most of them live on a company's servers and you reach them through a website or a chat app. What is new here is that the model file itself, the trained "weights", is published for anyone to download.

That one decision changes who gets to use this technology and how. It is worth slowing down to see what the model is and what "open" means in this case.

What Stable Diffusion is

The model comes from the CompVis research group, with computing power donated by Stability AI and support from LAION, a group that assembles large image datasets. According to the project's GitHub page, it learned from 512 by 512 pixel images drawn from part of a large web dataset called LAION-5B.

Technically it is a latent diffusion model. "Diffusion" describes how it works: during training, the model sees images gradually buried under random noise and learns to reverse the process, step by step, until a clean picture emerges. "Latent" means it does this work on a compressed version of the image rather than on every pixel, which is a large part of why it can run on ordinary hardware. To follow your words, it uses a text encoder called CLIP that turns the prompt into numbers the image model can use as a guide.

The README calls it relatively lightweight: an image network of about 860 million parameters plus a text encoder of about 123 million.

What "open" means here

Stability AI released the model under a license called CreativeML OpenRAIL-M. It allows both commercial and non-commercial use, but it comes with a list of uses you agree not to pursue, and it places the responsibility for legal and ethical use on whoever runs the model. Anyone who redistributes the model must pass the license along.

So this is not "open source" in the strictest software sense, because there are use restrictions. But the practical openness is real:

	Hosted image service	Stable Diffusion with public weights
Where it runs	The company's servers	Your computer or any server you choose
Cost per image	Set by the company	Your own electricity and hardware
Can you change it?	No	Yes: retrain it, combine it, build apps on it
Content rules	The company's filter	A safety checker you can adjust, plus the license
Can researchers inspect it?	Rarely	Yes

This matters for a few reasons. Artists and developers can now build tools around the model without asking permission. Researchers can study its behavior and its flaws directly. And the cost of trying ideas drops to nearly zero once you have the hardware. The project's own documentation is frank about the downside: the model mirrors the biases and misconceptions in its training data, and the default safety features can be changed by anyone running it. Stability AI's release includes an AI-based safety classifier on by default, and the reference script adds an invisible watermark so images can be identified as machine-made.

Writing your first prompts

Image prompts work best as short, concrete descriptions: the subject, the setting, the style and the light. Compare these two:

Prompt

a house

Prompt

a small red wooden cabin beside a lake at sunrise, mist on the water, autumn trees, soft golden light, watercolor painting

The first gives you something generic. The second tells the model what to draw, where, in what style and with what mood, and each part steers the result.

Try it: Pick one subject, such as "a lighthouse", and write three prompts that change only one thing each time: the time of day, then the art style, then the weather. Generate each and note which word had the biggest effect.

Can you run Stable Diffusion on your own computer?

Yes, if your computer has a suitable graphics card. The CompVis README says the model runs on a GPU (graphics processing unit) with at least 10 GB of video memory, and Stability AI says the released weights should use about 6.9 GB. Right now NVIDIA cards are the recommended choice; Stability AI says optimizations for AMD chips and Apple's M1 and M2 Macs are coming but are not part of this release.

The setup is aimed at people comfortable with a command line. You create a Python environment, download the weights from the CompVis page on Hugging Face, and run a script with your prompt. The reference script makes 512 by 512 images in 50 steps by default.

If you do not have the hardware or would rather not install anything, there are easier routes today. Stability AI points to a public demo on Hugging Face, a notebook built on Hugging Face's diffusers library that you can run in the cloud, and its own web app, DreamStudio. Those are good places to learn how prompts behave before you decide whether a local setup is worth it. Many expect simpler one-click installers to appear soon, now that the weights are public.