What is Google's Imagen, and how does it turn text into pictures?

Yesterday Google Research showed off Imagen, a system that draws pictures from written descriptions. Its sample gallery includes a corgi in sunglasses riding a bicycle through Times Square, a glass duck sculpture in front of a landscape painting, and a raccoon painted in the style of Rembrandt. Google says these images are more realistic, and closer to what the text asked for, than anything else it has tested.

The announcement comes shortly after OpenAI unveiled DALL-E 2, and Google compares the two directly. The most interesting part, though, is a finding about where the quality comes from: a better grasp of language, more than a bigger image model.

What Google announced, and when

On 23 May 2022, the Brain Team at Google Research posted a project page and a paper titled "Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding" on arXiv. Chitwan Saharia, William Chan and Mohammad Norouzi are listed as equal first contributors, with Jonathan Ho and David Fleet among the core contributors.

What they released is research, not a product: a description of the method, test results, a new benchmark and a gallery of hand-picked samples. There is no app, no demo and no code.

How Imagen works

Imagen is a text-to-image model: you type a description, called a prompt, and it produces a picture. It works in three stages, which you can think of as a reader, an artist who sketches, and two assistants who add detail.

Stage	What it does	Size
1. Text encoder (T5-XXL)	Reads the prompt and turns it into a list of numbers that capture its meaning	4.6 billion parameters, kept frozen
2. Base diffusion model	Creates a small 64×64-pixel image that matches those numbers	2 billion parameters
3. Two super-resolution models	Enlarge the image to 256×256, then to 1024×1024, adding fine detail	600 million and 400 million parameters

The image stages are diffusion models. A diffusion model learns to remove noise. During training it sees real pictures with random speckles added and learns to undo the damage step by step. To make a new image, it starts from pure noise and cleans it up, guided by the text, until a picture emerges, a little like a photograph slowly developing in a darkroom tray.

The reader is the surprise. T5-XXL is a language model Google trained earlier on text alone; it has never seen an image. Imagen uses it unchanged (frozen) to understand prompts. The team found that making this text model larger improved both the realism of the images and how well they matched the prompt much more than making the image model larger did. In plain terms, understanding exactly what was asked matters as much as drawing skill.

The image models learned from about 860 million picture-and-caption pairs: roughly 460 million from Google's internal datasets and about 400 million from LAION-400M, a public collection gathered from the web. The paper also introduces two technical improvements: dynamic thresholding, which keeps colors from washing out when the model is pushed to follow the prompt closely, and a faster image network it calls Efficient U-Net.

How good is it?

Image researchers often use a score called FID (Fréchet Inception Distance), which measures how statistically close a set of generated images is to real photos; lower is better. On the COCO dataset, a standard collection of everyday photos with captions, Imagen scores 7.27 without ever having trained on COCO. In the same comparison, Google lists DALL-E 2 at 10.39.

Because a single number cannot capture whether a picture follows a tricky instruction, Google also built DrawBench, about 200 prompts in 11 categories. They test things image models often get wrong: counting objects, placing them in the right spot relative to each other, writing words inside the image, rare words and long, detailed descriptions. Human raters compared pictures side by side. Against DALL-E 2, they preferred Imagen for matching the text in 7 of the 11 categories and for image quality in all 11.

Here is how you might test a system with prompts in the DrawBench spirit:

Prompt

Three yellow tulips in a blue vase, to the left of a red apple, on a wooden table.

Prompt

A street sign that reads "SLOW: TURTLE CROSSING" in front of a pond.

The first checks counting, colors and position at once; the second checks whether the model can spell.

Try it: Write five prompts that each test one skill: a number of objects, a color on a specific object, a left-right position, a word written in the scene, and an unusual combination (such as a cactus wearing a straw hat). If you have access to any image generator, run them and score each result yes or no. You will quickly see which skills are still hard.

Why it matters

Imagen strengthens a trend that DALL-E 2 started: diffusion models can now produce images that many people would take for photographs or professional illustrations. It also points to a practical lesson for builders. Large language models trained only on text, already common in AI labs, turn out to be powerful ready-made "readers" for image systems, so progress in language models may carry straight over to image making.

The new benchmark matters too. By publishing DrawBench, Google gives other researchers a shared set of hard prompts, which makes claims of "better" images easier to check.

Why isn't Google letting the public use Imagen?

Google says plainly that it has decided not to release the code or a public demo for now. It gives several reasons. Text-to-image tools could be misused, for example to create misleading images. The web-scraped data used for training, including LAION-400M, is known to contain pornographic images, racist language and harmful stereotypes, and only part of Imagen's data was filtered. The T5 text encoder, trained on unfiltered web text, brings the biases of large language models with it.

Google's own early checks found problems. Imagen does noticeably worse when asked to depict people. It tends to produce people with lighter skin, and images of different professions tend to follow Western gender stereotypes. Even scenes without people showed social and cultural biases.

The team says it wants to develop a way of sharing such models that allows outside review without unrestricted access, and to build better methods for measuring bias in image generators. For now, the gallery and the paper are the only ways to see Imagen at work. Readers who want to understand it can study the paper's sample prompts and its DrawBench list, which give a clear picture of what the system does well and where text-to-image models still struggle.