# What is zero-shot prompting?

Date: 2022-02-15 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/zero-shot-prompting/

**Answer:** Zero-shot prompting means asking a language model to do a task with only an instruction and no worked examples in the prompt. It works well for common tasks the model has seen many times, such as summarizing, translating or sorting text into obvious categories, and it is the natural starting point now that instruction-following models are the default.

"Shot" is the word researchers use for a worked example placed in a prompt. A few-shot prompt contains several solved examples before the real question. A one-shot prompt contains one. A zero-shot prompt contains none: you describe the task in plain words and let the model take it from there.

The terms come from the 2020 GPT-3 paper, which tested the model in all three settings. The authors called zero-shot the most convenient setting and also the most challenging one, since the model has to understand the task from the description alone. That balance has shifted in the past few weeks. Since January 27, OpenAI's instruction-following InstructGPT models are the default on its API, and they are trained specifically to act on plain requests. Zero-shot is now a sensible first try for many everyday jobs.

## Why a model can work without examples

A large language model has read an enormous amount of text during training. Summaries, translations, product reviews, emails and definitions all appear in that text countless times. When you ask for one of these familiar tasks, the model already knows what a good result tends to look like. The instruction only has to point it in the right direction.

Training can strengthen this ability. In a September 2021 paper, Google researchers fine-tuned a 137-billion-parameter model on more than 60 tasks, each phrased as a written instruction, a method they call instruction tuning. The tuned model, named FLAN, did better than GPT-3 in the zero-shot setting on 19 of the 25 tasks they checked. OpenAI's InstructGPT models reach a similar goal by a different route, learning from people who wrote and ranked answers.

## Three zero-shot prompts that usually work

Here are three tasks where an instruction is enough. Each one is common, has a clear goal and accepts more than one good answer.

**1. Summarizing**

```prompt
Summarize the following paragraph in one sentence for a busy manager.

Our office will move to the third floor starting March 7. Movers will pack desks on Friday afternoon, so please take personal items home by noon. Internet access may be down Monday morning until IT finishes the setup.
```

**2. Translating**

```prompt
Translate into French: "The library is closed on Sunday, but you can return books in the drop box by the front door."
```

**3. Sorting into obvious categories**

```prompt
Is the following customer review positive, negative or mixed? Answer with one word.

Review: Delivery was fast and the jacket is warm, but the zipper broke after a week.
```

In each case the model has seen this kind of job thousands of times, and the instruction names the audience, the target or the allowed answers. A word like "mixed" in the third prompt matters: it gives the model a sensible option for a review that is neither one thing nor the other.

> **Try it:** Write zero-shot prompts for three small tasks from your own week, such as shortening an email, translating a sign or labeling a message as urgent or not. For each one, change a single word of the instruction (the audience, the length or the allowed answers) and compare the two outputs.

## When does a zero-shot prompt fail?

Zero-shot prompts fail when the task depends on something only you know: a private format, house rules, unusual labels or a judgment call the instruction does not explain. The model fills the gap with the most typical answer, and the most typical answer is not yours.

Here is a prompt that looks clear but usually goes wrong:

```prompt
Tag this support email with the right code.

Email: Hi, I was charged twice for my February subscription. Can you refund one of the payments?
```

The model has no way to know your codes. It may answer "Billing," "Refund request," "#payments" or a whole sentence. Each is reasonable, and none matches a system that expects, say, `BIL-02` for duplicate charges.

The fix is to turn the zero-shot prompt into a one-shot or few-shot prompt by adding examples that show the codes and the format:

```prompt
Tag each support email with one code from this list: BIL-01 (wrong amount), BIL-02 (duplicate charge), ACC-01 (can't log in), SHP-01 (late delivery). Answer with the code only.

Email: My package was supposed to arrive Tuesday and it still isn't here.
Code: SHP-01

Email: Hi, I was charged twice for my February subscription. Can you refund one of the payments?
Code:
```

Now the model sees the list, the exact answer format and a solved case. The task has changed from guessing to matching.

Other warning signs that zero-shot is not enough:

- **The output format keeps changing.** If one run gives a list and the next gives a paragraph, show one example of the shape you want.
- **The task is unusual.** The GPT-3 paper found that puzzles such as unscrambling the letters of a word were much harder for the model with no examples than with them.
- **Accuracy matters and you can test it.** Run the same zero-shot prompt on ten inputs where you know the right answer. If it misses several, add two or three examples and run the test again.

A good working habit is to start zero-shot, look at what comes back, and add examples only where the instruction alone leaves the model guessing. That keeps prompts short when they can be short, and precise when they need to be.

## Sources

- Language Models are Few-Shot Learners: https://arxiv.org/abs/2005.14165
- Finetuned Language Models Are Zero-Shot Learners: https://arxiv.org/abs/2109.01652
- Aligning Language Models to Follow Instructions: https://openai.com/index/instruction-following/