# What is prompt engineering, and is it a real skill?

Date: 2022-07-05 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/prompt-engineering-real-skill/

**Answer:** Prompt engineering is the work of designing the text you give a language model so that it performs a task as well as possible, then testing and refining that text. It is a real, learnable skill, closer to clear writing plus careful experiment than to a trick, though whether it becomes a lasting job title is still an open question.

Anyone who has used GPT-3 for more than an afternoon has noticed that small changes in a prompt can change the result a lot. Ask for "a summary" and you get one thing; ask for "three bullet points for a busy manager" and you get something much more useful. The practice of finding the wording that works, and checking that it keeps working, has a name that is spreading quickly: prompt engineering.

The term sounds grand, and some people roll their eyes at it. It is worth separating the hype from the substance, because the substance is real and useful to anyone who works with language models.

## What the term means

A prompt is the text a language model receives before it starts writing. GPT-3 and similar models continue text, so the prompt sets up the task: an instruction, a few examples, the material to work on, and a place for the answer.

In a 2021 survey of prompting research, Pengfei Liu, Graham Neubig and colleagues at Carnegie Mellon University define prompt engineering as creating the prompt that gives the best performance on the task you care about. They describe two main shapes. A cloze prompt asks the model to fill in a blank ("Paris is the capital of ___"). A prefix prompt gives the start of a text for the model to continue, which suits left-to-right models like GPT-3.

The GPT-3 paper showed why this matters. Its authors gave the model tasks purely through text, with no retraining, in three settings: zero-shot (only an instruction), one-shot (one worked example) and few-shot (several). Results often improved as examples were added. The model was the same in every case; only the prompt changed.

## Is it a real skill?

Look at what a good prompt engineer actually does: states a task without ambiguity, chooses representative examples, anticipates how the model might misread the request, and tests the prompt on many inputs instead of trusting one good result. Those are the habits of a clear writer and a careful experimenter. They improve with practice, and people differ in how well they do them. By any ordinary definition, that is a skill.

The Carnegie Mellon survey says as much: writing prompts by hand takes time and experience, and even experienced designers may not find the best one. That second point is also why researchers are building methods to search for prompts automatically, for example by paraphrasing a starting prompt many ways and keeping the version that scores best.

## The debate over the job title

Whether "prompt engineer" becomes a lasting job, rather than a skill inside other jobs, is less clear, and there are reasonable arguments both ways.

On one side, companies building products on GPT-3 depend on prompts that are reliable across thousands of inputs. Someone has to design, test and maintain them, and that work takes real time.

On the other side, two trends could shrink the need for specialists. The first is automation, the automatic prompt search described above. The second is better instruction-following. Since January, OpenAI's default API models have been InstructGPT versions, trained with human feedback to follow plain instructions, and OpenAI reports that its labelers preferred their answers to those of the original GPT-3. If models understand ordinary requests well, elaborate prompt tricks matter less, while clear thinking about the task still matters.

A fair summary for today: prompting well is a valuable part of many roles, such as writing, support, research and software, and it is too early to say how many people will hold it as a title.

## How do you practise it?

Work the way a scientist does: change one thing at a time and compare. Here is a simple routine.

**Step 1. Pick one real task** with a result you can judge, for example "write a two-sentence product description for a handmade wooden cutting board."

**Step 2. Write five variations** of the prompt, each adding one element:

| Version | What changes |
|---|---|
| 1 | Bare task only |
| 2 | Add the audience ("for shoppers on a craft market website") |
| 3 | Add constraints ("two sentences, no exclamation marks, mention maple wood") |
| 4 | Add one example of a description you like, for a different product |
| 5 | Reframe as a document, with a label where the answer goes |

Version 5 might look like this:

```prompt
Product descriptions from a craft market website. Each is two calm, concrete sentences.

Product: Hand-thrown ceramic mug
Description: A sturdy stoneware mug, glazed in deep blue and shaped by hand in a small studio. It holds a generous cup of coffee and is safe in the dishwasher.

Product: Handmade maple cutting board
Description:
```

**Step 3. Run each version three times**, since GPT-3's output varies, and keep all fifteen results.

**Step 4. Score them** against a short checklist you write before you look: correct length, accurate details, tone, nothing invented.

**Step 5. Write one sentence** about which change helped most. Over a few weeks, these notes become your own guide to what works.

> **Try it:** Run the five-version routine on a task from your own work this week, and keep a simple log of the version, the score and the one change that made the biggest difference.

## Sources

- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing: https://arxiv.org/abs/2107.13586
- Language Models are Few-Shot Learners: https://arxiv.org/abs/2005.14165
- Aligning Language Models to Follow Instructions: https://openai.com/index/instruction-following/