Prompting

How do you keep a log of your best prompts?

· By Robert Scales · 4 min read

Short answer

Keep a simple spreadsheet or notes file with one row per prompt version, recording the task, the exact prompt, a short note on the result, a rating and a version number. Over a few weeks it becomes a personal library of prompts that are proven to work, and a record of what you changed to make them better.

Anyone who has used ChatGPT for a few weeks knows the feeling: you wrote a prompt last Tuesday that produced exactly the right kind of answer, and now you cannot remember how you phrased it. A prompt log solves this. It is nothing more than a place where you save the prompts that worked, with enough notes to use them again.

It matters more than it seems because small wording changes have large effects. OpenAI's launch post for ChatGPT notes that the model is sensitive to tweaks in how a question is phrased, and that asking the same thing again can produce a different result. If wording matters that much, the exact wording of your best prompts is worth keeping.

What to record

A useful log answers five questions about each prompt:

  • Task: what was this for? ("Summarize meeting notes into action items.")
  • Prompt: the exact text, copied in full, including any examples.
  • Result: one or two lines on what came back, good and bad.
  • Rating: a simple score, such as 1 to 5, for how usable the output was without edits.
  • Version: a number that goes up each time you change the prompt.

Two optional columns pay off over time: the date, and the tool and settings (ChatGPT, or a specific model in OpenAI's Playground with its temperature). Tools change, and a prompt that worked well in one may behave differently in another.

A worked example: one prompt, three versions

Here is how a log might look for a single task, turning messy meeting notes into a short list of action items. The results and ratings are illustrative.

VersionChange madeResult (illustrative)Rating
1"Summarize these meeting notes."A long paragraph; action items buried2
2Asked for bullet points of action items onlyClear list, but no owners or dates3
3Added a fixed format with owner and due date, notes set off with ###Ready to paste into an email5

And here is version 3 as it would be stored, with a placeholder in braces where the new material goes each time:

Prompt
From the meeting notes below, list every action item.
Format each one as: Owner | Task | Due date
If no due date was given, write "not set". Do not add items that are not in the notes.

###
{paste meeting notes here}
###

Two of the changes in version 3 follow advice from OpenAI's own guide to prompting its models: put the instruction first and separate it from the material with markers such as ###, and show the exact output format you want. The log makes it easy to see which change produced the jump in quality, so you can apply the same lesson to other prompts.

Where to keep it is a matter of taste. A spreadsheet works well because you can sort by rating or filter by task. A plain document with one heading per task works too. What matters is that it is one place you actually open.

Try it: Start a log today with three columns more than you think you need: task, prompt, result, rating, version. Add the next three prompts you use, and the next time one of them disappoints you, change one thing, save it as a new version and rate both.

Habits that keep the log useful

Log at the moment of success. The best time to save a prompt is right after it gives you something you would use. A minute later you will be busy using it.

Change one thing at a time. If you rewrite the whole prompt between versions, you will not know which change helped. OpenAI's guide suggests a similar order: start with a plain instruction, add a few examples if that is not enough, and only then consider heavier methods.

Record failures too. A low-rated version with a note such as "too vague about length" saves you from repeating the mistake.

Keep private material out. Store the prompt with placeholders, not with the real client names or figures you pasted in on the day.

How do you organize prompts so you can reuse them?

Group them by the job they do, not by the date you wrote them. A handful of categories covers most people's needs: writing (emails, summaries, rewrites), learning (explanations, quizzes), planning (outlines, checklists) and work-specific tasks such as product descriptions or lesson plans. Give each prompt a short, plain name, such as "Meeting notes to action items", so you can find it by searching.

Turn your best prompts into templates. Replace the parts that change each time with labeled placeholders in braces: {audience}, {tone}, {word limit}, {paste text here}. OpenAI's own prompt examples use the same convention for the variable part of a prompt. A template with clear slots is faster to fill in and harder to misuse than a prompt you have to edit by hand.

Finally, keep only the current best version on top, and the older ones underneath or in a separate tab. When a template stops working as well, perhaps after the tool changes, your history shows what you tried before, and you can start the next round of improvements from your best result rather than from scratch.

Sources

  1. ChatGPT: Optimizing Language Models for Dialogue — OpenAI, November 30, 2022
  2. Best practices for prompt engineering with OpenAI API — OpenAI Help Center

Get the free Guide to Artificial Intelligence

Sign up for the weekly newsletter and receive the free guide: what changed in AI this week, one lesson, one prompt to try. Unsubscribe in one click.

How we handle your address

Back to top

Display

Your choices are saved in this browser only. No data is sent.

Text size
100%
Colours
Reading
More settings
Accessibility statement