What is chain-of-thought prompting?

Ask a large language model a simple fact and it usually answers well. Ask it a word problem with three steps and it often blurts out a confident wrong number. Two research papers from this year show that a small change in the prompt makes a large difference: let the model write out its reasoning first, and it gets many more of these problems right.

That change is called chain-of-thought prompting. A chain of thought is simply the series of in-between steps that lead from a question to its answer, written in ordinary language, the way you might talk yourself through a problem on scrap paper.

Where the idea comes from

In January, Jason Wei, Denny Zhou and colleagues at Google Research posted a paper titled "Chain of Thought Prompting Elicits Reasoning in Large Language Models". They took the usual few-shot prompt, where the model sees a handful of solved examples before a new question, and changed one thing: each example answer showed its working instead of only the final number.

Their own illustration makes the difference clear. With plain examples, a model asked about a cafeteria that had 23 apples, used 20 and bought 6 more answered 27. With step-by-step examples, it wrote that 23 minus 20 leaves 3, that 3 plus 6 makes 9, and answered 9, which is correct.

The headline result came from Google's 540-billion-parameter PaLM model. With eight hand-written step-by-step examples, it reached a new best score on GSM8K, a set of grade-school math word problems, beating even a version of GPT-3 that had been specially trained for the task. Google's PaLM announcement in April reported 58% of these problems solved, close to the 60% average it cites for children aged 9 to 12.

Then, in late May, Takeshi Kojima and colleagues at the University of Tokyo and Google showed that you may not need the examples at all. Adding the single sentence "Let's think step by step" after the question was enough to trigger a chain of thought.

A worked example you can run

Here is a fresh word problem, first asked the plain way:

Q: A bakery bakes 4 trays of 12 muffins. It sells 30 muffins and gives 5 to a food bank. How many muffins are left?
A: The answer is

The prompt forces the model to commit to a number immediately. In Kojima's tests, this answer-at-once format solved only 17.7% of a set of multi-step arithmetic problems with text-davinci-002. Now the same question with the cue:

Q: A bakery bakes 4 trays of 12 muffins. It sells 30 muffins and gives 5 to a food bank. How many muffins are left?
A: Let's think step by step.

A good response now walks through the steps:

If you need just the number for a spreadsheet or an app, Kojima's team used a second short prompt: paste the reasoning back in and add "Therefore, the answer (arabic numerals) is" to pull out a clean final answer.

Limits worth knowing

Chain-of-thought prompting is not a cure-all. Wei's team found that it only helps very large models, around 100 billion parameters and up; smaller models wrote fluent but illogical steps and did worse than with plain prompts. The gains were biggest on harder, multi-step problems and small or even negative on one-step ones. And the written reasoning can be wrong: when the researchers examined 50 wrong answers from one model, about half the chains had only a minor slip, such as an arithmetic error, but the rest had serious mistakes in understanding the problem.

Why does 'let's think step by step' help?

The simplest explanation is space. A language model produces one token at a time, and each new token is shaped by everything written before it. When a prompt demands the answer immediately, the model has to land on it in a single move. When it writes the steps first, each step becomes part of the text it reads next, so the final answer can build on worked-out pieces instead of a leap. Wei's paper describes this as letting the model spend more computation on problems that need more steps.

A second likely reason is that the phrase points the model toward a familiar kind of text: worked explanations, in which ordered reasoning comes before the conclusion. The model then continues in that style. The exact wording matters: Kojima's team tested eight variations with text-davinci-002 on a set of arithmetic problems. "Let's think step by step" scored best, lifting accuracy from 17.7% to 78.7%, while "Let's think about this logically" reached 74.5% and phrases that did not invite reasoning did much worse.

It is not magic, and it does not turn a model into a calculator. Hand-written step-by-step examples still beat the one-line cue, and the model can still write convincing steps that lead to a wrong answer. The practical value is that the steps are visible, so you can check them, which is a good reason to use the cue whenever a question has more than one moving part.