Chain-of-Thought Prompting

Chain-of-thought (CoT) prompting is a technique that improves a language model's reasoning by encouraging it to generate intermediate steps before committing to a final answer. Introduced by Wei and colleagues at Google in 2022, it dramatically raised performance on tasks like arithmetic word problems, logical reasoning, and multi-hop question answering.

How it works

Instead of asking for an answer directly, you prompt the model to "think step by step" or you supply examples that show the reasoning written out. The model then produces a sequence of intermediate "thoughts" that decompose the problem before stating the result. Because each generated token conditions the next, spelling out the reasoning gives the model more computation and a scaffold to follow, which reduces errors on problems that fail under direct prompting.

An emergent behavior

A notable finding is that chain-of-thought gains are largely emergent: they appear in large models (roughly 100 billion parameters and up) and do little for small ones. Later "reasoning" models internalize CoT during training so they produce extended reasoning automatically, but the prompting technique remains a simple, free lever for any capable model. CoT also makes a model's logic legible, which helps users catch mistakes, though a stated rationale is not a guarantee of a correct or faithful one.

Chain-of-thought is a form of in-context learning applied at inference time, and longer reasoning chains consume more of the context window. Verbose intermediate steps do not prevent a model from confidently producing a hallucination, so outputs still warrant verification.