Definition
Beam search is a decoding algorithm that searches for a high-probability output sequence rather than sampling one token at a time. At each generation step it keeps the k most probable partial sequences — called the beam width or number of hypotheses — expands each one, and again prunes back to the best k. This lets it explore several promising paths in parallel and avoid the trap of greedy decoding, where committing to the single best token now can lead to a worse sequence overall.
Beam width and trade-offs
A beam width of 1 is just greedy decoding. Common values in machine translation and summarisation are 5 to 10. Wider beams explore more candidates and often find higher-likelihood sequences, but cost more compute and run slower. Counterintuitively, very large beams can actually produce worse, blander text, so width is tuned rather than maximised.
When it is used
Beam search shines on tasks with a clear 'correct' target — translation, transcription, structured generation — where you want the most likely answer and reproducibility matters. For open-ended chat and creative writing it tends to produce repetitive, generic output, which is why sampling methods are usually preferred there. As a self-hoster you choose per task: beam search for deterministic, high-precision jobs; sampling for variety.
Contrast it with the sampling approaches top-k sampling and top-p (nucleus sampling).
In Simple Terms
Beam search is a decoding algorithm that searches for a high-probability output sequence rather than sampling one token at a time. At each generation step…