Definition
Top-p sampling, also called nucleus sampling, is a decoding strategy that controls the randomness of a language model's output. At each step the model produces a probability for every possible next token; top-p keeps only the smallest set of tokens whose cumulative probability reaches a threshold p (for example 0.9 or 0.95), then samples the next token from that set. The discarded long tail of unlikely tokens is never chosen, which suppresses incoherent text while still allowing variety.
Why it adapts better than top-k
The key advantage of top-p is that the size of the candidate pool — the 'nucleus' — changes with the shape of the distribution. When the model is very confident, one or two tokens may already exceed the threshold, so the pool is tiny and output is nearly deterministic. When the model is uncertain and probability is spread across many tokens, the pool grows, permitting more diversity. This adaptiveness is why nucleus sampling is a common default in many local inference tools.
Practical use
Top-p is usually combined with a temperature setting and sometimes with top-k. A lower p produces tighter, more predictable answers; a higher p produces more creative, varied output. For a self-hoster, these knobs are exposed directly by local runtimes, giving you full control over generation behaviour with no external service involved.
Compare with top-k sampling, which fixes the number of candidates instead, and beam search, a non-sampling alternative.
In Simple Terms
Top-p sampling, also called nucleus sampling, is a decoding strategy that controls the randomness of a language model’s output. At each step the model produces…