Definition
Logit bias is a per-token override applied during decoding that nudges or forbids specific tokens regardless of what the model would otherwise predict. It gives an operator surgical control over generation: ban a word entirely, force a particular answer token, or steer away from a topic. Because it works at the lowest level of the sampling pipeline, it is one of the most precise levers available when self-hosting a model.
How it works
The model emits a vector of logits, one raw score per vocabulary token, just before the softmax converts them to probabilities. A logit bias adds a fixed value to chosen token IDs at that point. A large positive bias makes a token almost certain to be selected; a large negative bias (often negative infinity in practice) effectively removes it from the vocabulary for that generation. The adjustment is applied every step, so it persists across the whole sequence.
Practical uses
Operators use logit bias to enforce a closed set of answers (for example biasing only the tokens for "yes" and "no"), to suppress slurs or banned strings, or to discourage a model from emitting end-of-sequence too early. Because the bias is blunt and static, it can backfire: forcing a token the context does not support produces incoherent text. It is therefore best reserved for narrow, well-understood constraints rather than broad stylistic steering.
For schema-level control rather than single-token control, see Grammar-Constrained Decoding, and for the looping problem it can help fix, see Repetition Penalty.
In Simple Terms
Logit bias is a per-token override applied during decoding that nudges or forbids specific tokens regardless of what the model would otherwise predict. It gives…