Definition
A long context window refers to a language model whose maximum input length stretches well beyond the few-thousand-token windows of early transformers, often reaching 128K, 200K, or even one million tokens. This lets the model ingest entire books, multi-file codebases, or long conversation histories in a single forward pass instead of relying on external retrieval to feed it small chunks at a time.
Why length is not the whole story
Advertised window size is an upper bound on what a model can read, not a guarantee of what it can use. A model rated for a million tokens may still attend unevenly across that span, weighting the start and end of the input more than the middle. The practical, usable context is therefore often shorter than the headline number, and it is measured with retrieval and reasoning evaluations rather than assumed from the specification.
Cost and engineering trade-offs
Long windows are expensive. Self-attention scales quadratically with sequence length unless mitigated, and the per-token memory of cached keys and values grows linearly, so a long prompt can dominate both latency and GPU memory. Techniques such as sparse and sliding attention, cache quantization, and context compression exist specifically to make long windows affordable. For a sovereign operator running models on owned hardware, the window you can realistically serve is bounded by your VRAM, not by the model card.
See the context window for the underlying concept and the KV cache for the memory structure that grows with it.
In Simple Terms
A long context window refers to a language model whose maximum input length stretches well beyond the few-thousand-token windows of early transformers, often reaching 128K,…