Definition
Lost in the middle is a documented failure pattern in which a language model retrieves and reasons over information most reliably when it sits near the beginning or end of a long input, and noticeably worse when the relevant fact is buried in the middle. The term comes from a 2023 study by Liu and colleagues that tested multi-document question answering and key-value retrieval, finding a characteristic U-shaped accuracy curve as the position of the answer moved through the context.
What the research showed
Performance degraded substantially when the model had to access information located in the middle of a long context, and the effect persisted even for models explicitly marketed as long-context. In other words, simply having a large window does not mean a model attends to all of it evenly. This is a positional bias baked into how attention distributes weight across a sequence.
Practical implications
For anyone building retrieval or summarization pipelines on their own hardware, the lesson is to place the most important material where the model actually reads it. Ordering retrieved passages by relevance, putting key instructions at the top and a restated question at the bottom, and avoiding the habit of dumping huge undifferentiated blobs into the prompt all measurably improve results. It is a reminder that a long context window is an opportunity, not an automatic win, and that prompt structure remains a lever you control.
Related reading: long context window and the needle in a haystack evaluation that probes the same weakness.
In Simple Terms
Lost in the middle is a documented failure pattern in which a language model retrieves and reasons over information most reliably when it sits near…