In-Context Learning

In-context learning (ICL) is the ability of a large language model to perform a new task purely from information in its prompt, without any change to its underlying weights. Documented prominently in the 2020 GPT-3 paper "Language Models Are Few-Shot Learners," it is the mechanism that makes prompting feel like programming: you describe or demonstrate the task, and the model adapts on the fly.

Zero-shot, one-shot, and few-shot

The GPT-3 work framed ICL along a spectrum based on how many worked examples (demonstrations) you include. Zero-shot gives the model only a task description and no examples. One-shot provides a single example. Few-shot provides several, typically ten to a hundred that fit the context window. Generally, more relevant demonstrations improve accuracy, and larger models exploit those examples more effectively. The model is not "learning" in the training sense; it is conditioning its output on the pattern you supplied.

Why it matters

In-context learning is what lets a single downloaded model serve countless tasks without retraining, a major practical advantage for anyone running models on their own hardware. It also means prompt design directly governs quality, and that everything you place in context, trusted or not, shapes the response, which is why prompt-injection risk exists.

In-context learning operates entirely within the context window and at inference time, distinguishing it from fine-tuning, which changes weights. It is the foundation for chain-of-thought prompting, and its openness to injected text underlies prompt injection.