In-Context Learning

In-context learning (ICL) is the ability of a large language model to perform a new task purely from information in its prompt, without any change to its underlying weights. Documented prominently in the 2020 GPT-3 paper "Language Models Are Few-Shot Learners," it is the mechanism that makes prompting feel like programming: you describe or demonstrate the task, and the model adapts on the fly. Nothing is trained, nothing is saved — the "learning" lives entirely inside a single conversation.

Zero-shot, one-shot, and few-shot

The GPT-3 work framed ICL along a spectrum based on how many worked examples (demonstrations) the prompt includes. Zero-shot gives the model only a task description: "translate this sentence into French." One-shot adds a single worked example. Few-shot provides several — anywhere from a handful to dozens, bounded by what fits in the context window. Generally, more relevant demonstrations improve accuracy, and larger models exploit examples more effectively; the ability itself strengthens markedly with scale. Formatting matters too: consistent structure across demonstrations often helps as much as their content, because the model is locking onto a pattern.

What is actually happening

The model is not learning in the training sense — gradient descent never runs, and the weights that encode its knowledge are untouched. Instead, the transformer's attention mechanism conditions every generated token on the entire prompt, so demonstrations act as evidence about which of the many behaviors the model already contains you want right now. A useful mental model: pretraining built a vast library of latent skills; the context selects and configures one. That also explains ICL's limits — it can steer and combine what the model knows, but it cannot teach genuinely new knowledge the way fine-tuning can, and everything it "learned" evaporates when the context ends.

Why it matters to a self-hoster

In-context learning is what lets a single downloaded model serve countless tasks without retraining — a major practical advantage when the model runs on your own hardware through Ollama or llama.cpp. Instead of maintaining a fine-tuned variant per job, you maintain a library of prompts: one turns the model into a log-file triager for your miner fleet, another into a French translator, another into a JSON extractor. It is also the foundation that retrieval systems build on: RAG works precisely because a model can absorb retrieved documents in-context and reason over them immediately, and chain-of-thought prompting is ICL applied to reasoning style.

Treat those prompts as engineering artifacts, because that is what they are. A prompt that encodes a task deserves version control, a fixed set of test inputs, and a diff review when it changes — the same discipline as any configuration that drives production behavior. More advanced setups select demonstrations dynamically: rather than hard-coding examples, they retrieve the most similar solved cases from a library at request time and insert them into the prompt, so the few-shot examples are always relevant to the input at hand. That pattern — retrieval feeding demonstration — blurs into RAG, and it is one of the highest-leverage tricks available on local hardware, where you control both the example library and the model consuming it.

The double edge

The same openness that makes prompting powerful makes it exploitable: the model conditions on everything in context, trusted or not, which is why prompt injection exists — a malicious instruction embedded in a retrieved document is, to the model, just more context shaping its behavior. Prompt design therefore governs quality and safety simultaneously. Treat the context window the way a machinist treats stock in the chuck: whatever you clamp in is what gets machined, so be deliberate about every piece.