Prompt Tuning

Prompt tuning is one of the simplest parameter-efficient fine-tuning methods: it freezes the entire pretrained model and learns only a small set of continuous prompt vectors that are prepended to the input. Introduced by Lester and colleagues at Google in 2021, it asks a modest question — if we cannot afford to retrain the model, how much can we accomplish by training only a few extra input tokens? — and the answer turned out to be: at scale, nearly everything.

Trainable input tokens

A normal text prompt is made of tokens drawn from the model's fixed vocabulary, each mapped to a fixed embedding. Prompt tuning drops that restriction: the prepended "soft" tokens are free-floating vectors in embedding space, not tied to any real word, and they have their own trainable parameters. During training, examples flow through the frozen model, and gradients update only these prompt-token embeddings — the model's billions of weights never change. The learned vectors need not correspond to anything a human could type; they are whatever directions in embedding space best steer the frozen model toward the task. A typical learned prompt is tens of tokens long, meaning the trainable parameter count is measured in tens or hundreds of thousands — vanishingly small against the base model.

Scaling behaviour

The original paper's most cited finding is that prompt tuning becomes more competitive as model size grows. On small models it clearly lags full fine-tuning; at multi-billion-parameter scale the gap closes and the method rivals tuning the whole model, while touching a minuscule fraction of the parameters. The intuition is that a large pretrained model already contains the capabilities most tasks need, so the job shrinks from "teach the model" to "address what it already knows" — and a learned prompt is an address, not a lesson.

The soft-prompt family

Prompt tuning is the lightest member of the broader soft prompt family. Prefix tuning injects trainable vectors into every transformer layer rather than only the input, giving more steering capacity at more cost; P-tuning generates its virtual tokens with a small encoder network and interleaves them with the input. All share the same core move — freeze the model, learn a conditioning signal — and differ in where and how the learnable vectors are placed. LoRA-style adapters, by contrast, modify the model's internal weight pathways directly, which generally wins on quality but produces heavier per-task artifacts.

Why self-hosters should care

For someone running an open-weight model on their own hardware, prompt tuning's economics are attractive. Because each downstream task needs only its own short learned prompt, one frozen base model can serve many tasks: keep a single copy of the weights in VRAM and swap kilobyte-scale prompt vectors per request — a support-ticket classifier, a log summarizer, a French translation helper — with no model reload. Training is similarly humble, fitting on hardware that could never full-fine-tune the same model. The trade-off is capacity: a soft prompt can steer and specialize, but it cannot teach the model genuinely new knowledge or unfamiliar formats the way heavier adaptation can. As with any tuning, the craftsman's rule applies — try the cheapest method first and escalate only when the evaluation, run on your actual task, says you must.

Prompt tuning also illustrates a broader shift in how adaptation is understood: the boundary between "prompting" and "training" is a spectrum, not a wall. A hand-written system prompt, a few-shot example set, a learned soft prompt, an adapter, and a full fine-tune are all points on one axis of increasing cost and increasing control. Sovereign builders benefit from thinking in those terms — the question is never "should I fine-tune?" but "what is the cheapest point on the spectrum that meets my measured bar?" Prompt tuning earns its place as the first learned method past hand-prompting: real gradients, real specialization, and a footprint small enough to email.