Definition
Prompt tuning is one of the simplest parameter-efficient fine-tuning methods: it freezes the entire pretrained model and learns only a small set of continuous prompt vectors that are prepended to the input. Introduced by Lester and colleagues at Google in 2021, it was developed for tasks cast as text generation on T5-style models.
Trainable input tokens
The core idea is that the added prompt tokens have their own parameters, updated independently of the frozen base model. During training, gradients flow only into these prompt-token embeddings; the model's billions of weights never change. Unlike prefix tuning, which injects trainable vectors into every transformer layer, prompt tuning adds parameters only at the input embedding layer, making it the most parameter-light of the major soft-prompt approaches.
Scaling behaviour
A notable finding from the original paper is that prompt tuning becomes more competitive as model size grows. On small models it can lag full fine-tuning, but at multi-billion-parameter scale it closes the gap and rivals tuning the whole model, while training a tiny fraction of the parameters. Because each downstream task needs only its own short learned prompt, one frozen base model can be reused across many tasks, which keeps storage and deployment costs low for self-hosted setups.
Prompt tuning is a member of the broader soft prompt family, alongside prefix tuning and P-tuning, each of which differs in where and how the learnable vectors are placed.
In Simple Terms
Prompt tuning is one of the simplest parameter-efficient fine-tuning methods: it freezes the entire pretrained model and learns only a small set of continuous prompt…