Definition
P-tuning is a parameter-efficient fine-tuning method that adapts a frozen language model by learning continuous prompt embeddings, but it differs from related methods in how those embeddings are generated and placed. Proposed by Liu and colleagues in 2021, it was designed for natural-language understanding (NLU) tasks and works across model types, notably helping GPT-style models compete with BERT-style models on understanding benchmarks.
How it works
Rather than optimizing the prompt vectors directly, P-tuning passes trainable virtual tokens through a small prompt encoder, typically a bidirectional LSTM, which produces the embeddings fed into the model. Two features distinguish it from prefix tuning: the prompt tokens can be inserted anywhere in the input sequence rather than only at the start, and they are added only to the input rather than to every layer. Optional anchor tokens, which mark meaningful parts of the input, can further improve results.
Why it matters
P-tuning showed that learned continuous prompts are far more reliable than hand-crafted discrete prompts, removing much of the brittle trial-and-error of manual prompt engineering. A later revision, P-tuning v2, extended trainable prompts to every layer to strengthen performance on harder tasks and smaller models, narrowing the gap with full fine-tuning.
P-tuning belongs to the soft prompt family and is often compared directly with prompt tuning, which uses a simpler input-only prompt without a dedicated encoder.
In Simple Terms
P-tuning is a parameter-efficient fine-tuning method that adapts a frozen language model by learning continuous prompt embeddings, but it differs from related methods in how…