Definition
An adapter is a small, trainable neural module inserted between the layers of a large pretrained model to adapt it to a new task without retraining the whole network. First introduced for transformers by Houlsby et al. in 2019, adapters keep the original model weights frozen and learn only the inserted modules, making them a foundational parameter-efficient fine-tuning technique.
The bottleneck architecture
A classic adapter uses a bottleneck design: it first projects the layer's high-dimensional representation down to a much smaller dimension, applies a non-linear activation, then projects it back up to the original size, with a residual connection around the whole block. Because the down- and up-projection matrices are tiny relative to the model, an adapter adds only a small number of trainable parameters — Houlsby's work reported matching full fine-tuning accuracy while training roughly half as many task parameters as competing methods, with no change to the base model's structure.
Why adapters are useful
Adapters are modular and composable. You can train a separate adapter for each task or domain and swap them in and out at inference time over a single shared base model, which is far cheaper than maintaining a full fine-tuned copy per task. Follow-up methods such as AdapterFusion and Compacter extend the idea.
Adapters sit alongside LoRA as a core PEFT approach, and both are inserted into the transformer blocks of the underlying model rather than altering its pretrained weights.
In Simple Terms
An adapter is a small, trainable neural module inserted between the layers of a large pretrained model to adapt it to a new task without…