Foundation Model

A foundation model is, in the definition coined by Stanford's Center for Research on Foundation Models in 2021, any model trained on broad data — generally using self-supervision at scale — that can be adapted to a wide range of downstream tasks. The term captures a shift in how AI systems are built: rather than training a fresh model for each task, practitioners start from one large, general-purpose model and specialise it.

Why the name

The word "foundation" is deliberate. Such a model is itself incomplete — it is not designed for any single application — but it serves as the common base from which many task-specific systems are built through adaptation, fine-tuning, or prompting. Existing labels like large model, pretrained model, or self-supervised model each describe one technical facet but miss the broader role these models play as shared infrastructure.

Characteristics and trade-offs

Foundation models are typically very large neural networks — often billions to trillions of parameters — trained on enormous, broad datasets using self-supervised objectives that need no human labels. This scale yields striking generality and emergent capabilities, but it also concentrates capability, cost, and risk: training is expensive, behaviour can be hard to audit, and reliance on a handful of providers raises sovereignty concerns. Open-weight foundation models are central to the case for running capable AI on hardware you control.

Large language models are the best-known foundation models, but the category spans vision, audio, and mixed inputs. See the multimodal AI model for cross-modality systems, and the State Space Model (Mamba) for an efficient architecture used to build them.