Definition
State space duality (SSD) is the theoretical framework introduced with Mamba-2 that proves a deep equivalence between certain state-space models and a form of masked attention. It shows that the same sequence transformation can be computed two ways: as a linear-time recurrence that carries a fixed-size state, or as a quadratic-time attention-like matrix multiplication. The two are not approximations of each other; they are exactly the same operation expressed through different algebra.
The structured-matrix bridge
The connection runs through structured matrices. A state-space model whose state-transition is a scalar times the identity turns out to be equivalent to masked self-attention using a special structured mask (a 1-semiseparable matrix) that encodes the temporal decay of information. That matrix view is what reveals the duality: the recurrence and the attention computation are two factorizations of one structured matrix. This unifies two research lineages, state-space models and linear attention, that had developed largely in parallel.
Why it is useful
The practical payoff is speed and flexibility. Because the dual form is a structured matrix multiplication, Mamba-2 can route its work onto the highly optimized matrix-multiply hardware that modern GPUs are built around, rather than relying solely on a custom scan. Implementers can also choose the linear recurrence for cheap long-context inference or the quadratic form for efficient parallel training, picking whichever is faster for a given setting.
State space duality ties together several entries in this glossary. See selective state space, linear attention, and gated linear attention for the families it unifies.
In Simple Terms
State space duality (SSD) is the theoretical framework introduced with Mamba-2 that proves a deep equivalence between certain state-space models and a form of masked…