Definition
Selective state space is the core innovation of the Mamba architecture, sometimes called the S6 mechanism. Classical state-space models process a sequence with fixed parameters applied uniformly to every position, which makes them efficient but unable to reason about content: they cannot decide that one token matters and another does not. Selective state space fixes this by making the key state-space parameters functions of the current input rather than constants, turning a time-invariant system into a time-varying one.
What "selective" buys
Because the transition and input-projection terms now depend on the token being read, the model can selectively propagate or forget information along the sequence depending on what each token contains. In effect it gains a content-based gate, the ability previously associated mainly with attention, to focus on relevant history and discard the rest. This is what lets Mamba match or beat transformers of comparable size while keeping linear scaling in sequence length.
The engineering catch
Making the parameters input-dependent breaks the efficient convolution that fixed-parameter state-space models rely on for fast training. Mamba's authors recovered the lost speed with a hardware-aware parallel scan, an algorithm that computes the recurrence in a way tuned to how a GPU moves data through its memory hierarchy. The result keeps the linear-time, constant-memory inference of a recurrent model while training nearly as efficiently as a parallel one.
Selective state space is the engine behind Mamba and its descendants, and it underpins hybrid systems like Jamba. For related concepts, see state space duality, sub-quadratic attention, and Jamba.
In Simple Terms
Selective state space is the core innovation of the Mamba architecture, sometimes called the S6 mechanism. Classical state-space models process a sequence with fixed parameters…