Product Quantization

Product quantization (PQ) is a vector-compression technique for approximate nearest-neighbour search, introduced by Jegou, Douze, and Schmid in 2011. Instead of storing a full high-dimensional vector, PQ splits it into several lower-dimensional subvectors and quantizes each one separately against its own small codebook learned by k-means. The vector is then stored as a short list of centroid IDs, one per subvector, often just one byte each. For anyone running a local retrieval stack, PQ is the difference between an embedding index that fits in RAM and one that does not.

How the compression works

Suppose you have a 1,024-dimensional float vector. Split it into 8 subvectors of 128 dimensions, learn a 256-entry codebook for each subspace, and replace every subvector with the index of its nearest codebook centroid. The original 4,096-byte vector now fits in 8 bytes, a 512x compression. The full space of representable vectors is the Cartesian product of the per-subspace codebooks, hence the name: 256⁸ distinct codes from just 8 small tables. That is the trick that makes PQ dramatically better than plain vector quantization with a single codebook, which would need astronomically many centroids to describe the same space at similar fidelity.

Asymmetric distance computation

At query time, PQ estimates the distance between the raw float query and a compressed code without decompressing anything. The system precomputes a lookup table of distances from each query subvector to every centroid in its codebook, then scores each candidate by summing table entries, a handful of additions per vector. This asymmetric distance keeps the query at full precision while documents stay compressed, preserving more accuracy than quantizing both sides, and it turns distance computation into cache-friendly table lookups that scan millions of codes per second on ordinary CPUs.

What you give up

PQ is lossy. Each subvector snaps to its nearest centroid, so ranking quality degrades as compression tightens; the knobs are the number of subvectors and codebook size, trading bytes for recall. Standard practice compensates by over-fetching, retrieve a few hundred PQ-scored candidates, then re-rank the short list with exact float vectors or a stronger scorer such as a cross-encoder. Refinements like optimized PQ rotate the space before splitting so each subspace carries balanced information, squeezing out more accuracy at the same code size.

Why it matters for a sovereign stack

PQ is frequently paired with a coarse partitioning stage so queries scan only a fraction of the codes; that combination, IVF-PQ, described under the inverted file index (IVF), is a workhorse of large-scale open-source search libraries. The arithmetic is what makes self-hosted retrieval realistic: a million 1,024-dimensional float embeddings cost about 4 GB raw, but roughly 16 MB as 16-byte PQ codes, small enough for a modest home server or even the box already running your node to hold entire libraries of searchable knowledge in memory. Rather than renting a managed vector database and shipping your documents to someone else's cloud, PQ lets the whole index, and therefore the whole question of what you are reading and searching, stay on hardware you own.

Tuning PQ is mercifully mechanical. Hold out a set of queries with known-good results, then sweep the two knobs, subvector count and codebook bits, measuring recall at your over-fetch depth. Most corpora hit a knee where more compression starts costing real recall; sit just above it. Re-train codebooks when your embedding model changes, since codes are meaningless across models, and keep the raw vectors archived offline so you can always rebuild. An afternoon of this turns a research technique into settled infrastructure, which is exactly what a self-hosted stack needs its components to become.