Dense vs Sparse Retrieval

Dense and sparse retrieval are the two foundational ways to match a query against a document corpus, and understanding the contrast is essential before you build any self-hosted search system. The difference is in how each represents text as a vector.

Sparse retrieval

Sparse vectors have one dimension per vocabulary term, so they are huge (tens of thousands of dimensions) but almost entirely zero, with non-zero weights only for terms that appear. Classic sparse retrieval is lexical: BM25 and TF-IDF score documents by exact term overlap, weighting rare terms more heavily. Sparse search is fast, memory-light, fully interpretable (you can see exactly which words matched), and needs no GPU. Its weakness is zero semantic understanding: a search for "PSU fault" will not match "power supply failure" unless the words literally appear. Learned sparse methods such as SPLADE close part of that gap by using a language model to expand each document with related terms while keeping the vector sparse.

Dense retrieval

Dense vectors are low-dimensional (hundreds to low thousands) and mostly non-zero. Produced by a neural encoder, they place semantically similar text near each other in vector space, so paraphrases and synonyms match even with no shared words. The trade-offs are the mirror image of sparse: dense retrieval captures meaning but is opaque, heavier to compute, and can miss exact rare strings such as a part number or error code.

Hybrid is usually best

Because their failure modes are complementary, production systems typically run both and fuse the scores. Hybrid retrieval reliably beats either method alone, which is why a self-hosted stack worth building combines lexical and semantic signals.

For the indexing techniques that make dense search scale on your own hardware, see our entries on the inverted file index (IVF) and late interaction (ColBERT).