Hybrid Search

Hybrid search runs two complementary retrieval strategies in parallel and merges their results: a sparse keyword search (typically BM25) and a dense vector search over embeddings. Dense and sparse retrievers fail in opposite ways, so combining them catches what either alone would miss. It is a foundational pattern for self-hosted retrieval pipelines where you want both precision and recall without depending on a single black-box ranker — and it is usually the first upgrade worth making once a plain vector search starts missing obvious documents.

Why two retrievers beat one

BM25 excels at exact-match queries: product codes, part numbers, named entities, rare technical terms, and acronyms an embedding model has never meaningfully internalized. Dense vector search handles conceptual and paraphrased queries where the user's wording differs from the document's. The failure modes are mirror images. Ask a vector index for "BM1370 error code" and the embedding of an alphanumeric chip ID may land nowhere useful, while BM25 nails the literal string; ask BM25 "why does my hashboard run hot" and it misses the document titled "thermal management for ASIC boards," which the embedding matches easily. On a real technical corpus — a repair knowledge base full of chip IDs, error codes, and conceptual troubleshooting prose — queries of both kinds arrive constantly, which is why hybrid is the practical default rather than an optimization.

Fusing the results

The two retrievers produce scores on incompatible scales — BM25 scores are unbounded and corpus-dependent, cosine similarities live in a narrow band — so naive score mixing is fragile. Results are usually merged with a rank-based method such as Reciprocal Rank Fusion, which ignores raw scores entirely and combines documents by their position in each ranked list: simple, tuning-free, and robust. Run the two searches concurrently rather than sequentially, otherwise you double retrieval latency for no benefit. A reranking pass — a cross-encoder scoring each candidate against the query — can then refine the merged list, trading a little compute for a meaningful precision gain on the handful of passages that will actually reach the model's context window.

Building it sovereign

Tuning the balance

Hybrid pipelines have refreshingly few knobs, but the ones that exist repay attention. Rank fusion can be weighted toward the sparse or dense side when your corpus clearly favors one — a parts database leans keyword, a prose wiki leans semantic. Evaluate with a small gold set of real queries from your own logs rather than generic benchmarks; twenty representative queries with known-correct documents will expose more truth than any leaderboard. Domain vocabulary is the usual weak point: if your embedding model has never seen your jargon, the dense side underperforms and deserves a lower weight until you swap in a better-suited model. Re-run the same gold set after every such change, and keep the previous scores — retrieval tuning without measurement is just superstition with extra steps.

Every component of a hybrid pipeline runs comfortably on your own hardware: an open-source search engine or library for the BM25 side, a self-hosted vector database for the dense side (many now ship hybrid capability in one system), a local embedding model, and fusion logic that is a dozen lines of code. That matters for the same reason running your own node does — a private knowledge base feeding a local LLM should not route your queries and documents through anyone else's ranker, and hybrid's redundancy means you are not betting retrieval quality on any single similarity metric or model's blind spots. Quality still starts upstream: thoughtful chunking determines what either retriever can find at all. Pair all of it together and you have a robust on-premise RAG stack — keyword precision, semantic recall, and no external dependencies.