Vector Database

A vector database is a storage system purpose-built to hold embedding vectors and answer one question fast: which stored items are most similar to this one? Where a traditional relational database matches exact values in rows and columns, a vector database compares the geometric proximity of high-dimensional vectors, returning the items whose meaning is closest to a query. Ask it for "hashboard temperature fault" and it will surface a note titled "chain 2 overheating diagnosis" that shares almost no keywords — because the two sit near each other in embedding space.

How similarity search works

Every stored item — a document chunk, an image, a support ticket — is first converted into a vector by an embeddings model, typically a few hundred to a few thousand dimensions. The core operation is then nearest-neighbor search: given a query vector, find the stored vectors closest to it under a distance metric such as cosine similarity or Euclidean distance. Comparing a query against millions of vectors exhaustively is too slow for interactive use, so vector databases rely on Approximate Nearest Neighbor (ANN) indexes — most commonly HNSW, a layered graph structure navigated greedily from coarse to fine. ANN search skips the vast majority of candidates while returning results nearly identical to an exact scan, at a small fraction of the cost; the recall-versus-speed balance is tunable per query. Mature systems add metadata filtering (only documents from this manual, only tickets from this model family) and hybrid search that blends vector similarity with classic keyword scoring, which in practice beats either method alone.

The memory layer for private AI

Vector databases matter to self-hosters because they are the memory layer of retrieval-augmented generation. A language model's weights know nothing about your documents; RAG fixes that by embedding your corpus into a vector database, retrieving the most relevant chunks at question time, and handing them to a local LLM as context. The database is what makes the "retrieval" step fast and semantic rather than slow and literal. Retrieval quality can be further sharpened by a reranking stage that re-orders the candidates before they reach the model — the standard two-stage design for corpora where precision matters.

Running one on your own hardware

This is one of the easiest pieces of the sovereign AI stack to self-host. Capable open-source vector databases run entirely on local hardware, and for personal-to-small-business corpora — thousands to a few million chunks — commodity CPUs and modest RAM are entirely sufficient; no GPU is required for the search side. Several general-purpose databases have grown vector extensions as well, so a self-hoster may not even need a separate service. The practical workflow: chunk your documents sensibly, embed them with a locally run model, index, and query — your data, your index, your machine, with nothing leaving the premises. One craftsman's caveat: the index is downstream of the embedding model, so if you switch embedding models you must re-embed and re-index the whole corpus — vectors from different models live in incompatible spaces, and mixing them silently returns garbage.

For a concrete D-Central-flavored example: a repair bench that embeds years of diagnostic notes, board-revision quirks, and error-code writeups into a local vector database gets a searchable institutional memory that answers in seconds, works offline, and never ships a customer's serial numbers to a cloud API. That is the pattern worth copying — the database is small, boring, and reliable, and it turns a pile of documents into something a semantic search box or a local assistant can actually use. Like most sovereign infrastructure, its value compounds quietly: every document you add makes the next question cheaper to answer.