Embedding Dimension

The embedding dimension (or vector dimensionality) is the number of numeric values used to represent each item in an embedding. A 384-dimensional embedding describes every data point as a list of 384 numbers; a 1,536-dimensional one uses 1,536. It is the size of the vector space into which a model maps its inputs, and it is one of the few model parameters that a self-hoster feels directly in disk bytes, RAM, and query latency — every stored item and every comparison scales with it linearly.

The core trade-off

Dimensionality balances expressiveness against efficiency. More dimensions give the model more axes along which to separate meanings — finer distinctions between similar words, subtler visual detail, more nuance preserved — but every added dimension costs memory, storage, and compute for every single comparison, forever. Too few dimensions and the space gets crowded: genuinely different items are forced near each other because there is nowhere else to put them, and retrieval starts confusing things that should be distinct. Too many and you pay for capacity the data never uses, while distances themselves become less discriminating — in very high-dimensional spaces, the gap between the nearest and farthest neighbor shrinks relative to the distances involved, a facet of the curse of dimensionality that quietly erodes what "similar" means. The practical middle ground is why common text-embedding models cluster in the few-hundred-to-low-thousands range rather than growing without bound.

Concrete arithmetic for self-hosters

If you are building a local vector database or retrieval-augmented system, the dimension sets your budget. A vector of D float32 values costs 4D bytes: 384 dimensions is about 1.5 KB per item, 1,536 dimensions about 6 KB — so one million chunks of a personal document archive costs roughly 1.5 GB versus 6 GB of raw vectors before index overhead. Query cost scales the same way, since cosine similarity touches every component: doubling the dimension roughly doubles both the disk per vector and the work per comparison. Two mitigations are worth knowing. Quantization stores each component in fewer bits (int8 or less), cutting memory several-fold at a small accuracy cost. And some newer models are trained so their dimensions are ordered by importance, letting you truncate vectors to a prefix — keeping, say, the first 256 of 1,024 dimensions — and trade precision for speed after the fact without re-embedding.

Choosing well

The right dimension depends on the data's complexity and the hardware you intend to run on — and the honest method is empirical. Benchmark a smaller-dimensional model against a larger one on your own retrieval task; for many personal-scale corpora, the difference in answer quality is modest while the difference in resource use is not. A model whose index fits in RAM will outperform a "better" one that thrashes disk on every query. Picking a dimension that fits your machine is a practical sovereignty concern: it determines whether your AI stack runs comfortably on hardware you control or quietly pressures you back toward someone else's API. Note one detail that trips people: dimensions are not comparable across models — a 768-dimensional vector from one encoder shares no geometry with a 768-dimensional vector from another, so a vector store is married to the exact model that filled it, and changing models means re-embedding everything.

The embedding dimension is the size of the model's latent space, and comparing vectors within it is usually done with cosine similarity. Size it to your hardware, measure on your data, and let the smallest dimension that answers well win. As with sizing a PSU or a breaker circuit, the specification exists to serve the load — so measure the load first, then buy the number.