Best Local Embedding Models for RAG (2026): Self-Hosted & Ollama-Ready
Before a local AI can answer questions about your own notes, contracts or codebase, something has to turn that text into vectors it can search. That something is an embedding model — the first and most under-discussed layer of a self-hosted RAG stack. This is a comparison of the open, self-hostable embedding models worth running in 2026: what dimensions they output, how much text they take, how they score on MTEB (with the exact benchmark named, because the numbers are not interchangeable), how much VRAM they need, and which ones you can pull straight from Ollama.
Quick answer
A self-hosted RAG system ("chat with your own documents") has three layers: an EMBEDDING model that turns text into vectors, a vector database that stores and searches them, and a local LLM that writes the answer. This table compares the first layer — 20 open, self-hostable text-embedding models — by embedding dimensions, context length, MTEB benchmark score, VRAM and whether you can pull them straight from Ollama. It is sorted by VRAM (what fits your hardware), smallest first: bigger is not automatically better — a small model that fits your GPU and indexes fast often beats a leaderboard giant you can barely run.
Honest picks for 2026: for the easiest local start, nomic-embed-text pulls from Ollama and runs in ~0.3 GB; the best quality-per-VRAM is Qwen3-Embedding-0.6B (70.7 on MTEB-eng-v2, Apache-2.0, ~1.5 GB, Ollama-native), scaling up to Qwen3-Embedding-8B if you have the GPU; for multilingual + hybrid dense/sparse retrieval, bge-m3 (MIT, 8K context, 100+ languages); the lightest CPU-only option is all-MiniLM-L6-v2 (~0.1 GB). One trap: NV-Embed-v2 and jina-embeddings-v3 top their boards but are CC-BY-NC — NOT usable for commercial RAG. And MTEB scores are only comparable WITHIN the same benchmark (English 56-task vs MTEB-eng-v2 vs multilingual MMTEB are different scales — read the benchmark in each cell). Free CSV/JSON under CC BY 4.0.
Download CSV Download JSON REST API →
Ollama = one-command ollama pull. MTEB scores carry their benchmark; compare only within the same one. ~ = lower-confidence.
| Model | Params | Dims | Context | MTEB | VRAM | Ollama | License | Best for |
|---|---|---|---|---|---|---|---|---|
| all-MiniLM-L6-v2 sentence-transformers | 22.7M | 384 | 256 | 56.26 MTEB(eng) ORIGINAL 56-task mean ~56.26 (leaderboard, NOT on card; card lists only ArguAna 50.17) | 0.1 GB | all-minilm | Apache-2.0 | Ultra-light CPU-only RAG, prototyping, very high throughput; weakest accuracy + short 256-token limit. Ollama all-minilm:latest = :22m = this (L6-v2); :33m = L12-v2 |
| nomic-embed-text-v1.5 Nomic AI | 137M | 768 ↓ 512, 256, 128, 64 | 8,192 | 62.28 MTEB(eng) ORIGINAL 56-task mean = 62.28 (full 768-dim) | 0.3 GB | nomic-embed-text | Apache-2.0 | Default general-purpose English RAG on modest hardware; long 8K docs + MRL dimension truncation. NB: Ollama shows a 2K default num_ctx but the model max is 8192 — raise num_ctx |
| granite-embedding-278m-multilingual ML IBM | 278M | 768 | 512 | 48.2 MTEB(eng) Retrieval mean = 48.2; MIRACL 18-lang mean = 58.3 (IBM Granite Embedding paper) — retrieval-only, NOT full MTEB means | 0.6 GB | granite-embedding:278m | Apache-2.0 | Lightweight, truly-open (Apache-2.0) multilingual retrieval; tiny+fast, but only 12 languages and 512-token window. MUST pull granite-embedding:278m — bare/:latest/:30m is the 30M English-only model |
| paraphrase-multilingual-mpnet-base-v2 ML sentence-transformers | 278M | 768 | 512 | — Not on modern MTEB/MMTEB leaderboards — a legacy STS/paraphrase model predating MTEB-era retrieval benchmarking (expect materially weaker retrieval than E5/BGE/Granite) | 0.6 GB | paraphrase-multilingual | Apache-2.0 | Tiny, CPU-friendly multilingual semantic similarity / clustering / dedup; pull-and-go on Ollama (paraphrase-multilingual:278m), but weak for precision RAG. Model max 512 tokens though ST truncates to 128 by default |
| mxbai-embed-large-v1 mixedbread.ai | 335M | 1024 ↓ 512, 256 (MRL; supports arbitrary truncate_dim + binary/int8 quant) | 512 | 64.68 MTEB(eng) ORIGINAL 56-task mean = 64.68 | 0.7 GB | mxbai-embed-large | Apache-2.0 | Strong English retrieval; was SOTA for BERT-large size (Mar 2024); needs a retrieval query prompt prefix ('Represent this sentence for searching relevant passages:') |
| bge-large-en-v1.5 BAAI | 335M | 1024 | 512 | 64.23 MTEB(eng) ORIGINAL 56-task mean = 64.23 | 0.7 GB | bge-large | MIT | Proven, permissive (MIT, commercial-OK) English RAG baseline; short 512-token chunks; prepend the retrieval instruction to queries. Ollama bge-large:335m = this model |
| nomic-embed-text-v2-moe ML Nomic AI | 475M total / 305M active | 768 ↓ 256 (768->256) | 512 | 65.8 MIRACL nDCG@10 mean = 65.80 (multilingual); BEIR = 52.86 (English) — model-card retrieval numbers, NOT an MTEB mean | 1 GB | nomic-embed-text-v2-moe | Apache-2.0 | Multilingual RAG wanting big-model retrieval quality at small active-param (305M) inference cost; first general-purpose MoE embedder |
| multilingual-e5-large-instruct ML Microsoft (intfloat) | 560M | 1024 | 512 | 63.2 MMTEB multilingual mean ~63.2 (132 tasks) — ranked #1 among open models in the MMTEB paper | 1.1 GB | — | MIT | Top open MMTEB-multilingual retrieval at 560M; needs a query instruction prefix for peak quality; 512-token window. Only community (not official) Ollama uploads |
| multilingual-e5-large ML Microsoft (intfloat) | 560M | 1024 | 512 | 58.6 MMTEB multilingual mean ~58.6 (132 tasks) — ranked 4th in the MMTEB paper | 1.1 GB | — | MIT | Proven permissive multilingual RAG workhorse; no instructions, just query:/passage: prefixes; the safe MIT default — but 512-token window and only community (not official) Ollama uploads |
| bge-m3 ML BAAI | 568M | 1024 | 8,192 | 59.56 No single MTEB scalar on card — headline is MIRACL(18-lang)+MKQA(25-lang) SOTA. Community-reported MMTEB multilingual mean ~59.56. Uniquely does dense + sparse + ColBERT multi-vector in one model | 1.2 GB | bge-m3 | MIT | The self-host multi-vector specialist: ONE model doing dense + sparse (lexical) + native ColBERT-style late-interaction, 8K context — ideal for hybrid RAG + rerank. Ollama tag bge-m3:567m |
| jina-embeddings-v3 ML Jina AI | 570M | 1024 ↓ 32-1024 | 8,192 | 65.52 MTEB English mean ~65.52 (jina-reported, ~56-task original MTEB) — NOT multilingual MMTEB | 1.2 GB | — | CC-BY-NC-4.0 non-commercial | Excellent all-round multilingual dense retrieval: 8K context, task-LoRA + Matryoshka to 32d — BUT CC-BY-NC-4.0 blocks commercial use (contact Jina); late-interaction is a separate model (jina-colbert-v2) |
| snowflake-arctic-embed-l-v2.0 ML Snowflake | 568M total (303M non-embedding) | 1024 ↓ 256 (MRL + quantization-aware; ~128 bytes/vector) | 8,192 | 55.6 MTEB(eng) Retrieval nDCG@10 ~55.6 (retrieval-only, NOT a full MTEB mean); multilingual MIRACL nDCG@10 ~64.9 | 1.2 GB | snowflake-arctic-embed2 | Apache-2.0 | Multilingual long-context RAG that must not sacrifice English retrieval; compressible (MRL+quant) vectors. Ollama tag snowflake-arctic-embed2:568m |
| Qwen3-Embedding-0.6B ML Alibaba (Qwen) | 0.6B | 1024 ↓ 32-1024 (MRL, user-defined output dim) | 32,768 | 70.7 MTEB(eng, v2) mean = 70.70 (card table); MMTEB multilingual mean = 64.33 | 1.5 GB | qwen3-embedding:0.6b | Apache-2.0 | Best quality-per-VRAM small embedder — 70.7 MTEB(eng,v2) at ~1.5GB fp16, 32K context, Ollama-native; strong CPU/edge RAG |
| gte-large-en-v1.5 Alibaba (Tongyi) | 434M | 1024 | 8,192 | 65.39 MTEB(eng) ORIGINAL 56-task mean = 65.39 (SOTA within size class at release) | 1.5 GB | — | Apache-2.0 | Best small-footprint 8K-context English encoder — cheap/fast to serve at ~1.5GB; ideal high-throughput RAG default. Not in the official Ollama library |
| stella_en_1.5B_v5 NovaSearch | 1.5B | 1024 ↓ 512, 768, 1024, 2048, 4096, 6144, 8192 (separate trained projection heads; 1024 recommended) | 512 | 69.43 MTEB(eng, v2) mean ~69.43 (leaderboard; NOT on card). On the OLDER 56-task board it was ~71.19 — different benchmark, don't mix | 3.5 GB | — | MIT | Efficient English RAG with flexible output dims — near-7B quality at ~1.5B/3.5GB; but only 512-token inputs (chunk small); no official Ollama tag |
| Qwen3-Embedding-4B ML Alibaba (Qwen) | 4B | 2560 ↓ 32-2560 (MRL, user-defined output dim) | 32,768 | 74.6 MTEB(eng, v2) mean = 74.60 (card table); MMTEB multilingual mean = 69.45 | 9 GB | qwen3-embedding:4b | Apache-2.0 | Mid-tier multilingual RAG when 0.6B underfits but a 7-8B model won't fit the GPU; ~9GB fp16 |
| gte-Qwen2-7B-instruct ML Alibaba (Tongyi) | 7B | 3584 | 32,768 | 70.24 MTEB(eng) ORIGINAL 56-task mean = 70.24 (Jun 16 2024); C-MTEB (Chinese, 35 tasks) = 72.05 — not comparable to v2 boards | 15 GB | — | Apache-2.0 | Strong bilingual EN/ZH retrieval, 32K context, commercial-friendly Apache-2.0 (2024-era SOTA); no official Ollama tag |
| e5-mistral-7b-instruct Microsoft (intfloat) | 7B | 4096 | 4,096 | 66.63 MTEB(eng) ORIGINAL 56-task mean = 66.63 (arXiv:2401.00368; NOT stated on HF card — from the paper) | 15 GB | — | MIT | The original LLM-as-embedder; proven MIT-licensed English 7B retriever, but now superseded by Qwen3/gte-Qwen2. Max input 4096 tokens (not 32K) |
| NV-Embed-v2 NVIDIA | 7.85B | 4096 | 32,768 | 72.31 MTEB(eng) ORIGINAL 56-task mean = 72.31 (Aug 2024, #1 at launch) — NOT comparable to Qwen3's MTEB(eng,v2) | 16 GB | — | CC-BY-NC-4.0 non-commercial | Highest raw score on the original 56-task English MTEB — but CC-BY-NC-4.0 explicitly forbids commercial use; research/eval only, NOT for commercial RAG |
| Qwen3-Embedding-8B ML Alibaba (Qwen) | 8B | 4096 ↓ 32-4096 (MRL, user-defined output dim) | 32,768 | 75.22 MTEB(eng, v2) mean = 75.22 (card table); MMTEB multilingual mean = 70.58 (#1) — do NOT conflate; both distinct from the older 56-task board | 17 GB | qwen3-embedding:8b | Apache-2.0 | Top open-weight retrieval embedder in 2026 (#1 MMTEB), Apache-2.0, native Ollama pull; the default when ~17GB VRAM is available |
This is the retrieval layer of a self-hosted RAG stack. Store those vectors in a self-hosted vector database, pair it with a local runtime from the AI runtime comparison and a generative model from the local LLM database. Part of the D-Central local-AI knowledge base. MTEB scores from the HuggingFace leaderboard; verify the current model card + license before production use.
This completes the retrieval leg of the self-hosted stack. Feed the vectors to a runtime from the local AI runtime comparison, answer with a model from the local LLM database, and keep the whole pipeline on your own hardware — the reason Law-25-compliant RAG runs on-premises in the first place. Part of D-Central’s local-AI knowledge base.
Related products, repair, and setup paths
- how D-Central diagnoses ASIC repairs
- ASIC troubleshooting library
- ASIC manuals and repair guides
- replacement hashboards
- ASIC control boards
- ASIC power supplies
- S19 family replacement hashboard
- C52 replacement control board
- APW12 S19 power supply
- immersion cooling hub
- home immersion cooling guide
- ASIC miners for immersion planning
- ASIC cooling parts
- airflow shroud before immersion
- compare miner specs in the database
- ASIC repair support
Last reviewed July 17, 2026.