Best Embedding Models for RAG in 2026: Hosted vs Open-Source, Compared

The embedding model is the part of a RAG (retrieval-augmented generation) system that decides what your AI "finds" before it answers. It turns every document chunk and every user question into a vector, and retrieval returns the chunks whose vectors sit closest to the question. Pick a weak model and the right paragraph never reaches the LLM, no matter how good the LLM is.

This guide compares 15 embedding models, hosted APIs and open-source models you can run yourself, on retrieval quality, speed, price and license. It then tells you which one to pick for your situation, and what actually moves answer quality in the RAG systems we build for clients.

Short answer: for the best retrieval quality from an API, start with Voyage 4 or OpenAI text-embedding-3-large. For the best value, Voyage 3.5 Lite or OpenAI text-embedding-3-small at $0.02 per million tokens. To self-host with a permissive license, Qwen3 Embedding 8B (or 4B on smaller hardware) and BAAI/bge-m3 as the lightweight default.

The 2026 embedding model leaderboard

Ranked by ELO rating, with retrieval quality (nDCG@10, higher is better), latency, price per million tokens, vector dimensions and license. Figures as of March 2026; providers update models often, so check current pricing before you commit.

Prices for open-source models are typical hosted-inference prices; if you run them on your own hardware, the cost is your compute instead.

How to choose: match the model to your constraint

You need the best possible retrieval quality

Voyage 4 leads on both ELO and nDCG@10 (0.859), with low latency. OpenAI text-embedding-3-large is close behind and is the easy choice if you already run on OpenAI. Expect higher storage costs from its 3,072-dimension vectors.

You are cost-sensitive at scale

Voyage 3.5 Lite (nDCG@10 0.803) and OpenAI text-embedding-3-small (0.762) both cost $0.02 per million tokens. Voyage 3.5 Lite also uses 512-dimension vectors, which keeps your vector database small and fast.

Your data cannot leave your infrastructure

Self-host an open-source model. Qwen3 Embedding 8B has the best retrieval quality of the open models (nDCG@10 0.818, Apache 2.0). Qwen3 Embedding 4B keeps most of that quality (0.802) at half the latency and much lower hardware needs. See the open-source section below.

Your users write in many languages

Cohere Embed Multilingual v3 combines strong quality (0.781) with 7 ms latency and compact 512-dimension vectors. BAAI/bge-m3 is the open-source multilingual workhorse.

You need the lowest latency

Cohere's models (7 ms) and OpenAI's (9–10 ms) are the fastest hosted options. Among open models, Qwen3 Embedding 0.6B (23 ms) and bge-m3 (29 ms) are the quickest and run on modest hardware.

The best open-source embedding models to run locally

Self-hosting makes sense when data privacy rules (HIPAA, SOC 2, GDPR), air-gapped deployments or high, unpredictable API costs rule out hosted APIs. The gap between open and proprietary models has largely closed: the best open models now beat older proprietary endpoints.

1. Qwen3 Embedding 8B: the open-source quality leader

Apache 2.0 · ELO 1516 · nDCG@10 0.818 · 56 ms · 4,096 dimensions. If you have the hardware, this is the strongest open model for commercial use. The large vectors capture fine semantic detail, at the cost of more storage.

2. Qwen3 Embedding 4B: the sweet spot

Apache 2.0 · ELO 1496 · nDCG@10 0.802 · 28 ms · 2,560 dimensions. Half the latency of the 8B model with lower hardware requirements, and still above 0.80 on retrieval quality. The default we would test first for most self-hosted builds.

3. BAAI/bge-m3: the reliable lightweight default

MIT · ELO 1491 · nDCG@10 0.753 · 29 ms · 1,024 dimensions. Fast, multilingual, standard-size vectors and the most permissive license here, which makes legal approval easy.

4. Jina Embeddings v5 Text Small: top ranking, restrictive license

CC BY-NC 4.0 · ELO 1558 · nDCG@10 0.710 · 289 ms · 1,024 dimensions. It ranks second overall on ELO, but the non-commercial license means you need a commercial agreement to use it in a product. Good for internal tools and research.

5. Jina Embeddings v3: permissive fallback

Apache 2.0 · ELO 1491 · nDCG@10 0.766 · 93 ms · 1,024 dimensions. Superseded by v5, but its Apache 2.0 license keeps it relevant when you need a fully commercial-friendly Jina model.

6. Qwen3 Embedding 0.6B: for edge and constrained hardware

Apache 2.0 · ELO 1478 · nDCG@10 0.751 · 23 ms · 1,024 dimensions. Small enough to run almost anywhere, with respectable retrieval quality.

How to start self-hosting

Hugging Face Text Embeddings Inference (TEI) and Ollama both serve these models behind an HTTP endpoint with a single command. Pair the model with a vector database such as Qdrant (which runs locally in Docker), pgvector in Postgres, or a managed store.

What actually moves RAG quality in production

The leaderboard matters, but in the RAG systems we build for clients, the biggest wins rarely come from swapping the embedding model. They come from what you retrieve and how much of it you send to the LLM:

• Retrieve fewer, better chunks. For NorthernAI's branded CustomGPT portal, the assistant returned verbose answers padded with source references. Cutting retrieval from the default 20 results to 5 made answers more focused and reduced excess context.
• Size retrieval to your rate limits. In Celestix's multi-tenant RAG dashboard, the default 20 results triggered "maximum tokens per minute exceeded" errors under load. Lowering it to 16 stabilized responses without hurting answers.
• Keep local and hosted options swappable. For AskMovieBot we built local RAG (Ollama with Qdrant in Docker), hosted RAG and a fine-tuned model behind one configuration switch, so the client could compare quality and cost on real traffic instead of guessing.
• Evaluate on your own questions. Benchmarks like nDCG@10 are averages over public datasets. Before you commit, run 30–50 real questions from your users against two or three candidate models and compare what each one retrieves.

Choosing the embedding model is half of the decision; the other half is the LLM that writes the answer. See our leaderboard of the best LLMs for RAG.

Key takeaways

• Best quality (API): Voyage 4, then OpenAI text-embedding-3-large.
• Best value (API): Voyage 3.5 Lite or OpenAI text-embedding-3-small, both $0.02 per million tokens.
• Best open-source: Qwen3 Embedding 8B for quality, 4B for balance, bge-m3 as the lightweight default.
• Check the license: Jina v5 ranks high but is non-commercial without an agreement.
• Tune retrieval before swapping models: how many chunks you retrieve often matters more than which model ranks first.