Embedding and reranking models.

Two roles

Embedding models decide recall over the whole corpus; rerankers decide precision over the retrieved candidates

Self-hosting

Usually easier than self-hosting a generator — the strong open models are small and run on modest hardware

Dimensionality

Drives index size and memory. A larger vector is not automatically a better one on your corpus.

Sequence length

Caps how much text one vector may represent, which is what your chunking strategy has to respect

Switching cost

Vectors from two models are not comparable, so changing the embedding model means re-ingesting the entire corpus

How to choose

On an evaluation set of fifty to two hundred real queries from your own domain, not on a public leaderboard position

Provider-published specifications as of 1 September 2026, not figures we measured. Model families move quickly — check the provider's own documentation before committing to a number.

• Ignore the public leaderboard position.

  Embedding benchmarks are built from general-domain corpora. Your corpus is contracts, or clinical notes, or part numbers, or support tickets in three languages — and relative performance reorders substantially on domain-specific text. The leaderboard is a shortlist generator, not an answer.

• Build a small evaluation set from real queries.

  Fifty to two hundred real questions, each with the passage that should be retrieved, is enough to rank candidates decisively. This is typically a half-day of work, and it is the highest-leverage half-day in the whole retrieval build. Without it you are tuning on impressions.

• Check the practical constraints early.

  Dimensionality drives index size and memory. Maximum sequence length drives your chunking strategy. Whether the model can run on your own hardware drives whether your entire corpus has to be sent to a third party to be indexed — which for a private corpus is frequently the deciding factor and is often noticed far too late.

• Multilingual is a separate question.

  If your corpus and your queries are in different languages, you need a model trained for cross-lingual retrieval specifically. A strong English model with a translation step in front of it is a different system with different failure modes, and usually a worse one.

• Changing the embedding model means reindexing everything.

  Vectors from two different models are not comparable, so a swap is a full re-ingestion of the corpus. Budget for it, and build the ingestion pipeline so it can be re-run without ceremony — because you will change your mind at least once.

• Chunking usually matters more than the model.

  Teams routinely swap embedding models looking for a quality gain that was actually available by fixing chunk boundaries so passages stop being cut mid-clause. Test chunking first; it is cheaper and the effect is larger.

• Skipping the reranker to save money is a false economy.

  In our experience it is the cheapest large improvement available in most retrieval systems, precisely because it operates on a few dozen candidates rather than a few million documents.