Model families

Embedding and reranking models.

In a retrieval system, the generator gets the attention and the embedding model decides the outcome. If the right passage is not in the candidate set, no amount of frontier reasoning downstream will recover it — the model will answer confidently from what it was given.

SPECIFICATION

Two roles: Embedding models decide recall over the whole corpus; rerankers decide precision over the retrieved candidates
Self-hosting: Usually easier than self-hosting a generator — the strong open models are small and run on modest hardware
Dimensionality: Drives index size and memory. A larger vector is not automatically a better one on your corpus.
Sequence length: Caps how much text one vector may represent, which is what your chunking strategy has to respect
Switching cost: Vectors from two models are not comparable, so changing the embedding model means re-ingesting the entire corpus
How to choose: On an evaluation set of fifty to two hundred real queries from your own domain, not on a public leaderboard position

Provider-published specifications as of 1 September 2026, not figures we measured. Model families move quickly — check the provider's own documentation before committing to a number.

THE TWO JOBS

Embedding — recall

Turns text into a vector so that semantically similar passages sit close together. Its job is to make sure the right passage is somewhere in the top few dozen candidates. It runs over your entire corpus, so its cost and its dimensionality decide your index size and your ingestion bill.

Reranking — precision

Takes those candidates and scores each one against the query directly, which is far more accurate and far more expensive per item. Its job is to get the right passage into the top three, where the generator will actually attend to it. In our experience a cheap reranker over a wide candidate set beats an expensive embedding model over a narrow one.

HOW TO CHOOSE

Ignore the public leaderboard position. Embedding benchmarks are built from general-domain corpora. Your corpus is contracts, or clinical notes, or part numbers, or support tickets in three languages — and relative performance reorders substantially on domain-specific text. The leaderboard is a shortlist generator, not an answer.

Build a small evaluation set from real queries. Fifty to two hundred real questions, each with the passage that should be retrieved, is enough to rank candidates decisively. This is typically a half-day of work, and it is the highest-leverage half-day in the whole retrieval build. Without it you are tuning on impressions.

Check the practical constraints early. Dimensionality drives index size and memory. Maximum sequence length drives your chunking strategy. Whether the model can run on your own hardware drives whether your entire corpus has to be sent to a third party to be indexed — which for a private corpus is frequently the deciding factor and is often noticed far too late.

Multilingual is a separate question. If your corpus and your queries are in different languages, you need a model trained for cross-lingual retrieval specifically. A strong English model with a translation step in front of it is a different system with different failure modes, and usually a worse one.

THE TRAPS

Changing the embedding model means reindexing everything. Vectors from two different models are not comparable, so a swap is a full re-ingestion of the corpus. Budget for it, and build the ingestion pipeline so it can be re-run without ceremony — because you will change your mind at least once.

Chunking usually matters more than the model. Teams routinely swap embedding models looking for a quality gain that was actually available by fixing chunk boundaries so passages stop being cut mid-clause. Test chunking first; it is cheaper and the effect is larger.

Skipping the reranker to save money is a false economy. In our experience it is the cheapest large improvement available in most retrieval systems, precisely because it operates on a few dozen candidates rather than a few million documents.

QUESTIONS

QUESTIONS — 3

Yes, and it is usually easier than self-hosting a generator — the strong open embedding and reranking models are small, they run on modest hardware, and they are the part of the pipeline that touches every document you own. For a private corpus that is often reason enough to keep them in-house even when the generator is a hosted API.

On a domain corpus, materially — but less than fixing chunking and adding a reranker, which is why we do those first. The order we work in is: chunking, reranker, embedding model, then everything else. It reflects effect size per unit of effort.

You need somewhere to put the vectors, which for most systems is the Postgres you already run with pgvector. A dedicated vector database earns its place at a scale and query-pattern threshold that a lot of projects assume they have crossed and have not. See choosing a vector database.

Keep reading

Choosing a Vector Database

→

Technology

Evaluating AI Systems

→

30-minute technical call · no deck