What is reranking in AI search? Definition
On this page
Reranking is a second search step in which a model reorders a shortlist of retrieved passages by how well each one actually answers the user's question.
How it works
Search for AI answers usually runs in two stages:
- Retrieve. A fast method, such as embeddingEmbeddings are lists of numbers that represent the meaning of a piece of text, so software can find related content by measuring how close two lists are. similarity, keyword search or both, pulls a broad shortlist from a large index. It is quick because it compares compact representations, but it can be fooled by passages that use similar words without answering the question.
- Rerank. A slower, more careful model reads the query and each candidate passage together and gives every pair a relevance score. The shortlist is reordered and only the top few passages go to the language model.
Cohere's documentation describes its Rerank model this way: given a query and a list of documents, it "indexes the documents from most to least semantically relevant to the query" and returns a relevance score for each (Cohere).
In Anthropic's retrieval experiments, the team pulled 150 candidate chunks, used a reranker to keep the best 20, and found that adding reranking cut failed retrievals further than better chunkingChunking is splitting long documents into smaller passages before indexing, so an AI system can retrieve just the part of a page that answers a question. and embeddings alone. The same write-up notes the cost: reranking adds some latency and expense to every query, so scoring more candidates trades speed for accuracy (Anthropic).
Why it matters for your website
When a site assistant gives an answer that is close but wrong, the cause is often retrieval, not the language model: the right passage was found but ranked sixth, and only the top three were used. Reranking is one of the cheapest ways to fix that class of error, because it improves which content the model sees without changing the content itself.
For a website owner the questions are practical: does the assistant rerank, how many passages reach the model, and can you see which sources an answer used? Visible source links make retrieval problems easy to spot.
Example
A visitor asks: "Can I pay by invoice?"
| Rank | First-stage result | After reranking |
|---|---|---|
| 1 | "Invoices are emailed on the 1st of each month." | "Annual plans can be paid by bank transfer against an invoice." |
| 2 | "Download past invoices from Billing settings." | "Invoices are emailed on the 1st of each month." |
| 3 | "Annual plans can be paid by bank transfer against an invoice." | "Download past invoices from Billing settings." |
All three passages mention invoices, so the first stage ranks them by word overlap. The reranker reads the question and moves the passage that actually answers it to the top.
Related terms
- Retrieval-augmented generation (RAG): the pipeline reranking sits inside.
- Embeddings: the usual first-stage retrieval method.
- Vector database: returns the shortlist that gets reranked.
- Chunking: determines what each candidate passage contains.
Sources
- Rerank overview (Cohere documentation) — accessed October 20, 2026
- Introducing Contextual Retrieval (Anthropic, September 2024) — accessed October 20, 2026
Articles that use this term
AI search
AI site search: replacing the search box with answers
How AI site search turns a keyword box into short, sourced answers: how it works, where it goes wrong, and what to measure before and after you switch.