Recall@k, and why it is the number retrieval lives or dies by

Recall@k is the fraction of questions for which at least one passage that genuinely answers the question appears in the top k results. Ask 10 questions, check whether an answer-bearing passage landed in the top 5 for each, find it did for 7 of them, and recall@5 is 0.7. That is the whole definition.

Note what it deliberately ignores. Not where in the k the passage sat, not how many good passages came back, not whether the model used it. It is a yes-or-no question about presence, asked once per question and averaged, and its bluntness is the source of its usefulness.

The asymmetry that makes it the gate

Everything after the candidate stage operates on the candidate set. A reranker rescores the list it was handed and cannot introduce a passage that is not in it — the point of what a reranking stage does to a candidate list. Rank fusion merges 2 orderings and can only promote what one arm already found. The prompt cannot cite text it never received.

So the 2 failure modes are not symmetrical. A passage ranked 40th when it should be 3rd is a precision problem, and half a dozen cheap techniques fix it. A passage absent from the top 100 is invisible to all of them. That is why recall is measured first.

A worked calculation over 10 questions

Recall@5 is 6 of 10, or 0.6. Recall@20 is 8 of 10, or 0.8. Together they say more than either alone: for 2 of the 4 questions that failed at 5, the passage was already retrieved and merely ranked too low, so reordering could recover them. For questions 5 and 9 it was absent at 20, and for question 5 absent at 100 — a representation problem in the index, not a ranking one.

That reading is what makes the measure operational. Recall@20 is the ceiling on what any downstream stage can achieve; recall@5 is what you are delivering. The gap is the headroom a reranker has to work with, and if it is small, buying one will not help you.

Precision, and why it is the junior partner here

Precision@k is the other half of the pair: of the k passages returned, what fraction were relevant. It matters — prompt slots are finite, and passages that answer nothing crowd out the one that does. But a precision failure is a ranking problem with cheap fixes that operate on the list you already have.

The rule follows directly: raise k until recall stops improving, then work on precision inside that k. The other order means tuning the ordering of a candidate set that was missing the answer all along.

Reranking reorders. Fusion merges. Prompting instructs. None of them can retrieve a passage the first stage never fetched.

The k you measure at and the k you serve

These are different numbers and conflating them is a common error. Measure at a generous k — 50 or 100 — because that says what is reachable at all. Serve a small k, because every passage competes for attention and costs tokens. Report both, always naming the k: a bare "recall of 0.9" means nothing without it.

One more reason to keep measuring the generous k: it is the tripwire for changes that alter retrieval without touching it. A new embedding model, a corpus that doubled, or an edit to the rewrite step that reshapes a query before the search runs can move recall several points while every dashboard stays green and no request errors. That is the shape of a silent regression in a system that never errors, and a tracked recall number is the cheapest detector for it.

Where the question set comes from — how many questions, who labels them, how it stays honest as the corpus changes — is a separate discipline. What belongs here is the habit: pick the measure that gates everything else and put it in the build from the first week, which is how we scope measurement into MVP and product builds. The terms and diagnostics leaning on this one sit in retrieval and grounding, part of the engineering library.

• retrieval
• measurement
• ranking
• definitions