Library// topic
Retrieval and grounding: answering from your own content
In short
Everything that happens at query time — how a passage is chosen, ranked, filtered by permission and cited — and how you tell 'the wrong documents came back' apart from 'the right documents, badly used'; the plumbing that gets content into the index lives in the data cluster, and eval methodology lives in the release-gate cluster.
20 pages
diagnostics
- Adding one new source made unrelated answers worseNothing about the old passages changed. A new source ranks moderately well on a great many queries, and a fixed top-k has to give it those slots by taking them from something else.diagnostic11 min
- Facts in the middle of a long context get missedHold the passage set constant, walk the answer-bearing chunk from first position to last, and measure. The shape of the resulting curve is the diagnosis, and it takes about an hour to produce.diagnostic9 min
- It worked on five hundred documents and broke at fifty thousandNothing regressed when the corpus grew. A candidate budget that comfortably held the answer at 500 documents now competes against a hundred times as many near-neighbours, and k never moved.diagnostic11 min
- Queries with a part number or a clause reference come back emptyProse questions work and a single part number returns nothing useful. Identifiers are the one query class where similarity is structurally the wrong instrument.diagnostic11 min
- Recall collapsed the day you turned on permission filteringThe retrieval quality did not change. A filter applied after the nearest-neighbour search is eating a candidate set that was already fixed at k, and the users with least access feel it first.diagnostic10 min
- The assistant quoted a policy that was replaced last quarterThe old version is still indexed, or it is gone and a cache is still serving it, or both versions are live and the old one ranks higher. One probe separates them.diagnostic9 min
- The citation points at the wrong paragraphA misaligned citation is a pipeline bug with a deterministic cause; an invented one is a grounding failure. One exact-string search against the original file separates them in under 5 minutes.diagnostic10 min
- The model answered from memory and ignored the passage you gave itPlant a passage that contradicts common knowledge, then ask a question only that passage answers. Whichever the answer follows tells you whether you have an override or a payload that never arrived.diagnostic10 min
- The number is in a table and the assistant cannot find itThe figure is in the document and the assistant says it cannot find it. The row and its header stopped being connected somewhere between the file and the chunk.diagnostic10 min
- The right passages were retrieved and the answer is still wrongIf the answer-bearing passage was in the context and the answer is still wrong, retrieval tuning cannot help. Four synthesis defects produce this, and each leaves its own mark in the output.diagnostic10 min
- The same question returns different passages every runSome of the variation is approximate search doing exactly what it was built to do. The rest is a rewrite step, a moving index or a session-dependent filter — and only that half is worth fixing.diagnostic11 min
- You know the document is indexed and it still never comes backFour layers can quietly drop one document, and they look identical from the outside. Probe them in cost order — existence, filter, exact text, rank — and each probe eliminates exactly one.diagnostic10 min
definitions
- Chunk overlap: what it protects against and what it duplicatesOverlap is insurance against a chunk boundary landing in the middle of one idea. The premium is paid in duplicated candidates crowding a fixed number of prompt slots.definition6 min
- Groundedness: what separates a supported answer from a plausible oneGroundedness relates a claim to the evidence supplied with it. Correctness relates it to the world. Systems lose credibility in the gap between the two.definition6 min
- Metadata filters, and the difference between filtering and rankingA filter decides which passages may compete. A boost decides how they place. Treating one as the other produces empty result sets and permission leaks.definition6 min
- Multi-hop questions: the shape single-shot retrieval cannot serveThe test is dependency, not difficulty: if you cannot write the second search until the first answers, no amount of index tuning makes one retrieval enough.definition6 min
- Query rewriting: what happens to the user's words before the search runsOne name covers three unrelated operations: resolving what a follow-up refers to, widening vocabulary, and reshaping a question into something passage-shaped. Each fails differently.definition6 min
- Rank fusion: merging a keyword list and a vector list into one orderFusion combines the positions a document holds in two result lists, not the scores. That is the whole trick, and why hybrid search works without calibration.definition6 min
- Recall@k, and why it is the number retrieval lives or dies byRecall@k is the share of questions whose answer-bearing passage made the top k. It gates everything downstream, because no reranker or prompt recovers a passage that was never fetched.definition6 min
- What a reranking stage does to a candidate listA reranker rescores a list that already exists. It reads the query and each passage together, which a stored vector cannot — and it can only ever reorder what the first stage found.definition6 min
Other topics in Library
See allData readiness, pipelines and keeping the corpus trueThe unglamorous half of every AI build — whether the data is usable at all, how it gets in, how it stays fresh, and how deletions, duplicates and schema drift are handled before anything is indexed or inferred over; one-off migrations of a legacy dataset belong to the integration cluster.16 pagesChoosing the shape of the systemThe decision made before any code exists: what problem shape this actually is, whether it needs an agent loop, a fixed pipeline, a classifier or no model at all, how much autonomy the task can carry, and whether prompting, retrieval or training is the right lever — which process to automate first is an organisational question and sits in the adoption cluster.2 pagesDeciding what to build, and who builds itThe pre-engagement decision — whether to build, buy or configure, whether to hire in-house or bring in a partner, how studios, consultancies, agencies and freelancers actually differ, and how to justify the spend and run diligence on a vendor whose security posture rests on controls rather than certificates.6 pages
Working on something in this space?
Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.
Start the conversation