The record was deleted and the assistant still quotes it

Somebody deleted the supplier record on the 3rd. On the 24th the assistant quoted its payment terms, with a citation, out of a document nobody can open any more. Nothing errored across those 21 days: a pipeline built around inserts and updates has nothing to report when a record stops arriving.

Deletes fail differently from every other operation. A failed insert leaves a gap somebody eventually counts. A failed delete leaves the previous state, which is what a healthy system looks like. The work is not repair, it is location: finding the one hop where the delete stopped, out of the 5 or 6 most builds contain.

Put one tracer through the real delete path

Do not start from the record in the complaint. It may have been edited, re-exported, merged or re-created since, and each of those muddies the evidence. Start from a record you control.

1. Mint a tracer. Create 1 record with a nonsense token such as ZZQX-TRACER-0471 in a field that reaches indexed text. A token with no natural-language neighbours is exact-matchable in every store you are about to query.
2. Let it land completely. Wait one full cycle of every scheduled stage, then confirm the assistant can answer with it. If it never became answerable, stop: that is an ingest problem.
3. Write down where it lives. Note its identifier in each store — source id, raw payload, document id, chunk ids, vector ids, any summary mentioning it. Compiling the list usually surfaces a store nobody had on the diagram.
4. Delete it the way a user does, through the same button or API call the business uses. A delete issued straight against the database proves nothing about the path real deletions take.
5. Note the wall-clock time and wait one full cycle again, so a slow stage is not mistaken for a broken one.
6. Query every store from step 3 by exact token, working outward from the source. The first that still returns it is the guilty hop; everything downstream of it is innocent.
7. Ask the question that used to retrieve it. Clean stores plus a stale answer means a cache, or an answer never retrieved at all — the failure in answering from memory rather than context.

If every store clears but takes days, that is not a delete failure. It is lag, measured as an age distribution rather than hunted as a bug — the method is in answers that are a quarter out of date. If only the vector index lags, check whether the delete is queued behind a reindex still running after three days.

Five hops that eat a delete, and how each one looks from outside

One possibility sits underneath all 5 rows: the delete landed correctly, on 1 of the 3 records describing the same thing. Deleting a record does not delete an entity when nothing ever decided they were one — a customer under three spellings.

A soft delete is a flag, and a flag means nothing to a reader never told about it

The source did not lie. It sent the record with a field saying it is gone, and the connector — which maps the 6 fields it cares about and ignores the rest — passed it through as ordinary content. This is the cause most often misdiagnosed as a broken index, because the payload arrives complete.

• A boolean that flips. Easy to handle once known, invisible until somebody reads a payload from after the deletion.
• A status enum that gains a value. The record moves from active to archived, and a mapping knowing only active treats the new value as ordinary content.
• A validity window that closes. Nothing is flagged: an end date moves into the past, and correctness depends on the reader comparing it against now.
• A move rather than a change. The record goes to a deleted-items table the connector never queries, so it is neither present nor announced.
• Permission revoked instead of removed. The record exists, the service account still reads it, and every human calling it deleted is also right.

Which of these your source uses is a property of the source, not a preference, and it decides what change detection can see at all — the choice examined in how a pipeline decides something changed and change data capture for a document corpus.

An insert that fails leaves a hole somebody counts. A delete that fails leaves yesterday's answer, and nobody counts those.

The copies nobody drew on the architecture diagram

Every trace that comes back clean at the index and dirty at the interface ends in the same place: a copy that exists for speed and was never listed as a store.

• The answer cache. Keyed on question text, holding a generated response and its citations, and usually the last thing anyone purges.
• Autocomplete dictionaries. Built from indexed terms on their own schedule, so a deleted supplier keeps offering itself in the search box.
• Materialised summaries and entity profiles. Generated from documents, stored as text, then retrieved as though they were documents.
• Scheduled exports. A spreadsheet refreshed nightly, which nobody counts as part of the AI system until it quotes a deleted record.
• Search and prompt logs. Not serving traffic, but holding the content, and in scope for anything beyond a mechanical delete.

Give the delete a receipt from every store

The design that ends this class of bug is unglamorous: treat deletion as a record rather than an absence. Write a tombstone carrying the source id, the timestamp and the run that observed it, fan it out over the stored child ids, and require every store to return a verification count of 0 before the deletion is marked complete.

• Make it idempotent. Deleting twice must be a no-op and must not fail the run, for the reason in idempotent ingestion.
• Carry the run identifier. A deletion you cannot attribute to a run is one you cannot audit, which is the argument for lineage on every served passage.
• Alert on unacknowledged deletes. A tombstone past its propagation window with a store still unconfirmed is the shape of alert in what to alert on in an ingest pipeline.
• Keep the tracer running on a schedule. A delete path is only known to work on the day it was last exercised.

The full fan-out design is the guide on making a delete travel to the last cache. Where the vector index is a managed service, its deletion semantics and consistency window are somebody else's engineering decisions and can change under you — one of the things weighed in which capabilities are worth owning outright. The verification harness itself is ordinary software: a page that takes a record id and shows a row per store with a count, which we treat as part of the build in MVP and product builds.

What a clean trace still does not prove

A tracer that clears every store proves the mechanical path works today, for the shape of record you tested. It says nothing about copies outside the serving path: backups, replicas, prompt and trace logs, exports, anything a person downloaded. Answering an erasure request to a standard somebody else will audit is a different exercise, handled in carrying a deletion through every copy you made and what a retention schedule covers.

It also does not prove updates propagate. Deletion and modification travel different paths and fail independently, so a working delete path says nothing about a document updated at source that still serves old text. Both sit in data readiness and pipelines, part of the engineering library.

• deletion
• pipelines
• cache invalidation
• diagnosis