Retrieval · Short note

Retrieval fails quietly

The dangerous miss is a fluent answer written from the wrong page.

A search that returns nothing is obvious. Someone notices the blank. A search that returns the wrong three paragraphs is harder, because the model will write a smooth answer from them. The mistake is in the choice of source. The sentences are fine.

We have seen this with care documents. A wound note from a previous admission outranks this week’s plan, because the old note is longer and shares more words with the question. The answer looks finished. It describes the wrong week. A coordinator who is tired at handover may not catch it.

Good retrieval is mostly the unglamorous part. What you index. How you split a document. Which files this person is allowed to see. Whether the system will say the file does not contain the answer, and stop, when the passages are a poor match.

Fluency is cheap now. The page number is the product. If an agent cannot show which passage it used, it is not ready for a record someone else will rely on.

This is a short note. The longer essay is still being written.