Retrieval · Short note
Retrieval fails quietly
The dangerous miss is a fluent answer written from the wrong page.
A search that returns nothing is obvious. Someone notices the blank. A search that returns the wrong three paragraphs is harder, because the model will write a smooth answer from them. The mistake is in the choice of source. The sentences are fine.
We have seen this with care documents. A wound note from a previous admission outranks this week’s plan, because the old note is longer and shares more words with the question. The answer looks finished. It describes the wrong week. A coordinator who is tired at handover may not catch it.
Good retrieval is mostly the unglamorous part. What you index. How you split a document. Which files this person is allowed to see. Whether the system will say the file does not contain the answer, and stop, when the passages are a poor match.
Fluency is cheap now. The page number is the product. If an agent cannot show which passage it used, it is not ready for a record someone else will rely on.
This is a short note. The longer essay is still being written.