Retrieval means the model was handed documents, not that it looked anything up

The arrangement is simpler than the acronym suggests

A retrieval-based system does something quite mundane. When a question arrives, a search runs over a prepared collection of documents, some number of the most relevant passages are selected, and those passages are inserted into the model’s input alongside the question. The model then answers with that material in front of it.

Nothing about the model changes. It has not learned the documents, it does not consult a database, and it has no ability to go and check anything. It has been handed some text and asked to use it, which is a considerably weaker arrangement than the vocabulary implies.

That weakness is also the strength. Because the material is supplied at request time, it can be updated, restricted by permission, swapped out or audited, none of which is possible with knowledge baked into parameters.

Preparation is where most of the quality is decided

Before any of this works, the collection must be split into passages, because whole documents rarely fit and rarely have uniform relevance. Where those splits fall matters enormously: a passage cut mid-argument loses the qualification that made it accurate, and a table separated from its heading becomes meaningless.

Each passage is then converted into a numerical position so that similarity can be computed quickly across a large collection. This is the step that lets a query match a passage that shares no words with it, which is the main advantage over keyword search and also the main source of confident irrelevance.

None of this preparation is visible in the finished system, and it is where most of the difference between a good retrieval system and a poor one lives. The model at the end is frequently the least important component.

Every stage fails in its own way

The search may return nothing relevant, in which case the model answers from its parameters and typically does so without signalling the difference. It may return material that is topically similar but answers a different question, which is worse, because the model will use it. It may return the right passage ranked below the cutoff, so it never arrives.

Contradictions are handled poorly. If two retrieved passages disagree — an old policy and its replacement, for instance — nothing in the arrangement establishes which is authoritative, and the response may blend them into something that was never true.

And a citation attached to a claim is a weaker guarantee than it appears. It indicates which passage was supplied, not that the claim is supported by it, and the connection between the two is generated in the same way as everything else.

What it fixes and what it moves

Retrieval genuinely helps with recency, with material the model never saw, and with private collections that could not be trained on. It also makes an answer inspectable, which is often the more valuable property: a user who can read the source can check the claim, which is impossible when the answer comes from parameters alone.

What it does not do is make a system reliable. It replaces the question of whether the model knows something with the question of whether the search found it, and those failures are different rather than smaller. Both are silent.

It is worth noticing that the retrieval component is ordinary information retrieval, a field with decades of accumulated knowledge about ranking, evaluation and query understanding. Systems built by teams who ignore that history tend to rediscover its lessons the slow way.

Reading claims about it carefully

A system described as grounded in your documents is making a claim about its inputs rather than about its outputs. Grounding is an architectural description, not a guarantee of accuracy, and the two get conflated routinely in marketing and occasionally in technical writing.

The productive questions are about the pipeline. What was indexed, how it was split, how many passages are retrieved, what happens when nothing relevant is found, and whether the system will say so. Answers to those predict behaviour far better than anything about the model itself.

The vocabulary is unfortunate, since retrieval suggests an active act of looking something up. Nothing looks anything up. Something searched, something was pasted in, and something wrote an answer with it visible. Describing that chain accurately takes a sentence rather than an acronym, and the sentence is considerably more useful.