Retrieval-Augmented Generation
Retrieval-augmented generation grounds a model's answer in retrieved documents, so responses reflect your data rather than only training-time knowledge.
Definition
Retrieval-augmented generation is a technique that retrieves relevant passages from an external corpus and supplies them to a language model as context, so the generated answer can cite and use knowledge outside the model's parameters.
Context and scope
It is used for question answering over private documents, support assistants, and any application needing current or proprietary facts.
Common misconceptions
- retrieval guarantees the answer is factually correct
- a larger model removes the need for retrieval
Anti-patterns
- retrieving passages without regard to the question's intent
- passing retrieved text to the model without delimiting it from instructions
Related concepts
Authoritative source
Definition
Retrieval-augmented generation is a technique that retrieves relevant passages from an external corpus and supplies them to a language model as context, so the generated answer can cite and use knowledge outside the model’s parameters.
It is used for question answering over private documents, support assistants, and any application needing current or proprietary facts. Compared to simpler alternatives, this concept fits best when the trade-off it represents is the one the system actually cares about.
When to use it
A common use case is applying it at the boundary where the cost of getting it wrong is highest, and the scenario usually involves a concrete guarantee the system must uphold. Unlike an ad hoc approach, it makes that guarantee explicit, reviewable, and testable.
How it works
The quality ceiling is set by retrieval: if the relevant passage is never fetched, no amount of generation skill can recover it, so ranking and recall come first.
Chunking, embedding choice, and reranking all influence which passages surface, making the retrieval pipeline the primary lever for accuracy.
Trade-offs and alternatives
Retrieved content is untrusted input, so it must be separated from system instructions to reduce prompt injection through documents. The main drawback is the extra complexity it introduces, so the benefit should clearly outweigh that cost before a team adopts it.
Common misconceptions
A persistent misconception is that retrieval guarantees the answer is factually correct. Another is that a larger model removes the need for retrieval. For example, an engineer who assumes otherwise may design around a guarantee that does not actually exist, which only becomes obvious under load or failure.
Anti-patterns to avoid
Two recurring anti-patterns are to retrieving passages without regard to the question’s intent and to passing retrieved text to the model without delimiting it from instructions. Both tend to surface as hard-to-debug production issues rather than obvious test failures. Related concepts include Vector Databases, Semantic Search, Text Embeddings.