What Is RAG and Why Does It Matter?

Retrieval-augmented generation, or RAG, is a pattern that pairs a language model with a search step over a body of documents. Instead of asking the model to recall facts, you fetch the relevant passages first and ask the model to answer using them.

Why RAG instead of fine-tuning

Fine-tuning bakes knowledge into weights, which is slow, expensive, and hard to update. RAG keeps knowledge outside the model where it can be added, removed, and audited.

What a RAG pipeline looks like

1. Chunk and embed your documents.
2. Store embeddings in a vector database.
3. At query time, embed the question and retrieve the closest passages.
4. Feed the question and passages to a language model with a strict instruction to use only the provided context.

Where teams hit walls

• Chunking strategy (too small loses context, too large wastes tokens).
• Hybrid search (combining vector + keyword) usually beats pure vector.
• Re-ranking and citation tracking matter more than people expect.
• Evaluation: hallucination rates collapse only when you measure them.

The bigger signal

RAG remains the default architecture for any AI feature that needs to answer over private or fresh data. Long-context models help, but retrieval is rarely going away.