What is retrieval-augmented generation (RAG)?

Retrieval-augmented generation (RAG) is a technique where an AI system first looks up relevant documents, then writes its answer using what it found.

How it works

The name comes from a 2020 research paper by Patrick Lewis and colleagues, which combined a language model's built-in knowledge with passages retrieved from an external index at question time (Lewis et al.). Most production systems follow the same three steps:

1. Index. Your content (pages, help articles, PDFs) is split into passages by chunking, each passage is turned into embeddings, and the results are stored in a vector database.
2. Retrieve. When a question arrives, the system finds the passages closest to it, often refined with a reranking step.
3. Generate. The model receives the question plus the best passages and is told to answer from them, ideally citing which page each fact came from.

Cloud platforms package this as a managed service. Amazon's documentation, for example, describes knowledge bases that search your data at query time, feed the results to the model and can include citations so readers can check the original source (AWS).

RAG is not always necessary. Anthropic notes that if a knowledge base is smaller than about 200,000 tokens (roughly 500 pages), you can often place all of it in the prompt instead (Anthropic). Most company websites with a blog, docs and help center are larger than that.

Why it matters for your website

RAG is how a website assistant answers from your facts rather than from whatever the model learned in training:

• Freshness. Change a page, re-index it, and answers change. No model retraining.
• Control. You decide which content is in scope, and old pages can be excluded.
• Checkability. Answers can link to the page they came from, which is the basis of grounding.

The quality ceiling is set by your content and your retrieval. If the right passage is never retrieved, the best model still cannot use it.

Example

A visitor asks, "Is there a setup fee on the annual plan?"

1. The question is embedded and matched against the index.
2. Retrieval returns a passage from the pricing page ("Annual plans have no setup fee") and one from the billing FAQ about invoicing.
3. The model answers: "No. Annual plans have no setup fee," and links to the pricing page.

If nothing relevant is retrieved, a well-built system says it doesn't know and offers a contact option instead of inventing an answer.

Related terms

• Embeddings: the numeric form that makes search by meaning possible.
• Vector database: where embeddings are stored and searched.
• Chunking: how documents are split before indexing.
• Reranking: a second pass that puts the best passages first.
• Grounding: tying answers to verifiable sources.
• Knowledge base: the content a RAG system draws from.