What is chunking in RAG? Definition and examples

Chunking is splitting long documents into smaller passages before indexing, so an AI system can retrieve just the part of a page that answers a question.

How it works

In retrieval-augmented generation, each passage is turned into one embedding and searched on its own. A whole pricing page as a single embedding would be a blur of every plan; a passage per plan is sharp. Pinecone's guide frames the goal as chunks "big enough to contain meaningful information, while small enough to enable performant applications," and offers a test: does the chunk make sense to a person without the surrounding text (Pinecone)?

The common strategies, from simplest to smartest:

• Fixed-size. Cut every N tokens. Simple and predictable, and Pinecone suggests it as a sensible default.
• Recursive. Split on paragraphs, then sentences, then words, until pieces fit.
• Structure-based. Follow the document's own structure: HTML or Markdown headings, list items, table rows.
• Semantic. Start a new chunk where the topic changes, detected by comparing sentence embeddings.

A chunk can also lose context: "It costs $49 per month" means little without knowing which plan "it" is. Anthropic's Contextual Retrieval technique adds a short, generated note to each chunk about where it sits in the document before indexing. In Anthropic's tests this cut the rate of failed retrievals by 35%, by 49% when combined with keyword search, and by 67% with reranking added (Anthropic).

Why it matters for your website

You don't control the chunker, but you control the page it chunks. Pages that split cleanly get answered from accurately:

• One topic per section, under a descriptive heading ("Refunds on annual plans", not "More info").
• Self-contained paragraphs. Name the plan, product or feature instead of "this", "the above" or "see table".
• Real text, not images. Prices inside a graphic or a screenshot may never be indexed.
• Simple tables with header rows, so each row still makes sense alone.

Example

A pricing page section, chunked two ways:

Weak chunk (fixed-size cut mid-section):
"…and it includes priority support. Annual billing saves 20%. The next tier adds"

Better chunk (split on the heading, with context):
"Pricing page > Pro plan: The Pro plan costs $129 per month, or $103 per month
billed annually, and includes meeting booking and CRM sync."

A question like "How much is Pro if I pay yearly?" matches the second chunk directly. The first chunk never names the plan, so it may be retrieved for the wrong question or not at all.

Related terms

• Embeddings: each chunk becomes one embedding.
• Retrieval-augmented generation (RAG): the pipeline chunking feeds.
• Vector database: where chunks are stored and searched.
• Knowledge base: the documents being chunked.
• Reranking: picks the best chunks after retrieval.