- In short
- Chunking strategies trade off differently: fixed-size chunking is simple and predictable but can split a single idea across two chunks; semantic chunking splits at topic or meaning boundaries to preserve coherence at the cost of variable chunk sizes; and structural chunking splits along document structure such as headings or sections, which works well for well-formatted source documents. The right strategy depends on the source document's shape and consistency, not a universal default.
Three strategies, three trade-offs
Chunking is the pipeline stage that decides how source documents are cut into the passages retrieval will later search. The exam wants you to know the three main strategies and, more importantly, what each trades away, because there is no universally best choice, only a best choice for a given document shape. Fixed-size chunking cuts at a set token or character count. Semantic chunking cuts at topic or meaning boundaries. Structural chunking cuts along the document's own structure, headings, sections, clauses. Each is good at something and weak at something, and matching the strategy to the source is the skill.
Because chunking sits upstream, its trade-offs bound retrieval quality, exactly the upstream-bounds-downstream principle from the pipeline stages. A chunking choice mismatched to the document shows up later as a retrieval miss.
- Chunking strategy trade-offs
- The comparison among fixed-size chunking (simple and predictable but can split an idea), semantic chunking (coherent at meaning boundaries but variable in size), and structural chunking (aligned to headings or sections, best for well-formatted documents). The right choice depends on the source document's shape and consistency.
Fixed-size: simple but blind to meaning
Fixed-size chunking is the simplest: cut every N tokens. Its virtue is predictability, uniform chunk sizes, trivial implementation, no dependence on document structure. Its flaw is that the cut points ignore meaning, so a single idea can be split across two chunks. When a fact and the context that makes it findable land on opposite sides of a fixed boundary, retrieval may fetch one half without the other, and the answer suffers. Fixed-size is fine for uniform, prose-like content where arbitrary cuts rarely sever a critical idea, and risky where meaning is tightly bound to exact spans.
The exam's canonical warning is legal or contract text: fixed-size chunking there can split mid-clause, and a clause severed in the middle loses the meaning that depended on being whole. That is a case where the simplicity of fixed-size is not worth the cost.
Semantic and structural: coherence and structure fit
Semantic chunking cuts at topic or meaning boundaries, so each chunk holds a coherent unit of meaning rather than an arbitrary span. The gain is coherence, ideas stay intact; the cost is variable chunk sizes, which complicate downstream handling and can produce chunks too large or too small. Structural chunking cuts along the document's own structure, headings, sections, list items, and works well precisely when the source is well-formatted, because the document's structure already marks natural coherent boundaries. For a document with clean headings and sections, structural chunking gets coherence almost for free by following the author's own divisions.
So the choice is a fit question. Well-formatted documents with clear structure favour structural chunking. Documents where meaning boundaries do not align with any explicit structure favour semantic chunking. Uniform prose where cuts rarely matter can tolerate fixed-size. Reading the source's shape is what selects the strategy.
What the exam trips candidates on
Two traps. The first is using fixed-size chunking on legal or contract text where splitting mid-clause loses meaning, applying the simplest strategy where its blindness to meaning is most damaging. The second is assuming one chunking strategy is universally best regardless of the source document's structure, ignoring that the right choice is a fit to document shape. The credited answer matches the strategy to the source's shape and consistency.
Worked example
A team is building RAG over a corpus of commercial contracts with clearly numbered clauses and section headings. For simplicity they default to fixed-size chunking at 500 tokens. Early tests show the system sometimes retrieves half of a clause and answers as if the missing half did not exist. Recommend a chunking approach and justify it.
The symptom points straight at the fixed-size trap. Contracts carry meaning at the clause level, and a 500-token fixed cut lands wherever the counter reaches 500, with no regard for clause boundaries, so a single clause is regularly split across two chunks. When retrieval then fetches one half, the answer is built on a fragment and the severed half is invisible, which is exactly the 'retrieves half of a clause' behaviour observed. Fixed-size is precisely the wrong strategy for legal text, because its blindness to meaning is most damaging where meaning is tightly bound to whole clauses.
The corpus, though, has an advantage the team is ignoring: it is well-formatted, with numbered clauses and section headings. That makes structural chunking the strong fit, split along the document's own structure so each chunk is a whole clause or section, following the boundaries the drafters already marked. Structural chunking here buys coherence almost for free and eliminates the mid-clause splits. Where a clause is unusually long or the structure is irregular, semantic chunking (cutting at meaning boundaries) is the fallback that still preserves coherence, at the cost of variable chunk sizes. The recommendation is to move off fixed-size to structural chunking aligned to the clause and section structure, justified by the document shape, not to pick a strategy for implementation simplicity, which is the second trap of assuming one strategy fits regardless of structure.
Common misreadings to avoid
Misconception
Fixed-size chunking is a safe default for any corpus, including legal and contract text.
What's actually true
Misconception
There is one best chunking strategy that should be used regardless of the document.
What's actually true
How this shows up on the exam
Expect a corpus with a specific shape, legal clauses, structured docs, uniform prose, and a chunking choice to evaluate. The reliable reading is to match strategy to document shape: structural for well-formatted docs, semantic for meaning-bound content, fixed-size only where arbitrary cuts are harmless. This knowledge point builds on the pipeline stages, unlocks chunk size, overlap, and retrieval quality and metadata-enriched indexing, and underpins diagnosing retrieval failures rooted in chunking.
RAG over a corpus of clause-numbered contracts uses 500-token fixed-size chunking and keeps retrieving half a clause. What is the best fix?
People also ask
What are the main chunking strategies?
When should you use structural chunking?
Is fixed-size chunking safe for legal text?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.