- In short
- A RAG pipeline moves through ingestion, chunking, embedding, indexing, retrieval, and augmentation of the prompt with retrieved content. Retrieval happens before generation: retrieved chunks are inserted into context so Claude can ground its answer. The quality of every downstream stage is bounded by the quality of the chunking and indexing decisions made upstream, so a weak retrieval pipeline cannot be rescued by a stronger generation model.
The pipeline, stage by stage
Retrieval-augmented generation gives Claude access to a knowledge base it was not trained on by fetching relevant content and putting it in the prompt. The exam expects you to know the stages that pipeline moves through, because later knowledge points, chunking, indexing, retrieval strategy, are all decisions about specific stages. The pipeline runs: ingestion (bringing source documents in), chunking (splitting them into passages), embedding (turning chunks into vectors), indexing (storing them for search, often with metadata), retrieval (finding the relevant chunks for a query), and augmentation (inserting those chunks into the prompt). Each stage produces the input to the next.
This is a remember-level foundation, but it carries two ideas that matter for everything downstream: retrieval precedes generation, and upstream decisions bound downstream quality. Both reappear across the RAG topic.
- RAG pipeline core stages
- The sequence a retrieval-augmented generation pipeline runs through: ingestion, chunking, embedding, indexing, retrieval, and augmentation of the prompt with retrieved content. Retrieval precedes generation, and the quality of every downstream stage is bounded by the upstream chunking and indexing decisions.
Retrieval happens before generation
The ordering is the first load-bearing idea. Retrieval runs before generation: the pipeline finds relevant chunks and inserts them into the context, and only then does Claude generate an answer grounded in that retrieved content. This is what makes RAG 'grounded', the model is answering from the fetched passages, not from parametric memory alone. If retrieval brings the right passage, the model can ground on it; if retrieval misses, the model has nothing to ground on for that fact, no matter how the generation step is configured.
Seeing retrieval as the step that supplies the evidence clarifies why the retrieval stages matter so much. Generation can only work with what retrieval handed it. The answer's grounding is decided before generation begins.
Upstream decisions bound downstream quality
The second load-bearing idea is a bound: the quality of every downstream stage is capped by the chunking and indexing decisions made upstream. If chunking splits a fact away from the context that makes it findable, or indexing omits the metadata a query needs to filter on, retrieval cannot recover what was lost, and generation cannot recover what retrieval never fetched. Errors introduced early propagate forward and cannot be undone late. This is why the exam repeatedly frames retrieval problems as chunking and indexing problems.
The immediate corollary is the one candidates most need: a stronger generation model does not fix a weak retrieval pipeline. Upgrading the model improves how well the system reasons over what it was given, but it does nothing to change what retrieval fetched. If the needed chunk was never retrieved because of a chunking gap, a bigger model still never sees it. Retrieval-quality problems are fixed in the retrieval pipeline, not by the generation model, a theme that runs straight into diagnosing retrieval failures.
What the exam trips candidates on
Two traps. The first is assuming retrieval-quality problems can always be fixed by upgrading the generation model rather than the retrieval pipeline, which ignores that downstream quality is bounded upstream. The second is treating chunking as a one-time setup step rather than a design decision that must be revisited as content changes. The credited answer locates retrieval problems in the retrieval stages and treats chunking as an ongoing design decision.
Worked example
A support-knowledge RAG system gives incomplete answers: it often omits a detail that clearly exists in the source documents. An engineer proposes upgrading to a more capable generation model to fix the gaps. Trace the failure through the pipeline and evaluate the proposed fix.
Trace the answer back through the stages. The system produced an answer, so generation ran; the complaint is that a detail present in the source was omitted. For the model to include that detail, retrieval must first have fetched the chunk containing it and augmentation must have placed it in context. If the answer omits a detail that exists in the corpus, the most likely explanation is upstream: the chunk holding that detail was not retrieved, which usually traces to a chunking or indexing decision, the fact was split from its findable context, or the chunk was not indexed in a way the query could reach. Retrieval can only surface what chunking and indexing made findable, and generation can only ground on what retrieval surfaced.
That makes the proposed fix the first trap. Upgrading the generation model changes how well the system reasons over the chunks it receives, but it does nothing about which chunks retrieval fetched. If the detail's chunk was never retrieved, a more capable model still never sees it and still omits the detail, because downstream quality is bounded by the upstream chunking and indexing. The correct direction is to investigate and adjust the retrieval pipeline, revisit chunk boundaries and size so the detail stays with its context, and check the index and metadata, rather than spend the fix on the generation stage. The scenario also illustrates the second trap in the background: chunking was likely treated as set-once at launch, when it needs revisiting as the content evolves.
Common misreadings to avoid
Misconception
If a RAG system gives incomplete answers, upgrading the generation model will fix it.
What's actually true
Misconception
Chunking is a one-time setup step you configure once at launch.
What's actually true
How this shows up on the exam
Expect a RAG scenario with incomplete or wrong answers and a tempting 'upgrade the model' fix. The reliable reading is the six-stage pipeline, retrieval before generation, and upstream chunking/indexing bounding downstream quality, so retrieval problems are fixed in the retrieval pipeline. This foundational knowledge point unlocks chunking strategy trade-offs, connects to retrieval method selection by data shape, and sets up diagnosing retrieval failures rooted in chunking.
A RAG system omits details that exist in the source corpus. An engineer wants to upgrade the generation model to fix it. What is the sounder approach?
People also ask
What are the stages of a RAG pipeline?
Does retrieval happen before or after generation?
Can a better model fix poor retrieval?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.