- In short
- When a RAG answer is missing facts that exist in the source corpus, the root cause is often a chunking or indexing configuration issue, not a generation-model limitation. Symptoms include correct-seeming but incomplete answers, or answers that reference the wrong section of a source document. The fix is adjusting chunk boundaries, size, overlap, or indexed metadata, not adding more instructions asking the model to be thorough.
When the fact is in the corpus but not in the answer
This is the analyse-level capstone of the RAG task statement: a system produces answers that are missing facts you know exist in the source corpus, and you have to locate the cause. The exam's reliable diagnosis is that this is usually a chunking or indexing configuration problem, not a limitation of the generation model. The logic follows straight from the pipeline: generation can only ground on what retrieval fetched, and retrieval can only fetch what chunking and indexing made findable. So a fact present in the corpus but absent from the answer most likely never reached the model, because a chunking or indexing decision kept it out of the retrieved set.
The diagnostic move is to resist blaming the visible stage (the model, the prompt) and trace back to the upstream stage that actually determines what gets retrieved. That is where the fix lives.
- Diagnosing chunking-rooted retrieval failures
- Attributing missing or incomplete RAG answers, for facts that exist in the corpus, to chunking or indexing misconfiguration rather than a generation-model limitation, and fixing by adjusting chunk boundaries, size, overlap, or indexed metadata rather than by adding prompt instructions or blaming the embedding model.
The symptoms that point to chunking
Two symptom shapes signal a chunking root cause. The first is a correct-seeming but incomplete answer: the response is right as far as it goes but omits a fact that exists in the source. That omission usually means the chunk holding the fact was never retrieved, because it was split from the context that would have matched the query, a chunk size or overlap problem, or because it was not indexed to be findable. The second is an answer that references the wrong section of a source document, which points to chunk boundaries that grouped content misleadingly or metadata that misdirected retrieval.
Both symptoms share the tell that the needed information exists in the corpus yet did not make it into the answer. That gap between 'in the corpus' and 'in the answer' is the fingerprint of a retrieval-pipeline failure, and it is what distinguishes a chunking problem from, say, a hallucination, where the model invents content not in the source at all.
The fix is in the pipeline, not the prompt
Because the cause is upstream, the fix is upstream: adjust chunk boundaries so the fact stays with its context, tune chunk size and overlap so it is not split or diluted, or fix the indexed metadata so the chunk is findable and filterable. What does not work is adding instructions telling the model to 'be thorough' or 'search harder.' The model cannot use a chunk it never received, so no amount of prompt encouragement recovers a fact that retrieval never fetched. Prompting the model to search harder is the RAG analogue of the capability-bloat trap of fixing a structural problem with wording.
The second misdirected fix is blaming the embedding model. When a relevant fact is missed, it is tempting to conclude the embeddings are poor, but if the fact was split from its context by a chunk boundary, even a perfect embedding model would embed the fragment, not the whole fact, so the miss is a chunking artifact, not an embedding-quality one. Swapping embedding models leaves the boundary problem in place. The credited fix adjusts the chunking and indexing that actually determined what was retrievable.
What the exam trips candidates on
Two traps. The first is prompting the model to search harder when the needed chunk was never retrieved because of a chunking gap, which cannot recover a fact the model never received. The second is blaming the embedding model for a retrieval miss caused by chunk boundaries splitting the relevant fact from its context, which swaps a component that was not the cause. The credited answer traces the miss to chunking or indexing and fixes it there.
Worked example
A RAG system over a technical manual gives answers that are correct but consistently omit a safety caveat that appears in the manual right after each procedure. One engineer wants to add 'always include any relevant safety warnings' to the prompt; another wants to switch embedding models. Diagnose and choose the real fix.
Start from the fingerprint: the safety caveat exists in the corpus (it is right there in the manual) but is consistently absent from the answers. That gap between 'in the corpus' and 'in the answer' marks a retrieval-pipeline failure, not a generation problem, so the question is why the caveat's chunk is not being retrieved. The clue is positional: the caveat appears right after each procedure. If chunk boundaries fall between the procedure and its trailing caveat, the caveat lands in a separate chunk from the procedure text the query matches on, so retrieval fetches the procedure chunk and leaves the caveat chunk behind. That is a chunk-boundary and overlap problem, the caveat is split from the context that would have pulled it in.
Now the two proposed fixes. Adding 'always include relevant safety warnings' to the prompt is the first trap: the model cannot include a caveat it never received, and it never received it because the caveat chunk was not retrieved, so the instruction changes nothing. Switching embedding models is the second trap: a better embedding model still embeds whatever chunk it is given, and if the caveat is in a separate fragment split from its procedure, an improved embedding of that fragment does not make retrieval fetch it alongside the procedure query. The real fix is in the chunking: adjust the boundaries so each procedure and its trailing safety caveat stay in the same chunk, or add enough overlap that the caveat appears in the procedure's chunk, and confirm the caveat is indexed to be findable. Fixing the boundary so the caveat travels with its procedure is what actually gets it retrieved and into the answer.
Common misreadings to avoid
Misconception
If a RAG answer misses a fact, instruct the model to search harder or be more thorough.
What's actually true
Misconception
A retrieval miss means the embedding model is not good enough and should be swapped.
What's actually true
How this shows up on the exam
Expect a RAG system missing facts that exist in the corpus, with tempting fixes of 'prompt the model to be thorough' or 'change the embedding model.' The reliable reading is to trace the miss to chunking or indexing and fix boundaries, size, overlap, or metadata. This capstone builds on chunk size and overlap and metadata-enriched indexing, applies the upstream-bounds principle from RAG pipeline core stages, and shares its diagnose-before-fixing discipline with the failure taxonomy.
A RAG system over a manual reliably omits a safety caveat that appears right after each procedure in the source. What is the correct fix?
People also ask
Why does a RAG system miss facts that exist in the source?
Is a retrieval miss a generation-model problem?
Should you blame the embedding model for a retrieval miss?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.