- In short
- An LLM self review is when the same model instance that generated an output is then asked to critique it. Because that instance still holds the original reasoning in context, it tends to justify its choices rather than question them, so it catches far fewer issues than an independent review instance that starts with no prior context.
Why LLM self review falls short
An LLM self review is the natural first idea for catching mistakes: the model just produced an answer, so why not ask it to check that answer before you ship it? The problem is that the reviewing pass and the producing pass share the same context window. Every instruction, every intermediate reasoning step, and every trade-off the model talked itself into is still sitting in front of it. When a model reviews its own output in that state it does not scrutinise the work from scratch; it re-reads its own justification and nods along. That is why an LLM self review reliably under-performs independent review, and it is the central insight behind this knowledge point in Domain 4.
The exam frames this as a design property, not a quirk. You are expected to understand that the limitation comes from the architecture of the review, specifically from what the reviewer can see, rather than from the model occasionally getting things wrong. Two engineers can have the same skill, but the one who wrote the code is worse at spotting its flaws than the one who did not. The same asymmetry holds for a model judging text it just generated versus the same model judging text it has never seen before.
- LLM self review
- Asking the same model instance that produced an output to critique that output within the same context. Because the producing reasoning is still in the window, the instance is biased toward confirming its own work and catches fewer substantive issues than an independent reviewer.
The mechanism: retained context becomes confirmation bias
When the producing instance keeps its reasoning in context, three forces push it toward approval. First, it is anchored: the design decisions it already made look correct because it can see the chain of thought that led to them, and that chain reads as sound to its author. Second, reinforcement learning from human feedback trains assistant models to be agreeable and helpful, which nudges a model toward smoothing over a doubt rather than escalating it into a rejection. Third, there is no counter-evidence in the conversation: nothing argues against the output, so the path of least resistance is to confirm it.
The result is a review that polishes wording and tidies formatting while leaving the substantive logic untouched. It will rename a variable, add a comment, or suggest a cosmetic tweak, and then declare the work sound. A reviewer who did not write the work carries none of that baggage, which is exactly why mature engineering teams require code review by a second person, why scientific results are checked by people who did not run the experiment, and why editors do not edit their own manuscripts. The principle transfers cleanly to language models, with a different session standing in for a different person.
What an independent review instance sees differently
An independent review instance is a separate call, ideally a separate session, that receives only the artefact to be judged and the criteria to judge it against. It does not see the reasoning that produced the artefact, so it cannot be anchored by that reasoning. It reads the code or the extraction the way a new engineer would on their first day: interpreting what is actually written rather than what the author intended. Because it never agreed that the design was right, it is far more willing to say the design is wrong.
Anthropic makes the same recommendation in its evaluation guidance, where the standard pattern for grading model outputs notes that it is generally best practice to use a different model instance to evaluate than the instance used to generate the output. The independence is the entire source of value. You are not hoping a better prompt will make the author honest about its own mistakes; you are removing the bias at the structural level by ensuring the reviewer never had a stake in the answer. This is also why model-based grading, where a separate Claude instance scores a candidate response against a rubric, is a reliable evaluation building block in a way that self-scoring is not.
Inspect transcripts, not just the final answer
Part of why a model misses its own errors is that the producing instance tends to judge its final answer and its own visible reasoning, both of which can read as plausible even when something went wrong earlier in the work. In an agentic setting the model may call tools, and a mistake can hide in a tool call or a tool response rather than in the prose the model shows you. Anthropic's guidance on evaluating agents is to inspect the raw transcript, including tool calls and tool responses, instead of trusting the final summary, because a superficially convincing answer can mask a faulty step that a self-grading pass will happily approve.
The same guidance shapes how an independent reviewer should be set up. When you use one Claude instance to grade another, the grader should be calibrated against human experts so its judgements track what a person would decide, and it helps to give the grader an explicit way out, such as returning "Unknown" when the information is insufficient, rather than forcing a confident verdict it cannot support. For tool-using systems, pair that judgement with concrete metrics such as task accuracy, runtime, number of tool calls, token consumption, and tool errors, so the review rests on measured behaviour and not only on the model's own account of how it did.
A picture of the two review paths
The contrast is easiest to see as a flow. In both paths the same model produces the artefact; the only thing that changes is whether the reviewing call can see the reasoning that created it.
Why human review processes already solved this
It helps to remember that teams did not invent independent review because of language models; they invented it because authorship creates blind spots in people too. A pull request is reviewed by someone other than its author precisely so that approval is not granted by the person most motivated to approve. When you design an agentic system, you are recreating an organisation in miniature, and the same separation of duties applies. A model that both writes and signs off on its own work has no separation of duties at all, and the exam treats that as a recognisable anti-pattern. Recognising it lets you upgrade a fragile pipeline into a trustworthy one by adding a single independent step, which is the simplest, lowest-effort fix available and therefore the one the exam usually rewards.
How this is tested on the Claude Certified Architect exam
Domain 4, Prompt Engineering and Structured Output, accounts for 20 percent of the exam, and task statement 4.6 asks you to design multi-instance and multi-pass review architectures. This knowledge point is the foundation those architectures stand on. It surfaces most often inside Scenario 5, Claude Code for Continuous Integration, where a team wires Claude into a pipeline that both writes and reviews code and then wonders why obvious defects still reach the main branch. The exam wants you to recognise that a single instance grading its own output is structurally weak and to choose the option that introduces an independent reviewer.
You will rarely be asked to recite a definition of self review. You will be asked to read a scenario, notice that the only quality gate is the author grading itself, and identify that as the root cause of escaped defects. Distractors will tempt you toward tuning sampling parameters, swapping in a larger model, or adding more instructions to the self-review prompt. All of those leave the shared-context bias in place. The correct move is almost always to add a fresh instance that never saw the producing reasoning, because that is the only change that actually removes the bias.
Worked example
A CI pipeline asks one Claude instance to implement a database migration and then, in the same conversation, to review that migration before opening a pull request.
The implementation turn produces a migration that drops a column and recreates it with a new type, and the model explains in its reasoning that existing data can be safely discarded because the column is rarely used. The review turn runs in the same conversation, so the model still sees that justification. Asked to review, it confirms the migration looks correct, tidies a comment, and approves. The pull request merges and production data is lost.
Now change exactly one thing. The review runs as an independent instance that receives only the migration file and a rule stating that destructive migrations must preserve existing data or document a backfill. With no access to the earlier rarely-used rationalisation, the fresh instance reads the SQL literally, sees an unguarded column drop, and flags it as a data-loss risk that needs a backfill step.
Nothing about the model changed between the two runs. What changed is whether the reviewer could see, and be reassured by, the reasoning that created the problem. That single architectural choice, independent versus self review, is the difference between catching the defect and shipping it. It is also why the next knowledge points in this task statement build multi-pass and multi-instance structures on top of this idea rather than trying to make self review smarter.
When an LLM self review is still worth running
Self review is not worthless, and the exam expects nuance rather than a blanket rule. A producing instance can usefully catch its own syntax errors, obvious omissions against an explicit checklist, and output that violates a known schema, because those checks do not require the reviewer to disagree with a decision it already made. Self review is also cheap and fast, which makes it a reasonable first filter that removes noise before a more expensive independent pass looks at what remains.
The mistake is treating self review as sufficient. Use it to clean up the surface; use an independent instance to make the actual judgement about whether the work is correct. Holding both ideas at once, self review for cheap surface checks and independent review for substantive correctness, is exactly the kind of layered thinking task statement 4.6 is testing, and it sets up the multi-pass and confidence-routing patterns that follow.
How self review interacts with review criteria
The quality of any review, self or independent, also depends on the criteria the reviewer is handed, and the two factors interact in a way the exam likes to probe. Vague criteria make self review even weaker, because a model with no concrete standard to check against falls back on its own sense that the work looks fine, which is exactly the biased judgement you wanted to remove. Explicit, categorical criteria help an independent instance far more than they help a self-reviewing one, because the independent instance has no prior commitment to defend and can apply the standard literally.
This is why the false-positive trust problem and explicit categorical criteria sit alongside this knowledge point as related concepts: a robust review needs both an unbiased reviewer and a clear rubric, and self review undermines the first while doing nothing to supply the second. When you read an exam scenario, check both levers. If the reviewer is the author and the criteria are vague, two independent weaknesses are stacking, and the fix is to separate the reviewer and sharpen the standard rather than to reword the existing prompt.
Misconceptions to avoid
Misconception
If the model is capable enough, asking it to review its own output is just as good as a second reviewer.
What's actually true
Misconception
Self review fails because the model is inconsistent and randomly makes mistakes.
What's actually true
A continuous integration pipeline uses a single Claude instance to generate code and then, in the same session, to review that code before merging. Defects keep reaching the main branch despite the review step. What is the most accurate explanation?
People also ask
Can an LLM review its own output reliably?
Why is self review unreliable for AI-generated work?
What is an independent review instance?
Is self review as good as a second model checking the work?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
Model based grading
Why watch: Grading with a separate model instance is exactly the independent-review approach that avoids self-review bias.
More videos for this concept
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.