Output Evaluation and Validation·Task 2.2·Bloom: understand·Difficulty 2/5·8 min read·Updated 2026-07-14

Detecting Internal Contradictions in Long Outputs

Identify hallucinations, inconsistencies, and biases in responses

SUBy Solomon UdohReviewed by Solomon UdohAI-assisted · human-reviewed
In short
Internal contradictions occur when an early claim in a long output conflicts with a later one. Each passage can read as fine in isolation, which is why the contradiction hides in ordinary paragraph-by-paragraph reading. Catching it requires a dedicated consistency pass across the whole document, and numeric claims such as totals, market sizes, and dates are common sites for this drift.

The failure that a normal read cannot catch

Most review techniques assume the error is visible somewhere on the page. Internal contradiction is different: no single sentence is wrong. The Claude Certified Associate - Foundations (CCAO-F) exam highlights this pattern because it defeats the way people naturally read, which is sequentially, one paragraph at a time. A claim on page two and a conflicting claim on page eight can each be locally correct and mutually incompatible, and nothing in the act of reading forward will alert you to the clash.

This matters most in long outputs, where the distance between the two claims is exactly what hides the contradiction. The longer the document, the less likely you are to hold both ends in view at once, and the more room there is for a figure to drift between where it is introduced and where it is used.

Internal contradiction detection
Recognising that a long output can state one figure or claim early on and a conflicting one later, and that this only becomes visible with a dedicated consistency pass across the whole document. Each passage reads as fine in isolation, so the contradiction hides in ordinary paragraph-by-paragraph reading. Numeric claims such as totals, market sizes, and dates are frequent sites of the drift.

Why each passage reads fine alone

The reason contradictions are so easy to miss is that they are invisible locally. A paragraph that describes a "$2 billion addressable market" is internally coherent and correct-sounding. A later section that builds a projection on "the $2.6 billion market" is also internally coherent. Read either alone and nothing is wrong. The error exists only in the relationship between them, and relationships between distant passages are precisely what sequential reading does not surface. You finish each paragraph satisfied, and the conflict never announces itself.

The consistency pass

The remedy is a review pass with a different aim than reading for comprehension. A consistency pass holds the whole document in view and deliberately compares claims across sections: does the total in the summary match the sum of the parts, does the market size quoted up front match the one used in the projection, do the dates line up throughout. This is not a paragraph-by-paragraph read done more carefully; it is a distinct sweep whose only job is to reconcile claims that appear far apart. For long documents, this pass is not optional, because it catches the one class of error that ordinary reading structurally cannot.

Numbers are where the drift concentrates

Contradictions can be qualitative, but they cluster around numeric claims: totals, subtotals, market sizes, growth rates, and dates. Numbers are easy to restate slightly differently across a long generation, and the consequences of a mismatch are concrete, a projection built on the wrong base figure, a total that does not reconcile with its line items. So a consistency pass should treat every repeated or dependent figure as a checkpoint, confirming that the number introduced in one place is the same number used everywhere it recurs.

early vs late
the conflict lives between distant passages, not within one
reads fine alone
each passage is locally coherent
consistency pass
compare claims across the whole document

What the CCAO-F exam trips candidates on

The first trap is reviewing a long report section by section and never comparing figures across distant sections. This is the default reading mode, and it is exactly the mode a cross-section contradiction is built to survive. The credited answer adds a dedicated consistency pass, treating the section-by-section read as necessary but insufficient for a long document.

The second trap is assuming a document is internally consistent because each paragraph individually reads correctly. Local correctness is not global consistency. An output where every paragraph is fine can still contain a fatal contradiction between two of them. The exam rewards recognising that consistency is a property of the whole, checked across the whole, not inferred from the soundness of the parts.

Worked example

In a ten-page market analysis Claude produced, page two states the addressable market is 'roughly $2 billion' and page eight builds a five-year revenue projection on 'the $2.6 billion market.' You read the report section by section and it felt solid throughout. What went wrong, and how should the review have run?

Nothing in your section-by-section read felt wrong because nothing was locally wrong. Page two's "$2 billion" is a coherent, reasonable-sounding statement on its own, and page eight's projection off "the $2.6 billion market" is internally consistent as a piece of arithmetic. The defect is not inside either passage; it is in the relationship between them, and sequential reading never puts that relationship in front of you. By the time you reach page eight, page two's figure is long out of view.

The consequence is concrete: a five-year projection built on a base figure that contradicts the one the same document established six pages earlier. Whichever number is right, the projection is now unmoored from the analysis it is supposed to rest on, and a reader who trusts the projection inherits the error.

The review should have included a dedicated consistency pass in addition to the section read. That pass holds the whole document in view and treats every recurring figure, especially a market size the projection depends on, as a checkpoint to reconcile. Running it would have surfaced the $2 billion versus $2.6 billion clash immediately, because its only job is to compare claims across distant sections rather than to understand each section in turn. Numeric drift like this is exactly what the consistency pass exists to catch.

Common misreadings to avoid

Misconception

Reading a long document carefully section by section will catch its contradictions.

What's actually true

A cross-section contradiction is invisible to sequential reading because each passage is locally correct. Catching it requires a dedicated consistency pass that compares claims across the whole document, not a more careful paragraph-by-paragraph read.

Misconception

If every paragraph reads correctly, the document is internally consistent.

What's actually true

Local correctness is not global consistency. Every paragraph can be individually fine while two distant ones conflict. Consistency is a property of the whole and must be checked across the whole.

How this shows up on the exam

Domain 2 questions on this knowledge point describe a long output whose figures do not reconcile across sections and ask how the review should have caught it or what pass is missing. The dependable answer is a dedicated consistency pass across the whole document, with recurring numeric claims treated as checkpoints, rather than trusting a section-by-section read.

This pattern is one of the failure signatures from the hallucination pattern taxonomy and it complements spotting fabricated specifics, which targets a single uncited figure rather than two conflicting ones. It relates to the completeness half of accuracy and completeness as independent checks, and the numeric reconciliation it demands is often best done with code execution for numeric verification.

Check your understanding

A ten-page Claude analysis quotes a '$2 billion' market on page two and projects revenue off a '$2.6 billion' market on page eight. Your section-by-section read missed it. What would have caught this?

People also ask

How do you catch contradictions in a long AI output?
Run a dedicated consistency pass that holds the whole document in view and compares claims across distant sections, rather than only reading paragraph by paragraph.
Why do contradictions hide in long documents?
Each passage reads as fine in isolation, so nothing in ordinary sequential reading flags the conflict, and you rarely hold an entire long document in view at once.
What is a consistency pass?
A review pass aimed specifically at comparing claims across the whole document, checking that figures, totals, and dates in one section match those used in another.

Watch and learn

Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.

No videos curated for this concept yet

We are still curating the best official and community videos for this topic.

Official prep for this domain

Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.

References & primary sources

Adaptive study

Master this concept with Archie

Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.

Start studying