- In short
- Claude cross validation is a self-correction technique where you have the model extract an independently computed value, such as a calculated total summed from line items, alongside the value stated in the document, then compare them. When the two disagree you raise a conflict flag. It catches semantic errors, values that are wrong but correctly typed, which schema validation passes straight through.
What claude cross validation is
Claude cross validation is the practice of making the model check its own extraction by deriving the same fact two ways and comparing the results. Instead of pulling a single total off a document and trusting it, you ask the model for two figures: the total as stated on the page, and a total it computes independently by summing the underlying line items. If the two agree, you have strong evidence the figure is right. If they disagree, you have caught an error that no amount of schema validation would have flagged, because both numbers are perfectly valid numbers, just not the same one.
This matters because structured output, enforced through tool use and JSON schemas, solves a narrower problem than people assume. A schema guarantees the syntax of your data, that totals are numeric and required fields are present, but it is silent on the semantics, on whether the value is true. Claude cross validation is the layer that addresses semantics, and it does so without a second model or a human in the loop for the common case.
- Claude cross validation
- A self-correction technique in which the model extracts an independently computed value, such as a total summed from line items, alongside the value stated in the source, and a comparison between them raises a conflict flag when they disagree, surfacing semantic errors that schema validation cannot.
Why a passing schema is not enough
A JSON schema is a shape checker. It can insist that invoice_total is a number, that currency is one of three allowed codes, and that no required key is missing. Every one of those checks can pass while the value is wrong: the model could transcribe 1,240.00 as 1240 when the true total is 1180, and the schema would wave it through because 1180 and 1240 are both valid numbers. These are semantic errors, errors of meaning rather than form, and they are precisely the failures that slip past structural validation and into your downstream systems, where a finance team eventually finds them the expensive way.
Anthropic's reliability guidance treats this category as a first-class concern. Its recommendations to verify claims against the source and to cross-check with techniques like best-of-N are all variations on one idea: do not trust a single generation about a fact you care about; obtain a second, independent view and reconcile them. Cross validation is that idea specialised to numeric and relational extraction, where a second view is cheap to compute and the comparison is exact.
Extracting calculated values alongside stated values
The implementation is a schema design choice. You expand the extraction object so the model must return both the stated figure and an independently derived one, plus a boolean it sets when they conflict. Asking for the calculated value forces the model to actually do the arithmetic over the line items rather than simply copy the printed total, and the printed total is exactly the field most likely to contain a transcription or rounding error.
{
"name": "extract_invoice",
"input_schema": {
"type": "object",
"properties": {
"line_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"description": { "type": "string" },
"amount": { "type": "number" }
},
"required": ["description", "amount"]
}
},
"stated_total": { "type": "number" },
"calculated_total": { "type": "number" },
"conflict_detected": { "type": "boolean" }
},
"required": ["line_items", "stated_total", "calculated_total", "conflict_detected"]
}
}
Your code then performs the authoritative comparison; you never rely on the model alone to judge the match. You sum the amount values yourself, compare that to stated_total within a small tolerance for rounding, and treat any disagreement, or any record where the model set conflict_detected, as needing attention.
Two ways to enforce the extraction schema
The cross-validation schema above only helps if Claude reliably returns exactly that shape, and Anthropic gives you two distinct primitives for that guarantee. The first is structured outputs, configured with output_config.format, which constrains the response itself to valid JSON matching a schema you supply. The second is strict tool use, enabled with strict: true, which guarantees schema validation on the tool name and the tool inputs when the extraction rides inside a tool call. The two solve different problems: structured outputs govern the response format, while strict tool use governs tool-call parameter validation, and Anthropic's structured outputs documentation describes them as features you can use separately or together.
Either route gives the cross-validation fields a dependable substrate. Once the model is forced to emit line_items, stated_total, calculated_total, and conflict_detected as a well-typed object, your code can recompute and compare without first wrestling free-form text into shape. It is worth being precise about ownership here: calculated_total, stated_total, and conflict_detected are fields you define in your own schema, not built-in API parameters. The platform guarantees the structure you ask for; the semantics of checking a calculated figure against a stated one are entirely your design. That division is the whole reason a passing schema never implies a correct value, and why the comparison has to live in your validator rather than in the model's self-report.
How claude cross validation works in the pipeline
In the pipeline, cross validation sits after extraction and before acceptance. The extraction produces the line items, the stated total, and the calculated total. Your validator recomputes the sum from the items as the ground truth, compares it against the stated total, and branches on the outcome. Agreement means the record passes with high confidence. Disagreement means you have a semantic conflict, and now the retry effectiveness boundary tells you what to do next: if the line items or total were merely misread, a retry with the specific discrepancy can fix it; if the source document itself is internally inconsistent, no retry will reconcile it and the record must go to a human.
Cross validation beyond totals
A summed total is the clearest example, but the same two-source idea applies to any fact a document expresses more than once or implies through a relationship. The technique generalises wherever an independently derivable value can be set against a stated one.
- Counts against collections. If a document states it contains twelve invoices and your extraction returns a list, compare the stated count to the length of the extracted list. A mismatch means you either missed an item or the stated count is wrong, and either way the record needs attention.
- Dates against ordering. When a record carries a start date and an end date, a cross-check that the end is not before the start catches transposed or misread dates that are each individually valid.
- Parts against a whole. A reported subtotal, tax, and grand total should reconcile: subtotal plus tax should equal the grand total. Extracting all three and checking the arithmetic surfaces an error in any one of them.
- Derived ratios against stated ones. A document that prints both a percentage and the raw figures it is computed from gives you a free consistency check, since you can recompute the percentage and compare.
In every case the pattern is identical to the totals example: extract enough to derive the fact two ways, then let deterministic code, not the model's self-assessment, decide whether they agree. The more redundancy a document carries, the more of these checks you can run for almost no additional cost.
Setting the tolerance correctly
A naive equality check between a calculated and a stated value will raise false conflicts constantly, because real documents round. A subtotal printed to the penny will not exactly equal a sum of line items that were themselves rounded, and a strict comparison would flag a perfectly correct invoice. The tolerance is therefore a real design parameter, not an afterthought. Set it too tight and you flood the review queue with rounding noise, eroding trust in the conflict flag until people start ignoring it. Set it too loose and a genuine error, a sixty-dollar discrepancy hiding under a generous threshold, slips through as agreement.
The right tolerance is domain-specific and worth calibrating against real data. For currency, a tolerance of a cent or two usually absorbs legitimate rounding while still catching meaningful errors. For counts, the only sensible tolerance is zero, because there is no such thing as a rounded number of invoices. Whatever you choose, make the tolerance explicit and visible in your validator rather than buried in a comparison operator, because it encodes a judgement about how much disagreement your business is willing to treat as noise, and that judgement deserves to be reviewable.
Designing conflict flags for inconsistent sources
Not every disagreement is the model's fault. Real documents contain genuine internal inconsistencies: a printed total that does not match its own line items because of a vendor error, a form where two sections contradict each other. A well-designed extraction captures this directly with a conflict_detected boolean so the model can report inconsistency in the source rather than silently picking one value and hiding the contradiction. That honesty is the same instinct behind allowing the model to express uncertainty: surfacing a conflict is far safer than papering over it with a confident-looking number. Records where the source itself conflicts belong in a review queue, because no automated correction can decide which of two contradictory figures the business should trust.
Worked example
An accounts-payable pipeline extracts invoices for automatic payment. One invoice lists line items that sum to 1,180.00, but the printed total reads 1,240.00 because of a supplier error. A plain schema-validated extraction returns 1,240.00 and the schema passes, so the wrong amount is queued for payment.
With cross validation in place, the schema now demands three things instead of one: the line items, the stated_total of 1240.00, and a calculated_total the model derives by summing the items, which comes out to 1180.00. Your validator does not trust either figure on faith; it sums the amount fields itself, gets 1180.00, and compares that to the stated 1240.00.
The two disagree by sixty dollars, well beyond any rounding tolerance, so the validator sets the record aside as a conflict rather than queuing it for payment. This is a semantic error the original schema accepted without complaint, because 1240.00 is a flawless number; it is simply the wrong one.
Now the retry boundary decides the next step. If the gap had been caused by the model misreading a line item, a corrective turn citing the specific discrepancy would let it re-read and reconcile. But here the source itself is inconsistent, the printed total genuinely does not match the printed items, so no retry can resolve which figure is correct. The record routes to a human, who contacts the supplier. The pipeline has prevented an incorrect payment that a schema-only design would have authorised silently.
Common misconceptions
Misconception
If the extraction passes JSON schema validation, the values are correct and can be trusted downstream.
What's actually true
Misconception
Cross validation means asking the model whether its own answer is correct.
What's actually true
How it shows up on the exam
The structured data extraction scenario is the natural home for this knowledge point, and the exam exploits a tempting fallacy: that enforcing a schema makes the data correct. A question will describe a pipeline whose extractions all validate cleanly yet still send wrong totals downstream, and ask for the fix. The assessable answer is to derive the total a second way and compare, recording a conflict when they diverge, because the failure is semantic and lives entirely outside what a schema can see. Distractors will push you toward stricter schemas, lower temperature, or more examples, all of which improve form or consistency but none of which check the value against an independent source.
An accounts-payable pipeline extracts invoice totals with a tool_use schema. Every extraction validates cleanly, yet finance reports that several queued payments do not match the invoices' line items. What change most directly catches these errors before payment?
People also ask
Does a passing JSON schema mean the extracted data is correct?
How do you catch an LLM extraction that is the wrong value but the right type?
What is a conflict_detected field used for?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
Code based grading
Why watch: Comparing computed values against stated values in code is the cross-validation that catches semantic errors schemas miss.
More videos for this concept
References & primary sources
- Anthropic Docs: Reduce hallucinations (verify with citations, best-of-N)Primary source
- Anthropic Engineering: Building effective agents (evaluator-optimizer)Docs
- Anthropic Docs: Tool use overview (structured output)Docs
- Anthropic Docs: Structured outputs (output_config.format, strict tool use)Primary source
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.