- In short
- Diagnosing underspecification means reading a production prompt for what it fails to say: checking, for every requirement the output must meet, whether the prompt states it or merely hopes for it. Non-deterministic variation across otherwise identical requests is the primary symptom. The fix is to make the implicit explicit: restate the goal, name the format, and bound the constraint.
Reading a prompt for what it does not say
Writing a well-described prompt is one skill; diagnosing an existing one that misbehaves is another. The Claude Certified Architect - Professional (CCAR-P) exam treats this diagnostic work as an analyse-level competency. The move is counterintuitive: instead of reading the prompt for what it says, you read it for what it fails to say. A production prompt is producing inconsistent output, and the question is which unstated requirement is causing it. This is the applied, backward-looking counterpart to the Description competency, which describes how to write a prompt well in the first place.
The discipline is systematic, not intuitive. For every requirement the output must meet, you ask a single question: does the prompt state this, or does it merely hope for it? Every requirement the prompt only hopes for is a diagnosed gap.
- Underspecification gap diagnosis
- The practice of reading a production prompt for what it fails to say, by checking for each requirement the output must meet whether the prompt states it or merely hopes for it. The characteristic symptom is non-deterministic output variation across otherwise identical requests. The fix is to make the implicit explicit: restate the goal, name the format, and bound the constraint.
The symptom: variation across identical requests
The primary symptom of an underspecification gap is non-deterministic output variation across otherwise identical requests. When the same prompt, given effectively the same input, produces outputs that differ in ways that matter, the most likely cause is a requirement the prompt never pinned down. The model is filling a silent gap with an assumption, and because the assumption is not fixed, the output varies from call to call. Recognising this symptom is what points the diagnosis at the prompt rather than at the model.
This is important because the symptom is easy to misread. Inconsistent output looks like the model being unreliable, which tempts a team into blaming model quality or reaching for a bigger tier. But if the requirement was never stated, no model can be relied on to honour it consistently, and a more capable model will still improvise where the prompt is silent. The variation is a specification symptom, and treating it as a model symptom sends the fix in the wrong direction.
The fix: make the implicit explicit
The fix for a diagnosed gap is direct: make the implicit explicit. Restate the goal alongside the instruction so the model is not inferring what you are trying to achieve. Name the format so the shape of the output is fixed rather than assumed. Bound the constraint so the rule the output must respect is stated, not hoped for. Each of these closes a specific gap by turning something the model was left to guess into something the prompt now dictates. Once the requirement is stated, the model no longer has a gap to fill, and the variation that came from filling it stops.
What the exam trips candidates on
The two traps are the two wrong responses to inconsistent output. The first is fixing inconsistent output by adding more examples instead of first checking whether the requirement was ever stated. A scenario will show a team piling on few-shot examples to force consistency; the credited reading first diagnoses the missing requirement and states it. The second is assuming a single instance of bad output is a model quality problem rather than a prompt specification gap. A scenario will present one bad output as evidence the model is failing; the credited reading asks whether the requirement the output missed was ever specified.
Worked example
A prompt that classifies support tickets and writes a one-line rationale is producing rationales that are sometimes one sentence, sometimes a paragraph, and sometimes omitted entirely. An engineer proposes adding ten few-shot examples of good rationales to fix it. Diagnose the real problem and propose the correct fix.
The engineer's instinct treats the symptom as something to overwhelm with examples, which is the first trap. Before adding anything, the diagnosis reads the prompt for what it fails to say. The requirement is clear from the observed variation: the rationale should have a specific length and should always be present. Checking the prompt, neither is stated. It asks for "a rationale" without saying how long or that it is mandatory. That silence is the gap, and the model is filling it differently each time: one sentence here, a paragraph there, nothing at all when it decides the classification is self-evident.
Adding ten examples might nudge the model toward a length, but it does so by demonstration rather than by specification, leaving the actual requirement unstated and the behaviour still probabilistic. The correct fix makes the implicit explicit. Restate the goal, every ticket gets a rationale. Name the format, exactly one sentence. Bound the constraint, the rationale is required and must not be omitted. With the requirement stated, the variation resolves at its source, and if a couple of examples are still useful for tone, they now supplement a specified requirement rather than substituting for one.
The second trap would have appeared if the team had seen a single omitted rationale and concluded the model was unreliable. The omission is not a model failure; it is the predictable result of a requirement the prompt never stated.
Common misreadings to avoid
Misconception
Inconsistent output is best fixed by adding more few-shot examples.
What's actually true
Misconception
A single bad output means the model is unreliable and needs upgrading.
What's actually true
How this shows up on the exam
Analyse-level questions describe a prompt producing inconsistent or occasionally wrong output and ask for the diagnosis and fix. The reliable answer reads the prompt for the unstated requirement, identifies the underspecification gap behind the variation, and makes the implicit explicit rather than adding examples or blaming the model.
This knowledge point applies the Description competency diagnostically and connects to system prompt structural components, since the gaps you find usually map to a missing part of that structure. When a diagnosed constraint is high-stakes, stating it may not be enough, which is where structural enforcement of guardrails takes over.
A classification prompt's one-line rationales come back inconsistently: sometimes a sentence, sometimes a paragraph, sometimes missing. An engineer wants to add ten few-shot examples to fix it. What should the architect do first?
People also ask
How do you diagnose an underspecified prompt?
What causes inconsistent output across identical requests?
How do you fix an underspecification gap?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.