Model-Based vs Deterministic Check Selection for the CCAR-P Exam

Two kinds of check, chosen by how well-defined the rule is

Every checkpoint on the guarded request path is implemented as one of two things: a model-based check that reasons about language, or a deterministic check that applies a fixed rule. The CCAR-P exam treats choosing between them as an apply-level skill, because the right answer depends entirely on the nature of the thing being checked. The organising question is simple: can the condition be exhaustively enumerated, or not?

If a condition can be written down completely - these exact terms, this schema, this length - a deterministic rule is the better tool. If it cannot - novel jailbreaks, toxicity, policy nuance that arrives in phrasings you never anticipated - a model-based classifier is the better tool. The exam rewards matching the check type to the decision point rather than reaching for one type everywhere.

Where each type fits along the path

At input screening, a model-based classifier earns its place when you are catching jailbreak or prompt-injection patterns that cannot be captured with a finite list of rules. A deterministic check fits when the rule is defined - a banned-term blocklist, a length limit, a format validator - because it is faster, predictable, and cannot be argued out of its verdict.

At output screening, a judge model is right when you are evaluating qualities that need language understanding, like toxicity or policy compliance. A deterministic check is right when you are looking for a known string, a forbidden field, or a schema violation that a validator catches with certainty.

At tool-call authorization, the answer is almost always deterministic. Authorization is a decision you must be able to prove and replay - an allowlist of permitted actions, an identity check, a scope validation. A model-based authorization decision is rarely appropriate, because it cannot be audited with the same certainty and can be rephrased into approving something it should have denied.

Why the two types fail differently

A model-based classifier can be evaded. A determined user can phrase an input so it slips past even the strongest judge model, because the classifier reasons about language and language can be manipulated. A deterministic rule fails the opposite way: it is brittle. It blocks exactly what it was programmed to detect and nothing more, so it misses anything it did not anticipate and over-blocks anything that merely resembles a restricted pattern.

Because their failure modes are opposite, the two are deployed in series. No single control catches everything, so you place a classifier to cover the open-ended cases a rule would miss, and a rule to cover the defined cases a classifier might be talked out of. Identifying what each one misses is the point: every gap should be deliberately covered by a different control rather than left open.

When a built-in classifier refuses

Not every screening decision is one you build. Anthropic's own streaming classifiers can intervene on a response, and the Messages API surfaces that as a refusal stop reason carrying a details object with a policy category and a readable explanation. On newer models the category names a class such as cyber or bio content; on models that do not return the details object, your handler must tolerate its absence and fall back to generic handling, and both fields are null when the refusal maps to no named category. Two practical rules follow. Read the category and route different refusal classes differently, rather than treating every refusal as one undifferentiated event. And once a refusal arrives, reset the conversation context before continuing - remove or rephrase the turn that triggered it, because sending the next request on the same refused context simply returns further refusals. Verify the current category list against the platform documentation rather than hardcoding it.

What the CCAR-P exam trips candidates on

The first trap is reaching for a judge model to authorize an action. A scenario describes a refund or a data-write that needs authorizing, and offers a "policy judge model" as an option. It is the wrong choice: a deterministic allowlist plus an identity check is both faster and provably correct, and unlike a classifier it can be replayed for an auditor. Authorization wants certainty, not judgement.

The second trap is monoculture - relying on one check type everywhere. Use only classifiers and you leave defined rules to a control that can be rephrased around; use only deterministic rules and you leave novel attacks to a control that only catches what it was built to catch. The credited designs layer both, and can say precisely which check covers which gap. The exam is testing whether you match the tool to the decision and then close the residual gap with the complementary tool.

Common misreadings to avoid

How this shows up on the exam

Domain 5 items present a checkpoint and ask which check type belongs there, or present a full design and ask what is misassigned. The reliable method is to ask whether the condition can be exhaustively enumerated: if yes, deterministic; if no, model-based - with authorization defaulting to deterministic almost every time. Then check that no gap is left uncovered by the complementary type.

This choice sits inside the three-point guarded request path, which defines the checkpoints you are assigning check types to, and it pairs with fail-open vs fail-closed design, which decides what each check does when it errors. It also underpins indirect injection via retrieved content and tool output, where the same classifier chosen for user input must be reused on retrieved content, and token-budget exhaustion as an attack surface, which is a textbook deterministic length check.