Designing the Feedback Loop: Evaluator and Escalation (CCAR-P)

Closing the loop deliberately

The four-stage architecture shape ends in a feedback loop, and this knowledge point is where you design that loop rather than merely noting it should exist. The Claude Certified Architect - Professional (CCAR-P) exam treats the choice as an apply-level skill because there are two distinct mechanisms for closing a system, and matching the wrong one to a failure mode leaves the failure uncaught. An evaluator-optimizer loop catches failures a rule or rubric can judge; a human-escalation path catches failures that need judgment. Choosing between them, or combining them, is how you turn a system that produces output into a system that can catch its own errors.

An evaluator-optimizer feedback loop checks the output against a rubric and triggers a refinement pass when the check fails. A human-escalation path routes low-confidence or high-stakes outputs to a person before delivery. Both are feedback loops in the four-stage sense; they differ in what kind of failure they can catch, which is exactly what the design decision turns on.

The evaluator-optimizer loop: for checkable failures

When a failure mode can be expressed as a rule or a rubric, an automated evaluator-optimizer loop is the efficient mechanism. The evaluator scores the output against the rubric, does the JSON conform to the required schema, does the extraction include all mandatory fields, does the answer cite a source, and when the output fails, a refinement pass regenerates it. The loop is bounded on both ends: it exits when the output passes the quality criterion or when it hits a retry limit, so a case the model cannot fix does not spin forever. This closes the loop without human cost, and it scales to high volume. The condition is that the failure has to be genuinely checkable: if you can write down what "correct" means precisely enough for a rubric to judge, the evaluator loop fits.

The human-escalation path: for judgment failures

Some failure modes cannot be reduced to a rule. Whether an answer strikes the wrong tone, carries a subtle bias, or creates legal risk is a judgment, not a rubric check, and an evaluator built to score it will miss the cases that matter. For those, the feedback loop is a human-escalation path: low-confidence or high-stakes outputs are routed to a person before they are delivered. The human is the mechanism because the failure requires human judgment to detect. Using an automated evaluator here creates the appearance of a control without the substance, since the evaluator cannot actually judge the thing that fails.

A human gate is placed by the risk and reversibility of the action it guards, not applied uniformly. Gate before any irreversible or high-stakes step a system would otherwise take autonomously, and sample lower-stakes actions rather than pausing on every one, so the review effort lands where a wrong call actually costs something. Treating every output as needing a human, or none of them, both miss this calibration.

No feedback path is the baseline failure

Before choosing between the two mechanisms, the architecture has to have one at all. A design that ends at a response-composer step, with neither an evaluator nor an escalation path, has no way to catch or correct systematic errors after launch. It will produce plausible output indefinitely while quietly making the same mistake, because nothing in the system is positioned to notice. Adding a feedback path is not a refinement; it is the difference between a system that can improve and one that cannot.

What the CCAR-P exam trips candidates on

The exam tests two traps. The first is drawing an architecture that ends at a response-composer step with no escalation or evaluator path at all. A scenario will present a clean pipeline that stops at output, and the credited answer notices the missing feedback path and adds an evaluator or escalation appropriate to the failure mode.

The second is adding an evaluator-optimizer loop for a failure mode that actually requires human judgment, such as bias, tone, or legal risk, rather than a rubric check. A scenario will offer an automated rubric as the fix for a judgment-based failure, and the reliable reading recognises that no rubric can reliably catch that failure and routes it to human escalation instead.

Common misreadings to avoid

How this shows up on the exam

Domain 1 questions on this knowledge point present an architecture and ask what feedback mechanism it needs, or critique a mismatched one. The reliable reading first checks that a feedback path exists at all, then matches the mechanism to the failure mode: rule- or rubric-checkable failures to an automated evaluator-optimizer loop, judgment-based failures to a human-escalation path, often both in one system.

This design builds on the four-stage architecture shape, which establishes that the loop must exist, and on automation vs augmentation, which decides when a human belongs in the chain. The evaluator-optimizer mechanism is one of the workflow sub-patterns, and the whole loop is a stage in end-to-end trade-off analysis.