The Five Evaluation Dimensions for Production Claude Systems

Why one metric never tells the whole story

A system that answers correctly can still be the wrong system to ship. It might answer correctly but too slowly for the workflow it sits in, or correctly but at a monthly bill that blows past the budget signed off at approval, or correctly while quietly leaking a field it should never have touched. The Claude Certified Architect - Professional (CCAR-P) exam treats production evaluation as a five-part question precisely because architects who fixate on a single number keep discovering the other four the hard way, in production, after launch.

The five dimensions are accuracy, latency, cost, safety, and security. They are not a ranked list where accuracy matters most and the rest are optional. They are five independent axes, and a deployment is only production-ready when it clears the threshold on all of them at once. This knowledge point is a remember-level foundation: name the five, keep safety and security apart, and understand that each carries its own metric.

The five dimensions, one at a time

Accuracy asks whether the output matches the expected or correct result. For a field-extraction task that means the extracted values are right; for a summary it means the summary is faithful to the source. Accuracy is the dimension most teams instinctively measure, and it is genuinely necessary, but it is only the first of five.

Latency asks how fast the response arrives, and in production the number that matters is the tail, not the median. A demo that runs one request at a time hides the slow requests that breach a service-level agreement under concurrent load. Latency is a design target you set against the workflow's tolerance, and it lives on its own axis from accuracy: a correct answer that arrives too late has still failed the user.

Cost asks what the workload spends, priced separately for input and output tokens and projected to production volume rather than a demo's handful of calls. A system can be accurate and fast and still be infeasible because the bill exceeds the ceiling.

Safety asks whether the content itself is harmful, unsafe, or ungrounded, a fabricated claim, an unsafe instruction, a confident answer with no basis. Security asks a different question entirely: whether data is exposed to logs it should not reach, or accessed by a user who is not authorized to see it. Conflating the two is a classic error, because the fix for each is different, redirect the model's content behaviour versus lock down the data path.

Each dimension carries its own threshold

Because the dimensions are independent, each needs a metric and a numeric bar of its own. Accuracy might be gated at 99% on structured fields, latency at a p95 under three seconds, cost at a monthly ceiling, safety at a maximum tolerated rate of ungrounded outputs, and security at zero tolerance for cross-record leakage. A single blended "quality score" hides exactly the failure the five-dimension framing is designed to surface: a system that looks fine on average while one axis is quietly out of bounds.

Each dimension also carries its own grading method, and picking it follows the behaviour being checked rather than the stakes. Accuracy on known fields, latency against a target, cost against a ceiling, and cross-record leakage are all deterministic, so they are graded in code even when the stakes are high, a security check that is checkable by scanning for identifiers stays a code-based eval, not a human review. High stakes do not automatically escalate a check to a human. A single dimension can also need two methods at once: for a claims system, the safety axis splits into a code-based check on the binary "did it auto-deny" action and an LLM-judge check on whether the summary is faithful to the source, because faithfulness needs interpretation a function cannot supply. Mapping the method to the behaviour, not to the perceived risk, is what keeps the eval suite both cheap and trustworthy.

What the exam trips candidates on

Two traps recur. The first is treating accuracy as the only metric that matters and skipping cost, latency, safety, or security evaluation entirely. A scenario will describe a system that scores well on correctness and ask whether it is ready to ship; the credited reading refuses to certify it without evidence on the other four dimensions.

The second is conflating safety and security as if they were one dimension. A scenario will describe an incident, a fabricated response versus a leaked identifier, and the wrong answer applies the mitigation for the other category. Safety failures call for grounding and content controls; security failures call for data-handling and access controls. Keeping the two apart is the whole point of listing them separately.

Common misreadings to avoid

How this shows up on the exam

Domain 4 uses this knowledge point as the vocabulary the rest of the domain is built on. Expect questions that hand you a system strong on one dimension and ask what a complete evaluation still needs, or that describe an incident and ask you to name which dimension failed. The reliable move is to run the five-dimension checklist and keep safety and security distinct.

From here the natural next step is understanding that these dimensions are defined before you write production code, not audited afterward. The dimensions also connect forward to choosing the right eval type for each one, to cost and latency modeling for two of the five axes, and to what a production system must log so every dimension stays observable after launch.