The Four Categories a Production System Must Log

You can only investigate what you logged

When a production Claude system does something wrong, the investigation depends entirely on what was recorded at the time. If the data path involved was never logged, there is simply no evidence, and the incident becomes unreconstructable. The CCAR-P exam frames production logging as four specific categories that together capture enough to reconstruct almost any incident, and this remember-level knowledge point is about naming those categories and understanding why logging only errors is not enough.

The four categories

The request category records what went in: the model version used, the input token count, and a prompt identifier that says which prompt produced this call. The response category records what came back: the output token count, the latency, and the stop reason (why generation ended). The context category records the surrounding circumstances: the user role, the session ID, and whether caching applied to the request. The outcome category records what happened next: whether the downstream system accepted the output, and any rejection signals it produced.

Together these four span the whole lifecycle of a call, in, out, around, and after, which is what makes them sufficient to reconstruct an incident. Drop any one and a class of questions becomes unanswerable: without the outcome you cannot tell whether the output was actually used, without the context you cannot tell which user or session was affected, and so on.

Why logging only errors is not enough

The instinct carried over from traditional systems is to log errors and timeouts, the things that visibly break. For an LLM system that is precisely the wrong scope, because the dangerous failure mode is not a crash, it is a subtly wrong response that throws no error at all. Nothing in an error log flags a confidently incorrect answer, because mechanically nothing errored. To reconstruct that kind of failure, why a particular call produced a wrong-but-not-crashed output, you need the request, response, context, and outcome, none of which an error-only log captures. Logging only errors leaves the most important failures invisible.

Build it before you need it, for incidents and for approval

Two forces make this a design-time obligation rather than a later addition. First, observability built after the first incident cannot answer questions about what already happened: the historical log data was never designed to capture the field the incident turns out to hinge on, so the instrumentation arrives too late for the very incident that prompted it. The logging has to exist before the incident it will be used to investigate.

Second, a trustworthy audit trail is increasingly a precondition for approval, not just a debugging aid. Security reviewers are less and less willing to let an autonomous agent act at all without a reliable record of what it did, because an action taken but not logged is, to a reviewer, an action that cannot be allowed. So the four categories are not only how you debug, they are part of what earns the system permission to run.

What the exam trips candidates on

The first trap is logging only errors and timeouts, missing the request, response, context, and outcome categories needed to reconstruct a subtly wrong, not crashed, response. A scenario will show a team that logged exceptions only; the credited reading points out that the dangerous LLM failure never throws an exception, so the four categories must be logged to investigate it.

The second trap is adding observability instrumentation only after the first incident, then discovering the historical log data cannot answer the question the incident raised. A scenario will show a team scrambling to instrument after the fact; the correct answer establishes that logging must be designed in before incidents occur, because it cannot be retrofitted onto data that was never captured.

Common misreadings to avoid

How this shows up on the exam

Domain 4 questions on this knowledge point present a team with error-only logging, or one retrofitting observability after an incident. The reliable moves are to log all four categories, request, response, context, outcome, on every call, to build the logging before incidents rather than after, and to recognize the audit trail as a precondition for approving autonomous action.

This is the foundation of the monitoring task statement and the practical answer to the silent-failure problem in the core failure taxonomy. The raw records it captures are exactly what request-level tracing feeds into aggregate dashboards, the shared trace ID it enables underpins orchestrator-workers failure attribution, and logging every control's behaviour makes layered reliability controls observable.