Breaking Behavior Changes Across Model Releases (CCDV-F)

A newer model is not a drop-in replacement

It is tempting to treat a model upgrade the way you treat a routine dependency bump: newer is better, so swap it in and move on. The CCDV-F exam wants you to reject that instinct. The central fact of this knowledge point is that a newer model can change output style, format adherence, or edge-case behavior, even when it is stronger on aggregate benchmarks. Overall capability and behavioural compatibility are different properties, and a release can raise the first while quietly changing the second in ways your application feels.

This matters because your application does not consume "a capable model" in the abstract; it consumes a specific model's concrete behaviour through prompts and parsing that were built and tuned against that behaviour. When you selected a tier using Opus, Sonnet, and Haiku use cases, you fixed a point in a capability family, but the underlying model at that tier can be revised. A revision can phrase things differently, adhere to a requested format a little more or less strictly, or handle an unusual input in a new way. None of that shows up as "worse", so none of it announces itself.

Where behavior drifts

The drift shows up in three places worth naming, because exam scenarios usually point at one of them. The first is output style: the same prompt can yield differently worded, differently structured, or differently formatted prose. If a downstream step reads that prose loosely, a style shift can throw it off. The second is format adherence. A newer model might follow a requested structure more or less rigidly than the old one, so JSON that always parsed can occasionally arrive shaped differently, or a field the old model reliably omitted now appears.

The third, and the most dangerous, is edge-case behavior. Most inputs behave the same across releases, which is exactly what lulls a team into assuming compatibility. The differences hide in the long tail: the unusual input, the ambiguous request, the boundary condition that the old model handled one way and the new one handles another. Because these cases are rare, a quick manual check almost never surfaces them, and the change only appears once real traffic hits the tail. That tail sensitivity is why the discipline in this knowledge point is built around systematic evaluation rather than spot-checking.

Pin the version, upgrade deliberately

The defence is a two-part discipline. First, applications pin a model version so behaviour is stable and reproducible. Pinning means naming a specific model version in your requests rather than an alias that could resolve to whatever is newest, so a release on Anthropic's side cannot silently change what your application receives. With a pinned version, an upgrade becomes a deliberate act you schedule, not an event that happens to you.

Second, you revalidate with evals before upgrading. An eval suite is a set of representative inputs with expected properties for the output, run automatically so you can compare two model versions objectively. Before adopting a new release, you run the suite against it and compare the results to the pinned version, looking specifically for the style, format, and edge-case shifts described above. Only when the new version passes do you promote it. This is the same evaluation muscle the exam emphasises elsewhere, applied here to migration: you do not trust that the new model is compatible, you measure whether it is.

Prompts tuned to old quirks

There is a subtler part of migration that the knowledge point calls out explicitly: migration includes checking prompts that depended on quirks of the prior version. Over time, prompts accumulate small workarounds shaped by how one specific model behaved. A phrase added because the old model kept over-explaining, an instruction repeated because it otherwise got ignored, a formatting nudge that compensated for a particular habit. These are invisible couplings to the old model's behaviour.

When the model changes, those couplings can misfire. A workaround for a habit the new model no longer has can now push it in the wrong direction, and an instruction that fought an old tendency can become redundant or even counterproductive. So a migration is not only "does the new model pass the evals", it is also "which of my prompt's compensations were for the old model, and do they still make sense". Reviewing prompts through that lens is part of the revalidation, not a separate afterthought, and it connects to the token accounting in token usage tracking because a new model can also change how many tokens the same task consumes.

What the exam tests you on

The CCDV-F exam trips candidates on two closely related mistakes. The first is upgrading the model in production and assuming behavior is unchanged. A scenario describes a team that swapped to a newer, more capable release directly in production because it scored better overall, and then saw a downstream integration break or output quality regress on certain inputs. The credited answer identifies the missing discipline: they should have pinned the old version, evaluated the new one, and promoted it only after it passed, rather than assuming a stronger model behaves identically.

The second trap is skipping the eval pass after a model migration. This is the same failure framed from the process side: even a team that intends to migrate carefully can shortcut the validation because the new model "is just better". The exam wants you to treat the eval pass as non-negotiable, because compatibility is a property you verify, not one you infer from a benchmark. When a question offers an option that upgrades based purely on overall capability or headline scores, treat that as the distractor; the option that revalidates against the application's own cases before promoting is the one to credit.

Putting it to work

This knowledge point closes task statement 5.3 by adding a time dimension to model selection. The tier you chose in Opus, Sonnet, and Haiku use cases and the balance you struck in quality, latency, and cost tradeoffs are both decisions about a specific model version, and this knowledge point reminds you that the version underneath can change. Pinning and revalidation are what make those earlier decisions durable rather than accidental.

To apply it, remember the sequence: pin, evaluate, review prompts, then promote. Never let a model change reach production on the strength of a benchmark alone, and never skip the eval pass just because the new release is supposed to be better. When an exam scenario shows a broken integration after an upgrade, the answer is almost always that this discipline was missing.