Sampling and Non-Determinism in Claude for the CCDV-F Exam

What sampling is

In the previous knowledge point you saw that a model generates text by predicting the next token from everything before it. That prediction is not a single certainty. At each step the model produces a distribution over many possible next tokens, some far more probable than others. Sampling is the act of choosing an actual token from that distribution. Because there is a choice involved, and because that choice can favour variety, the same prompt run twice can pick different tokens and diverge into different outputs.

This is the heart of the concept for the Claude Certified Developer - Foundations (CCDV-F) exam: outputs are sampled, not looked up. The model is not a function that maps a prompt to one fixed string. It is a process that draws from probable continuations, and that draw is where variability enters.

Non-determinism: same prompt, different output

Because sampling introduces a choice, model output is non-deterministic. Send the exact same prompt twice and you may get two different answers. That is not a malfunction and not a sign that your prompt is wrong. It is the expected behaviour of a system that samples among probable next tokens. The CCDV-F exam explicitly tests candidates on this, because the instinct to expect byte-identical output across runs is so strong and so wrong.

The mental shift the exam wants is from "the model returns the answer" to "the model returns an answer drawn from a space of plausible ones." Two answers can both be correct, well-formed, and on-task while differing in wording, ordering, or detail. Once you hold that, non-determinism stops being surprising and starts being something you design around.

The randomness dial: focused versus varied

You are not stuck with whatever variability the model happens to produce. Sampling exposes a randomness setting, commonly a temperature, that shifts the balance. Lower randomness makes the model favour the most probable tokens, which produces output that is more focused, more predictable, and more repeatable across runs. Higher randomness lets the model reach for less probable tokens more often, which produces more variety and creativity at the cost of consistency.

The design guidance follows the task. For structured, format-sensitive, or factual work where you want the same shape every time, such as returning JSON in a fixed schema or extracting a specific field, lean toward lower randomness. For brainstorming, drafting, or generating diverse alternatives, higher randomness is an asset. But note the trap: lowering the setting reduces variation, it does not necessarily guarantee identical output on every run. The safe posture is to treat run-to-run difference as always possible and let the randomness dial narrow it, not eliminate the need to handle it.

Combining more examples with lower randomness is often the most reliable recipe for consistent output, which is why this concept pairs so naturally with model options and shot-based prompting: few-shot examples pin the format while the randomness setting pins how tightly the model sticks to the most probable continuation.

Why evaluation has to change

The most consequential implication, and the one the exam rewards, is what non-determinism does to testing. If output can vary run to run, then a single passing run tells you almost nothing about reliability. It might have been the lucky draw. The moment you accept that output is sampled, your evaluation approach has to account for variation: run a prompt multiple times, judge against acceptance criteria that allow for legitimate wording differences, and look at the distribution of outcomes rather than one snapshot.

This is why "it worked when I tried it" is a dangerous claim for an AI feature. Deterministic software either passes or fails a given input consistently; a sampled model can pass on Tuesday and fail on Wednesday with the identical prompt. Evaluating for non-determinism means asking "how often does this hold?" not "did it hold once?" That discipline carries directly into production, where the same variability is why an integrated SDK call must validate each response rather than trusting that the output looks the way it did during a single test.

Why this knowledge point matters

This concept sits at Bloom level understand and difficulty 2. You are not just recalling that output varies; you are expected to explain why it varies, how the randomness setting shifts it, and what that means for how you test. It builds directly on tokens, context windows, and next-token generation, because sampling is precisely the choice made at each next-token step you learned about there.

Get this right and two later habits become natural: you tune the randomness setting to the job instead of leaving it at a default and hoping, and you evaluate AI features the way you would evaluate a noisy measurement, across many runs, rather than trusting a single green result.

Common misreadings to avoid

How it shows up on the exam

CCDV-F questions here typically describe a developer surprised that output changed between runs, or a team that shipped after one passing test and hit intermittent failures. The correct answer always comes back to the two grounded facts: sampling means identical prompts can yield different outputs, and non-determinism means evaluation must account for run-to-run variation. Distractors lean on the deterministic-software intuition, that same input gives same output, or that one passing run is sufficient. Recognise those as the traps and the questions become straightforward.