Claude Prompt Testing: CCDV-F Eval and Debugging Guide

Claude prompt testing, the systematic practice of running prompts against defined inputs and checking whether outputs meet specified criteria, cuts across two CCDV-F exam domains. Domain 6, Prompt and Context Engineering, carries 11.0% of the 53-item exam. Domain 4, Eval, Testing, and Debugging, adds another 2.6%. Together they account for roughly one question in eight. More importantly, Domain 2, Applications and Integration, which carries 33.1%, regularly embeds evaluation choices inside application design scenarios. This guide covers the eval patterns, regression workflows, and debugging techniques the exam tests, with working code throughout.

What is claude prompt testing and why does CCDV-F examine it?

Claude prompt testing is the practice of validating a prompt's behaviour across a representative distribution of inputs, not just the examples you used when writing it. The criteria can be structural (does the model return valid JSON?), semantic (does the answer address the question correctly?), or behavioural (does the model refuse an out-of-scope request?).

The CCDV-F exam tests this skill because untested prompts are a principal source of production failures in Claude-powered applications. Per the official CCDV-F exam guide (sourced 8 July 2026), Domain 4 specifically targets failure analysis, regression testing, and how to act on evaluation signals. Domain 6 reaches into how prompt design choices affect output quality and consistency across diverse inputs, including how context placement and few-shot examples interact with output reliability.

Passing the exam requires knowing not only how to write a strong prompt, but how to prove it is reliable at scale. As of 3 June 2026, over 10,000 individuals hold a Claude Partner Network certification, and practitioners who can build testable, observable systems are increasingly distinguishable from those who cannot.

How does CCDV-F weight testing across its eight domains?

The eight CCDV-F domains distribute 100 percentage points across skills a practising developer is expected to demonstrate. Domains 4 and 6 carry the most direct testing content, but Domain 2 (Applications and Integration, 33.1%) is the single largest domain overall and frequently includes scenario items where the correct architecture choice depends on an evaluation strategy.

Domain 2's 33.1% weight makes it the most tested area. Many Domain 2 items embed an implicit evaluation question: the right architecture often depends on how you plan to verify that the system is working. Candidates who treat evaluation as a standalone skill, separate from application design, tend to under-prepare for this interaction.

For a structured treatment of all domains, our Claude Certification Concepts library maps each area to exam task statements.

What eval patterns does the CCDV-F exam test?

The exam tests three broad evaluation categories. Each suits a different output type and cost profile, and the exam expects you to know when to use which.

Rule-based evals check structural properties programmatically. They are fast, cheap, and deterministic. Use them to verify schema conformance, length limits, forbidden string presence, and format correctness before spending tokens on more expensive checks.

import json

def check_json_output(raw_output: str, required_keys: list) -> dict:
    try:
        parsed = json.loads(raw_output)
    except json.JSONDecodeError:
        return {"pass": False, "error": "invalid_json"}

    missing = [k for k in required_keys if k not in parsed]
    return {"pass": len(missing) == 0, "missing_keys": missing}

Model-as-judge evals use a second Claude call to score outputs on dimensions that are hard to capture programmatically: coherence, relevance, tone, and completeness. They introduce a second model's biases and cost more per sample. The exam typically asks you to justify when this overhead is warranted given the application's error cost.

import anthropic, json

client = anthropic.Anthropic()

def llm_judge(question: str, candidate_answer: str, rubric: str) -> dict:
    prompt = (
        f"Evaluate the candidate answer against the rubric.\n"
        f"Question: {question}\nAnswer: {candidate_answer}\nRubric: {rubric}\n\n"
        "Return JSON with keys: score (1 to 5) and rationale (string)."
    )
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=256,
        messages=[{"role": "user", "content": prompt}]
    )
    return json.loads(response.content[0].text)

Human review remains the ground truth for ambiguous outputs, but it does not scale to production volume. Choose it when the stakes of a false positive or false negative are high enough that automated classifiers cannot be trusted. The exam rewards choosing human review selectively rather than reflexively.

The exam distinguishes between these three modes by asking which is appropriate for a described scenario. A scenario asking you to validate that an extraction pipeline returns fields in the correct schema rewards a rule-based answer. A scenario asking you to measure whether a customer-facing response is empathetic rewards a model-as-judge or human review answer, depending on the stated volume and cost constraints.

How do you build a regression test suite for Claude prompts?

A regression suite is a fixed dataset of inputs with known-good or known-bad expected outputs. You run it every time you modify a prompt to detect regressions before they reach users.

A well-structured suite has four components: representative input cases, expected criteria (not necessarily exact strings), a comparison function, and a threshold for acceptable pass rate.

TEST_CASES = [
    {
        "input": "Summarise this contract in under 50 words.",
        "doc": "<contract>Sample contract text.</contract>",
        "checks": [
            {"type": "max_words", "value": 55},
            {"type": "no_forbidden_tokens", "tokens": ["I cannot", "As an AI"]},
        ]
    }
]

def run_suite(cases: list, prompt_fn, threshold: float = 0.90) -> dict:
    results = [evaluate_case(c, prompt_fn) for c in cases]
    pass_rate = sum(r["pass"] for r in results) / len(results)
    return {"pass_rate": pass_rate, "passed": pass_rate >= threshold, "results": results}

Setting the threshold at 0.90 is a defensible production bar. A 100% requirement on a small test set is fragile and over-fits to the examples you happened to write. A 90% pass rate on a large, diverse set is stronger evidence of generalisation across real-world input variation. For CI integration, the suite runs as a pre-merge gate:

python run_evals.py --suite prompts/summarise_suite.json --threshold 0.90 || exit 1

The Prompt Engineering & Structured Output domain covers the prompt patterns that interact most with regression test design, including how few-shot examples shape output distributions and where schema constraints help or hurt.

What debugging techniques does the CCDV-F exam test?

When a prompt fails, the exam expects a structured diagnostic path rather than trial-and-error edits. The three-step approach the exam consistently rewards:

Step 1: Classify the failure. Is the output structurally wrong (invalid format, schema violation), semantically wrong (correct format but wrong answer), or behaviourally wrong (refusal, hallucination, scope violation)? Each class has a different root cause and a different proportionate fix.

Step 2: Isolate the variable. Change one element at a time: instruction wording, example selection, context length, output schema, or system prompt scope. Multiple simultaneous changes prevent attribution. Root-cause tracing, not broad rewrites, is what the exam rewards.

Step 3: Measure net pass rate. A fix that resolves one failing case while breaking two previously passing cases is not a fix. Always run the full regression suite after any prompt edit and report net pass rate change.

A common exam trap is misclassifying context placement failures as wording failures. When Claude ignores relevant content located near the middle of a long prompt, that reflects attention dilution, not unclear instructions. The proportionate fix is restructuring context placement, not rewriting the task description. See Context Management & Reliability for the patterns the exam tests in Domain 6 and Domain 5.

How does prompt testing extend to agentic and tool-using systems?

When prompts drive agents, testing becomes harder because outputs are actions, not strings. A prompt that works correctly in isolation may misroute tool calls or trigger agentic loop anti-patterns when composed with other components.

Domain 1 (Agents and Workflows, 14.7%) and Domain 8 (Tools and MCPs, 10.6%) both include scenario items where the implicit question is how you would verify the agent behaves correctly. The exam does not require you to memorise specific testing frameworks, but it expects you to recognise when a given eval approach fits an agentic context.

Three patterns the exam rewards:

Stub tools during testing so the eval suite does not trigger live side-effects. A get_customer_record tool can return a fixture; a send_email tool can log to a file instead of sending.

Assert tool call sequences, not just final answers. If an agent must call validate_input before process_payment, your eval should check that ordering:

def assert_tool_order(trace: list, first: str, second: str) -> bool:
    names = [s["name"] for s in trace if s.get("type") == "tool_call"]
    if second not in names:
        return True
    idx_first = next((i for i, n in enumerate(names) if n == first), None)
    idx_second = names.index(second)
    return idx_first is not None and idx_first < idx_second

Inject adversarial inputs that test refusal behaviour. Requests that cross safety or scope boundaries should produce a structured refusal, not a hallucinated answer. This connects to Domain 7 (Security and Safety, 8.1%), which the exam treats as directly relevant to how you design and test prompts in production.

The distinction between testing a prompt in isolation versus testing a composed agentic system is itself an exam-relevant concept. Isolated prompt tests catch wording and format failures early and cheaply. System-level tests catch integration failures that only appear when the prompt interacts with memory, tool state, or coordinator logic. A mature testing strategy runs both, with isolated tests in every pre-commit check and system-level tests in nightly or pre-release runs.

What should you practise before sitting the CCDV-F exam?

The CCDV-F is 53 items scored 100 to 1000, with a passing mark of 720. Unlike the CCAR-F architect exam, CCDV-F has no scenario bank; items are written directly against the skills in each domain. The raw-to-scaled conversion is not published, so the exam rewards thorough domain coverage rather than targeting a specific raw score.

The domain weights give clear preparation priorities. Domain 2 (33.1%), Domain 5 (16.8%), and Domain 1 (14.7%) together account for roughly 65% of the exam. Domains 4 and 6 combined (13.6%) reward candidates who can distinguish eval strategies and trace prompt failures under timed conditions. Scenario-based practice matters more than definition review because the exam tests practical judgment, not recall.

AI Skill Certs' CCDV-F prep offers adaptive study, Archie tutoring, and full 53-item practice exams in the real format, scored on the 100-to-1000 scale with 720 as the passing bar. AI Skill Certs is independent of and not affiliated with or endorsed by Anthropic.