- In short
- A Skill bundles executable code with an instruction set, so an untrusted Skill is a supply-chain risk: its malicious logic can run the moment it is invoked, before any conversation-level input or output screening can see it. The control is a pre-execution audit for anomalous calls and out-of-scope behaviour, backed by a runtime sandbox that runs the Skill with least privilege and no standing credentials, ending in an explicit verdict - approve, reject, or remediate and re-audit - with trust granted only to vetted, signed sources.
Why a Skill is a different class of risk
There is a common objection in the field: "Skills are a black box. I cannot see everything inside one until it runs, so how am I supposed to trust it?" The CCAR-P exam treats resolving that objection as an evaluate-level skill, because the answer is not "screen it like any other input" - it is to build a control that compensates for the fact that a Skill is code you cannot fully inspect at runtime.
A Skill is reusable, distributable code paired with an instruction set, bundled together and dropped into your environment. That distribution model is precisely what makes it a supply-chain risk rather than a conversation risk. An untrusted Skill can carry a code-execution exploit - logic that runs commands, reaches out to the network, or touches files the moment it is invoked. The malicious behaviour was baked into the bundle upstream, so it is a fundamentally different threat from an instruction a user types.
- Skill supply-chain auditing
- The control pattern for third-party Skills, whose bundled code can execute the moment the Skill is invoked - before conversation-level screening can see it. It combines a pre-execution audit for anomalous calls and out-of-scope behaviour with a runtime sandbox that runs the Skill under least privilege and no standing credentials, ending in an explicit verdict and backed by a trusted-source policy.
Why conversation screening cannot help
Input and output screening watch the conversation - the messages flowing to and from the model. A Skill's threat does not live in the conversation; it lives in the bundle. The malicious logic can include hidden instructions your input filters and prompt screening cannot see, because those controls inspect the dialogue, not the code that was dropped into your environment. Output monitoring might catch a downstream effect after the fact, but by then the code has already run and any damage - a credential read, a network exfiltration - is done.
That timing is the whole problem. The harmful action happens at invocation, upstream of every conversation-level control. So the defence has to move earlier in the chain, before the Skill is ever trusted and called. This is why the same architects who screen retrieved content for indirect injection still need a separate, earlier control for Skills: the injection defence screens text entering the context, while the Skill defence must inspect and confine code entering the environment.
Audit first, then confine at runtime
Before you trust and call a Skill, audit it: open the bundle and read it for two things. First, anomalous calls - network requests, shell execution, file-system access, credential reads. Second, out-of-scope operations - behaviour that does not match the job the Skill claims to perform. A formatting Skill that phones home is out of scope; a summariser that writes to disk is out of scope. The Skill's stated purpose is your audit baseline, and anything beyond it is a finding to investigate.
The audit alone is not enough, and knowing why is the deeper half of this knowledge point. Your audit tells you what is in the bundle you read - but a Skill that passes review clean can still reach out at runtime to fetch code that was never in the package you inspected. That is why the gate needs a net. Run Skills with least privilege and in a sandbox: limited file access, limited network, no standing credentials they do not need. The audit decides what gets in; the runtime confinement contains it if the audit missed something. You use both, because neither works perfectly alone - the sandbox is the fail-closed backstop for the audit's blind spot.
Trusted sources and an explicit verdict
Two disciplines complete the pattern. First, constrain where Skills may come from: trust only a vetted internal registry, verified publishers, and signed releases. A trusted-source policy shrinks the surface you must audit and stops untrusted bundles before they reach review. And do not assume the platform screens Skills for you - verify what automated vetting actually exists, read the documentation, and confirm exactly what any scanning does and does not catch.
Second, every audit must end in an explicitly recorded verdict: approve, reject, or remediate. Approve means it is clean and cleared for use. Reject means it does not enter the environment. Remediate means you found a fixable problem - strip the offending call, sandbox the operation, pin a safer version - and then re-audit before use. The recorded verdict plus the trusted-source policy are the compensating controls that let you act responsibly on a component you can never fully see.
What the CCAR-P exam trips candidates on
The first trap is assuming that because a Skill passed conversation-level input and output screening, its bundled code is safe to execute. The scenario shows a Skill flowing through the normal guardrails without incident and asks whether it is trustworthy. The credited reading is that those guardrails inspect the conversation, not the code, so they say nothing about what the bundle does when invoked - the risk was never in a place they could look.
The second trap is trusting that the platform automatically vets every Skill for malicious behaviour. Candidates who accept "the platform handles it" without verifying what the vetting covers take the bait. The credited stance is candour: confirm the actual scope of any automated scanning rather than assuming it catches everything, and layer your own audit and sandbox regardless.
Worked example
A team wants to adopt a third-party document-formatting Skill from a public source. It passed through their input and output screening without any flags, so an engineer proposes enabling it in production. As the architect, how do you evaluate this and what do you require before it runs?
The screening result is reassuring for the wrong reason. Input and output screening watch the conversation; a Skill's risk lives in the bundled code that runs at invocation, which those controls never inspect. So "it passed screening" tells you nothing about whether the formatting Skill also opens a network connection or reads credentials the moment it runs. The risk was never on the path the guardrails watch.
Before it runs, require the full pattern. Audit the bundle against its stated purpose: a formatter should format, so any network request, shell execution, filesystem write, or credential read is an out-of-scope finding to investigate. If the audit is clean, still confine it - run it in a sandbox with least privilege, limited file and network access, and no standing credentials - because a clean bundle can fetch further code at runtime that the audit never saw. Prefer a vetted registry or signed release over an arbitrary public source, and do not assume the platform vetted it; confirm what any scanning actually covers.
End with a recorded verdict. If the audit finds an out-of-scope call, the verdict is reject, or remediate-and-re-audit if the issue is fixable. Only an explicit approve, backed by the sandbox and a trusted source, clears it for production. The whole point is acting responsibly on a component you can never fully inspect at runtime.
Common misreadings to avoid
Misconception
A Skill that passed the system's input and output screening is safe to execute.
What's actually true
Misconception
The platform automatically vets every Skill for malicious behaviour, so no separate control is needed.
What's actually true
How this shows up on the exam
Domain 5 items describe adopting a third-party Skill and ask what control the architecture needs. The reliable method is to name the risk as a supply-chain risk - code in a bundle that runs at invocation - and require the two-part defence: a pre-execution audit against the Skill's stated purpose, plus a least-privilege sandbox as the runtime backstop, ending in a recorded verdict and constrained to trusted sources. Any answer that leans on conversation screening or on assumed platform vetting is incomplete.
This is the sharpest application of the LLM system risk taxonomy to a distribution-model threat, and it extends the injection reasoning from indirect injection via retrieved content and tool output to code rather than text. The sandbox is a concrete instance of the fail-open vs fail-closed principle - confinement is the fail-closed net for anything the audit misses.
A third-party Skill passed the team's input and output screening cleanly. Which evaluation and control set should the architect require before it runs in production?
People also ask
Why is a Skill a supply-chain risk rather than a prompt-injection risk?
What do you audit a Skill for before using it?
Why sandbox a Skill that passed its audit?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.