- In short
- Prompt caching is most effective when the system prompt is long and stable across many requests. A cached prefix is billed at a fraction of the standard input rate on later reads, while the first write costs more than standard input, so caching pays off only when reads are frequent enough to outweigh the write. Caching also introduces a consistency window: if cached content needs to reflect live state, the cache can serve stale information.
Caching as a lever, and when it pays
Prompt caching is one of the strongest cost and latency levers in the three-input model, but only under specific conditions. The condition that makes it work is a system prompt that is long and stable across many requests. When a large prefix, detailed instructions, schemas, reference material, is identical on request after request, caching that prefix lets the API skip reprocessing it, cutting both the input cost and the latency of every subsequent call. The savings scale with two things: how long the cached prefix is and how often it is reused.
The exam frames this at the understand level, which means you need the economics and the risk, not just the headline. The headline is that caching cuts cost; the substance is knowing when it does and when it quietly does not.
- Prompt caching
- A cost and latency lever that stores the processed prefix of a prompt so later requests reusing that prefix are billed at a fraction of the standard input rate. It is most effective for long, stable system prompts, and it introduces a consistency window in which cached content can lag live state.
The read-write economics
Caching is not free on the first use. The initial write of a cached prefix costs more than standard input processing, and later reads of that prefix are billed at a fraction of the standard input rate. That asymmetry sets the break-even. If a prefix is written once and read many times, the cheap reads swamp the one expensive write and caching wins big. If a prefix is written but rarely reused, the write cost is never recovered and caching loses money relative to just paying standard input each time.
This is why "long and stable, across many requests" is the exact condition. Long makes the prefix worth caching; stable means the same prefix is reused rather than rewritten; many requests means enough reads to clear the write cost. Remove any of the three, a short prefix, a prefix that changes each call, low request frequency, and the economics tip against caching.
Frequency has a specific mechanical floor: a cached prefix lives for only a bounded time-to-live before it expires and the next call has to pay the write again. The default cache window is short, on the order of a few minutes, so a workload whose requests arrive further apart than that window keeps re-writing the cache instead of reading it, and never realises consistent savings. The rule of thumb is that requests must recur inside the cache lifetime for caching to pay off, so a steady, high-frequency stream benefits far more than sporadic traffic even when the prefix itself is long and stable.
The consistency window
The risk half of the knowledge point is consistency. A cache holds a snapshot of the prefix; between the write and a refresh, that snapshot is what every read sees. If the cached content is genuinely static, instructions, schemas, that is fine. But if the cached content needs to reflect live state, current prices, current inventory, a changing status, the cache can serve stale information for the duration of the consistency window. The saving is real, but it is bought with a staleness risk that is unacceptable when the content must be current.
The design rule that follows is to cache the stable part and keep the live part out of the cache, fetched fresh per request, which is the same static-versus-live-state separation the retrieval topic makes. Cache the long stable instructions; do not cache the field that has to be up to the second.
What the exam trips candidates on
Two traps. The first is enabling caching on a system prompt that changes on nearly every request, where the write cost outweighs any read savings, misapplying a lever whose whole value depends on reuse. The second is ignoring the consistency risk of caching content that needs to reflect current, changing state, which trades cost for staleness in exactly the case where staleness is unacceptable. The credited answer caches only long, stable, frequently reused prefixes and keeps live-state content out of the cache.
Worked example
A team wants to cut cost on a pricing assistant. Its prompt has two parts: a 6,000-token block of stable product descriptions and policy text identical on every call, and a short section holding the customer's current cart total and today's live promotional prices, which change per request and through the day. They propose caching the entire prompt. Assess.
Caching the whole prompt is half right and half dangerous, and splitting it is the point. The 6,000-token block of product descriptions and policy text is long and stable and appears on every call, so it is an ideal caching candidate: written once, read across many requests at a fraction of the input rate, easily clearing the higher write cost and cutting both cost and latency. Caching that block is exactly what the lever is for.
The live section is the problem. The cart total and today's promotional prices change per request and through the day, so they are live state, and caching them creates a consistency window in which the cache serves a stale total or an outdated promo price until it is refreshed. In a pricing assistant, serving yesterday's promotional price or a stale cart total is a real correctness failure, which is the second trap. The correct design caches the stable 6,000-token prefix and keeps the live cart-and-promo section out of the cache, supplying it fresh on each request. That captures the large, safe saving from the stable block while avoiding staleness on the fields that must be current, cache the static part, fetch the live part.
Common misreadings to avoid
Misconception
Turning on prompt caching always reduces cost, so it should be enabled on every prompt.
What's actually true
Misconception
Caching is safe for any content because it just stores a copy to reuse.
What's actually true
How this shows up on the exam
Expect a scenario weighing caching on a prompt that is either unstable or mixes stable and live content, and a question about whether or how to cache. The reliable reading is that caching pays off for long, stable, frequently reused prefixes and that live-state content must stay uncached to avoid staleness. This knowledge point builds on the cost and latency levers, pairs with live-state vs static knowledge retrieval, and is often the dominant lever in the dual-constraint justification capstone when the system prompt is long and stable.
A pricing assistant's prompt has a large stable block of policy text and a small section holding live promotional prices that change through the day. How should caching be applied?
People also ask
When is prompt caching most effective?
How does cached-prefix pricing work?
What is the consistency risk of caching?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
Official prep for this domain
Anthropic's own free prep module for this part of the syllabus, on the official prep course. Free with an Anthropic Academy sign-in.
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.