- In short
- Live A/B testing routes real users to the new version and measures real downstream behaviour, but exposes some users to a potential regression. Shadow testing runs the new version in parallel on a copy of live traffic, logs its outputs without ever serving them to a user, and scores them offline. Use live testing when the deployment can absorb bounded exposure and traffic is high enough to reach significance quickly; use shadow testing when a single bad output carries too much risk, traffic is too low for a live split, or the deployment is regulated and cannot expose users to an unvalidated change.
Full concept guide coming soon
We are building the in-depth, exam-aligned guide for this knowledge point. In the meantime, explore the prerequisites and related concepts below, watch the official Anthropic Academy lessons, and start an adaptive study session to master it with Archie.
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
No videos curated for this concept yet
We are still curating the best official and community videos for this topic.
References & primary sources
Adaptive study
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.