- In short
- The false positive trust problem is the dynamic where a high false-positive rate in a single finding category makes users distrust and ignore the system as a whole, not just that one category. The standard fix is to temporarily disable the offending category so trust recovers while you iterate on its prompt.
What the false positive trust problem is
The false positive trust problem is what happens when one category of findings is wrong often enough that users stop believing the entire system. It is a knowledge point about human behaviour as much as about prompts, which is why the exam places it at the analyse level. You are not asked to recall a fact; you are asked to diagnose why a technically capable review tool has been switched off in practice, and to locate the failure in a single category rather than in the model overall.
The trap is to treat trust as if it were tracked separately for each kind of finding. It is not. A developer who gets burned three times by a bogus security warning does not carefully preserve their confidence in the tool's logic-bug findings. They form one global judgement, the tool cries wolf, and that judgement then applies to every comment it posts. A single bad category can therefore zero out the value of several good ones.
- False positive trust problem
- The dynamic in which a high false-positive rate in one finding category collapses user trust in the whole system, causing accurate findings everywhere to be ignored. The remedy is to disable the noisy category while you iterate on its prompt, then re-enable it.
Why trust collapses across categories
Trust is expensive to build and cheap to lose because humans use it as a shortcut. The entire point of an automated reviewer is to let a person stop checking everything themselves. The moment the tool proves unreliable in any visible way, that shortcut breaks, and the rational response is to go back to reading the code directly and treating the tool's output as background noise. Notice that this reaction is correct for the user even though it throws away genuine value, which is exactly what makes the problem so corrosive.
Alert fatigue is the mechanism underneath. Each false positive demands that a human read it, investigate whether it is real, conclude that it is not, and dismiss it. That loop is not free; it costs attention and a little frustration every time. Run it often enough and people stop paying the cost: they skim, they batch-dismiss, and eventually they mute. Industry experience with automated reviewers is blunt about this, noting that inaccurate comments early on significantly damage user trust and that, at scale, false alarms push users to ignore alerts altogether. Precision is prioritised over recall in practice precisely because the cost of lost trust dwarfs the cost of a missed finding in most review settings.
The counter-intuitive fix: disable before you debug
The instinctive response to a noisy category is to keep it running and tune it in place, because turning off a check feels like lowering standards. The exam wants you to override that instinct. Leaving the bad category live means every review cycle spends more of your users' trust, and trust lost during the debugging window may not come back even after the prompt is fixed. You can be improving the prompt and still be losing the user.
Temporarily disabling the offending category breaks that bleed. With the noise gone, the remaining categories get a clean run and users re-learn that the tool is worth reading. Meanwhile you iterate on the disabled category's prompt against a labelled set offline, where a wrong answer costs you nothing and no one's trust is on the line. When its false-positive rate is back under your bar, you re-enable it into a system that still has credibility to extend to it. The sequence is disable, iterate, re-enable, and the order is the whole point.
- Disable the category with the runaway false-positive rate so the live system stops emitting noise.
- Iterate on that category's prompt offline, ideally with explicit criteria and concrete examples, measured against a labelled evaluation set.
- Re-enable it only once its measured false-positive rate is acceptable, returning it to a system that still has trust to spend.
Measuring the rate before you re-enable
"Re-enable once its false-positive rate is acceptable" only means something if you can measure that rate, which is why this knowledge point leans on Anthropic's guidance to define success criteria and build evaluations before you trust a change. The disabled category does not come back because the prompt feels better; it comes back because a number cleared a bar you set in advance.
The practical shape is a labelled evaluation set. Collect a representative batch of the inputs the category will meet in production and label each one for ground truth: this case genuinely contains the issue, that case is benign. Run the candidate prompt across the set offline and count how often it flags a benign case. That count over the total is the false-positive rate, and now "acceptable" is a threshold you chose rather than an adjective you hoped for. Because the work happens offline against fixed data, you can iterate the prompt freely, compare versions on the same cases, and never put a live user's trust at stake while you experiment.
Production signals feed the same loop. Anthropic's managed Code Review attaches a thumbs-up and thumbs-down to every comment and aggregates the reactions to tune the reviewer, and its pipeline runs a verification step that checks each candidate finding against the actual code behaviour specifically to filter out false positives before a human ever sees them. Both are the same instinct as the offline eval: turn the vague worry "is this category noisy" into a measured quantity, then act on the measurement rather than the impression.
Tracing a trust collapse in practice
Worked example
A nightly Claude review job covers security, performance, logic and style. Adoption craters in a month and the team asks to switch the whole thing off.
The complaint arrives as a blanket verdict: the bot is useless and nobody reads it. An architect who accepts that framing will either kill a valuable tool or waste weeks re-prompting all four categories at once. The analyse-level move is to refuse the blanket framing and look at the findings per category.
Pulling the logs apart tells a sharp story. The logic, performance and style categories are fine, with dismissal rates in line with a healthy reviewer. The security category, however, is flagging routine input handling as injection risk on roughly half its comments. Developers met those bogus warnings first and most often, formed a global it cries wolf judgement, and that judgement then suppressed their attention to the three healthy categories. The system was not broadly broken; one category broke it.
The remedy follows the order exactly. Security is disabled that afternoon, and within days dismissal rates on the other three categories fall as developers start reading again. The security prompt is rebuilt offline with explicit categorical criteria and verified against a labelled set of real and benign cases, and only when its false-positive rate clears the bar is it switched back on. Crucially, the rescue depended on diagnosing a single category rather than re-tuning the model wholesale. That diagnostic instinct, not the prompt edit, is what the exam is testing.
The numbers make the spread vivid. A category wrong on roughly half its comments means a developer meets a bogus warning about as often as a real one, and after a handful of those the rational move is to stop reading that stream, then to stop reading every stream. Disabling security did not lower the team's standards; it removed the single input that had been teaching developers the tool could not be trusted, and the three accurate categories inherited the credibility the bad one had been quietly burning.
When the diagnosis is harder
The clean case has one obviously noisy category and three healthy ones. Reality is sometimes messier, and the analyse-level skill is knowing how to respond when it is. If two categories are both noisy, disable both rather than agonising over which did more damage; trust is global, so a partial cleanup leaves a partial leak. If the categories cannot be told apart at all, because findings arrive as one undifferentiated stream, the first fix is structural: tag each finding with its category so you can measure and disable at that grain in the first place. You cannot isolate a category you cannot name.
Full disablement is also not the only dial. Anthropic's Code Review guidance points at a softer move for borderline areas, where instead of skipping a category outright you raise the bar so it only speaks when it is near-certain and the issue is severe. That keeps a thin stream of high-value findings alive while you rebuild confidence. Choosing between a hard disable and a raised bar depends on how toxic the category is: one wrong sixty percent of the time should go dark immediately, while one that is merely a little too eager can often be throttled in place. What never works is leaving a high false-positive category running at full volume and hoping users stay patient, because the trust they spend in the meantime does not refund itself when the prompt finally improves.
Common misreadings to avoid
Misconception
Users distrust only the specific category that produces false positives, so the other categories are unharmed.
What's actually true
Misconception
You should keep the noisy category running while you fix it, because disabling a check lowers your standards.
What's actually true
Misconception
The fix for a noisy category is to lower the whole system's sensitivity so it reports fewer findings everywhere.
What's actually true
How it shows up on the exam
Expect a scenario, not a definition. A stem will describe a review or extraction system whose adoption is falling, often naming one category that is clearly noisier than the rest, and ask for the most effective response. Distractors will tempt you to re-prompt everything at once, to add more categories, or to accept that the model is simply unreliable. The credited answer isolates the high false-positive category, takes it offline to restore trust, and iterates on it separately. Reading the situation as a single-category trust collapse, rather than a whole-system failure, is the analytical skill being assessed.
The Bloom's verb matters for predicting the answer shape. This is an analyse task, not a remember task, so a stem that merely defines false positives and asks for the definition back is the wrong target; the credited stems hand you symptoms, often a falling adoption curve alongside per-category dismissal data, and make you locate the cause. It is also not an evaluate task: you are not weighing precision against recall in the abstract, you are diagnosing one concrete failure and prescribing the disable-iterate-re-enable sequence. Spotting that the question wants a diagnosis, not a definition or a trade-off, is half the work.
An AI reviewer runs four finding categories. Over a month, developers begin ignoring all of its comments. Logs show three categories have normal dismissal rates, but the performance category is wrong on about 60 percent of its comments. What is the most effective next step?
A code-review bot emits findings as one undifferentiated stream with no category tags. Adoption is falling and the team suspects the security warnings are the problem, but cannot prove it. What should they do first?
People also ask
Why do false positives in one category reduce trust in all categories?
Should you disable a noisy review category while fixing the prompt?
How do false positives cause alert fatigue?
Watch and learn
Official Anthropic Academy lessons first, then hand-picked walkthroughs. Videos load only when you press play.
Running the eval
Why watch: Eval results expose false-positive rates per category, the signal behind the trust problem this KP describes.
More videos for this concept
References & primary sources
Master this concept with Archie
Practice it inside an adaptive study session. Archie, your Socratic AI tutor, tracks your mastery with Bayesian Knowledge Tracing and schedules the perfect next review.