FALSE CONFIDENCE METRICS

Threshold Drift

Threshold Drift (n.) When alert thresholds gradually loosen until they no longer catch real problems.

Explanation

Teams set thresholds. Alerts fire too often. They loosen the threshold. Repeat. Eventually the threshold is so loose it only catches catastrophic failures — missing everything in between.

Operational Example

A fraud alert originally fires at 100 dollars. Too many false positives. Raised to 500. Then 1000. Then 5000. Now it only catches the largest fraud events. Everything under 5000 goes undetected.

Why It Matters

Threshold drift is invisible. Nobody decides to stop catching fraud. It happens one adjustment at a time.

What Most Teams Get Wrong

They adjust thresholds to reduce noise without measuring what they stop catching.

What Strong Teams Do Differently

Track what falls below threshold after every adjustment. Measure the cost of what you stop detecting.

Related Terms

By Hasan Jaffal
The Second Mind — Weekly writing on AI, risk, and decisions.