Understanding the detection alert precision and base-rate calculator — concept and principle
Intrusion detection systems, antivirus engines and fraud filters are often described by their detection rate and false positive rate. Those numbers sound reassuring, but what analysts actually experience is precision: the share of alerts that are real. This calculator combines the event volume, how rare attacks are and the detector's two error rates to show how many alerts you get per day and how many of them are true.
How it is calculated
- Malicious events = events × malicious rate.
- True alerts (TP) = malicious events × detection rate (recall, or true positive rate).
- False alerts (FP) = benign events × false positive rate, where benign events = events × (1 − malicious rate).
- Alerts per day = TP + FP.
- Precision = TP ÷ (TP + FP) × 100%.
Enter all rates as percentages.
The principle
This is Bayes' theorem in counting form. When attacks are rare, the benign events vastly outnumber the malicious ones, so even a tiny false positive rate applied to a huge benign pool produces more alerts than the near-perfect detection applied to a small malicious pool. Stefan Axelsson described this "base-rate fallacy" for intrusion detection in 2000: the false positive rate, not the detection rate, is what limits usefulness. Precision depends on the base rate; recall and false positive rate do not.
Worked example
- Inputs: 1,000,000 events per day, malicious rate 0.01%, detection rate 99%, false positive rate 0.1%.
- Malicious events = 1,000,000 × 0.0001 = 100.
- True alerts = 100 × 0.99 = 99.
- Benign events = 999,900; false alerts = 999,900 × 0.001 = 999.9.
- Alerts per day = 99 + 999.9 = 1,098.9.
- Precision = 99 ÷ 1,098.9 = 9.01%. About nine out of ten alerts are false alarms.
| Events per day | Alerts per day |
|---|---|
| 250,000 | 274.725 |
| 500,000 | 549.45 |
| 750,000 | 824.175 |
| 1e+06 | 1,098.9 |
| 1.25e+06 | 1,373.625 |
| 1.5e+06 | 1,648.35 |
| 2e+06 | 2,197.8 |
How to read the result
A precision of 9% means analysts must look at about 11 alerts to find one real attack. Cutting the false positive rate from 0.1% to 0.01% here would drop false alerts to about 100 per day and raise precision to about 50%. Improving detection from 99% to 99.9% would barely change precision. Also track how many alerts your team can handle each day.
Common mistakes
- Judging a detector by its detection rate alone.
- Using a test set with half attacks and half benign events, which hides the base-rate effect seen in production.
- Forgetting that the 1% of attacks missed (false negatives) still matters even when precision improves.
Frequently asked questions
What is the difference between precision and recall?
Recall is the share of real attacks you catch; precision is the share of alerts that are real attacks.
How can precision be improved?
Lower the false positive rate with tuning, allow-lists, correlation of several signals, or a second-stage check before alerting.
Sources: S. Axelsson, "The Base-Rate Fallacy and the Difficulty of Intrusion Detection", ACM Transactions on Information and System Security 3(3), 2000 · T. Fawcett, "An Introduction to ROC Analysis", Pattern Recognition Letters 27(8), 2006