Understanding the Bonferroni correction calculator — concept and principle
This calculator adjusts the significance level when you run several hypothesis tests at once. It gives the Bonferroni and Šidák thresholds, the risk of at least one false positive without correction, and, if you enter p-values, which ones remain significant and their adjusted values. Use it for A/B tests with several variants, multiple outcomes or subgroup analyses.
How it is calculated
Bonferroni threshold = α ÷ m
Šidák threshold = 1 − (1 − α)^(1/m)
Risk of any false positive, uncorrected (independent tests) = 1 − (1 − α)^m
Bonferroni-adjusted p-value = min(1, m × p)
- A test is significant after correction if p ≤ α ÷ m, which is the same as m × p ≤ α.
The principle
Each test at α = 0.05 has a 5 % chance of a false positive when there is no real effect. Run many tests and the chance that at least one is a false alarm grows: with 5 independent tests it is about 22.6 %. The Bonferroni inequality says the probability of any of several events is at most the sum of their probabilities, so testing each at α ÷ m keeps the overall risk at or below α, whether or not the tests are independent. The Šidák threshold is exact for independent tests and slightly less strict.
Worked example
Default inputs: α = 0.05, m = 5, p-values 0.003, 0.012, 0.021, 0.04, 0.31.
- Bonferroni threshold = 0.05 ÷ 5 = 0.01.
- Šidák threshold = 1 − 0.95^(1/5) = 0.0102062.
- Uncorrected risk = 1 − 0.95⁵ = 22.62 %.
- Only 0.003 is at or below 0.01: 1 / 5 significant (0.003).
- Adjusted p-values = 5 × p, capped at 1: 0.015, 0.06, 0.105, 0.2, 1.
| Overall significance level α | Bonferroni threshold α ÷ m |
|---|---|
| 0.0125 | 0.0025 |
| 0.025 | 0.005 |
| 0.0375 | 0.0075 |
| 0.05 | 0.01 |
| 0.0625 | 0.0125 |
| 0.075 | 0.015 |
| 0.1 | 0.02 |
How to read the result
A p-value that loses significance after correction is not proof of no effect; it means the evidence is not strong enough given how many tests were run. Bonferroni is conservative, especially with many or correlated tests, so it can miss real effects. For large screens, methods that control the false discovery rate, such as Benjamini–Hochberg, are common alternatives. Decide on the correction before looking at the data.
Common mistakes
- Choosing m after the fact, counting only the tests that looked interesting.
- Comparing adjusted p-values with α ÷ m (that corrects twice).
- Forgetting that each extra outcome or subgroup is another test.
- Treating non-significant as "no difference".
Frequently asked questions
What is the Bonferroni correction?
Dividing the significance level by the number of tests, so the overall false-positive risk stays at most α.
When should I use it?
When you run several tests and any single false positive would mislead your conclusion.
Is Bonferroni too strict?
It can be with many tests; Holm's method is uniformly more powerful and still controls the same error rate.
Sources: NIST/SEMATECH e-Handbook of Statistical Methods, §7.4.7.3 Bonferroni's method