Understanding the confusion matrix metrics calculator — concept and principle
A confusion matrix counts how a binary classifier's predictions line up with the truth: true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN). These four counts give every common quality metric for spam filters, screening tests and machine-learning models. This calculator returns F1, accuracy, precision, recall, specificity, balanced accuracy and the Matthews correlation coefficient (MCC).
How it is calculated
- Precision = TP ÷ (TP + FP): of everything flagged positive, how much was right.
- Recall (sensitivity) = TP ÷ (TP + FN): of all real positives, how many were found.
- Specificity = TN ÷ (TN + FP): of all real negatives, how many were correctly cleared.
- Accuracy = (TP + TN) ÷ total.
- F1 = 2 × precision × recall ÷ (precision + recall), the harmonic mean of the two.
- Balanced accuracy = (recall + specificity) ÷ 2.
- MCC = (TP×TN − FP×FN) ÷ √((TP+FP)(TP+FN)(TN+FP)(TN+FN)).
The principle
Each metric answers a different question. Accuracy is easy to inflate when one class dominates: if only 1% of cases are positive, predicting "negative" every time scores 99%. Precision and recall focus on the positive class, and the harmonic mean in F1 punishes a model that is strong on one but weak on the other. MCC is the Pearson correlation between predicted and actual labels, so it uses all four cells and is high only when the model does well on both classes.
Worked example
- Counts: TP = 85, FP = 10, FN = 15, TN = 890 (1,000 cases).
- Precision = 85 ÷ 95 = 89.47%; recall = 85 ÷ 100 = 85%; specificity = 890 ÷ 900 = 98.89%.
- Accuracy = 975 ÷ 1,000 = 97.5%.
- F1 = 2 × 0.8947 × 0.85 ÷ 1.7447 = 0.8718.
- Balanced accuracy = (0.85 + 0.9889) ÷ 2 = 91.94%.
- MCC = (85×890 − 10×15) ÷ √(95×100×900×905) = 75,500 ÷ 87,965 = 0.8583.
| True positives | F1 score |
|---|---|
| 21.2 | 0.6296296296 |
| 42.5 | 0.7727272727 |
| 63.8 | 0.8360655738 |
| 85 | 0.8717948718 |
| 106 | 0.8947368421 |
| 128 | 0.9107142857 |
| 170 | 0.9315068493 |
How to read the result
Here accuracy (97.5%) looks better than F1 (0.87) because negatives outnumber positives nine to one. F1 and MCC range up to 1; MCC runs from −1 (always wrong) through 0 (no better than chance) to +1 (perfect). Choose the metric by cost: when missing a case is costly, watch recall; when false alarms are costly, watch precision.
Common mistakes
- Reporting accuracy alone on imbalanced data.
- Swapping FP and FN; a false negative is a real positive the model missed.
- Comparing F1 values measured at different decision thresholds.
Frequently asked questions
Is recall the same as sensitivity?
Yes, recall, sensitivity and true positive rate are the same quantity.
What is a good F1 score?
It depends on the task and baseline. Compare it with a simple baseline on the same data.
Why is MCC sometimes undefined?
If any row or column of the matrix sums to zero, the denominator is zero; many tools then report 0.
Sources: B. W. Matthews, Comparison of the predicted and observed secondary structure of T4 phage lysozyme, Biochimica et Biophysica Acta, 1975 · D. Chicco and G. Jurman, The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy, BMC Genomics, 2020