The measures
- Accuracy = (TP + TN) / everything. It's misleading when one answer is rare: a filter that marks nothing as spam is 99 % accurate if only 1 % of emails are spam.
- Precision = TP / (TP + FP). When it says yes, how often is it right?
- Recall, also called sensitivity, = TP / (TP + FN). Of the real yeses, how many did it find?
- Specificity = TN / (TN + FP). Of the real noes, how many did it correctly leave alone?
- F1 = 2TP / (2TP + FP + FN), the harmonic mean of precision and recall. It's only high when both are.
- F-beta weights recall β times as much as precision. Use β = 2 when missing a case is worse than a false alarm, such as safety checks, and β = 0.5 when false alarms are worse.
- Matthews correlation coefficient (MCC) uses all four counts and runs from −1 (always wrong) through 0 (no better than chance) to +1 (perfect). It stays honest when the classes are unbalanced.
- Balanced accuracy is the average of recall and specificity.
How sure can you be?
A test set is a sample, so each rate has some uncertainty. The tool shows a 95 % Wilson score interval for accuracy, precision and recall, which behaves well even for small samples and rates near 0 or 100 %. With only 10 test cases, 7 correct gives an interval of about 40 % to 89 %, which is why small evaluations can't separate two similar models.
When a measure would divide by zero, for example precision when the model never says yes, the tool shows it as undefined rather than guessing. Some libraries report 0 in that case instead.