How can you read the confusion matrix

Confusion Matrix - Confusion matrix

Table layout to visualize performance; is also known as the error matrix
State positive (P)
the number of really positive cases in the data
Condition negative (N)
the number of true negative cases in the data

truly positive (TP)
Eq. with hit
true negative (TN)
Eq. with correct rejection
false positive (FP)
Eq. with false alarm, Type I errors, or underestimation
false negative (FN)
Eq. with failure, type II error, or overestimation

Sensitivity, recall, hit rate or true positive rate (TPR)
Specificity, Selectivity, or True Negative Rate (TNR)
Precision or Positive Predictive Value (PPV)
negative predictive value (NPV)
Miss Rate or False Negative Rate (FNR)
Fallout or False Positive Rate (FPR)
False Detection Rate (FDR)
incorrect omission rate (FOR)
Prevalence Threshold (PT)
Threat Score (TS) or Critical Success Index (CSI)

Accuracy (ACC)
balanced accuracy (BA)
F1 score
is the harmonious means for precision and sensitivity:
Matthews Correlation Coefficient (MCC)
Fowlkes-Mallows Index (FM)
Informedness or bookmaker informedness (BM)
Markedness (MK) or DeltaP (Δp)

Swell: Fawcett (2006), Piryonesi and El-Diraby (2020), Powers (2011), Ting (2011), CAWCR, D. Chicco and G. Jurman (2020, 2021), Tharwat (2018).

In the field of machine learning and in particular the problem of statistical classification, one Confusion matrix Also known as an error matrix, is a specific table layout that enables the visualization of the performance of an algorithm, typically an awake learning (in unsupervised learning it is usually called an appropriate matrix ). Each row of the matrix represents the instances in an actual class, while each column represents the instances in a predicted class or vice versa - both variants can be found in the literature. The name arises from the fact that it is easy to see if the system is confusing two classes (that is, often one is labeled incorrect).

It is a special type of contingency table with two dimensions ("actual" and "predicted") and identical sets of "classes" in both dimensions (each combination of dimension and class is a variable in the contingency table).

example

With a sample of 12 pictures, 8 from cats and 4 from dogs, cats belonging to class 1 and dogs to class 0,

Actual = [1,1,1,1,1,1,1,1,0,0,0,0],

Let us assume that a classifier that distinguishes between cats and dogs is trained. We take the 12 pictures and run them through the classifier. The classifier makes 9 accurate predictions and misses 3: 2 cats incorrectly predicted as dogs (first 2 predictions) and 1 dog incorrectly predicted as cats (last prediction).

Prediction = [0,0,1,1,1,1,1,1,0,0,0,1]

With these two labeled sets (Actuals and Predictions) we can create a confusion matrix that summarizes the results of testing the classifier:

cat dog
cat 6 2
dog 1 3

In this confusion matrix, the system judged of the 8 cat pictures that 2 were dogs and of the 4 dog pictures that 1 was cats. All correct predictions are on the diagonal of the table (highlighted in bold) so that the table can be easily checked visually for prediction errors as they are represented by off-diagonal values.

In terms of sensitivity and specificity, the confusion matrix is ​​as follows:

P. N.
P. TP FN
N. FP TN

Table of confusion

Comparison of mean accuracy and false negative (overestimation) percentage of five machine learning classification models (multiple classes). Models # 1, # 2, and # 4 have very similar accuracy, but different levels of false negatives or overestimation.

In predictive analysis, there is one Confusion table (sometimes called Called confusion matrix ) a table with two rows and two columns showing the number of false positives , false negative , true positive and true negative is stated . This allows for a more detailed analysis than just a fraction of the correct classifications (accuracy). The accuracy will lead to misleading results if the data set is not balanced. that is, when the number of observations in different classes varies greatly. For example, if there are 95 cats and only 5 dogs in the data, a particular classifier can classify all observations as cats. The overall accuracy would be 95%, but in more detail the classifier would have a detection rate (sensitivity) of 100% for the cat class but a detection rate of 0% for the dog class. In such cases, the F1 score is even more unreliable and would result in over 97.4%, while being informed removes such distortions and results in 0 as the probability of an informed decision for any form of guessing (always guess the cat). The confusion matrix is ​​not limited to binary classification and can also be used in classifiers with multiple classes.

According to Davide Chicco and Giuseppe Jurman, the Matthews Correlation Coefficient (MCC) is the most informative metric for evaluating a confusion matrix.

Assuming the above confusion matrix, the corresponding confusion table for the cat class would be:

cat Non-cat
cat 6 real positives 2 false negatives
Non-cat 1 false positive 3 true negatives

The final table of confusion would contain the averages for all classes combined.

Let's define an experiment P. positive instances and N negative instances for a particular condition. The four results can be expressed in a 2 × 2- Confusion matrix be formulated:

Expected condition
Total population Expected condition
positive
Expected condition
negative
Accuracy (ACC) = Σ Correctly positive + Σ Correctly negative /. Σ total population
Current state positive Really positive False negative ,
Type II error
True positive rate (TPR), recall, sensitivity (SEN), detection probability, performance = Σ True positive /. Σ Indeed positive False negative rate (FNR), miss rate = Σ false negative /. Σ Indeed positive
Current state negative False positive ,
Type I error
Really negative False positive rate (FPR), failure, probability of false positive = Σ false positive /. Σ Indeed negative Specificity (SPC), selectivity, true negative rate (TNR) = Σ true negative /. Σ Indeed negative
Prevalence = Σ Actually positive /. Σ total population Positive predictive value (PPV), precision = Σ Correctly positive /. Σ Predicted positively False Skip Rate (FOR) = Σ False Negative /. Σ Predicted negatively Positive probability ratio (LR +) = TPR /. FPR Negative probability ratio (LR−) = FNR /. TNR
Diagnostic Odds Ratio (DOR) = LR + /. LR−
False detection rate (FDR) = Σ false positive /. Σ Predicted positively Negative predictive value (NPV) = Σ True negative /. Σ Negatively predicted Matthews Correlation Coefficient (MCC) =
√ TPR · TNR · PPV · NPV - √ FNR · FPR · FOR · FDR
F. 1 Score = 2 PPV TPR /. PPV + TPR = 2 · Precision · Callback /. Precision + recall

Confusion matrices with more than two categories

The confusion matrices discussed above have only two conditions: positive and negative. In some areas, confusion matrices can have more categories. The following table, for example, summarizes the communication of a whistled language between two speakers, whereby zero values ​​have been omitted for the sake of clarity.

I e a Ö u
I fifteen 1
e 1 1
a 79 5
Ö 4 fifteen 3
u 2 2

See also

References