Open main menu
Home
Random
Recent changes
Special pages
Community portal
Preferences
About Wikipedia
Disclaimers
Incubator escapee wiki
Search
User menu
Talk
Dark mode
Contributions
Create account
Log in
Editing
Confusion matrix
(section)
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
==Example== Given a sample of 12 individuals, 8 that have been diagnosed with cancer and 4 that are cancer-free, where individuals with cancer belong to class 1 (positive) and non-cancer individuals belong to class 0 (negative), we can display that data as follows: {| class="wikitable" style="text-align:center;" !Individual number !1 !2 !3 !4 !5 !6 !7 !8 !9 !10 !11 !12 |- ! style=background:#eeeebb | Actual classification | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 |} Assume that we have a classifier that distinguishes between individuals with and without cancer in some way, we can take the 12 individuals and run them through the classifier. The classifier then makes 9 accurate predictions and misses 3: 2 individuals with cancer wrongly predicted as being cancer-free (sample 1 and 2), and 1 person without cancer that is wrongly predicted to have cancer (sample 9). {| class="wikitable" style="text-align:center;" !Individual number !1 !2 !3 !4 !5 !6 !7 !8 !9 !10 !11 !12 |- ! style=background:#eeeebb | Actual classification | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 |- ! style=background:#bbeeee | Predicted classification | style=background:#aadddd | 0 | style=background:#aadddd | 0 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#aadddd | 0 | style=background:#aadddd | 0 | style=background:#aadddd | 0 |} Notice, that if we compare the actual classification set to the predicted classification set, there are 4 different outcomes that could result in any particular column. One, if the actual classification is positive and the predicted classification is positive (1,1), this is called a true positive result because the positive sample was correctly identified by the classifier. Two, if the actual classification is positive and the predicted classification is negative (1,0), this is called a false negative result because the positive sample is incorrectly identified by the classifier as being negative. Third, if the actual classification is negative and the predicted classification is positive (0,1), this is called a false positive result because the negative sample is incorrectly identified by the classifier as being positive. Fourth, if the actual classification is negative and the predicted classification is negative (0,0), this is called a true negative result because the negative sample gets correctly identified by the classifier. We can then perform the comparison between actual and predicted classifications and add this information to the table, making correct results appear in green so they are more easily identifiable. {| class="wikitable" style="text-align:center;" !Individual number !1 !2 !3 !4 !5 !6 !7 !8 !9 !10 !11 !12 |- ! style=background:#eeeebb | Actual classification | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ffffcc | 1 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 | style=background:#ddddaa | 0 |- ! style=background:#bbeeee | Predicted classification | style=background:#aadddd | 0 | style=background:#aadddd | 0 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#ccffff | 1 | style=background:#aadddd | 0 | style=background:#aadddd | 0 | style=background:#aadddd | 0 |- ! Result | style=background:#ffdddd | {{abbr|FN|False negative}} | style=background:#ffdddd | {{abbr|FN|False negative}} | style=background:#ccffcc | {{abbr|TP|True positive}} | style=background:#ccffcc | {{abbr|TP|True positive}} | style=background:#ccffcc | {{abbr|TP|True positive}} | style=background:#ccffcc | {{abbr|TP|True positive}} | style=background:#ccffcc | {{abbr|TP|True positive}} | style=background:#ccffcc | {{abbr|TP|True positive}} | style=background:#ffcccc | {{abbr|FP|False positive}} | style=background:#bbeebb | {{abbr|TN|True negative}} | style=background:#bbeebb | {{abbr|TN|True negative}} | style=background:#bbeebb | {{abbr|TN|True negative}} |} The template for any binary confusion matrix uses the four kinds of results discussed above (true positives, false negatives, false positives, and true negatives) along with the positive and negative classifications. The four outcomes can be formulated in a 2Γ2 ''confusion matrix'', as follows: {| class="wikitable" style="border:none; background:transparent; text-align:center;" align="center" | rowspan="2" style="border:none;" | | style="border:none;" | | colspan="2" style="background:#bbeeee;" | '''Predicted condition''' |- | style="background:#eeeeee;" | [[Statistical population|Total population]] <br /><span style="white-space:nowrap;">= P + N</span> | style="background:#ccffff;" | '''Positive (PP)''' | style="background:#aadddd;" | '''Negative (PN)''' |- | rowspan="2" {{verth|va=middle|cellstyle=background:#eeeebb;|'''Actual condition'''}} | style="background:#ffffcc;" | '''Positive (P)''' | style="background:#ccffcc;" | '''[[True positive]] (TP) <br />''' | style="background:#ffdddd;" | '''[[False negative]] (FN) <br />''' |- | style="background:#ddddaa;" | '''Negative (N)''' | style="background:#ffcccc;" | '''[[False positive]] (FP) <br />''' | style="background:#bbeebb;" | '''[[True negative]] (TN) <br />''' |- | colspan="4" style="border:none;" | <sup>Sources: </sup><ref>{{Cite book |last=Provost |first=Foster |title=Data science for business: what you need to know about data mining and data-analytic thinking |last2=Fawcett |first2=Tom |date=2013 |publisher=O'Reilly |isbn=978-1-4493-6132-7 |edition=1. ed., 2. release |location=Beijing KΓΆln}}</ref><ref> {{cite journal|last=Fawcett|first=Tom|date=2006|title=An Introduction to ROC Analysis|url=http://people.inf.elte.hu/kiss/11dwhdm/roc.pdf|journal=Pattern Recognition Letters|volume=27|issue=8|pages=861β874|doi=10.1016/j.patrec.2005.10.010|bibcode=2006PaReL..27..861F |s2cid=2027090 }}</ref><ref> {{cite journal|last=Powers|first=David M. W.|date=2011|title=Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation|url=https://www.researchgate.net/publication/228529307|journal=Journal of Machine Learning Technologies|volume=2|issue=1|pages=37β63}}</ref><ref> {{cite book|last=Ting|first=Kai Ming|title=Encyclopedia of machine learning|date=2011|publisher=Springer|isbn=978-0-387-30164-8|editor1-last=Sammut|editor1-first=Claude|doi=10.1007/978-0-387-30164-8|editor2-last=Webb|editor2-first=Geoffrey I.}}</ref><ref> {{cite web|last1=Brooks|first1=Harold|last2=Brown|first2=Barb|last3=Ebert|first3=Beth|last4=Ferro|first4=Chris|last5=Jolliffe|first5=Ian|last6=Koh|first6=Tieh-Yong|last7=Roebber|first7=Paul|last8=Stephenson|first8=David|date=2015-01-26|title=WWRP/WGNE Joint Working Group on Forecast Verification Research|url=https://www.cawcr.gov.au/projects/verification/|access-date=2019-07-17|website=Collaboration for Australian Weather and Climate Research|publisher=World Meteorological Organisation}}</ref><ref> {{cite journal|vauthors=Chicco D, Jurman G|date=January 2020|title=The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation|journal=BMC Genomics|volume=21|issue=1|page=6-1β6-13|doi=10.1186/s12864-019-6413-7|pmc=6941312|pmid=31898477 |doi-access=free }}</ref><ref> {{cite journal|author=Tharwat A.|date=August 2018|title=Classification assessment methods|journal=Applied Computing and Informatics|volume=17 |pages=168β192 |doi=10.1016/j.aci.2018.08.003|doi-access=free}}</ref> |} The color convention of the three data tables above were picked to match this confusion matrix, in order to easily differentiate the data. Now, we can simply total up each type of result, substitute into the template, and create a confusion matrix that will concisely summarize the results of testing the classifier: {| class="wikitable" style="border:none; background:transparent; text-align:center;" align="center" | rowspan="2" style="border:none;" | | style="border:none;" | | colspan="2" style="background:#bbeeee;" | '''Predicted condition''' |- | style="background:#eeeeee;" | Total 8 + 4 = 12 | style="background:#ccffff;" | '''Cancer'''<br/>7 | style="background:#aadddd;" | '''Non-cancer'''<br/>5 |- | rowspan="2" {{verth|va=middle|cellstyle=background:#eeeebb;|'''Actual condition'''}} | style="background:#ffffcc;" | '''Cancer'''<br/>8 | style="background:#ccffcc;" | 6 | style="background:#ffdddd;" | 2 |- | style="background:#ddddaa;" | '''Non-cancer'''<br/>4 | style="background:#ffcccc;" | 1 | style="background:#bbeebb;" | 3 |} In this confusion matrix, of the 8 samples with cancer, the system judged that 2 were cancer-free, and of the 4 samples without cancer, it predicted that 1 did have cancer. All correct predictions are located in the diagonal of the table (highlighted in green), so it is easy to visually inspect the table for prediction errors, as values outside the diagonal will represent them. By summing up the 2 rows of the confusion matrix, one can also deduce the total number of positive (P) and negative (N) samples in the original dataset, i.e. <math>P=TP+FN</math> and <math>N=FP+TN</math>.
Edit summary
(Briefly describe your changes)
By publishing changes, you agree to the
Terms of Use
, and you irrevocably agree to release your contribution under the
CC BY-SA 4.0 License
and the
GFDL
. You agree that a hyperlink or URL is sufficient attribution under the Creative Commons license.
Cancel
Editing help
(opens in new window)