Editing Machine learning (section)

== Model assessments ==
Classification of machine learning models can be validated by accuracy estimation techniques like the [[Test set|holdout]] method, which splits the data in a training and test set (conventionally 2/3 training set and 1/3 test set designation) and evaluates the performance of the training model on the test set. In comparison, the K-fold-[[Cross-validation (statistics)|cross-validation]] method randomly partitions the data into K subsets and then K experiments are performed each respectively considering 1 subset for evaluation and the remaining K-1 subsets for training the model. In addition to the holdout and cross-validation methods, [[Bootstrapping (statistics)|bootstrap]], which samples n instances with replacement from the dataset, can be used to assess model accuracy.<ref>{{cite journal|last1=Kohavi|first1=Ron|title=A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection|journal=International Joint Conference on Artificial Intelligence|date=1995|url=https://ai.stanford.edu/~ronnyk/accEst.pdf|access-date=26 March 2023|archive-date=12 July 2018|archive-url=https://web.archive.org/web/20180712102706/http://web.cs.iastate.edu/~jtian/cs573/Papers/Kohavi-IJCAI-95.pdf|url-status=live}}</ref>

In addition to overall accuracy, investigators frequently report [[sensitivity and specificity]] meaning true positive rate (TPR) and true negative rate (TNR) respectively. Similarly, investigators sometimes report the [[false positive rate]] (FPR) as well as the [[false negative rate]] (FNR). However, these rates are ratios that fail to reveal their numerators and denominators. [[Receiver operating characteristic]] (ROC) along with the accompanying Area Under the ROC Curve (AUC) offer additional tools for classification model assessment. Higher AUC is associated with a better performing model.<ref>{{cite journal|last1=Catal|first1=Cagatay|title=Performance Evaluation Metrics for Software Fault Prediction Studies|journal=Acta Polytechnica Hungarica|date=2012|volume=9|issue=4|url=http://www.uni-obuda.hu/journal/Catal_36.pdf|accessdate=2 October 2016}}</ref>