Editing Receiver operating characteristic (section)

==ROC curves beyond binary classification==
The extension of ROC curves for classification problems with more than two classes is cumbersome.  Two common approaches for when there are multiple classes are (1) average over all pairwise AUC values<ref name="HandTill01">{{cite journal |last1=Till |first1=D.J. |last2=Hand |first2=R.J. |year=2001 |title=A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems |journal=Machine Learning |volume=45 |issue=2 |pages=171–186 |doi=10.1023/A:1010920819831 |doi-access=free}}</ref> and (2) compute the volume under surface (VUS).<ref name="Mossman99">{{cite journal |last=Mossman |first=D. |year=1999 |title=Three-way ROCs |journal=Medical Decision Making |volume=19 |issue=1 |pages=78–89 |doi=10.1177/0272989x9901900110 |pmid=9917023 |s2cid=24623127}}</ref><ref name="Ferri03">{{cite conference |last1=Ferri |first1=C. |last2=Hernandez-Orallo |first2=J. |last3=Salido |first3=M.A. |year=2003 |title=Volume under the ROC Surface for Multi-class Problems |pages=108–120 |book-title=Machine Learning: ECML 2003}}</ref> To average over all pairwise classes, one computes the AUC for each pair of classes, using only the examples from those two classes as if there were no other classes, and then averages these AUC values over all possible pairs. When there are {{math|''c''}} classes there will be {{math|''c''(''c'' − 1) / 2}} possible pairs of classes.

The volume under surface approach has one plot a hypersurface rather than a curve and then measure the hypervolume under that hypersurface. Every possible decision rule that one might use for a classifier for {{math|''c''}} classes can be described in terms of its true positive rates {{math|(TPR{{sub|1}},&nbsp;.&nbsp;.&nbsp;.&nbsp;,&nbsp;TPR{{sub|''c''}})}}. It is this set of rates that defines a point, and the set of all possible decision rules yields a cloud of points that define the hypersurface.  With this definition, the VUS is the probability that the classifier will be able to correctly label all {{math|''c''}} examples when it is given a set that has one randomly selected example from each class.  The implementation of a classifier that knows that its input set consists of one example from each class might first compute a goodness-of-fit score for each of the {{math|''c''{{sup|2}}}} possible pairings of an example to a class, and then employ the [[Hungarian algorithm]] to maximize the sum of the {{math|''c''}} selected scores over all {{math|''c''!}} possible ways to assign exactly one example to each class.

Given the success of ROC curves for the assessment of classification models, the extension of ROC curves for other supervised tasks has also been investigated. Notable proposals for regression problems are the so-called regression error characteristic (REC) Curves <ref name="bij2003regression">{{cite conference |last1=Bi |first1=J. |last2=Bennett |first2=K.P. |year=2003 |title=Regression error characteristic curves |url=https://www.aaai.org/Papers/ICML/2003/ICML03-009.pdf |book-title=Twentieth International Conference on Machine Learning (ICML-2003). Washington, DC.}}</ref> and the Regression ROC (RROC) curves.<ref name="hernandez2013rroc">{{cite journal |last=Hernandez-Orallo |first=J. |year=2013 |title=ROC curves for regression |journal=Pattern Recognition |volume=46 |issue=12 |pages=3395–3411 |doi=10.1016/j.patcog.2013.06.014 |bibcode=2013PatRe..46.3395H |hdl-access=free |hdl=10251/40252|s2cid=15651724 }}</ref> In the latter, RROC curves become extremely similar to ROC curves for classification, with the notions of asymmetry, dominance and convex hull. Also, the area under RROC curves is proportional to the error variance of the regression model.