Editing Supervised learning (section)

=== Bias–variance tradeoff ===
{{Main|Bias–variance tradeoff}}

A first issue is the tradeoff between ''bias'' and ''variance''.<ref>S. Geman, E. Bienenstock, and R. Doursat (1992). [http://delta-apache-vm.cs.tau.ac.il/~nin/Courses/NC06/VarbiasBiasGeman.pdf Neural networks and the bias/variance dilemma]. Neural Computation 4, 1–58.</ref> Imagine that we have available several different, but equally good, training data sets. A learning algorithm is biased for a particular input <math>x</math> if, when trained on each of these data sets, it is systematically incorrect when predicting the correct output for <math>x</math>. A learning algorithm has high variance for a particular input <math>x</math> if it predicts different output values when trained on different training sets. The prediction error of a learned classifier is related to the sum of the bias and the variance of the learning algorithm.<ref>G. James (2003) Variance and Bias for General Loss Functions, Machine Learning 51, 115-135. (http://www-bcf.usc.edu/~gareth/research/bv.pdf)</ref> Generally, there is a tradeoff between bias and variance. A learning algorithm with low bias must be "flexible" so that it can fit the data well. But if the learning algorithm is too flexible, it will fit each training data set differently, and hence have high variance. A key aspect of many supervised learning methods is that they are able to adjust this tradeoff between bias and variance (either automatically or by providing a bias/variance parameter that the user can adjust).