Editing Supervised learning (section)

===Empirical risk minimization===
{{Main|Empirical risk minimization}}

In empirical risk minimization, the supervised learning algorithm seeks the function <math>g</math> that minimizes <math>R(g)</math>. Hence, a supervised learning algorithm can be constructed by applying an [[Optimization (mathematics)|optimization algorithm]] to find <math>g</math>.

When <math>g</math> is a conditional probability distribution <math>P(y|x)</math> and the loss function is the negative log likelihood: <math>L(y, \hat{y}) = -\log P(y | x)</math>, then empirical risk minimization is equivalent to [[Maximum likelihood|maximum likelihood estimation]].

When <math>G</math> contains many candidate functions or the training set is not sufficiently large, empirical risk minimization leads to high variance and poor generalization. The learning algorithm is able to memorize the training examples without generalizing well (overfitting).