Editing Empirical risk minimization (section)

== Formal definition ==
In general, the risk <math>R(h)</math> cannot be computed because the distribution <math>P(x, y)</math> is unknown to the learning algorithm. However, given a sample of [[Independent and identically distributed random variables|iid]] training data points, we can compute an [[Estimate (statistics)|estimate]], called the ''empirical risk'', by computing the average of the loss function over the training set; more formally, computing the expectation with respect to the [[empirical measure]]:
: <math>\! R_\text{emp}(h) = \frac{1}{n} \sum_{i=1}^n L(h(x_i), y_i).</math>

The empirical risk minimization principle<ref>V. Vapnik (1992). [https://papers.nips.cc/paper_files/paper/1991/file/ff4d5fbbafdf976cfdc032e3bde78de5-Paper.pdf ''Principles of Risk Minimization for Learning Theory.'']</ref> states that the learning algorithm should choose a hypothesis <math>\hat{h}</math> which minimizes the empirical risk over the hypothesis class <math>\mathcal H</math>:
: <math>\hat{h} = \underset{h \in \mathcal{H}}{\operatorname{arg\, min}}\, R_{\text{emp}}(h).</math>
Thus, the learning algorithm defined by the empirical risk minimization principle consists in solving the above [[Mathematical optimization|optimization]] problem.