Editing Machine learning (section)

== {{anchor|Generalisation}} Theory ==
{{Main|Computational learning theory|Statistical learning theory}}
A core objective of a learner is to generalise from its experience.<ref name="bishop2006">{{citation|first= C. M. |last= Bishop |author-link=Christopher M. Bishop |year=2006 |title=Pattern Recognition and Machine Learning |publisher=Springer |isbn=978-0-387-31073-2}}</ref><ref name="Mohri-2012">{{Cite Mehryar Afshin Ameet 2012}}</ref> Generalisation in this context is the ability of a learning machine to perform accurately on new, unseen examples/tasks after having experienced a learning data set. The training examples come from some generally unknown probability distribution (considered representative of the space of occurrences) and the learner has to build a general model about this space that enables it to produce sufficiently accurate predictions in new cases.

The computational analysis of machine learning algorithms and their performance is a branch of [[theoretical computer science]] known as [[computational learning theory]] via the [[probably approximately correct learning]]  model. Because training sets are finite and the future is uncertain, learning theory usually does not yield guarantees of the performance of algorithms. Instead, probabilistic bounds on the performance are quite common. The [[bias–variance decomposition]] is one way to quantify generalisation [[Errors and residuals|error]].

For the best performance in the context of generalisation, the complexity of the hypothesis should match the complexity of the function underlying the data. If the hypothesis is less complex than the function, then the model has under fitted the data. If the complexity of the model is increased in response, then the training error decreases. But if the hypothesis is too complex, then the model is subject to [[overfitting]] and generalisation will be poorer.<ref name="alpaydin">{{Cite book |author=Alpaydin, Ethem |title=Introduction to Machine Learning |url=https://archive.org/details/introductiontoma00alpa_0 |year=2010 |publisher=The MIT Press |place=London |isbn=978-0-262-01243-0 |access-date=4 February 2017 |url-access=registration }}</ref>

In addition to performance bounds, learning theorists study the time complexity and feasibility of learning. In computational learning theory, a computation is considered feasible if it can be done in [[Time complexity#Polynomial time|polynomial time]]. There are two kinds of [[time complexity]] results: Positive results show that a certain class of functions can be learned in polynomial time. Negative results show that certain classes cannot be learned in polynomial time.