Editing Minimum description length (section)

{{Short description|Model selection principle}}
'''Minimum Description Length''' ('''MDL''') is a [[model selection]] principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of [[Occam's razor]]. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data. 

MDL has its origins mostly in [[information theory]] and has been further developed within the general fields of statistics, theoretical computer science and machine learning, and more narrowly [[computational learning theory]].

Historically, there are different, yet interrelated, usages of the definite noun phrase "''the'' minimum description length ''principle''" that vary in what is meant by ''description'':
* Within [[Jorma Rissanen]]'s theory of learning, a central concept of information theory, models are statistical hypotheses and descriptions are defined as universal codes.
* Rissanen's 1978<ref>{{cite journal|last1=Rissanen|first1=J.|date=September 1978|title=Modeling by shortest data description|journal=Automatica|volume=14|issue=5|pages=465–471|doi=10.1016/0005-1098(78)90005-5}}</ref> pragmatic first attempt to automatically derive short descriptions, relates to the [[Bayesian Information Criterion]] (BIC).
* Within [[Algorithmic Information Theory]], where the description length of a data sequence is the length of the smallest program that outputs that data set.  In this context, it is also known as 'idealized' MDL principle and it is closely related to [[Solomonoff's theory of inductive inference]], which is that the best model of a data set is represented by its shortest [[self-extracting archive]].