Editing Overfitting (section)

== Benign overfitting ==
Benign overfitting describes the phenomenon of a statistical model that seems to generalize well to unseen data, even when it has been fit perfectly on noisy training data (i.e., obtains perfect predictive accuracy on the training set). The phenomenon is of particular interest in [[deep neural networks]], but is studied from a theoretical perspective in the context of much simpler models, such as [[linear regression]]. In particular, it has been shown that [[overparameterization]] is essential for benign overfitting in this setting. In other words, the number of directions in parameter space that are unimportant for prediction must significantly exceed the sample size.<ref>Bartlett, P.L., Long, P.M., Lugosi, G., & Tsigler, A. (2019). Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 117, 30063 - 30070.</ref>