Editing Boosting (machine learning) (section)

== Convex vs. non-convex boosting algorithms ==

Boosting algorithms can be based on [[Convex optimization|convex]] or non-convex optimization algorithms.  Convex algorithms, such as [[AdaBoost]] and [[LogitBoost]], can be "defeated" by random  noise such that they can't learn basic and learnable combinations of weak hypotheses.<ref>P. Long and R. Servedio.  25th International Conference on Machine Learning (ICML), 2008, pp. 608--615.</ref><ref name=long-criticism>{{cite journal |last1=Long |first1=Philip M. |last2=Servedio |first2=Rocco A. |date=March 2010 |title=Random classification noise defeats all convex potential boosters |url=https://www.cs.columbia.edu/~rocco/Public/mlj9.pdf |archive-url=https://ghostarchive.org/archive/20221009/https://www.cs.columbia.edu/~rocco/Public/mlj9.pdf |archive-date=2022-10-09 |url-status=live |journal=Machine Learning |volume=78 |issue=3 |pages=287–304 |doi=10.1007/s10994-009-5165-z |s2cid=53861 |access-date=2015-11-17|doi-access=free }}</ref> This limitation was pointed out by Long & Servedio in 2008.  However, by 2009, multiple authors demonstrated that  boosting algorithms based on non-convex optimization, such as [[BrownBoost]], can learn from noisy datasets and can specifically learn the underlying classifier of the Long–Servedio dataset.