Editing Nonlinear dimensionality reduction (section)

=== Principal curves and manifolds ===

[[File:SlideQualityLife.png|thumb|300px| Application of principal curves: Nonlinear quality of life index.<ref>{{cite journal |first1=A. N. |last1=Gorban |first2=A. |last2=Zinovyev |arxiv=1001.1122 |title=Principal manifolds and graphs in practice: from molecular biology to dynamical systems |journal=[[International Journal of Neural Systems]] |volume=20 |issue=3 |year=2010 |pages=219–232 |doi=10.1142/S0129065710002383 |pmid=20556849 |s2cid=2170982 }}</ref> Points represent data of the [[United Nations|UN]] 171 countries in 4-dimensional space formed by the values of 4 indicators: [[Gross domestic product|gross product per capita]], [[life expectancy]], [[infant mortality]], [[tuberculosis]] incidence. Different forms and colors correspond to various geographical locations. Red bold line represents the '''principal curve''', approximating the dataset. This principal curve was produced by the method of [[elastic map]]. <ref>A. Zinovyev, [http://bioinfo-out.curie.fr/projects/vidaexpert/ ViDaExpert] - Multidimensional Data Visualization Tool [[Curie Institute (Paris)|Institut Curie]], Paris.</ref>]]

'''[[Principal curve]]s and manifolds''' give the natural geometric framework for nonlinear dimensionality reduction and extend the geometric interpretation of PCA by explicitly constructing an embedded manifold, and by encoding using standard geometric projection onto the manifold. This approach was originally proposed by [[Trevor Hastie]] in his 1984 thesis,<ref>{{cite thesis |first=T. |last=Hastie |url=https://apps.dtic.mil/dtic/tr/fulltext/u2/a148833.pdf |archive-url=https://web.archive.org/web/20190802195018/https://apps.dtic.mil/dtic/tr/fulltext/u2/a148833.pdf |url-status=live |archive-date=August 2, 2019 |title=Principal Curves and Surfaces |type=PhD  |publisher=Stanford Linear Accelerator Center, Stanford University |date=November 1984 }}</ref> which he formally introduced in 1989.<ref>{{cite journal|author1-last=Hastie|author1-first=T. |author1-link=Trevor Hastie|author2-last=Stuetzle|author2-first=W. |title=Principal Curves|journal=[[Journal of the American Statistical Association]]|date=June 1989|volume=84|issue=406|pages=502–6|doi=10.1080/01621459.1989.10478797 |url=https://web.stanford.edu/~hastie/Papers/Principal_Curves.pdf}}</ref> This idea has been explored further by many authors.<ref>{{cite book |editor-link=Alexander Nikolaevich Gorban |editor-first=A. N. |editor-last=Gorban |editor2-first=B. |editor2-last=Kégl |editor3-first=D. C. |editor3-last=Wunsch |editor4-first=A. |editor4-last=Zinovyev |url=https://www.researchgate.net/publication/271642170 |title=Principal Manifolds for Data Visualisation and Dimension Reduction |series=Lecture Notes in Computer Science and Engineering (LNCSE) |volume=58 |publisher=Springer |year=2007 |isbn=978-3-540-73749-0 }}</ref>
How to define the "simplicity" of the manifold is problem-dependent, however, it is commonly measured by the intrinsic dimensionality and/or the smoothness of the manifold. Usually, the principal manifold is defined as a solution to an optimization problem. The objective function includes a quality of data approximation and some penalty terms for the bending of the manifold. The popular initial approximations are generated by linear PCA and Kohonen's SOM.