Editing Principal component analysis (section)

=== Nonlinear PCA ===
[[File:Elmap breastcancer wiki.png|thumb|300px| Linear PCA versus nonlinear Principal Manifolds<ref>[[Alexander Nikolaevich Gorban|A. N. Gorban]], A. Y. Zinovyev, [https://arxiv.org/abs/0809.0490 "Principal Graphs and Manifolds"], In: ''Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods and Techniques'', Olivas E.S. et al Eds. Information Science Reference, IGI Global: Hershey, PA, USA, 2009. 28–59.</ref> for [[Scientific visualization|visualization]] of [[breast cancer]] [[microarray]] data: a) Configuration of nodes and 2D Principal Surface in the 3D PCA linear manifold. The dataset is curved and cannot be mapped adequately on a 2D principal plane; b) The distribution in the internal 2D non-linear principal surface coordinates (ELMap2D) together with an estimation of the density of points; c) The same as b), but for the linear 2D PCA manifold (PCA2D). The "basal" breast cancer subtype is visualized more adequately with ELMap2D and some features of the distribution become better resolved in comparison to PCA2D. Principal manifolds are produced by the [[elastic map]]s algorithm. Data are available for public competition.<ref>{{cite journal |last1=Wang |first1=Y. |last2=Klijn |first2=J. G. |last3=Zhang |first3=Y. |last4=Sieuwerts |first4=A. M. |last5=Look |first5=M. P. |last6=Yang |first6=F. |last7=Talantov |first7=D. |last8=Timmermans |first8=M. |last9=Meijer-van Gelder |first9=M. E. |last10=Yu |first10=J. |title=Gene expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer |journal=[[The Lancet]] |volume=365 |issue=9460 |pages=671–679 |year=2005 |doi=10.1016/S0140-6736(05)17947-1 |pmid=15721472 |s2cid=16358549 |display-authors=etal}} [https://www.ihes.fr/~zinovyev/princmanif2006/ Data online]</ref> Software is available for free non-commercial use.<ref>{{cite web |first=A. |last=Zinovyev |url=http://bioinfo-out.curie.fr/projects/vidaexpert/ |title=ViDaExpert – Multidimensional Data Visualization Tool |work=[[Curie Institute (Paris)|Institut Curie]] |location=Paris }} (free for non-commercial use)</ref>]]

Most of the modern methods for [[nonlinear dimensionality reduction]] find their theoretical and algorithmic roots in PCA or K-means. Pearson's original idea was to take a straight line (or plane) which will be "the best fit" to a set of data points. [[Trevor Hastie]] expanded on this concept by proposing '''Principal [[curve]]s'''<ref>{{cite journal|author1-last=Hastie|author1-first=T. |author1-link=Trevor Hastie|author2-last=Stuetzle|author2-first=W. |title=Principal Curves|journal=[[Journal of the American Statistical Association]]|date=June 1989|volume=84|issue=406|pages=502–506|doi=10.1080/01621459.1989.10478797 |url=https://web.stanford.edu/~hastie/Papers/Principal_Curves.pdf}}</ref> as the natural extension for the geometric interpretation of PCA, which explicitly constructs a manifold for data [[approximation]] followed by [[Projection (mathematics)|projecting]] the points onto it. See also the [[elastic map]] algorithm and [[principal geodesic analysis]].<ref>A.N. Gorban, B. Kegl, D.C. Wunsch, A. Zinovyev (Eds.), [https://www.researchgate.net/publication/271642170_Principal_Manifolds_for_Data_Visualisation_and_Dimension_Reduction_LNCSE_58 Principal Manifolds for Data Visualisation and Dimension Reduction],
LNCSE 58, Springer, Berlin – Heidelberg – New York, 2007. {{isbn|978-3-540-73749-0}}</ref> Another popular generalization is [[kernel PCA]], which corresponds to PCA performed in a reproducing kernel Hilbert space associated with a positive definite kernel.

In [[multilinear subspace learning]],<ref name="Vasilescu2003">{{cite conference
 |first1=M.A.O. |last1=Vasilescu
 |first2=D. |last2=Terzopoulos 
 |url=http://www.cs.toronto.edu/~maov/tensorfaces/cvpr03.pdf 
 |title=Multilinear Subspace Analysis of Image Ensembles
 |conference=Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’03)
 |location=Madison, WI
 |year=2003
 }}</ref><ref name="Vasilescu2002tensorfaces">{{cite book
 |first1=M.A.O. |last1=Vasilescu
 |first2=D. |last2=Terzopoulos 
 |url=http://www.cs.toronto.edu/~maov/tensorfaces/Springer%20ECCV%202002_files/eccv02proceeding_23500447.pdf 
 |title=Multilinear Analysis of Image Ensembles: TensorFaces
 |series=Lecture Notes in Computer Science 2350; (Presented at Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark)
 |publisher=Springer, Berlin, Heidelberg
 |doi=10.1007/3-540-47969-4_30
 |isbn=978-3-540-43745-1
 |year=2002
 }}</ref><ref name="MPCA-MICA2005">{{cite conference
 |first1=M.A.O. |last1=Vasilescu
 |first2=D. |last2=Terzopoulos 
 |url=http://www.media.mit.edu/~maov/mica/mica05.pdf 
 |title=Multilinear Independent Component Analysis
 |conference=Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05) 
 |location=San Diego, CA
 |date=June 2005
 |volume=1
 |pages=547–553}}</ref> PCA is generalized to [[multilinear principal component analysis|multilinear PCA]] (MPCA) that extracts features directly from tensor representations. MPCA is solved by performing PCA in each mode of the tensor iteratively. MPCA has been applied to face recognition, gait recognition, etc. MPCA is further extended to uncorrelated MPCA, non-negative MPCA and robust MPCA.

''N''-way principal component analysis may be performed with models such as [[Tucker decomposition]], [[PARAFAC]], multiple factor analysis, co-inertia analysis, STATIS, and DISTATIS.