Editing Principal component analysis (section)

=== Non-negative matrix factorization ===
[[File:Fractional Residual Variances comparison, PCA and NMF.pdf|thumb|500px|Fractional residual variance (FRV) plots for PCA and NMF;<ref name="ren18"/> for PCA, the theoretical values are the contribution from the residual eigenvalues. In comparison, the FRV curves for PCA reaches a flat plateau where no signal are captured effectively; while the NMF FRV curves decline continuously, indicating a better ability to capture signal. The FRV curves for NMF also converges to higher levels than PCA, indicating the less-overfitting property of NMF.]] [[Non-negative matrix factorization]] (NMF) is a dimension reduction method where only non-negative elements in the matrices are used, which is therefore a promising method in astronomy,<ref name="blantonRoweis07">{{Cite journal|arxiv=astro-ph/0606170|last1= Blanton|first1= Michael R.|title= K-corrections and filter transformations in the ultraviolet, optical, and near infrared |journal= The Astronomical Journal|volume= 133|issue= 2|pages= 734–754|last2= Roweis|first2= Sam |year= 2007|doi= 10.1086/510127|bibcode = 2007AJ....133..734B|s2cid= 18561804}}</ref><ref name="zhu16"/><ref name="ren18"/> in the sense that astrophysical signals are non-negative. The PCA components are orthogonal to each other, while the NMF components are all non-negative and therefore constructs a non-orthogonal basis.

In PCA, the contribution of each component is ranked based on the magnitude of its corresponding eigenvalue, which is equivalent to the fractional residual variance (FRV) in analyzing empirical data.<ref name = "soummer12">{{Cite journal|arxiv=1207.4197|last1= Soummer|first1= Rémi |title= Detection and Characterization of Exoplanets and Disks Using Projections on Karhunen-Loève Eigenimages|journal= The Astrophysical Journal Letters |volume= 755|issue= 2|pages= L28|last2= Pueyo|first2= Laurent|last3= Larkin | first3 = James|year= 2012|doi= 10.1088/2041-8205/755/2/L28|bibcode = 2012ApJ...755L..28S |s2cid= 51088743}}</ref> For NMF, its components are ranked based only on the empirical FRV curves.<ref name = "ren18">{{Cite journal|arxiv=1712.10317|last1= Ren|first1= Bin |title= Non-negative Matrix Factorization: Robust Extraction of Extended Structures|journal= The Astrophysical Journal|volume= 852|issue= 2|pages= 104|last2= Pueyo|first2= Laurent|last3= Zhu | first3 = Guangtun B.|last4= Duchêne|first4= Gaspard |year= 2018|doi= 10.3847/1538-4357/aaa1f2|bibcode = 2018ApJ...852..104R |s2cid= 3966513|doi-access= free}}</ref> The residual fractional eigenvalue plots, that is, <math> 1-\sum_{i=1}^k \lambda_i\Big/\sum_{j=1}^n \lambda_j</math> as a function of component number <math>k</math> given a total of <math>n</math> components, for PCA have a flat plateau, where no data is captured to remove the quasi-static noise, then the curves drop quickly as an indication of over-fitting (random noise).<ref name="soummer12"/> The FRV curves for NMF is decreasing continuously<ref name="ren18"/> when the NMF components are constructed [[Non-negative matrix factorization#Sequential NMF|sequentially]],<ref name="zhu16">{{Cite arXiv|last=Zhu|first=Guangtun B.|date=2016-12-19|title=Nonnegative Matrix Factorization (NMF) with Heteroscedastic Uncertainties and Missing data |eprint=1612.06037|class=astro-ph.IM}}</ref> indicating the continuous capturing of quasi-static noise; then converge to higher levels than PCA,<ref name="ren18"/> indicating the less over-fitting property of NMF.