Editing Linear discriminant analysis (section)

{{Short description|Method used in statistics, pattern recognition, and other fields}}
{{Distinguish|latent Dirichlet allocation}}
[[File:Linear discriminant analysis plot.png|thumb|Linear discriminant analysis on a two dimensional space with two classes. The Bayes boundary is calculated based on the true data generation parameters, the estimated boundary on the realised data points.<ref>{{Cite web |last=Holtel |first=Frederik |date=2023-02-20 |title=Linear Discriminant Analysis (LDA) Can Be So Easy |url=https://towardsdatascience.com/linear-discriminant-analysis-lda-can-be-so-easy-b3f46e32f982 |access-date=2024-05-18 |website=Medium |language=en}}</ref>]]
'''Linear discriminant analysis''' ('''LDA'''), '''normal discriminant analysis''' ('''NDA'''), '''canonical variates analysis''' ('''CVA'''), or '''discriminant function analysis''' is a generalization of '''Fisher's linear discriminant''', a method used in [[statistics]] and other fields, to find a [[linear combination]] of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a [[linear classifier]], or, more commonly, for [[dimensionality reduction]] before later [[statistical classification|classification]].

LDA is closely related to [[analysis of variance]] (ANOVA) and [[regression analysis]], which also attempt to express one [[dependent variable]] as a linear combination of other features or measurements.<ref name="Fisher:1936">{{cite journal |last=Fisher |first=R. A. |author-link=Ronald Fisher |title=The Use of Multiple Measurements in Taxonomic Problems |journal=[[Annals of Eugenics]] |volume=7 |pages=179–188 |year=1936 |hdl=2440/15227|doi=10.1111/j.1469-1809.1936.tb02137.x |issue=2|url=https://digital.library.adelaide.edu.au/dspace/bitstream/2440/15227/1/138.pdf |hdl-access=free }}
</ref><ref name="McLachlan:2004">{{cite book |title=Discriminant Analysis and Statistical Pattern Recognition |first1=G. J. |last1=McLachlan |publisher=Wiley Interscience |isbn=978-0-471-69115-0 |year=2004 |mr=1190469}}</ref> However, ANOVA uses [[categorical variable|categorical]] [[independent variables]] and a [[continuous variable|continuous]] [[dependent variable]], whereas discriminant analysis has continuous [[independent variables]] and a categorical dependent variable (''i.e.'' the class label).<ref>Analyzing Quantitative Data: An Introduction for Social Researchers, Debra Wetcher-Hendricks, p.288</ref> [[Logistic regression]] and [[probit regression]] are more similar to LDA than ANOVA is, as they also explain a categorical variable by the values of continuous independent variables. These other methods are preferable in applications where it is not reasonable to assume that the independent variables are normally distributed, which is a fundamental assumption of the LDA method.

LDA is also closely related to [[principal component analysis]] (PCA) and [[factor analysis]] in that they both look for linear combinations of variables which best explain the data.<ref name="Martinez:2001">{{cite journal |last1=Martinez |first1=A. M. |last2=Kak |first2=A. C. |title=PCA versus LDA |journal=[[IEEE Transactions on Pattern Analysis and Machine Intelligence]] |volume=23 |issue=2 |pages=228–233 |year=2001 |url=http://www.ece.osu.edu/~aleix/pami01.pdf |doi=10.1109/34.908974 |access-date=2010-06-30 |archive-date=2008-10-11 |archive-url=https://web.archive.org/web/20081011071459/http://www.ece.osu.edu/~aleix/pami01.pdf |url-status=dead }}</ref> LDA explicitly attempts to model the difference between the classes of data. PCA, in contrast, does not take into account any difference in class, and factor analysis builds the feature combinations based on differences rather than similarities. Discriminant analysis is also different from factor analysis in that it is not an interdependence technique: a distinction between independent variables and dependent variables (also called criterion variables) must be made.

LDA works when the measurements made on independent variables for each observation are continuous quantities. When dealing with categorical independent variables, the equivalent technique is discriminant correspondence analysis.<ref name="Abdi 2007">Abdi, H. (2007) [http://www.utdallas.edu/~herve/Abdi-DCA2007-pretty.pdf "Discriminant correspondence analysis."] In: N.J. Salkind (Ed.): ''Encyclopedia of Measurement and Statistic''. Thousand Oaks (CA): Sage. pp.&nbsp;270–275.</ref><ref name="Perriere 2003">{{cite journal | last1 = Perriere | first1 = G. | last2 = Thioulouse | first2 = J. | year = 2003 | title = Use of Correspondence Discriminant Analysis to predict the subcellular location of bacterial proteins | journal = Computer Methods and Programs in Biomedicine | volume = 70 | issue = 2| pages = 99–105 | doi=10.1016/s0169-2607(02)00011-1| pmid = 12507786 }}</ref>

Discriminant analysis is used when groups are known a priori (unlike in [[cluster analysis]]). Each case must have a score on one or more quantitative predictor measures, and a score on a group measure.<ref name="buy">Büyüköztürk, Ş. & Çokluk-Bökeoğlu, Ö. (2008). [https://ejer.com.tr/wp-content/uploads/2021/01/ejer_2008_issue_33.pdf Discriminant function analysis: Concept and application]. Egitim Arastirmalari - Eurasian Journal of Educational Research, 33, 73-92. </ref> In simple terms, discriminant function analysis is classification - the act of distributing things into groups, classes or categories of the same type.