Editing Independent component analysis (section)

== History and background ==

The early general framework for independent component analysis was introduced by Jeanny Hérault and Bernard Ans from 1984,<ref>{{cite journal | last1 = Hérault | first1 = J. | last2 = Ans | first2 = B. | year = 1984 | title = Réseau de neurones à synapses modifiables : Décodage de messages sensoriels composites par apprentissage non supervisé et permanent | journal = Comptes Rendus de l'Académie des Sciences, Série III | volume = 299 | pages = 525–528 }}</ref> further developed by Christian Jutten in 1985 and 1986,<ref name="jutten85">Ans, B., Hérault, J., & Jutten, C. (1985). Architectures neuromimétiques adaptatives  : Détection de primitives. ''Cognitiva 85'' (Vol. 2, pp. 593-597). Paris: CESTA.</ref><ref>Hérault, J., Jutten, C., & Ans, B. (1985). Détection de grandeurs primitives dans un message composite par une architecture de calcul neuromimétique en apprentissage non supervisé. ''Proceedings of the 10th Workshop Traitement du signal et ses applications'' (Vol. 2, pp. 1017-1022). Nice (France): GRETSI.</ref><ref>Hérault, J., & Jutten, C. (1986). Space or time adaptive signal processing by neural networks models. ''Intern. Conf. on Neural Networks for Computing'' (pp. 206-211). Snowbird (Utah, USA).</ref> and refined by Pierre Comon in 1991,<ref name="pc91">P.Comon, Independent Component Analysis, Workshop on Higher-Order Statistics, July 1991, republished in J-L. Lacoume, editor, Higher Order Statistics, pp. 29-38. Elsevier, Amsterdam, London, 1992. [https://hal.archives-ouvertes.fr/hal-00346684 HAL link]</ref> and popularized in his paper of 1994.<ref name="comon94">Pierre Comon (1994) Independent component analysis, a new concept? http://www.ece.ucsb.edu/wcsl/courses/ECE594/594C_F10Madhow/comon94.pdf</ref> In 1995, Tony Bell and [[Terry Sejnowski]] introduced a fast and efficient ICA algorithm based on [[infomax]], a principle introduced by Ralph Linsker in 1987. A link exists between maximum-likelihood estimation and Infomax approaches.<ref name="card97">J-F.Cardoso, "Infomax and Maximum Likelihood for source separation", IEEE Sig. Proc. Letters, 1997, 4(4):112-114.</ref> A quite comprehensive tutorial on the maximum-likelihood approach to ICA has been published by J-F. Cardoso in 1998.<ref name="card98">J-F.Cardoso, "Blind signal separation: statistical principles", Proc. of the IEEE, 1998, 90(8):2009-2025.</ref>

There are many algorithms available in the literature which do ICA. A largely used one, including in industrial applications, is the FastICA algorithm, developed by Hyvärinen and Oja,<ref>{{Cite journal|last1=Hyvärinen|first1=A.|last2=Oja|first2=E.|date=2000-06-01|title=Independent component analysis: algorithms and applications|url=http://www.cse.msu.edu/~cse902/S03/icasurvey.pdf|journal=Neural Networks|language=en|volume=13|issue=4|pages=411–430|doi=10.1016/S0893-6080(00)00026-5|pmid=10946390 |s2cid=11959218 |issn=0893-6080}}</ref> which uses the [[negentropy]] as cost function, already proposed 7 years before by Pierre Comon in this context.<ref name=comon94/> Other examples are rather related to [[blind source separation]] where a more general approach is used. For example, one can drop the independence assumption and separate mutually correlated signals, thus, statistically "dependent" signals. Sepp Hochreiter and [[Jürgen Schmidhuber]] showed how to obtain non-linear ICA or source separation as a by-product of [[regularization (mathematics)|regularization]] (1999).<ref name="HochreiterSchmidhuber1999">{{cite journal|last1=Hochreiter|first1=Sepp|last2=Schmidhuber|first2=Jürgen|title=Feature Extraction Through LOCOCODE|journal=Neural Computation|volume=11|issue=3|year=1999|pages=679–714|issn=0899-7667|doi=10.1162/089976699300016629 |pmid=10085426|s2cid=1642107|url=ftp://ftp.idsia.ch/pub/juergen/lococode.pdf |archive-url=https://web.archive.org/web/20170706020004/ftp://ftp.idsia.ch/pub/juergen/lococode.pdf|archive-date=2017-07-06|url-status=dead|access-date=24 February 2018 }}</ref> Their method does not require a priori knowledge about the number of independent sources.