Editing Density estimation

{{Short description|Estimate of an unobservable underlying probability density function}}
{{for|the signal processing concept|spectral density estimation}}
{{more citations needed|date=August 2012}}

[[File:KernelDensityGaussianAnimated.gif|thumb|350px|Demonstration of density estimation using [[Kernel density estimation]]: The true density is a mixture of two Gaussians centered around 0 and 3, shown with a solid blue curve. In each frame, 100 samples are generated from the distribution, shown in red. Centered on each sample, a Gaussian kernel is drawn in gray. Averaging the Gaussians yields the density estimate shown in the dashed black curve.]]

In [[statistics]], '''probability density estimation''' or simply '''density estimation''' is the construction of an [[estimate]], based on observed [[data]], of an unobservable underlying [[probability density function]].  The unobservable density function is thought of as the density according to which a large population is distributed; the data are usually thought of as a random sample from that population.<ref>Alberto Bernacchia, Simone Pigolotti, Self-Consistent Method for Density Estimation, Journal of the Royal Statistical Society Series B: Statistical Methodology, Volume 73, Issue 3, June 2011, Pages 407–422, https://doi.org/10.1111/j.1467-9868.2011.00772.x</ref>

A variety of approaches to density estimation are used, including [[Parzen window]]s and a range of [[data clustering]] techniques, including [[vector quantization]]. The most basic form of density estimation is a rescaled [[histogram]].

== Example ==
[[File:P glu given diabetes.png|thumb|right|360px|Estimated density of ''p'' (glu {{pipe}} diabetes=1) (red), ''p''&nbsp;(glu {{pipe}} diabetes=0) (blue), and ''p''&nbsp;(glu) (black)]]
[[File:P diabetes given glu.png|thumb|right|360px|Estimated probability of ''p''(diabetes=1 {{pipe}} glu)]]
[[File:Glu opt.png|thumb|right|360px|Estimated probability of ''p''&nbsp;(diabetes=1 {{pipe}} glu)]]

We will consider records of the incidence of [[diabetes]]. The following is quoted verbatim from the [[data set]] description:

:''A population of women who were at least 21 years old, of [[Pima people|Pima]] Indian heritage and living near Phoenix, Arizona,  was tested for [[diabetes mellitus]] according to [[World Health Organization]] criteria.  The data were collected by the US National Institute of Diabetes and Digestive and Kidney Diseases. We used the 532 complete records.<ref>{{cite web|url=https://stat.ethz.ch/R-manual/R-patched/library/MASS/html/Pima.tr.html|title=Diabetes in Pima Indian Women - R documentation}}</ref><ref>{{cite journal|author=Smith, J. W., Everhart, J. E., Dickson, W. C., Knowler, W. C. and Johannes, R. S.|year=1988|title=Using the ADAP learning algorithm to forecast the onset of diabetes mellitus|journal=Proceedings of the Symposium on Computer Applications in Medical Care (Washington, 1988)|editor=R. A. Greenes|pages=261–265|place=Los Alamitos, CA|pmc=2245318}}</ref>''

In this example, we construct three density estimates for "glu" ([[Blood plasma|plasma]] [[glucose]] concentration), one [[Conditional probability|conditional]] on the presence of diabetes,
the second conditional on the absence of diabetes, and the third not conditional on diabetes.
The conditional density estimates are then used to construct the probability of diabetes conditional on "glu".

The "glu" data were obtained from the MASS package<ref>{{cite web|url=https://cran.r-project.org/web/packages/MASS/index.html|title=Support Functions and Datasets for Venables and Ripley's MASS}}</ref> of the [[R programming language]]. Within R, <code>?Pima.tr</code> and <code>?Pima.te</code> give a fuller account of the data.

The [[mean]] of "glu" in the diabetes cases is 143.1 and the standard deviation is 31.26.
The mean of "glu" in the non-diabetes cases is 110.0 and the standard deviation is 24.29.
From this we see that, in this data set, diabetes cases are associated with greater levels of "glu".
This will be made clearer by plots of the estimated density functions.

The first figure shows density estimates of ''p''(glu | diabetes=1), ''p''(glu | diabetes=0), and ''p''(glu).
The density estimates are kernel density estimates using a Gaussian kernel. That is, a Gaussian density function is placed at each data point, and the sum of the density functions is computed over the range of the data.

From the density of "glu" conditional on diabetes, we can obtain the probability of diabetes conditional on "glu" via [[Bayes' rule]]. For brevity, "diabetes" is abbreviated "db." in this formula.

:<math> p(\mbox{diabetes}=1|\mbox{glu})
 = \frac{p(\mbox{glu}|\mbox{db.}=1)\,p(\mbox{db.}=1)}{p(\mbox{glu}|\mbox{db.}=1)\,p(\mbox{db.}=1) + p(\mbox{glu}|\mbox{db.}=0)\,p(\mbox{db.}=0)}
</math>

The second figure shows the estimated posterior probability ''p''(diabetes=1 | glu). From these data, it appears that an increased level of "glu" is associated with diabetes.

== Application and purpose ==
A very natural use of density estimates is in the informal investigation of the properties of a given set of data. Density estimates can give a valuable indication of such features as skewness and multimodality in the data. In some cases they will yield conclusions that may then be regarded as self-evidently true, while in others all they will do is to point the way to further analysis and/or data collection.<ref>{{Cite book|title=Density Estimation for Statistics and Data Analysis|last=Silverman|first=B. W.|publisher=Chapman and Hall.|year=1986|isbn=978-0412246203|url=https://archive.org/details/densityestimatio00silv_0}}</ref>

[[File:Gumbel distribtion.png|thumb|300px|Histogram and density function for a Gumbel distribution <ref>[https://www.waterlog.info/cumfreq.htm A calculator for probability distributions and density functions]</ref>]]

An important aspect of statistics is often the presentation of data back to the client in order to provide explanation and illustration of conclusions that may possibly have been obtained by other means. Density estimates are ideal for this purpose, for the simple reason that they are fairly easily comprehensible to non-mathematicians.

More examples illustrating the use of density estimates for exploratory and presentational purposes, including the important case of bivariate data.<ref>Geof H., Givens (2013). Computational Statistics. Wiley. p. 330. {{ISBN|978-0-470-53331-4}}.</ref>

Density estimation is also frequently used in [[anomaly detection]] or [[novelty detection]]:<ref>{{cite journal|last1=Pimentel|first1=Marco A.F.|last2=Clifton|first2=David A.|last3=Clifton|first3=Lei|last4=Tarassenko|first4=Lionel|title=A review of novelty detection|journal=Signal Processing|date=2 January 2014|volume= 99|issue=June 2014|pages=215–249|doi=10.1016/j.sigpro.2013.12.026}}</ref> if an observation lies in a very low-density region, it is likely to be an anomaly or a novelty.

* In [[hydrology]] the [[histogram]] and estimated density function of rainfall and river discharge data, analysed with a [[probability distribution]], are used to gain insight in their behaviour and frequency of occurrence.<ref>[https://www.waterlog.info/density.htm An illustration of histograms and probability density functions]</ref> An example is shown in the blue figure.

==Kernel density estimation==
{{excerpt|Kernel density estimation}}

== See also ==
* [[Kernel density estimation]]
* [[Mean integrated squared error]]
* [[Histogram]] 
* [[Multivariate kernel density estimation]]
* [[Spectral density estimation]]
* [[Kernel embedding of distributions]]
* [[Generative model]]
* [[Order statistic#Application: Non-parametric Density Estimation|Application of Order Statistics: Non-parametric Density Estimation]]
* [[Probability distribution fitting]]

== References ==
{{reflist}}
'''Sources'''
* {{cite book|author=Brian D. Ripley|title=Pattern Recognition and Neural Networks|place=Cambridge|publisher=Cambridge University Press|year=1996|url=https://books.google.com/books?id=2SzT2p8vP1oC|isbn=978-0521460866}}
* [[Trevor Hastie]], [[Robert Tibshirani]], and Jerome Friedman. ''The Elements of Statistical Learning''. New York: Springer, 2001. {{isbn|0-387-95284-5}}. ''(See Chapter 6.)''
* Qi Li and Jeffrey S. Racine. ''Nonparametric Econometrics: Theory and Practice''. Princeton University Press, 2007, {{isbn|0-691-12161-3}}. ''(See Chapter 1.)''
* D.W. Scott. ''Multivariate Density Estimation. Theory, Practice and Visualization''. New York: Wiley, 1992.
* [[Bernard Silverman|B.W. Silverman]]. ''Density Estimation''. London: Chapman and Hall, 1986. {{isbn|978-0-412-24620-3}}

==External links==
* [http://creem2.st-andrews.ac.uk/software/ CREEM: Centre for Research Into Ecological and Environmental Modelling] Downloads for free density estimation software packages [https://web.archive.org/web/20060315201547/http://www.ruwpa.st-and.ac.uk/distance/ ''Distance 4''] (from Research Unit for Wildlife Population Assessment "RUWPA") and [https://web.archive.org/web/20160303195223/http://www.ruwpa.st-and.ac.uk/estimating.abundance/ ''WiSP''].
* [https://www.ics.uci.edu/~mlearn/MLSummary.html UCI Machine Learning Repository Content Summary] ''(See "Pima Indians Diabetes Database" for the original data set of 732 records, and additional notes.)''
* MATLAB code for [http://www.mathworks.com/matlabcentral/fileexchange/14034 one dimensional ]  and  [http://www.mathworks.com/matlabcentral/fileexchange/17204 two dimensional ] density estimation 
* [https://libagf.sourceforge.net libAGF] C++ software for [[variable kernel density estimation]].

{{Statistics|inference}}

[[Category:Estimation of densities|*]]
[[Category:Nonparametric statistics]]