Editing Information bottleneck method (section)

=== Density estimation ===

{{Main|Density estimation}}

Since the bottleneck method is framed in probabilistic rather than statistical terms, the underlying probability density at the sample points <math>X = {x_i} \,</math>must be estimated. This is a well known problem with multiple solutions described by [[Bernard Silverman|Silverman]].<ref name=":2" /> In the present method, joint sample probabilities are found by use of a [[Stochastic matrix|Markov transition matrix]] method and this has some mathematical synergy with the bottleneck method itself.

The arbitrarily increasing distance metric <math>f \,</math> between all sample pairs and [[distance matrix]] is <math>d_{i,j}=f \Big ( \Big| x_i - x_j \Big | \Big )</math> . Then transition probabilities between sample pairs <math>P_{i,j}=\exp (- \lambda d_{i,j} ) \,</math> for some <math>\lambda > 0 \,</math>must be computed. Treating samples as states, and a normalised version of <math>P \,</math> as a Markov state transition probability matrix, the vector of probabilities of the 'states' after <math>t \,</math> steps, conditioned on the initial state <math>p(0) \,</math>, is <math>p(t)=P^t p(0) \,</math>. The equilibrium probability vector <math>p(\infty ) \,</math> given, in the usual way, by the dominant eigenvector of matrix <math>P \,</math> which is independent of the initialising vector <math>p(0) \,</math>. This Markov transition method establishes a probability at the sample points which is claimed to be proportional to the probabilities' densities there.

Other interpretations of the use of the eigenvalues of distance matrix <math>d \,</math> are discussed in Silverman's ''Density Estimation for Statistics and Data Analysis''.<ref name=":2">{{cite book|last = Silverman|first = Bernie|title = Density Estimation for Statistics and Data Analysis|series = Monographs on Statistics and Applied Probability|publisher = Chapman & Hall|year = 1986|isbn = 978-0412246203|author-link = Bernie Silverman|bibcode = 1986desd.book.....S|url-access = registration|url = https://archive.org/details/densityestimatio00silv_0}}</ref>