Editing Mixture model (section)

====Gaussian mixture model====
[[File:nonbayesian-gaussian-mixture.svg|right|250px|thumb|Non-Bayesian Gaussian mixture model using [[plate notation]].  Smaller squares indicate fixed parameters; larger circles indicate random variables.  Filled-in shapes indicate known values.  The indication [K] means a vector of size ''K''.]]

A typical non-Bayesian [[Gaussian distribution|Gaussian]] mixture model looks like this:

:<math>
\begin{array}{lcl}
K,N &=& \text{as above} \\
\phi_{i=1 \dots K}, \boldsymbol\phi &=& \text{as above} \\
z_{i=1 \dots N}, x_{i=1 \dots N} &=& \text{as above} \\
\theta_{i=1 \dots K} &=& \{ \mu_{i=1 \dots K}, \sigma^2_{i=1 \dots K} \}  \\
\mu_{i=1 \dots K} &=& \text{mean of component } i \\
\sigma^2_{i=1 \dots K} &=& \text{variance of component } i \\
z_{i=1 \dots N} &\sim& \operatorname{Categorical}(\boldsymbol\phi) \\
x_{i=1 \dots N} &\sim& \mathcal{N}(\mu_{z_i}, \sigma^2_{z_i})
\end{array}
</math>

{{clear}}
[[File:bayesian-gaussian-mixture.svg|right|300px|thumb|Bayesian Gaussian mixture model using [[plate notation]].  Smaller squares indicate fixed parameters; larger circles indicate random variables.  Filled-in shapes indicate known values.  The indication [K] means a vector of size ''K''.]]

A Bayesian version of a [[Gaussian distribution|Gaussian]] mixture model is as follows:

:<math>
\begin{array}{lcl}
K,N &=& \text{as above} \\
\phi_{i=1 \dots K}, \boldsymbol\phi &=& \text{as above} \\
z_{i=1 \dots N}, x_{i=1 \dots N} &=& \text{as above} \\
\theta_{i=1 \dots K} &=& \{ \mu_{i=1 \dots K}, \sigma^2_{i=1 \dots K} \}  \\
\mu_{i=1 \dots K} &=& \text{mean of component } i \\
\sigma^2_{i=1 \dots K} &=& \text{variance of component } i \\
\mu_0, \lambda, \nu, \sigma_0^2 &=& \text{shared hyperparameters} \\
\mu_{i=1 \dots K} &\sim& \mathcal{N}(\mu_0, \lambda\sigma_i^2) \\
\sigma_{i=1 \dots K}^2 &\sim& \operatorname{Inverse-Gamma}(\nu, \sigma_0^2) \\
\boldsymbol\phi &\sim& \operatorname{Symmetric-Dirichlet}_K(\beta) \\
z_{i=1 \dots N} &\sim& \operatorname{Categorical}(\boldsymbol\phi) \\
x_{i=1 \dots N} &\sim& \mathcal{N}(\mu_{z_i}, \sigma^2_{z_i})
\end{array}
</math><math></math>
[[File:Parameter estimation process infinite Gaussian mixture model.webm|thumb|end=49|Animation of the clustering process for one-dimensional data using a Bayesian Gaussian mixture model where normal distributions are drawn from a [[Dirichlet process]]. The histograms of the clusters are shown in different colours. During the parameter estimation process, new clusters are created and grow on the data. The legend shows the cluster colours and the number of datapoints assigned to each cluster.]]