Editing Bayesian inference (section)

===Parametric formulation: motivating the formal description===

By parameterizing the space of models, the belief in all models may be updated in a single step. The distribution of belief over the model space may then be thought of as a distribution of belief over the parameter space. The distributions in this section are expressed as continuous, represented by probability densities, as this is the usual situation. The technique is, however, equally applicable to discrete distributions.

Let the vector <math>\boldsymbol{\theta}</math> span the parameter space. Let the initial prior distribution over <math>\boldsymbol{\theta}</math> be <math>p(\boldsymbol{\theta} \mid \boldsymbol{\alpha})</math>, where <math>\boldsymbol{\alpha}</math> is a set of parameters to the prior itself, or ''[[Hyperparameter (Bayesian statistics)|hyperparameter]]s''. Let <math>\mathbf{E} = (e_1, \dots, e_n)</math> be a sequence of [[Independent and identically distributed random variables|independent and identically distributed]] event observations, where all <math>e_i</math> are distributed as <math>p(e \mid \boldsymbol{\theta})</math> for some <math>\boldsymbol{\theta}</math>. [[Bayes' theorem]] is applied to find the [[posterior distribution]] over <math>\boldsymbol{\theta}</math>:

<math display="block">\begin{align}
 p(\boldsymbol{\theta} \mid \mathbf{E}, \boldsymbol{\alpha}) &= \frac{p(\mathbf{E} \mid \boldsymbol{\theta}, \boldsymbol{\alpha})}{p(\mathbf{E} \mid \boldsymbol{\alpha})} \cdot p(\boldsymbol{\theta} \mid \boldsymbol{\alpha}) \\
  &= \frac{p(\mathbf{E} \mid \boldsymbol{\theta}, \boldsymbol{\alpha})}{\int p(\mathbf{E} \mid \boldsymbol{\theta}, \boldsymbol{\alpha}) p(\boldsymbol{\theta} \mid \boldsymbol{\alpha}) \, d\boldsymbol{\theta}} \cdot p(\boldsymbol{\theta} \mid \boldsymbol{\alpha}),
\end{align}</math>
where
<math display="block">p(\mathbf{E} \mid \boldsymbol{\theta}, \boldsymbol{\alpha}) = \prod_k p(e_k \mid \boldsymbol{\theta}).</math>