Editing Empirical Bayes method (section)

===Robbins' method: non-parametric empirical Bayes (NPEB)===

[[Herbert Robbins|Robbins]]<ref name=Robbins/> considered a case of sampling from a [[mixture distribution|mixed distribution]], where probability for each <math>y_i</math> (conditional on <math>\theta_i</math>) is specified by  a [[Poisson distribution]],

:<math>p(y_i\mid\theta_i)={{\theta_i}^{y_i} e^{-\theta_i} \over {y_i}!}</math>

while the prior on ''θ'' is unspecified except that it is also [[i.i.d.]] from an unknown distribution, with [[cumulative distribution function]] <math>G(\theta)</math>.  Compound sampling arises in a variety of statistical estimation problems, such as accident rates and clinical trials.{{Citation needed|date=February 2012}}  We simply seek a point prediction of <math>\theta_i</math> given all the observed data.  Because the prior is unspecified, we seek to do this without knowledge of ''G''.<ref name=CL/>

Under [[squared error loss]] (SEL), the [[conditional expectation]] E(''θ''<sub>''i''</sub>&nbsp;|&nbsp;''Y''<sub>''i''</sub>&nbsp;=&nbsp;''y''<sub>''i''</sub>) is a reasonable quantity to use for prediction.  For the Poisson compound sampling model, this quantity is

:<math>\operatorname{E}(\theta_i\mid y_i) = {\int (\theta^{y_i+1} e^{-\theta} / {y_i}!)\,dG(\theta) \over {\int (\theta^{y_i} e^{-\theta} / {y_i}!)\,dG(\theta}) }.</math>

This can be simplified by multiplying both the numerator and denominator by <math>({y_i}+1)</math>, yielding

:<math> \operatorname{E}(\theta_i\mid y_i)= {{(y_i + 1) p_G(y_i + 1) }\over {p_G(y_i)}},</math>

where ''p<sub>G</sub>'' is the marginal probability mass function obtained by integrating out ''θ'' over ''G''.

To take advantage of this, Robbins<ref name=Robbins/> suggested estimating the marginals with their empirical frequencies (<math> \#\{Y_j\}</math>), yielding the fully non-parametric estimate as:

:<math> \operatorname{E}(\theta_i\mid y_i) \approx (y_i + 1) { {\#\{Y_j = y_i + 1\}} \over {\#\{ Y_j = y_i\}} },</math>

where <math>\#</math> denotes "number of". (See also [[Good–Turing frequency estimation]].)

;Example – Accident rates

Suppose each customer of an insurance company has an "accident rate" Θ and is insured against accidents; the probability distribution of Θ  is the underlying distribution, and is unknown.  The number of accidents suffered by each customer in a specified time period has a [[Poisson distribution]] with expected value equal to the particular customer's accident rate.  The actual number of accidents experienced by a customer is the observable quantity. A crude way to estimate the underlying probability distribution of the accident rate Θ is to estimate the proportion of members of the whole population suffering 0, 1, 2, 3, ... accidents during the specified time period as the corresponding proportion in the observed random sample.  Having done so, it is then desired to predict the accident rate of each customer in the sample.  As above, one may use the [[conditional probability|conditional]] [[expected value]] of the accident rate Θ given the observed number of accidents during the baseline period.  Thus, if a customer suffers six accidents during the baseline period, that customer's estimated accident rate is 7 × [the proportion of the sample who suffered 7 accidents] / [the proportion of the sample who suffered 6 accidents].  Note that if the proportion of people suffering ''k'' accidents is a decreasing function of ''k'', the customer's predicted accident rate will often be lower than their observed number of accidents.

This [[Shrinkage (statistics)|shrinkage]] effect is typical of empirical Bayes analyses.