Editing Rate–distortion theory (section)

===Memoryless (independent) Gaussian source with squared-error distortion===
If we assume that <math>X</math> is a [[normal distribution|Gaussian]] random variable with [[variance]] <math>\sigma^2</math>, and if we assume that successive samples of the signal <math>X</math> are [[stochastically independent]] (or equivalently, the source is ''[[memorylessness|memoryless]]'', or the signal is ''uncorrelated''), we find the following [[analytical expression]] for the rate–distortion function:

:<math> R(D) = \begin{cases}
  \frac{1}{2}\log_2(\sigma_x^2/D ), & \text{if } 0 \le D \le \sigma_x^2 \\
              0, & \text{if } D > \sigma_x^2. 
                      \end{cases}
</math>&nbsp;&nbsp;&nbsp;<ref>{{harvnb|Cover|Thomas|2012|p=310}}</ref>

The following figure shows what this function looks like:

[[File:Rate distortion function.png|400px]]

Rate–distortion theory tell us that 'no compression system exists that performs outside the gray area'. The closer a practical compression system is to the red (lower) bound, the better it performs. As a general rule, this bound can only be attained by increasing the coding block length parameter. Nevertheless, even at unit blocklengths one can often find good (scalar) [[Quantization (signal processing)|quantizers]] that operate at distances from the rate–distortion function that are practically relevant.<ref>{{cite book| first = Thomas M. |last=Cover |first2=Joy A. |last2=Thomas |chapter=10. Rate Distortion Theory | title = Elements of Information Theory | publisher = Wiley |orig-year=2006 |year=2012 |chapter-url={{GBurl|VWq5GG6ycxMC|p=301}} |isbn=978-1-118-58577-1 |edition=2nd}}</ref>

This rate–distortion function holds only for Gaussian memoryless sources. It is known that the Gaussian source is the most "difficult" source to encode: for a given mean square error, it requires the greatest number of bits. The performance of a practical compression system working on&mdash;say&mdash;images, may well be below the <math>R \left(D \right)</math> lower bound shown.