Editing Quantization (signal processing) (section)

==Noise and error characteristics{{anchor|Noise|Error}}==

===Additive noise model===
A common assumption for the analysis of quantization error is that it affects a signal processing system in a similar manner to that of additive [[white noise]] – having negligible correlation with the signal and an approximately flat [[power spectral density]].<ref name=Bennett/><ref name=GrayNeuhoff/><ref name=Widrow1>{{cite journal | last=Widrow | first=B. |author-link=Bernard Widrow| title=A Study of Rough Amplitude Quantization by Means of Nyquist Sampling Theory | journal=IRE Transactions on Circuit Theory | publisher=Institute of Electrical and Electronics Engineers (IEEE) | volume=3 | issue=4 | year=1956 | issn=0096-2007 | doi=10.1109/tct.1956.1086334 | pages=266–276| hdl=1721.1/12139 | s2cid=16777461 | hdl-access=free }}</ref><ref name=Widrow2>[[Bernard Widrow]], "[http://www-isl.stanford.edu/~widrow/papers/j1961statisticalanalysis.pdf Statistical analysis of amplitude quantized sampled data systems]", ''Trans. AIEE Pt. II: Appl. Ind.'', Vol. 79, pp. 555–568, Jan. 1961.</ref> The additive noise model is commonly used for the analysis of quantization error effects in digital filtering systems, and it can be very useful in such analysis. It has been shown to be a valid model in cases of high-resolution quantization (small <math>\Delta</math> relative to the signal strength) with smooth PDFs.<ref name=Bennett/><ref name=MarcoNeuhoff>{{cite journal | last1=Marco | first1=D. | last2=Neuhoff | first2=D.L. | title=The Validity of the Additive Noise Model for Uniform Scalar Quantizers | journal=IEEE Transactions on Information Theory | publisher=Institute of Electrical and Electronics Engineers (IEEE) | volume=51 | issue=5 | year=2005 | issn=0018-9448 | doi=10.1109/tit.2005.846397 | pages=1739–1755| s2cid=14819261 }}</ref>

Additive noise behavior is not always a valid assumption. Quantization error (for quantizers defined as described here) is deterministically related to the signal and not entirely independent of it. Thus, periodic signals can create periodic quantization noise.  And in some cases, it can even cause [[limit cycle]]s to appear in digital signal processing systems. One way to ensure effective independence of the quantization error from the source signal is to perform ''[[dither]]ed quantization'' (sometimes with ''[[noise shaping]]''), which involves adding random (or [[pseudo-random]]) noise to the signal prior to quantization.<ref name=GrayNeuhoff/><ref name=Widrow2/>

===Quantization error models===
In the typical case, the original signal is much larger than one [[least significant bit]] (LSB). When this is the case, the quantization error is not significantly correlated with the signal and has an approximately [[uniform distribution (continuous)|uniform distribution]]. When rounding is used to quantize, the quantization error has a [[mean]] of zero and the [[root mean square]] (RMS) value is the [[standard deviation]] of this distribution, given by <math>\scriptstyle {\frac{1}{\sqrt{12}}}\mathrm{LSB}\ \approx\ 0.289\,\mathrm{LSB}</math>. When truncation is used, the error has a non-zero mean of <math>\scriptstyle {\frac{1}{2}}\mathrm{LSB}</math> and the RMS value is <math>\scriptstyle {\frac{1}{\sqrt{3}}}\mathrm{LSB}</math>. Although rounding yields less RMS error than truncation, the difference is only due to the static (DC) term of <math>\scriptstyle {\frac{1}{2}}\mathrm{LSB}</math><nowiki>. The RMS values of the AC error are exactly the same in both cases, so there is no special advantage of rounding over truncation in situations where the DC term of the error can be ignored (such as in AC-coupled systems). In either case, the standard deviation, as a percentage of the full signal range, changes by a factor of 2 for each 1-bit change in the number of quantization bits.  The potential signal-to-quantization-noise power ratio therefore changes by 4, or </nowiki><math>\scriptstyle 10\cdot \log_{10}(4)</math>, approximately 6&nbsp;dB per bit.

At lower amplitudes the quantization error becomes dependent on the input signal, resulting in distortion. This distortion is created after the anti-aliasing filter, and if these distortions are above 1/2 the sample rate they will alias back into the band of interest. In order to make the quantization error independent of the input signal, the signal is dithered by adding noise to the signal. This slightly reduces signal-to-noise ratio, but can completely eliminate the distortion.

===Quantization noise model===
[[File:Frequency spectrum of a sinusoid and its quantization noise floor.gif|thumb|300px|Comparison of quantizing a sinusoid to 64 levels (6 bits) and 256 levels (8 bits). The additive noise created by 6-bit quantization is 12 dB greater than the noise created by 8-bit quantization. When the spectral distribution is flat, as in this example, the 12 dB difference manifests as a measurable difference in the noise floors.]]

Quantization noise is a [[Model (abstract)|model]] of quantization error introduced by quantization in the ADC. It is a rounding error between the analog input voltage to the ADC and the output digitized value. The noise is non-linear and signal-dependent. It can be modeled in several different ways.

In an ideal ADC, where the quantization error is uniformly distributed between −1/2 LSB and +1/2 LSB, and the signal has a uniform distribution covering all quantization levels, the [[Signal-to-quantization-noise ratio]] (SQNR) can be calculated from

:<math>\mathrm{SQNR} = 20 \log_{10}(2^Q) \approx 6.02 \cdot Q\ \mathrm{dB} \,\!</math>

where Q is the number of quantization bits.

The most common test signals that fulfill this are full amplitude [[triangle wave]]s and [[sawtooth wave]]s.

For example, a [[16-bit]] ADC has a maximum signal-to-quantization-noise ratio of 6.02 × 16 = 96.3&nbsp;dB.

When the input signal is a full-amplitude [[sine wave]] the distribution of the signal is no longer uniform, and the corresponding equation is instead

:<math> \mathrm{SQNR} \approx  1.761 + 6.02 \cdot Q \ \mathrm{dB} \,\!</math>

Here, the quantization noise is once again ''assumed'' to be uniformly distributed.  When the input signal has a high amplitude and a wide frequency spectrum this is the case.<ref>{{cite book
  | last = Pohlman
  | first =Ken C.
  | title = Principles of Digital Audio 2nd Edition
  | publisher = SAMS
  | date = 1989
  | page = 60
  | isbn =9780071441568
 | url = https://books.google.com/books?id=VZw6z9a03ikC&pg=PA37}}</ref>  In this case a 16-bit ADC has a maximum signal-to-noise ratio of 98.09&nbsp;dB.  The 1.761 difference in signal-to-noise only occurs due to the signal being a full-scale sine wave instead of a triangle or sawtooth.

For complex signals in high-resolution ADCs this is an accurate model. For low-resolution ADCs, low-level signals in high-resolution ADCs, and for simple waveforms the quantization noise is not uniformly distributed, making this model inaccurate.<ref>{{cite book
  | last = Watkinson
  | first = John
  | title = The Art of Digital Audio 3rd Edition
  | publisher = [[Focal Press]]
  | date = 2001
  | isbn = 0-240-51587-0}}</ref> In these cases the quantization noise distribution is strongly affected by the exact amplitude of the signal.

The calculations are relative to full-scale input. For smaller signals, the relative quantization distortion can be very large. To circumvent this issue, analog [[companding]] can be used, but this can introduce distortion.