Editing Round-off error (section)

=== Calculating roundoff error in IEEE standard ===

Suppose the usage of round-to-nearest and IEEE double precision.

* Example: the decimal number <math>(9.4)_{10}=(1001.{\overline{0110}})_{2}</math> can be rearranged into <math display="block">+1.\underbrace{0010110011001100110011001100110011001100110011001100}_\text{52 bits}110 \ldots \times 2^{3}</math> 
Since the 53rd bit to the right of the binary point is a 1 and is followed by other nonzero bits, the round-to-nearest rule requires rounding up, that is, add 1 bit to the 52nd bit. Thus, the normalized floating-point representation in IEEE standard of 9.4 is 
<math display="block">fl(9.4)=1.0010110011001100110011001100110011001100110011001101 \times 2^{3}.</math>

* Now the roundoff error can be calculated when representing <math>9.4</math> with <math>fl(9.4)</math>. 
This representation is derived by discarding the infinite tail <math display="block">0.{\overline{1100}} \times 2^{-52}\times 2^{3} = 0.{\overline{0110}} \times 2^{-51} \times 2^{3}=0.4 \times 2^{-48}</math> 
from the right tail and then added <math>1 \times 2^{-52} \times 2^{3}=2^{-49}</math> in the rounding step. 
:Then <math>fl(9.4) = 9.4-0.4 \times 2^{-48} + 2^{-49} = 9.4+(0.2)_{10} \times 2^{-49}</math>. 
:Thus, the roundoff error is <math>(0.2 \times 2^{-49})_{10}</math>.