Editing IBM hexadecimal floating-point (section)

== Precision issues ==
Since the base is 16, there can be up to three leading zero bits in the binary significand. That means when the number is converted into binary, there can be as few as 21 bits of precision. Because of the "wobbling precision" effect, this can cause some calculations to be very inaccurate. This has caused considerable criticism.<ref name="Warren_2013">{{cite book |title=Hacker's Delight |title-link=Hacker's Delight |author-first=Henry S. |author-last=Warren Jr. |date=2013 |orig-year=2002 |edition=2 |chapter=The Distribution of Leading Digits|publisher=[[Addison Wesley]] - [[Pearson Education, Inc.]] |isbn=978-0-321-84268-8 |id=0-321-84268-5 |pages=385–387}}</ref>

A good example of the inaccuracy is representation of decimal value 0.1. It has no exact binary or hexadecimal representation. In hexadecimal format, it is represented as 0.19999999...<sub>16</sub> or 0.0001 1001 1001 1001 1001 1001 1001...<sub>2</sub>, that is:

:{|
|- style="text-align:center"
|style="width:20px;text-align:center;background-color:#FC9"|S
|style="width:90px;text-align:center;background-color:#99F"|Exp
|style="width:250px;text-align:center;background-color:#9F9"|Fraction
|style="text-align:center;background-color:#FFF"|&nbsp;
|- style="text-align:center"
|style="text-align:center;background-color:#FEC"|{{mono|0}}
|style="text-align:center;background-color:#CCF"|{{mono|100 0000}}
|style="text-align:center;background-color:#CFC"|{{mono|0001 1001 1001 1001 1001 1010}}
|style="text-align:center;background-color:#FFF"|&nbsp;
|}

This has only 21 bits, whereas the binary version has 24 bits of precision.

Six hexadecimal digits of precision is roughly equivalent to six decimal digits (i.e. (6 − 1) log<sub>10</sub>(16) ≈ 6.02). A conversion of single precision hexadecimal float to decimal string would require at least 9 significant digits (i.e. 6 log<sub>10</sub>(16) + 1 ≈ 8.22) in order to convert back to the same hexadecimal float value.