Editing Precision (computer science)

{{More citations needed|date=March 2007}}

In [[computer science]], the '''precision''' of a numerical quantity is a measure of the detail in which the quantity is expressed. This is usually measured in bits, but sometimes in decimal digits. It is related to [[precision (arithmetic)|precision in mathematics]], which describes the number of digits that are used to express a value.

Some of the standardized precision formats are:
* [[Half-precision floating-point format]]
* [[Single-precision floating-point format]]
* [[Double-precision floating-point format]]
* [[Quadruple-precision floating-point format]]
* [[Octuple-precision floating-point format]]

Of these, octuple-precision format is rarely used. The single- and double-precision formats are most widely used and supported on nearly all platforms. The use of half-precision format and [[minifloat]] formats has been increasing especially in the field of [[machine learning]] since many machine learning algorithms are inherently error-tolerant.

==Rounding errors==
{{Further|Floating-point arithmetic}}

Precision is often the source of [[rounding error]]s in [[computation]]. The number of bits used to store a number will often cause some loss of accuracy. An example would be to store "sin(0.1)" in IEEE single-precision floating point standard. The error is then often magnified as subsequent computations are made using the data (although it can also be reduced).

==See also==
* [[Approximate computing]]
* [[Arbitrary-precision arithmetic]]
* [[Extended precision]]
* [[Granularity]]
* [[IEEE754]] (IEEE floating point standard)
* [[Integer (computer science)]]
* [[Minifloat]] (8-bit and less)
* [[Significant figures]]
* [[Truncation]]

==References==
{{Reflist|60em}}

[[Category:Approximations]]
[[Category:Computer data]]