Precision (numbers)

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

In mathematics, precision describes the level of exactness in a number's digits, such as number 54.6 having precision 1 (one decimal digit). A number with end zeroes ("00") has a negative precision, such as 500 having precision -2, or 4,000 as precision -3. A whole number (not ending in "0") has precision 0.

The level of precision differs from the word "accuracy" of a number, which means the correctness or truth of a number as used to describe an item or count.