VerifiedEvidence: highv1.0.0

Machine Precision

The smallest difference between two numbers that can be reliably distinguished by a computer's arithmetic system.

Last reviewedDarrin Baines IP Ltd

Concept Architecture

Concept

Theoretically, Machine Precision is the smallest numerical difference that a computer can reliably distinguish within its floating-point arithmetic system. It defines the practical limit of numerical accuracy imposed by finite-precision representation and is a fundamental concept in numerical analysis, scientific computing and computational mathematics. Machine Precision determines the accuracy with which arithmetic operations can be performed and influences the stability and reliability of numerical algorithms.

Mathematically, Machine Precision is commonly characterised by machine epsilon (�), which represents the smallest positive floating-point number that, when added to one, produces a result distinguishably greater than one. Machine Precision depends on the floating-point format and the number of bits allocated to the significand. Under IEEE 754 double-precision arithmetic, machine epsilon is approximately 2.22 ? 10???.

In practice, Machine Precision is important in health economics because decision-analytic models, optimisation algorithms, Monte Carlo simulations, regression analyses and probabilistic sensitivity analyses all rely on floating-point computation. Understanding machine precision enables analysts to establish appropriate convergence criteria, numerical tolerances and stopping rules while ensuring computational accuracy and numerical stability.

Purpose

Used to quantify the numerical limits of floating-point computation, evaluate computational accuracy, establish numerical tolerances and support reliable implementation of health economic models.

Mathematical Formulae

Primary Formula

Machine Epsilon

� = min { � > 0 : 1 + � > 1 }

Supporting Formulae

Floating-Point Relative Error

Relative Error � �

Approximate IEEE Double Precision

� � 2??� � 2.220446049250313 ? 10???

Related Mathematical Methods

  • Floating-point arithmetic
  • Finite precision
  • Binary representation
  • Rounding error
  • Numerical stability
  • Loss of significance

Example

Using IEEE double-precision arithmetic:

1 + 2.220446049250313 ? 10???

is distinguishable from

1

whereas adding a substantially smaller value may produce no change because the increment is below the available floating-point precision.

This value therefore represents the practical Machine Precision of the numerical system.

Excel Implementation

FunctionExample FormulaHealth Economics Application
Formula=2^-52Calculate IEEE double-precision machine epsilon.
ABS=ABS((1+A2)-1)Demonstrate the smallest detectable numerical increment.
IF=IF(ABS(A2)<2^-52,"Below Machine Precision","Representable")Compare numerical values with machine precision.
ROUND=ROUND(A2,15)Investigate the effect of finite precision on reported values.
EXP=EXP(A2)Evaluate functions near numerical precision limits.

VBA (Optional)

Automate evaluation of numerical tolerances, identify computations approaching machine precision limits and generate diagnostics for floating-point stability within health economic models.

Sources

  • IEEE. IEEE Standard for Floating-Point Arithmetic (IEEE 754).
  • Higham NJ. Accuracy and Stability of Numerical Algorithms.
  • Goldberg D. What Every Computer Scientist Should Know About Floating-Point Arithmetic. ACM Computing Surveys.
  • Wilkinson JH. Rounding Errors in Algebraic Processes.
  • Press WH, Teukolsky SA, Vetterling WT, Flannery BP. Numerical Recipes: The Art of Scientific Computing.
  • Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.

Library

Publications

1
  • Book

    Accuracy and Stability of Numerical Algorithms — Nicholas J. Higham, 2nd Edition ed., 2002 (Society for Industrial and Applied Mathematics (SIAM))

    The standard reference on floating-point error, conditioning, stability and the numerical reliability of matrix, polynomial and linear-algebra algorithms.

    BookView source →

Frequently Asked Questions (6)

  • What is machine precision?

    The smallest difference between two numbers that can be reliably distinguished by a computer's arithmetic system.

    Source: Higham NJ. Accuracy and Stability of Numerical Algorithms. 2nd ed. SIAM; 2002. doi:10.1137/1.9780898718027.

  • What does machine precision quantify?

    Machine precision quantifies the smallest difference between two numbers that can be reliably distinguished by a computer's arithmetic system. It marks the finest gap the arithmetic can tell apart, below which two values appear the same to the machine. Measuring this smallest distinguishable difference is what machine precision does By putting a number on the finest gap the arithmetic can tell apart, machine precision gives a concrete measure of the resolution limit within which every finite-precision computation must work.

    Source: Higham NJ. Accuracy and Stability of Numerical Algorithms. 2nd ed. SIAM; 2002. doi:10.1137/1.9780898718027.

  • Why does machine precision set a limit on distinguishing numbers?

    Machine precision sets a limit because a computer's arithmetic holds numbers to a fixed number of digits, so differences smaller than the smallest reliably distinguishable gap cannot be told apart. Two numbers closer than this appear identical to the arithmetic system. This threshold on distinguishing values is exactly what machine precision measures By putting a number on the finest gap the arithmetic can tell apart, machine precision gives a concrete measure of the resolution limit within which every finite-precision computation must work.

    Source: Higham NJ. Accuracy and Stability of Numerical Algorithms. 2nd ed. SIAM; 2002. doi:10.1137/1.9780898718027.

  • What system does machine precision describe?

    Machine precision describes a computer's arithmetic system, giving the smallest difference between two numbers that system can reliably distinguish. It is a property of how the machine represents and compares numbers rather than of any particular calculation. Characterising the arithmetic system's finest distinguishable difference is the role of machine precision By putting a number on the finest gap the arithmetic can tell apart, machine precision gives a concrete measure of the resolution limit within which every finite-precision computation must work.

    Source: Higham NJ. Accuracy and Stability of Numerical Algorithms. 2nd ed. SIAM; 2002. doi:10.1137/1.9780898718027.

  • What does a smaller machine precision indicate?

    A smaller machine precision indicates a finer resolution, meaning the computer's arithmetic system can reliably distinguish numbers that are closer together. Because machine precision is the smallest such distinguishable difference, a smaller value reflects more precise arithmetic. This link between the size of machine precision and the arithmetic's resolution is central to its meaning By putting a number on the finest gap the arithmetic can tell apart, machine precision gives a concrete measure of the resolution limit within which every finite-precision computation must work.

    Source: Higham NJ. Accuracy and Stability of Numerical Algorithms. 2nd ed. SIAM; 2002. doi:10.1137/1.9780898718027.

  • How does machine precision relate to finite precision?

    Machine precision is the smallest difference between two numbers a computer's arithmetic can reliably distinguish, while finite precision is the limitation arising from representing values with a fixed number of digits. Machine precision quantifies how coarse that finite precision is. The two are connected, since machine precision puts a number on the resolution limit that finite precision imposes By putting a number on the finest gap the arithmetic can tell apart, machine precision gives a concrete measure of the resolution limit within which every finite-precision computation must work.

    Source: Higham NJ. Accuracy and Stability of Numerical Algorithms. 2nd ed. SIAM; 2002. doi:10.1137/1.9780898718027.

Trust Record

Verified by Dr Darrin Baines

British health economist

Professional identity: darrinbaines.org

Verification date: 3 Apr 2026

Content version: 1.0.0

Canonical Identity

Persistent URI
https://healtheconomics.wiki/concept/machine-precision
Term code
CS-NA-FPA-003

Stable URI · Machine-readable · Resolvable · CC BY 4.0