Numerical Method

Numerical Method: Concept Architecture

Orientation and learning roadmap

Numerical methods convert mathematical problems into finite computations that can be executed and checked. This page explains approximation, error, conditioning, stability, convergence, stopping rules, and the main families of numerical tasks. It then connects method choice and verification to health-economic models, worked calculations, spreadsheet use, reproducibility, and decision consequences.

Numerical methods solve problems that resist exact calculation

Many equations, integrals, differential equations, optimization problems, and matrix systems have no convenient closed-form solution at the required scale. A numerical method produces an approximation using a finite sequence of arithmetic and logical operations.

The method is distinct from both the mathematical model and its software implementation. A sound model can be solved poorly, and a flawless implementation can accurately solve the wrong mathematical problem.

Inputs, outputs, and tolerances define the computational task

A numerical problem specifies input data, an exact mathematical target, and the output quantity required. The numerical method adds an approximation scheme, control parameters, and a stopping or refinement rule.

Accuracy requirements should be expressed in units meaningful to the final use. A tolerance on an internal residual may not guarantee an adequately accurate cost, QALY, probability, or policy decision.

Absolute and relative errors answer different questions

Absolute error measures the distance between an approximation and the exact value in the quantity's units. Relative error scales that distance by the exact value and is useful when magnitudes vary widely.

[ e_{abs}=|\widehat x-x|, \qquad e_{rel}=\frac{|\widehat x-x|}{|x|},\quad x\neq0. ]

Relative error is undefined at zero and unstable near zero. A combined criterion such as (e_{abs}\leq a+r|x|) can protect both small and large values.

Error has several distinct sources

Numerical error is not one undifferentiated quantity. Representation, rounding, truncation, discretization, iteration, sampling, and implementation errors arise from different mechanisms and require different remedies.

• Representation error occurs when an input or intermediate real number is not exactly representable in the chosen numeric format.
• Roundoff error arises from finite-precision arithmetic operations.
• Truncation error arises when an infinite mathematical process is replaced by a finite approximation.
• Discretization error arises when a continuous problem is represented on a finite grid or time step.
• Iteration error remains when an iterative process stops before reaching its limiting solution.
• Simulation error arises when stochastic expectations are estimated from finitely many random draws.
• Implementation error arises when code does not match the specified algorithm.

Conditioning belongs to the underlying mathematical problem

Conditioning describes how strongly the exact solution changes when the inputs are perturbed. An ill-conditioned problem can amplify unavoidable measurement or representation error even under a stable algorithm.

For a scalar function, a local relative condition measure may be written as:

[ \kappa_f(x)=\left|\frac{x f'(x)}{f(x)}\right|, ]

where the expression is defined. Conditioning should be evaluated at the problem and output level rather than inferred from the apparent complexity of the formula.

Stability belongs to the algorithm

Numerical stability describes whether an algorithm avoids unnecessarily amplifying errors while computing the intended result. Backward stability means the computed result is the exact solution to a nearby input problem.

A stable algorithm cannot recover information lost in severely ill-conditioned inputs, but it prevents avoidable degradation. Comparing algebraically equivalent formulations often reveals important stability differences.

Consistency and stability support convergence

Consistency means that the discrete numerical rule approaches the governing mathematical relation as its step or grid scale shrinks. Stability controls how errors propagate through the computation.

For many well-posed approximation problems, suitable consistency and stability together support convergence to the exact solution. The precise theorem and norms depend on the problem class, so this principle should not be applied as a universal slogan.

Order describes asymptotic error reduction

A method has order (p) when its dominant error behaves like a constant times (h^p) as the resolution parameter (h) approaches zero in the asymptotic regime. Halving (h) then reduces the leading error by approximately (2^p).

[ E(h)=Ch^p+o(h^p). ]

Observed order can be distorted by roundoff, irregular solutions, discontinuities, solver tolerances, or steps that are not yet sufficiently small. Formal order alone does not determine efficiency or stability.

A refinement study estimates observed order

Composite trapezoidal integration of (f(x)=x^2) on ([0,1]) has exact integral (1/3) and error (h^2/6). For (h=0.1,0.05,0.025), the absolute errors are approximately (0.00166667), (0.000416667), and (0.000104167).

Each halving reduces error by a factor of four, giving observed order:

[ p\approx\log_2\left(\frac{E(h)}{E(h/2)}\right)=2. ]

This clean behavior is a verification case, not a guarantee that the same order appears for every integrand or implementation.

Residual and solution error are not interchangeable

A residual measures how closely an approximate solution satisfies the original equation. Solution error measures distance from the exact solution, and a small residual implies a small solution error only when conditioning supports that inference.

For (A\widehat{\mathbf x}=\mathbf b-\mathbf r), the residual is (\mathbf r=\mathbf b-A\widehat{\mathbf x}). An ill-conditioned matrix can produce a small residual alongside a large forward error.

Root-finding turns equations into zero searches

Solving (f(x)=0) can use bracketing, fixed-point, Newton, secant, or hybrid methods. Bracketing methods provide strong interval safeguards when a sign change and continuity apply, while derivative-based methods can converge faster near a suitable root.

Method choice depends on derivative availability, smoothness, multiplicity, bracketing information, and evaluation cost. A small function value does not prove closeness to a root when the function is flat or poorly scaled.

Newton's method illustrates rapid local convergence

For (f(x)=x^2-2), Newton's update for (\sqrt2) is the average of (x_n) and (2/x_n). Starting at (x_0=1), the first four iterates are (1.5), (1.416666667), (1.414215686), and (1.41421356237469).

[ x_{n+1}=x_n-\frac{x_n^2-2}{2x_n} =\frac{1}{2}\left(x_n+\frac{2}{x_n}\right). ]

The fourth iterate differs from (\sqrt2) by about (1.59\times10^{-12}). This quadratic local convergence depends on a simple root, a sufficiently good start, and accurate derivative behavior.

Linear systems require structure-aware solvers

Direct methods such as Gaussian elimination and matrix factorizations produce a solution in finitely many arithmetic stages, subject to roundoff. Iterative methods improve a sequence of approximations and can exploit sparsity in large systems.

Pivoting, scaling, symmetry, positive definiteness, sparsity, and conditioning guide method selection. Explicit matrix inversion is usually inferior to solving the factorized system for a specified right-hand side.

Interpolation estimates values within represented data

Interpolation constructs a function that matches supplied values at selected nodes. Polynomial, piecewise polynomial, spline, and shape-preserving methods differ in smoothness, local control, oscillation, and computational cost.

Interpolation error depends on node placement and function regularity as well as spacing. Evaluating outside the node range is extrapolation and generally carries different, often greater, uncertainty.

Numerical differentiation amplifies high-frequency error

Finite differences approximate derivatives from nearby function values. Smaller steps reduce truncation error initially but increase cancellation and sensitivity to noise.

[ f'(x)\approx\frac{f(x+h)-f(x-h)}{2h}. ]

Analytic derivatives, automatic differentiation, or complex-step methods where valid can improve accuracy. The derivative method should match the smoothness and precision of the available function evaluations.

Quadrature approximates accumulated quantities

Numerical quadrature replaces an integral with a weighted sum of function values. Newton–Cotes, Gaussian, adaptive, and Monte Carlo approaches suit different dimensionality, smoothness, and evaluation structures.

[ \int_a^b f(x),dx\approx\sum_{j=1}^{m}w_jf(x_j). ]

Sharp peaks, discontinuities, infinite domains, singularities, and oscillation require tailored transformations or subdivision. A solver's estimated error should be tested against refinement and known cases.

Differential-equation solvers advance dynamic states

Ordinary differential-equation methods approximate a continuous trajectory from rates and initial conditions. Euler, Runge–Kutta, multistep, and implicit families differ in order, stability, adaptivity, and cost.

Stiff systems can force explicit methods to take tiny steps for stability even when accuracy would permit larger steps. Events, discontinuities, positivity, and conserved quantities require explicit handling and checks.

Optimization methods search for best feasible decisions

Numerical optimization approximates minimizers or maximizers of objective functions, possibly under constraints. Gradient, Newton, trust-region, sequential quadratic, derivative-free, stochastic, and global methods provide different guarantees.

A termination flag does not establish global optimality, feasibility at meaningful precision, or validity of the objective. Nonconvexity, scaling, initialization, derivative accuracy, and constraint residuals must be assessed.

Eigenvalue methods reveal dynamic and matrix structure

Eigenvalues and eigenvectors describe invariant directions, long-run transitions, stability, variance structure, and dimensional reduction. Iterative methods can target selected eigenpairs without computing a full decomposition.

Repeated or closely clustered eigenvalues can make individual eigenvectors unstable even when the invariant subspace is meaningful. Residual norms and orthogonality checks should accompany reported decompositions.

Monte Carlo methods trade deterministic grids for sampling

Monte Carlo methods estimate expectations using random or quasi-random samples and are especially useful in high-dimensional uncertainty propagation. Their stochastic error usually falls in proportion to (N^{-1/2}) under ordinary independent sampling.

Variance reduction, correlation, effective sample size, rare events, and random-number quality affect precision. A fixed seed supports reproducibility but does not replace an empirical Monte Carlo error assessment.

Adaptive methods spend effort where it is needed

Adaptive algorithms change step size, grid density, quadrature nodes, or iteration controls in response to estimated local error. They can achieve a target accuracy with fewer evaluations than a uniformly fine discretization.

Error estimators are themselves approximations and may fail near discontinuities, singularities, or unrecognized stiffness. Minimum and maximum step safeguards and independent refinement remain important.

Stopping rules must match the requested output

Iterative methods stop using residuals, step changes, estimated errors, gradient norms, probability limits, or evaluation budgets. A single generic tolerance rarely controls every output component equally.

Absolute and relative criteria should reflect variable scales and final decision sensitivity. Hitting an iteration limit is a diagnostic outcome, not a converged solution.

Floating-point arithmetic changes exact identities

Finite-precision arithmetic can break associativity, absorb small addends, overflow, underflow, and lose digits through cancellation. Operation order and hardware-supported fused operations may change the last bits of a result.

Stable formulations, scaling, compensated summation, appropriate libraries, and higher-precision benchmarks reduce risk. Display formatting does not increase stored accuracy.

Verification and validation answer different questions

Code verification asks whether software implements the intended algorithm, while solution verification asks whether numerical approximations adequately solve the mathematical problem. Validation asks whether the mathematical and computational model adequately represents the real system for its intended use.

Passing a numerical benchmark establishes neither empirical validity nor policy relevance. These layers should be documented separately so computational success is not mistaken for model truth.

Manufactured and exact solutions provide strong tests

An exact solution, simplified special case, or manufactured solution enables direct measurement of numerical error. Tests should span ordinary, boundary, degenerate, and difficult cases rather than one favorable example.

Automated regression tests can preserve benchmark values, invariants, rates of convergence, and failure behavior across code changes. Test tolerances must be justified and should not be widened merely to make a failing result pass.

Independent implementations strengthen confidence

Agreement between two genuinely independent methods or software implementations can reveal coding and algorithm defects. Independence is weak when both routes use the same library, formula, data transformation, or hidden default.

Disagreement should be traced through intermediate quantities, units, conventions, and tolerances. Matching rounded outputs alone may conceal offsetting errors.

Health-economic models contain many numerical layers

Economic evaluation may combine survival extrapolation, numerical integration, state-transition recursion, microsimulation, calibration, optimization, probabilistic sensitivity analysis, and value-of-information estimation. Each layer contributes its own approximation and can interact with the others.

Numerical tolerances should be set so their combined effect is small relative to meaningful incremental costs, outcomes, and net benefit. Parameter and structural uncertainty remain separate from numerical error.

Incremental results deserve direct accuracy checks

Health-economic decisions often depend on differences between large total costs or total QALYs. Even small relative errors in each strategy can become large relative errors in the incremental result.

Analysts should reconcile incremental components, avoid premature rounding, use matched simulations where suitable, and refine the numerical method until decision outputs stabilize. Strategy totals alone are insufficient verification targets.

Cycle length is a numerical choice with modeling consequences

Discrete-time health models approximate events and rewards within cycles. Shortening the cycle can reduce temporal discretization error, but it also changes transition conversions, runtime, and possibly the structural interpretation of simultaneous events.

A cycle-length analysis should compare state occupancy, events, costs, QALYs, and incremental decisions. Half-cycle correction can reduce some timing bias but is not a universal substitute for refinement.

Extrapolation can dominate numerical precision

A highly accurate numerical evaluation of an unsupported long-term extrapolation is still unreliable for decision making. Numerical error, statistical parameter uncertainty, and structural extrapolation uncertainty should be quantified and communicated separately.

Solver precision should not be used to imply confidence beyond the evidence. Sensitivity to survival family, hazards, cure assumptions, and horizon may exceed any integration or rounding error.

Excel is useful for transparent small-scale checks

A spreadsheet can expose iterations, residuals, step sizes, and refinement tables cell by cell. It is useful for teaching, small deterministic methods, and independent checks of selected calculations.

Large sparse systems, stiff equations, high-dimensional simulation, adaptive algorithms, and high-stakes optimization are better handled in controlled numerical software. Spreadsheet formulas should avoid volatile behavior, hidden rounding, and circular calculations unless deliberately designed and tested.

A disciplined numerical-method workflow

Reliable numerical work begins with a mathematical target and an output-level accuracy requirement. It ends with evidence that the computation is stable, converged, reproducible, and adequate for the decision. A practical sequence is:

1. Specify the problem. Define inputs, equations, domains, units, constraints, and required outputs.
2. Set accuracy goals. State absolute, relative, probabilistic, or decision-level tolerances before calculation.
3. Assess conditioning. Identify inputs and outputs that inherently amplify perturbations.
4. Choose the method. Match smoothness, dimension, sparsity, stiffness, constraints, randomness, and available derivatives.
5. Select controls. Set resolution, step sizes, tolerances, iteration limits, precision, and random streams.
6. Verify implementation. Use exact cases, manufactured solutions, residuals, invariants, and independent calculations.
7. Demonstrate convergence. Refine grids, steps, tolerances, samples, or iterations and estimate observed behavior.
8. Test stability. Compare formulations, scaling, precision, ordering, starting values, and difficult inputs.
9. Trace decision impact. Propagate numerical discrepancies into incremental outcomes and conclusions.
10. Report reproducibly. Preserve algorithms, code, software, versions, settings, hardware-relevant details, and tests.

Common errors and safeguards

Numerical outputs can look precise even when the method has not converged or the implementation is unstable. Strong safeguards connect error control to the mathematical target and final decision.

• Reporting many digits without an error estimate or refinement study.
• Treating a small residual as proof of a small solution error in an ill-conditioned problem.
• Confusing formal order with observed accuracy on the problem at hand.
• Choosing a method by familiarity without checking stiffness, smoothness, sparsity, or constraints.
• Tightening tolerances while leaving a dominant discretization or model error unchanged.
• Using a solver success flag as the only verification evidence.
• Comparing two calculations that share the same hidden implementation defect.
• Ignoring underflow, overflow, cancellation, and operation-order effects.
• Treating simulation noise as parameter uncertainty.
• Rounding totals before calculating incremental costs or effects.
• Validating the real-world model without first verifying the numerical solution.
• Claiming policy certainty from a numerically precise but structurally uncertain extrapolation.

Reproducible reporting makes numerical claims auditable

A complete record identifies the mathematical problem, algorithm, discretization, numeric precision, scaling, starting values, stopping criteria, tolerances, random-number generator, seeds, software, libraries, versions, and relevant hardware or parallel settings. It should preserve warnings, failures, fallback methods, and any manual intervention.

Verification evidence should include benchmark cases, residuals, invariants, refinement results, observed order where relevant, sensitivity to controls, and independent implementations. Health-economic reporting should additionally show that numerical uncertainty is negligible relative to parameter and structural uncertainty and cannot plausibly reverse the decision at the reported precision.