Topic

Uncertainty and sensitivity analysis

Model results are uncertain because the inputs and assumptions behind them are uncertain. Deterministic sensitivity analysis varies inputs one or two at a time, often shown in a tornado diagram, while probabilistic sensitivity analysis samples all parameters together from distributions such as the beta and Dirichlet. Scenario analysis tests alternative assumptions.

Concepts in this topic

• Antithetic VariatesAntithetic variates are a variance reduction method that pairs each random draw with its mirror image so errors tend to cancel in Monte Carlo simulation.
• Base CaseThe base case is a model's preferred set of inputs and assumptions and the primary results they produce, against which sensitivity analyses are compared.
• Bayesian Analysis CEABayesian cost-effectiveness analysis updates prior evidence on costs and health effects with new data to estimate net benefit and decision uncertainty.
• Beta DistributionThe beta distribution is a probability distribution on values between zero and one, used in decision models to represent uncertainty in probabilities.
• Cholesky DecompositionCholesky decomposition factorises a real symmetric positive-definite matrix into a lower-triangular matrix and its transpose.
• Common Random NumbersCommon random numbers is a simulation variance-reduction method that uses corresponding random input streams across alternative strategies to improve the precision of their estimated difference when paired outputs are positively correlated.
• Confidence EllipseA graphical region, typically on a cost-effectiveness plane, showing where the true joint value of incremental cost and effect is expected to fall.
• Control VariatesControl variates is a Monte Carlo variance-reduction method that adjusts a simulated estimate using a correlated auxiliary quantity with an independently known expectation.
• ConvergenceConvergence is the property of an iterative sequence, numerical approximation, estimator, or simulation output approaching a defined limiting value or stable target as iterations, resolution, or sample size increase.
• CorrelationCorrelation is the tendency of two variables to vary together, with the form, direction, and strength of that association defined by the measure and population used.
• Deterministic Sensitivity AnalysisDeterministic sensitivity analysis evaluates how a model's results and decisions change when selected inputs or assumptions are assigned specific alternative values while all other stated conditions are controlled.
• Dirichlet DistributionThe Dirichlet distribution is a multivariate distribution for proportions that sum to one, used for uncertain transition probabilities from a model state.
• Extreme Value AnalysisA statistical approach characterising the behaviour of the most extreme, rather than typical, values a variable can take, based on extreme value distribution theory.
• First-Order IndexA first-order index in variance-based sensitivity analysis is the fraction of model-output variance explained by changes in the conditional mean when one uncertain input varies on its own under the specified input distribution.
• Global Sensitivity AnalysisGlobal sensitivity analysis, or GSA, evaluates how uncertainty in model inputs contributes to variation in model outputs when inputs vary across their full specified ranges or distributions.
• Importance SamplingA variance reduction technique for Monte Carlo simulation that deliberately oversamples important regions of a distribution, then reweights the samples to correct for this.
• Interaction EffectA difference in one variable’s effect across levels of another variable, defined on a specified outcome and measurement scale.
• Latin Hypercube SamplingA sampling technique dividing each input's range into equally probable intervals and sampling each exactly once, giving more even coverage than simple random sampling.
• Main EffectThe portion of output variance attributable to a single input parameter considered on its own, distinguished from its interaction effects with others.
• Model AveragingA statistical technique combining predictions from several plausible candidate models, weighted by their relative statistical support, rather than relying on one selected model.
• Monte Carlo ErrorMonte Carlo error is the random numerical difference between a finite-simulation estimate and the target quantity it would approach with sufficiently many valid draws under the specified model.
• Monte Carlo IntegrationA numerical technique estimating a complex integral's value by sampling random points repeatedly and averaging them, useful when no closed-form solution exists.
• Morris MethodA global sensitivity-screening method that samples one-input-at-a-time elementary effects across multiple model-input settings to identify influential or varying inputs.
• Multivariate SamplingThe simultaneous drawing of random values for multiple correlated input parameters from their joint probability distribution, rather than sampling each independently.
• Normal DistributionA normal distribution is a continuous, symmetric probability distribution characterised by a mean that sets its centre and a positive standard deviation that sets its spread.
• Numerical MethodA numerical method is a systematic computational procedure that approximates the solution of a mathematical problem through finite arithmetic operations, with accuracy assessed through error, convergence, stability, and conditioning.
• One-Way SensitivityA deterministic sensitivity analysis that varies one model input across a specified range while keeping the remaining inputs at their base-case settings, then recalculates the model outcome and any decision threshold.
• One-Way Sensitivity AnalysisOne-way sensitivity analysis (OWSA) varies one model input at a time, holding the rest at base case, to show which inputs drive cost-effectiveness results.
• Parameter DistributionA parameter distribution is a probability model for the plausible values of an uncertain model input, specified from evidence and assumptions so its uncertainty can be propagated through an analysis.
• Parameter UncertaintyParameter uncertainty is uncertainty about the true values of quantities used as model inputs because those values are estimated from incomplete or imperfect evidence.
• Probabilistic Sensitivity AnalysisProbabilistic sensitivity analysis (PSA) characterises parameter uncertainty by assigning probability distributions to uncertain model inputs and propagating joint samples through the model.
• Scenario AnalysisScenario analysis evaluates how model results and decisions change across explicitly defined, internally coherent combinations of alternative assumptions, inputs, structures, or future conditions.
• Sensitivity AnalysisSensitivity analysis examines how conclusions change when uncertain inputs, assumptions or model structures are varied.
• Simulation MethodA general approach to estimating a model's results by repeatedly running it computationally, rather than solving for results using closed-form equations.
• Sobol IndicesVariance-based global sensitivity measures that attribute a model output’s variance to uncertain inputs and their interactions under specified input distributions.
• Sobol SequenceA method generating a sequence of points that fills a multidimensional space more evenly than simple random sampling, improving simulation efficiency.
• Stochastic AnalysisAn analysis in which a model incorporates randomness explicitly, so repeated runs under the same inputs can produce different results, unlike a deterministic model.
• Stratified SamplingA sampling technique dividing a population into distinct subgroups, called strata, and drawing a separate sample from each to ensure adequate representation.
• Structural Sensitivity AnalysisA sensitivity analysis testing how results change under alternative structural assumptions, such as a different survival extrapolation function, rather than parameter values.
• Structural UncertaintyUncertainty about the appropriate form, pathways, states or mechanisms of a model, including plausible alternative structures that can change its predictions or decisions.
• Threshold AnalysisA sensitivity analysis technique identifying the value at which an input parameter would need to be set for a model's conclusion to change.
• Tornado DiagramA horizontal bar chart displaying one-way sensitivity analysis results, bars ranked from largest to smallest range, resembling a tornado shape.
• Total Effect IndexA variance-based sensitivity measure quantifying an input's total contribution to output variance, including its individual effect and all interactions with other inputs.
• Two-Way Sensitivity AnalysisAn analysis varying two parameters together across a grid of combinations, typically shown as a contour plot or heat map of results.
• Uncertainty QuantificationThe formal process of measuring how much confidence should be placed in a model's results, given the sources of uncertainty affecting it.
• Variance ReductionVariance reduction is the use of simulation sampling or estimator design to lower Monte Carlo error for a specified quantity without changing the quantity being estimated.
• Variance-Based SensitivityGlobal sensitivity analysis techniques, including Sobol indices, decomposing total output variance into components attributable to each parameter and their interactions.