Tornado Diagram

Tornado Diagram in Cost-Effectiveness Analysis

A tornado diagram stacks one horizontal bar per model input, and each bar spans the values a cost-effectiveness result takes when that input alone moves between the low and high ends of its range. Sorting the widest bar to the top gives the funnel shape behind the name and shows at a glance which inputs move the ICER or net benefit most. The diagram is the usual display for a one-way sensitivity analysis in health technology assessment, and the NICE manual (PMG36) names tornado histograms as a possible way to present deterministic sensitivity analysis results. The chart adds no calculation of its own, so what a committee sees depends on choices made before it is drawn, above all the outcome axis and the input ranges.

Parts of a tornado diagram

The ISPOR-SMDM task force report on parameter estimation and uncertainty gives the standard layout. The outcome runs along the horizontal axis, the parameters are arrayed down the vertical axis, and each bar covers the outcome range produced by that parameter's range. A vertical line through all the bars marks the base-case result, and commonly the longest bar is placed at the top with the others in descending order of length.

The report asks for a legend or table giving the upper and lower bound of each parameter with its justification from the evidence, and accepts a table or a sentence in the text instead of the chart. It adds that threshold values can be combined with the tornado presentation by marking them on the bars.

$$D_j = \left| Y\left(\theta_j^{U}, \theta_{-j}^{0}\right) - Y\left(\theta_j^{L}, \theta_{-j}^{0}\right) \right|$$

where $D_j$ is the bar width for input $j$, $Y$ is the model outcome on the horizontal axis, $\theta_j^{L}$ and $\theta_j^{U}$ are the low and high values of input $j$, and $\theta_{-j}^{0}$ holds every other input at its base-case value. Bars can be shaded by input end or by direction of the result, and the legend has to say which, because a low input value can produce a high outcome.

Choosing the outcome on the axis

The outcome on the axis changes the order of the bars as well as their lengths. McCabe and colleagues observe that tornado diagrams frequently use the ICER rather than net benefit as the axis scale, which brings complications because ICERs are neither continuously nor uniquely defined. Vreman and colleagues add that when an input value makes incremental QALYs exactly zero, no ICER exists to plot, so the classic diagram shows zero or leaves the run out.

Expressing the bars in incremental net monetary benefit removes that gap and the non-linearity that comes only from the ratio. Where the ICER is kept, Vreman and colleagues suggest showing which quadrant of the cost-effectiveness plane each result falls in, for example with transparent bars where the new therapy dominates.

The two scales can rank the same inputs differently. Take an input that moves only incremental cost and another that moves only incremental QALYs.

$$D_j^{\mathrm{ICER}} = \frac{\Delta C_j^{U} - \Delta C_j^{L}}{\Delta E_0}, \qquad D_j^{\mathrm{INMB}} = \Delta C_j^{U} - \Delta C_j^{L}$$

where $\Delta C_j^{L}$ and $\Delta C_j^{U}$ are the lower and higher incremental costs at the two ends of cost input $j$, and $\Delta E_0$ is the base-case incremental QALY gain.

$$D_k^{\mathrm{ICER}} = \Delta C_0 \left( \frac{1}{\Delta E_k^{L}} - \frac{1}{\Delta E_k^{U}} \right), \qquad D_k^{\mathrm{INMB}} = \lambda \left( \Delta E_k^{U} - \Delta E_k^{L} \right)$$

where $\Delta E_k^{L}$ and $\Delta E_k^{U}$ are the lower and higher incremental QALYs at the two ends of QALY input $k$, both positive, $\Delta C_0$ is the base-case incremental cost, taken as positive, and $\lambda$ is the cost-effectiveness threshold in GBP per QALY. On the ICER scale a cost bar is divided by the base-case QALY gain while a QALY bar depends on reciprocals, so the ranking can change with the axis.

Worked example with four inputs and a GBP 25,000 threshold

All numbers in this example are illustrative. A new drug is compared with standard care per patient, with costs and QALYs discounted. Incremental cost is the extra drug cost plus the extra cost of managing adverse events, less hospital costs avoided. The threshold is GBP 25,000 per QALY, the lower end of the range in PMG36.

In the base case, incremental cost is 9,000 + 1,500 − 1,500 = GBP 9,000 and the QALY gain is 0.50, so the ICER is 9,000 ÷ 0.50 = GBP 18,000 per QALY. INMB is 25,000 × 0.50 − 9,000 = GBP 3,500.

The drug cost runs give 6,000 ÷ 0.50 = GBP 12,000 and 12,000 ÷ 0.50 = GBP 24,000, with INMB of 12,500 − 6,000 = GBP 6,500 and 12,500 − 12,000 = GBP 500. The adverse event runs give incremental costs of GBP 8,000 and GBP 13,000, and the hospital runs GBP 9,500 and GBP 8,500. For the QALY gain, the low value gives 9,000 ÷ 0.35 = GBP 25,714 and 25,000 × 0.35 − 9,000 = minus GBP 250, and the high value gives 9,000 ÷ 0.65 = GBP 13,846 and 25,000 × 0.65 − 9,000 = GBP 7,250. The swing formulas agree, since 6,000 ÷ 0.50 = 12,000 and 25,000 × 0.30 = 7,500.

Reading the four bars

On the ICER axis the drug cost bar sits on top with the QALY gain a close second, and on the INMB axis those two swap places. The QALY gain bar is lopsided on the ICER axis, running GBP 4,154 below the base case and GBP 7,714 above it, while on the INMB axis it sits GBP 3,750 either side.

Marking GBP 25,000 on the chart changes the reading. The longest ICER bar, drug cost, stays wholly below the threshold, while the second and third, QALY gain and adverse event cost, both cross it. Setting INMB to zero gives switching values of 12,500 − 9,000 + 1,500 = GBP 5,000 for adverse event cost and 9,000 ÷ 25,000 = 0.36 for the QALY gain, both inside their ranges, and GBP 12,500 for drug cost, outside its range. Hospital cost avoided cannot switch the decision, since INMB reaches zero only if hospital costs rose by GBP 2,000. At GBP 35,000 per QALY, the upper end of the PMG36 range, no bar crosses, since the highest ICER in the table is GBP 26,000. The low ends of hospital cost avoided and of QALY gain give the high ICER, so shading by input end and by result direction disagree for those two bars.

What sets the length of each bar

Because the chart only rescales the endpoint runs, a bar is as long as the range assigned to its input allows. The ISPOR-SMDM report and the HEMA report on uncertainty both treat arbitrary variation, such as plus or minus 20%, as a measure of sensitivity rather than of uncertainty.

Vreman and colleagues looked at the 20 NICE technology appraisals published in the first half of 2019 whose committee papers included the company model. In 14 of them the deterministic analysis used arbitrary ranges for several parameters, between plus or minus 10% and 50% or of unknown size, and each of those models applied one percentage to all parameters. In their case study, two cost inputs varied by 50% and 25% would have sat in a different order under a common range.

Correlation is a separate source of error. Vreman and colleagues point out that moving one of the two correlated parameters of a Weibull or lognormal survival curve to the end of its range, with the other at base case, cancels the correlation and gives an incorrect estimate.

Reading a tornado diagram in an appraisal

PMG36, section 4.7.17, says tornado histograms may be a useful way to present deterministic analyses of individual or multiple correlated parameters, but that deterministic analysis is less appropriate for decision making if the model is non-linear. Section 4.7.22 defines a switching value as the value an input would need to take for the decision to change at a given threshold.

The HEMA report calls tornado diagrams familiar and easy to interpret but advises caution, since they examine inputs individually, may rely on poorly justified ranges and generally do not preserve correlations. It recommends reading them alongside analyses of joint uncertainty, including probabilistic sensitivity analysis and, where appropriate, EVPI or EVPPI. In the worked example the longest ICER bar, drug cost, never crossed the threshold, while the longest INMB bar, QALY gain, crossed zero.

Stepwise, distributional and probabilistic tornado diagrams

Vreman and colleagues set out three extensions that keep the tornado layout. A stepwise analysis records the outcome at intermediate steps between the base case and each end, which exposes non-linearity and marginal effects. A distributional analysis places those steps at percentiles of each parameter's probability distribution, so values in the tails are shown as less likely. A probabilistic analysis fixes the input of interest at each step and samples all other inputs, which carries their correlations into the result. With 12 parameters, 21 steps and 10,000 iterations per step their case study needed 2.5 million iterations, and the classic and probabilistic diagrams put the parameters in an entirely different order.

McCabe and colleagues proposed the probabilistic form with net benefit as the outcome, probabilistic one-way sensitivity analysis, reporting conditional expected incremental net benefit at specific parameter values. PMG36, section 4.7.21, notes that probabilistic univariate sensitivity analysis may be explored to incorporate the likelihood of a parameter taking its upper and lower bound values. Vreman and colleagues separate this from EVPPI, which also fixes one parameter while sampling the rest but values reducing uncertainty in it. Rankings from variance-based sensitivity analysis, such as the total-effect index, answer a different question, the share of output variance due to an input, including its interactions.

Tornado diagrams beside spider plots and other displays

Eschenbach compared spiderplots, used by engineering economists, with tornado diagrams, used by decision analysts, and judged their advantages complementary. In his account the simpler tornado diagram can summarise the total impact of many independent variables, while a spiderplot shows more about fewer variables, including the change needed to reach break-even.

Inputs that interact need a two-way sensitivity analysis, and a change of model structure belongs in scenario analysis. The HEMA report says that the cost-effectiveness plane, the cost-effectiveness acceptability curve and the tornado diagram each give useful information, but none gives a complete account of uncertainty alone.

Common misreadings of tornado diagrams

Most errors come from reading the bars as statements about probability or about the decision. Four recur often.

1. Treating bar length as uncertainty. A long bar may reflect a wide range chosen by convention, and a short bar may hide an input that is both influential and poorly estimated.
2. Reading bar share as probability. McCabe and colleagues note that the proportion of a bar lying in positive net benefit does not indicate the probability that the parameter takes a value giving positive net benefit.
3. Ranking by length when the question is the decision. The longest bar can sit wholly on one side of the threshold while a shorter one crosses it.
4. Ignoring the scale. Bars for QALY inputs on the ICER axis are asymmetric and can vanish where incremental QALYs reach zero, and their order can differ from the INMB order.

Sources

• Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making. 2012;32(5):722-732. https://doi.org/10.1177/0272989X12458348
• Eschenbach TG. Spiderplots versus tornado diagrams for sensitivity analysis. Interfaces. 1992;22(6):40-46. https://doi.org/10.1287/inte.22.6.40
• McCabe C, Paulden M, Awotwe I, Sutton A, Hall P. One-way sensitivity analysis for probabilistic cost-effectiveness analysis: conditional expected incremental net benefit. PharmacoEconomics. 2020;38(2):135-141. https://doi.org/10.1007/s40273-019-00869-3
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.7.17, 4.7.21 and 4.7.22. https://www.nice.org.uk/process/pmg36
• Steuten L, Thavorn K, Cipriano LE, Guertin JR, McQueen B, Schmidler G, Sullivan S, Wailoo A, Sculpher M. Methods to Analyze and Present Uncertainty in Economic Evaluation to Support HTA Decisions (HEMA Report 2). Health Economics Methods Advisory; 2026. Sections 4.3 and 6.4. https://hemamethods.org/wp-content/uploads/2026/10/HEMA-Report-2_For-Publication_100726.pdf
• Vreman RA, Geenen JW, Knies S, Mantel-Teeuwisse AK, Leufkens HGM, Goettsch WG. The application and implications of novel deterministic sensitivity analysis methods. PharmacoEconomics. 2021;39(1):1-17. https://doi.org/10.1007/s40273-020-00979-3