Double Counting

Double Counting in Health Economic Evaluation

Double counting is the analysis error of entering a single change in health or resource use into an appraisal twice, usually because two separate measures pick up the same underlying effect. In health economics the familiar cases are a cost-benefit analysis that adds avoided medical spending to a willingness-to-pay value that already reflects that spending, and a cost-utility model that subtracts an adverse event decrement already present in the trial utilities. Most slips inflate net benefit or flatter the incremental cost-effectiveness ratio, so a programme can look worth funding when a careful sum says otherwise, although an adverse event decrement subtracted twice works the other way and penalises the treatment that causes more events. UK guidance for appraisal therefore asks analysts to state what each value contains before values are added together.

The welfare logic of counting each effect once

Welfare economics judges a project by the sum of the gains and losses of everyone it affects, with each final effect valued once. A change in health or resources spreads through several markets, and a measure taken at one point often already contains what happens further along the chain. The European Commission's Guide to Cost-Benefit Analysis of Investment Projects (2014) gives the standard example. An irrigation project raises both farmland values and farm incomes, and only one should be counted, because the owner could either sell the land or keep it and take the gains as a stream of income.

The same guide leaves out indirect effects in secondary markets, because if those markets are efficient the effects are already captured by each shadow price, and adding them usually results in double counting. Net benefit is computed as follows.

$$NB = \sum_{k=1}^{K} V_{k} - C$$

where $NB$ is net benefit, $V_{k}$ is the money value of the $k$th distinct final effect, $K$ is the number of distinct effects and $C$ is the value of the resources used. Double counting adds a term $D$, the value of an effect already inside some $V_{k}$, so the reported figure becomes $NB + D$ while true net benefit is unchanged. A cost counted twice lowers the reported figure by its value instead.

What counts as a duplicate depends on the perspective. A welfarist cost-benefit analysis sums every person's valuation. The NICE manual (PMG36) instead counts health effects for patients or, when relevant, other people (mainly carers), and takes costs from the viewpoint of the NHS and personal social services (section 4.2.7). This extra-welfarist reference case avoids some duplicates by design and creates others at its boundaries, as the productivity debate below shows.

Willingness to pay and earnings in cost-benefit analysis

The US Department of Health and Human Services guidelines for regulatory impact analysis (2016) allow averted costs of illness to be added to willingness to pay values or monetised QALYs, as long as the same savings are not counted elsewhere. They note that willingness-to-pay studies typically may capture out-of-pocket costs and lost earnings, and possibly informal care by household members, but not costs paid by third parties such as medical expenses paid by insurance. Savings in those third-party costs can be added, after the analyst has checked what the study already captures.

Monetised QALYs are harder to read, because the guidelines describe the extent to which they include costs as highly uncertain. Only occasionally do studies that estimate health-related quality of life tell respondents to assume that insurance will offset their medical costs and lost income. In the absence of more specific information, the guidelines allow third-party medical costs to be added to monetised QALYs but not lost productivity or income. The value of statistical life page makes the same point about lost earnings and mortality risk.

The HM Treasury Green Book (2026) states that an economic transfer, such as a social security payment, incurs a cost to the payer and an equal benefit to the recipient, and so does not in itself make society better or worse off. Transfers may be entered on both sides, where they cancel in net present social value but push the benefit-cost ratio towards one, or left out altogether. Counting a fall in sickness benefit payments as a gain to society, with no matching loss to recipients, records a transfer on one side only.

Utilities, adverse events and productivity in cost-utility analysis

In a cost-utility analysis the risk lies mostly in the denominator. NICE Decision Support Unit Technical Support Document 12, published in 2011, states that decrements for grade 3 to 4 adverse events are particularly important, while decrements for grade 1 to 2 events must not be double counted, because the cohort that supplied the main health state utility values may already have included patients who experienced them. The arithmetic of subtracting a disutility from a health state utility value that already contains the loss is set out on those pages.

Tilling and colleagues, in a 2009 Sheffield discussion paper later published in PharmacoEconomics in 2012, recount that the 1996 Washington Panel argued that people answering health state valuation questions take account of the effect of ill health on their income, so that adding productivity changes to the cost side would be a form of double counting. In an online time trade-off survey of EQ-5D states in the Netherlands, the authors found that spontaneous inclusion of income effects did not significantly change valuations at the aggregate level. They read this as suggesting that earlier studies valuing productivity costs with the human capital approach or the friction cost method had not counted those costs twice.

PMG36 excludes productivity costs from the reference case (sections 4.2.9 and 4.4.23), allowing them to be presented separately when they may be a critical part of a technology's value. If QALYs did contain income effects, that separate presentation would duplicate part of the health gain, and a societal perspective that counts productivity on the cost side faces the same question.

Carers bring a related risk. PMG36 counts health effects for carers when relevant (section 4.2.7) and allows the time cost of family care that the NHS or social services would otherwise have provided, shown separately (section 4.4.24). A carer time cost and a carer QALY loss count one harm twice only when both measure the same loss, so an analysis should say which loss each figure represents.

Wellbeing values beside QALYs in UK appraisal

HM Treasury's Wellbeing Guidance for Appraisal, supplementary Green Book guidance published in July 2021, addresses a newer overlap. Its annex entry on physical and mental health recommends QALY or DALY evidence where available and warns that using subjective wellbeing values in addition would pose a significant risk of double counting. In its youth programme example, crime reduction is valued using Home Office estimates that rest on QALYs, so no further wellbeing effects of crime are added.

The guidance also shows that related values need not overlap. The wellbeing gain from moving into work is counted alongside the productivity gain, valued at average income, because the two measure different things. Its appraisal checklist asks whether care has been taken to avoid double counting outcomes.

Double counting in consequence tables and pooled trials

The NICE guideline manual (PMG20) asks committees using a cost-consequence analysis to ensure, where possible, that the different sets of consequences do not double count costs or effects. The same logic applies to multi-criteria decision analysis, where two criteria that capture one underlying benefit give it more weight than the panel intended.

Evidence synthesis has its own form of the error. The Cochrane Handbook (chapter 23, section 23.3.4) warns that entering several comparisons from a multi-arm trial into one meta-analysis, with the same comparator group each time, double counts the participants in that group and creates a unit-of-analysis error. It recommends combining groups into a single pair-wise comparison and notes that network meta-analysis handles multi-arm studies appropriately.

Worked example with a back pain programme

All numbers in this example are illustrative. A health authority considers a GBP 1,600,000 programme for 1,000 adults with chronic low back pain. A contingent valuation survey finds a mean willingness to pay of GBP 1,200 per participant, and the survey told respondents that the programme would also help them return to work. The NHS expects to avoid GBP 300,000 of hospital costs, made up of 1,000 bed days at GBP 300 each. Participants regain GBP 150,000 of earnings.

1. Correct benefits. Willingness to pay totals 1,000 × 1,200, or GBP 1,200,000. The NHS savings fall on a third party, so they are added, giving benefits of GBP 1,500,000.

2. Correct result. Net benefit is 1,500,000 minus 1,600,000, or minus GBP 100,000, and the benefit-cost ratio is 1,500,000 divided by 1,600,000, or about 0.94. The programme fails a cost-benefit test.

3. First double count. Adding the GBP 150,000 of regained earnings counts what respondents had already valued, raising benefits to GBP 1,650,000.

4. Second double count. Valuing the 1,000 released bed days at GBP 300 each as a further benefit counts the GBP 300,000 saving twice, raising benefits to GBP 1,950,000.

5. Reported result. Net benefit becomes 1,950,000 minus 1,600,000, or GBP 350,000, and the ratio becomes 1,950,000 divided by 1,600,000, or about 1.22. The overstatement is 150,000 plus 300,000, or GBP 450,000, enough to reverse the sign.

6. The cost-utility view. Suppose the programme also yields 20 QALYs and the NHS bears its cost. The net cost is 1,600,000 minus 300,000, or GBP 1,300,000, and the ICER is 1,300,000 divided by 20, or GBP 65,000 per QALY. Deducting the bed-day saving a second time gives 1,000,000 divided by 20, or GBP 50,000 per QALY.

7. Interpretation. Both ICERs lie above the GBP 25,000 to GBP 35,000 per QALY range in PMG36 (sections 6.3.7 and 6.3.8), so the decision is unchanged, but in a closer case a GBP 15,000 per QALY error could bring an ICER within the range.

Telling double counting apart from legitimate counting

Not every repeated figure is an error. A transfer entered as both a cost and a benefit is permitted by the Green Book, and the same transfer shown again in distributional analysis, which the Green Book requires, answers a separate question about who gains and who loses.

Correlated values are not necessarily duplicates either. A QALY gain sits beside NHS cost savings in every cost-utility analysis without overlap. For sensitivity analysis, the European Commission guide notes that correlated variables give rise to distortions and double counting, so the model should isolate independent variables before testing them.

Frequent sources of double counting in health appraisals

Five sources of double counting recur in health work. Each adds a second value for a consequence already counted.

1. Medical costs added to willingness to pay. Costs borne by respondents may already sit inside a willingness-to-pay value, so only third-party costs are added.
2. Earnings added to monetised QALYs. Third-party medical costs may be added, but not lost income or productivity.
3. Mild adverse events subtracted from trial utilities. Grade 1 to 2 decrements may already sit inside utilities measured in patients on treatment.
4. Wellbeing values stacked on QALYs. Where QALY evidence exists, the wellbeing guidance prefers it.
5. A shared comparator entered twice in a meta-analysis. The groups of a multi-arm trial should be combined into one comparison.

Sources

• Ara R, Wailoo A. NICE DSU Technical Support Document 12: the use of health state utility values in decision models. Decision Support Unit, School of Health and Related Research, University of Sheffield; July 2011. Section 3.2.1. https://www.sheffield.ac.uk/nice-dsu/tsds/utilities
• European Commission, Directorate-General for Regional and Urban Policy. Guide to Cost-Benefit Analysis of Investment Projects: Economic appraisal tool for Cohesion Policy 2014-2020. December 2014. Sections 2.8.10 and 2.9.1. https://ec.europa.eu/regional_policy/sources/studies/cba_guide.pdf
• Higgins JPT, Eldridge S, Li T. Chapter 23: Including variants on randomized trials (last updated October 2019). In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane; 2024. Section 23.3.4. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-23
• HM Treasury. The Green Book (2026). Updated 28 October 2026. https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026
• HM Treasury. Wellbeing Guidance for Appraisal: Supplementary Green Book Guidance. 26 July 2021. https://www.gov.uk/government/publications/green-book-supplementary-guidance-wellbeing
• National Institute for Health and Care Excellence. Developing NICE guidelines: the manual (PMG20). Published 31 October 2014, last updated 23 October 2025. Chapter 7. https://www.nice.org.uk/process/pmg20/chapter/incorporating-economic-evaluation
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Sections 4.2.7, 4.2.9, 4.4.23, 4.4.24, 6.3.7 and 6.3.8. https://www.nice.org.uk/process/pmg36
• Tilling C, Krol M, Tsuchiya A, Brazier J, van Exel J, Brouwer W. The impact of losses in income due to ill health: does the EQ-5D reflect lost earnings? HEDS Discussion Paper 09/04. School of Health and Related Research, University of Sheffield; 2009. https://mpra.ub.uni-muenchen.de/29837/
• Tilling C, Krol M, Tsuchiya A, Brazier J, van Exel J, Brouwer W. Does the EQ-5D reflect lost earnings? PharmacoEconomics. 2012;30(1):47-61. https://doi.org/10.2165/11539910-000000000-00000
• US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation. Guidelines for Regulatory Impact Analysis. 2016. https://aspe.hhs.gov/sites/default/files/migrated_legacy_files/171981/HHS_RIAGuidance.pdf