Budget Impact Model

Budget Impact Model Structure and Validation

A budget impact model is the spreadsheet or software that does the arithmetic of a budget impact analysis. It holds the population, treatment mix, cost and timing inputs and turns them into yearly spending under two scenarios. Its design decides which questions the analysis can answer, because a model that cannot track people entering and leaving the treated group cannot show how a curative or life-extending technology changes spending over time. The choice between a cost calculator, a cohort model and a patient-level model, and between open and closed populations, can change the answer. Estimating the eligible population, uptake and prices is covered on the budget impact analysis page, and the output on the budget impact page.

What a budget impact model contains

The 2012 ISPOR Budget Impact Analysis Good Practice II Task Force, reported by Sullivan and colleagues, lists the key elements of the analysis as the size of the eligible population, the current mix of treatments, the expected mix after the new intervention is introduced, the cost of each treatment mix and any expected changes in condition-related costs. The report adds that the design should take account of the features of the health care system, possible access restrictions, the expected uptake of the new intervention and the use and effects of current and new interventions.

The Canadian budget impact guidelines published by the Patented Medicine Prices Review Board (PMPRB) in June 2020 express the same structure as two scenarios. The Reference Scenario is projected from the existing environment, and the New Drug Scenario assumes the new drug has been added to the plan's formulary. Both are forecasts, in which analysts anticipate market growth and treatment distribution even without the new drug, then estimate the new drug's market share and its effect on the shares of every other treatment strategy. The difference between the two scenario totals in each year is the budget impact.

Time horizon and reporting periods

A budget impact model runs over a short time horizon because its user plans the next few budget cycles. A 2018 methodological review of national and transnational guidelines by Foroutan and colleagues reports that the ISPOR guidance recommends a base-case horizon of 1 to 5 years.

National guidance narrows this further. The PMPRB guidelines state that four years of data are typically presented, a one-year baseline period followed by a three-year forecast, with all results reported for 12-month periods. They allow a longer forecast when a drug has longer-term costs or savings or takes several years to reach a steady state. National methodological guidelines for India, published by Prinja and colleagues in 2021, allow results for a minimum of 1 year and a maximum of 4 years, on the grounds that uncertainty beyond that would be too high. The PMPRB guidelines also state that results should be neither discounted nor inflated, because the budget holder needs the amounts expected to be paid in each year.

Choosing a computing framework

The ISPOR report recommends a simple cost calculator wherever possible, because it is easy for budget holders to use. When changes in the size of the eligible population, the mix of disease severity or treatment patterns cannot be credibly captured that way, it recommends a cohort or patient-level condition-specific model that accounts for people entering and leaving the eligible population over time. The options differ in how they represent people moving through the pathway.

The Indian guidelines give examples of when the calculator approach is not enough, such as costs of managing side effects, and additional costs incurred because a treatment prolongs survival. They describe the modelling approach in those cases as close to that of a cost-effectiveness analysis, with the difference that only costs of resource use enter the result, not health benefits. The choice between cohort and individual-level structures follows the general rule in the ISPOR-SMDM state-transition modelling report by Siebert and colleagues. A cohort simulation suits cases where the problem fits a manageable number of states, an individual-level model suits cases where a valid representation would need an unmanageable number, and validity should not be sacrificed for simplicity.

The PMPRB guidelines take a more restrictive line for drug plan submissions. They recommend the simplest design that gives accurate, repeatable results and state that, because budget impact analyses do not consider the clinical impact of treatment, a disease model using techniques such as Markov modelling should not be needed, while accepting that more complex calculations may be required for the disease dynamics of the treated population.

Open and closed populations

A budget impact model also has to decide whether its population is open or closed. The PMPRB guidelines recommend that the model generally assume an open population, in which patients can be added to or removed from the analysis over the time horizon. They single out cases in which the new drug changes the size of the treated population, for example when lower mortality leaves more people on treatment or a curative treatment leaves fewer, and they note that mortality and disease progression matter for the same reason. A closed population may suit a select group that is not expected to change over time, such as people with a rare disease.

The Indian guidelines likewise define the population as an open cohort in which new patients enter every year because they develop the condition or come to meet the eligibility criteria. An open population is where a calculator and a cohort model start to diverge.

Worked example of a curative therapy in an open population

This example is illustrative, and the numbers are invented to show the mechanics. A payer funds chronic treatment for a condition at GBP 3,000 per person per year. At the start of year 1, 1,000 people are on treatment. Each year 10% of those on chronic treatment leave the pool through death or loss of eligibility, and 100 newly diagnosed people join, so in the reference scenario the pool stays at 1,000 and spending is GBP 3,000,000 a year. A one-off curative therapy costing GBP 20,000 per course is listed, and in the new scenario it is given to 10% of the pool in year 1 and 20% in years 2 and 3. Cured people leave the pool and cost the payer nothing further for this condition.

A cohort model updates the size of the pool each year as follows.

$$N_{t+1} = (1 - u_t)\,N_t\,(1 - m) + I$$

where $N_t$ is the number on chronic treatment at the start of year $t$, $u_t$ is the share cured in year $t$, $m$ is the annual exit probability (0.10) and $I$ is the number of new entrants each year (100). New-scenario spending in year $t$ is $u_t N_t$ times GBP 20,000 plus $(1 - u_t) N_t$ times GBP 3,000.

1. In year 1 the 100 people cured cost GBP 2,000,000 and 900 people on chronic treatment cost GBP 2,700,000, a total of GBP 4,700,000 and a budget impact of GBP 1,700,000.
2. The pool at the start of year 2 is 900 x 0.90 + 100 = 910.
3. In year 2 the 182 cured cost GBP 3,640,000 and 728 on chronic treatment cost GBP 2,184,000, a total of GBP 5,824,000 and a budget impact of GBP 2,824,000.
4. The pool at the start of year 3 is 728 x 0.90 + 100 = 755.2 (an expected value, not a head count).
5. In year 3 the 151.04 cured cost GBP 3,020,800 and 604.16 on chronic treatment cost GBP 1,812,480, a total of GBP 4,833,280 and a budget impact of GBP 1,833,280.

A cost calculator that applies the same uptake shares to a fixed eligible count of 1,000 gives the same year 1 result, then GBP 6,400,000 in years 2 and 3 (200 cured at GBP 20,000 plus 800 on chronic treatment at GBP 3,000), a budget impact of GBP 3,400,000 in each of those years.

Over three years the calculator overstates the budget impact by GBP 2,142,720, about 34% of the cohort result, because it keeps paying for people who have already been cured. The cohort model also shows the shape a budget holder needs, since the impact peaks in year 2 and falls as the pool shrinks, so a longer horizon could show annual savings. Neither result says whether the cure is good value.

Designing the model for the budget holder

The PMPRB guidelines ask for the model to be built in readily available software such as Microsoft Excel, with all choices and assumptions explained, and for the interactive model to accompany the report so that reviewers can change drug prices, markups, professional fees and co-payments for their own province or test price rebates. Foroutan and colleagues report that most of the guidelines they reviewed recommend an Excel-based model over more complicated software.

The ISPOR report adds that the software should offer options to include or exclude different categories of costs, and that reporting should give input values and calculations in enough detail for another modeller to replicate the analysis.

Validation and uncertainty in the model

Validation checks that the model calculates what it claims to and that its outputs are believable to the people who will use them. The ISPOR report states that model validation should include at least face validity with decision makers and verification of the calculations. The PMPRB guidelines ask for validity to be assessed and documented, for equations and parameters to be checked against their sources, for programming code to be documented and quality-assured, and point to the ISPOR-SMDM task force recommendations as a reference. The review by Foroutan and colleagues summarises that face, internal and external validity have to be checked and documented.

Uncertainty in a budget impact model is handled mainly through scenarios. The ISPOR report recommends scenario analysis of alternatives chosen from the perspective of the decision maker. The PMPRB guidelines ask for deterministic sensitivity analysis and scenario analyses, testing at minimum the size of the market, market shares, the price of the new drug and comparator prices, and state that probabilistic sensitivity analysis may be considered but is not required. Their reasons are that data for many parameters are too limited for their uncertainty to be meaningfully quantified, and that much of the uncertainty is structural.

Boundaries with related concepts

The model is the calculation tool, the budget impact analysis is the wider exercise of framing, data gathering, modelling and reporting, and the budget impact is the number produced. A budget impact model is a health economic model that shares many inputs with a cost-effectiveness model but answers a different question. The PMPRB guidelines describe budget impact analysis as focused on the financial impact of a new drug, with value to the health system examined through other analyses such as cost-effectiveness analysis. Concepts that use the model's output, such as a budget envelope or an economically justifiable price, have their own pages.

Where budget impact models mislead

A budget impact model is a conditional forecast. Its results are only as good as the market share and population forecasts, which before launch can rest on expert opinion or on experience in markets where the technology is already reimbursed.

• A calculator is always simpler and therefore better. Simplicity is preferred only while it gives credible results; the worked example shows a calculator overstating the impact of a curative therapy by about a third over three years.
• A more complex model is more accurate. A Markov or patient-level structure adds states and assumptions that are harder for a budget holder to check, and the PMPRB guidelines warn against complexity that the population dynamics do not require.
• A cumulative total is enough. The year-by-year profile matters because budgets are set annually; the same three-year total can hide a peak in year 2.
• A favourable budget impact means good value. The model counts only costs to one budget holder and says nothing about health gained.

Sources

• Sullivan SD, Mauskopf JA, Augustovski F, Jaime Caro J, Lee KM, Minchin M, Orlewska E, Penna P, Rodriguez Barrios JM, Shau WY. Budget impact analysis: principles of good practice. Report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value in Health. 2014;17(1):5-14. https://doi.org/10.1016/j.jval.2013.08.2291
• Patented Medicine Prices Review Board. Budget Impact Analysis Guidelines: Guidelines for Conducting Pharmaceutical Budget Impact Analyses for Submission to Public Drug Plans in Canada. 2nd ed. Ottawa: PMPRB; June 2020. https://publications.gc.ca/site/eng/9.885893/publication.html
• Foroutan N, Tarride JE, Xie F, Levine M. A methodological review of national and transnational pharmaceutical budget impact analysis guidelines for new drug submissions. ClinicoEconomics and Outcomes Research. 2018;10:821-854. https://doi.org/10.2147/CEOR.S178825
• Prinja S, Chugh Y, Rajsekar K, Muraleedharan VR. National methodological guidelines to conduct budget impact analysis for health technology assessment in India. Applied Health Economics and Health Policy. 2021;19(6):811-823. https://doi.org/10.1007/s40258-021-00668-y
• Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. https://doi.org/10.1016/j.jval.2012.06.014