Averaged-cohort gap in constant-risk life expectancy for two strata

Signature

q_bar = w * q_1 + (1 - w) * q_2; L_strat = w * (1 / q_1 - 0.5) + (1 - w) * (1 / q_2 - 0.5); L_avg = 1 / q_bar - 0.5; gap_L = L_strat - L_avg

Function

Population mean outcome of a cohort model run by stratum against one cohort at averaged inputs

Computational function

• Computational function: two risk strata run as separate cohorts against one averaged cohort, with life expectancy, QALY gains and ICERs

  Takes the share of a population in a low-risk stratum, the annual death probabilities of a low-risk and a high-risk stratum, a proportional cut in those probabilities from treatment, a utility per year alive and a one-off treatment cost, and returns the life expectancies of six cohort runs (each stratum and one averaged cohort, with and without treatment), the population QALY gain and ICER from the stratified runs, the same results from the averaged cohort, and the relative errors of the averaged cohort. It carries out the article's worked example. Each stratum gain is the difference of two values of the live L(q), HE-FM-ABS-001 less half a cycle, and equals e divided by (1 minus e) q because the half cycle cancels (derived here); the stratified QALY gain weights those gains with the shares of HE-FM-ATE-001, the ICERs follow HE-FM-ICER-001, and the life-expectancy shortfall is gap_L of HE-FM-COHM-001. The calculator line uses the closed form 1/q minus one half; the Excel, R and Python code instead run the Alive and Dead cohort trace numerically for a fixed number of cycles, so they also check the closed form, and the code takes any number of strata. The inputs differ from the formula's symbols, which describe one comparison at a time.

  Inputs and outputs: w is the share of the cohort in the low-risk stratum, q_low and q_high the annual death probabilities of the two strata, e the proportional cut from treatment, u the utility per year alive and c the one-off treatment cost per patient (cost in the R and Python code). q_bar is the share-weighted mean probability. L_low_0, L_low_1, L_high_0, L_high_1, L_avg_0 and L_avg_1 are the life expectancies in years of each stratum and of the averaged cohort without (0) and with (1) treatment. dQ_strat and dQ_avg are the population QALY gains from the stratified runs and from the averaged cohort, ICER_strat and ICER_avg the matching ICERs in currency per QALY, LE_strat_0 the stratified life expectancy without treatment, and under_LE, under_dQ and over_ICER the shortfalls in life expectancy and QALY gain and the excess in the ICER of the averaged cohort, as proportions of the stratified values. In the code n_cyc is the number of cycles in each trace.

  Assumption: Each run is a two-state Alive and Dead cohort with annual cycles and a constant death probability, counted at the start of each cycle less half a cycle, with nothing discounted; the treatment multiplies the death probability by 1 minus e in every cycle and costs c once. The averaged cohort is run at q_bar and at (1 minus e) times q_bar, which equals the share-weighted mean of the treated probabilities. With 2,000 cycles the untraced tail is below 1E-33 of a year at the article's probabilities.

  Worked example (Article's two equal risk groups): The stratified runs give a QALY gain of 2.4 and an ICER of GBP 8,333 per QALY; the averaged cohort gives 1.33 QALYs and GBP 15,000, understating life expectancy without treatment by 46% and the QALY gain by 44% and overstating the ICER by 80%, as in the article. w = 0.5; q_low = 0.05; q_high = 0.25; e = 0.2; u = 0.8; c = 20000; q_bar = 0.15; L_low_0 = 19.5; L_low_1 = 24.5; L_high_0 = 3.5; L_high_1 = 4.5; L_avg_0 = 6.1667; L_avg_1 = 7.8333; dQ_strat = 2.4; dQ_avg = 1.3333; ICER_strat = 8333.33; ICER_avg = 15000; LE_strat_0 = 11.5; under_LE = 0.4638; under_dQ = 0.4444; over_ICER = 0.8

  Worked example (80% of the population at low risk): With illustrative shares of 80% and 20% the stratified ICER is about GBP 5,952 per QALY and the averaged cohort's GBP 9,000, about 51% too high (computed here for illustration). w = 0.8; q_low = 0.05; q_high = 0.25; e = 0.2; u = 0.8; c = 20000; q_bar = 0.09; L_low_0 = 19.5; L_low_1 = 24.5; L_high_0 = 3.5; L_high_1 = 4.5; L_avg_0 = 10.6111; L_avg_1 = 13.3889; dQ_strat = 3.36; dQ_avg = 2.2222; ICER_strat = 5952.38; ICER_avg = 9000; LE_strat_0 = 16.3; under_LE = 0.349; under_dQ = 0.3386; over_ICER = 0.512

  Worked example (Equal stratum probabilities): When both strata have a probability of 0.15 the averaged cohort reproduces the stratified results and all three relative errors are zero, a limiting case. w = 0.5; q_low = 0.15; q_high = 0.15; e = 0.2; u = 0.8; c = 20000; q_bar = 0.15; L_low_0 = 6.1667; L_low_1 = 7.8333; L_high_0 = 6.1667; L_high_1 = 7.8333; L_avg_0 = 6.1667; L_avg_1 = 7.8333; dQ_strat = 1.3333; dQ_avg = 1.3333; ICER_strat = 15000; ICER_avg = 15000; LE_strat_0 = 6.1667; under_LE = 0; under_dQ = 0; over_ICER = 0

  Excel: Put the cycle numbers 0 to 1999 in a helper column named CycleIndex. With the probabilities in DeathProbLow and DeathProbHigh, the share in ShareLow, the cut in RiskCut, the utility in UtilityAlive and the cost in TreatCost, the trace life expectancy of the low-risk stratum without treatment is =SUMPRODUCT((1-DeathProbLow)^CycleIndex)-0.5 in a cell named LifeExpLowStd and with treatment =SUMPRODUCT((1-(1-RiskCut)*DeathProbLow)^CycleIndex)-0.5 in LifeExpLowTreat, with LifeExpHighStd and LifeExpHighTreat built the same way. The averaged cohort uses =SUMPRODUCT((1-MeanDeathProb)^CycleIndex)-0.5 in LifeExpAvgStd and =SUMPRODUCT((1-(1-RiskCut)*MeanDeathProb)^CycleIndex)-0.5 in LifeExpAvgTreat, with MeanDeathProb =ShareLow*DeathProbLow+(1-ShareLow)*DeathProbHigh. The stratified QALY gain is =UtilityAlive*(ShareLow*(LifeExpLowTreat-LifeExpLowStd)+(1-ShareLow)*(LifeExpHighTreat-LifeExpHighStd)) in QALYGainStrat and its ICER =TreatCost/QALYGainStrat in ICERStrat; =UtilityAlive*(LifeExpAvgTreat-LifeExpAvgStd) in QALYGainAvg and =TreatCost/QALYGainAvg in ICERAvg give the averaged-cohort results, and =ICERAvg/ICERStrat-1 the excess. SUMPRODUCT evaluates the power of the range as an array in classic Excel without array entry.

  R: cohort_le <- function(q, n_cyc = 2000) { m <- (1-q)^(0:(n_cyc-1)); sum(m)-0.5 }; strat_vs_avg <- function(w, q, e, u, cost, n_cyc = 2000) { le0 <- sapply(q, cohort_le, n_cyc = n_cyc); le1 <- sapply((1-e)*q, cohort_le, n_cyc = n_cyc); q_bar <- sum(w*q); dQ_strat <- u*sum(w*(le1-le0)); dQ_avg <- u*(cohort_le((1-e)*q_bar, n_cyc)-cohort_le(q_bar, n_cyc)); c(LE_strat_0 = sum(w*le0), L_avg_0 = cohort_le(q_bar, n_cyc), dQ_strat = dQ_strat, dQ_avg = dQ_avg, ICER_strat = cost/dQ_strat, ICER_avg = cost/dQ_avg) } Base R only; strat_vs_avg(c(0.5, 0.5), c(0.05, 0.25), 0.2, 0.8, 20000) returns 11.5, 6.1667, 2.4, 1.3333, 8333.33 and 15000, and the vector arguments take any number of strata.

  Python: def cohort_le(q, n_cyc=2000): return sum((1-q)**t for t in range(n_cyc))-0.5 and def strat_vs_avg(w, q, e, u, cost, n_cyc=2000): le0 = [cohort_le(x, n_cyc) for x in q]; le1 = [cohort_le((1-e)*x, n_cyc) for x in q]; q_bar = sum(a*b for a, b in zip(w, q)); dQ_strat = u*sum(a*(y1-y0) for a, y0, y1 in zip(w, le0, le1)); dQ_avg = u*(cohort_le((1-e)*q_bar, n_cyc)-cohort_le(q_bar, n_cyc)); return sum(a*y for a, y in zip(w, le0)), cohort_le(q_bar, n_cyc), dQ_strat, dQ_avg, cost/dQ_strat, cost/dQ_avg Plain Python with no imports; strat_vs_avg([0.5, 0.5], [0.05, 0.25], 0.2, 0.8, 20000) returns the same six values.

  Test (Trace life expectancy of the low-risk stratum): With DeathProbLow at 0.05 the cell LifeExpLowStd is 19.5 years to six decimal places. Expected result TRUE. Starting CycleIndex at 1, which counts at the end of each cycle, gives 18.5 and FALSE, and leaving out the half cycle gives 20 and FALSE. Excel check =ABS(LifeExpLowStd-19.5)<0.000001

  Test (Stratified ICER of the article's example): With the article's inputs ICERStrat is 20,000 divided by 2.4. Expected result TRUE. An ICERStrat cell that takes the averaged cohort's QALY gain, or the simple mean of the two stratum ICERs, returns GBP 15,000 and FALSE. Excel check =ABS(ICERStrat-20000/2.4)<0.01

  Test (Low-risk stratum gain against its closed form): The traced gain LifeExpLowTreat minus LifeExpLowStd is compared with RiskCut divided by the product of 1 minus RiskCut and DeathProbLow, the closed form derived here, which is 5 years at the article's inputs. Expected result TRUE. A LifeExpLowTreat cell that leaves out the half cycle while LifeExpLowStd keeps it gives 5.5 and FALSE. Excel check =ABS((LifeExpLowTreat-LifeExpLowStd)-RiskCut/((1-RiskCut)*DeathProbLow))<0.000001

  Common error (Running the average patient): Feeding the mean probability of 0.15 into one cohort gives an ICER of GBP 15,000 per QALY against GBP 8,333 from the stratified runs, 80% too high, and hides the fivefold difference between the stratum ICERs of GBP 5,000 and GBP 25,000 that the article reports.

  Source: Davis S, Stevenson M, Tappenden P, Wailoo A. NICE DSU Technical Support Document 15: Cost-effectiveness modelling using patient-level simulation. Sheffield, Decision Support Unit, ScHARR, University of Sheffield; April 2014. Section 3.1 (pages 14 and 15), on the biased estimate from average characteristics and on separate subgroup recommendations or a weighted average across subgroups. Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. Section Starting Cohort, on running the model once for each stratum and aggregating across strata.

  q_bar = w * q_low + (1 - w) * q_high; L_low_0 = 1 / q_low - 0.5; L_low_1 = 1 / ((1 - e) * q_low) - 0.5; L_high_0 = 1 / q_high - 0.5; L_high_1 = 1 / ((1 - e) * q_high) - 0.5; L_avg_0 = 1 / q_bar - 0.5; L_avg_1 = 1 / ((1 - e) * q_bar) - 0.5; dQ_strat = u * (w * (L_low_1 - L_low_0) + (1 - w) * (L_high_1 - L_high_0)); dQ_avg = u * (L_avg_1 - L_avg_0); ICER_strat = c / dQ_strat; ICER_avg = c / dQ_avg; LE_strat_0 = w * L_low_0 + (1 - w) * L_high_0; under_LE = 1 - L_avg_0 / LE_strat_0; under_dQ = 1 - dQ_avg / dQ_strat; over_ICER = ICER_avg / ICER_strat - 1

Try this function

Implementations

• Excel

  Averaged-cohort life-expectancy gap from named stratum cells

  With the first stratum's share in ShareLow and the two death probabilities in DeathProbLow and DeathProbHigh, the four formulas return q_bar in a cell named MeanDeathProb, L_strat in LifeExpStrat, L_avg in LifeExpAvg and gap_L in GapLifeYears.

  =ShareLow*DeathProbLow+(1-ShareLow)*DeathProbHigh; =ShareLow*(1/DeathProbLow-0.5)+(1-ShareLow)*(1/DeathProbHigh-0.5); =1/MeanDeathProb-0.5; =LifeExpStrat-LifeExpAvg

Assumptions

• Constant death probability within each stratum run to absorption

  Each stratum is a two-state Alive and Dead cohort with the same death probability in every cycle, run until everyone has died and counted at the start of each cycle less half a cycle, as Sonnenberg and Beck describe for a matrix solution. Where probabilities change with age the stratum totals come from the trace of HE-CF-CSIM-001 and the closed form does not apply.

• Fixed and exhaustive stratum shares at model start

  The two strata are defined by characteristics known at the start, do not overlap and together make up the cohort, so their shares are w and 1 minus w; every member of a stratum has its death probability, the homogeneity within states that the ISPOR-SMDM report asks for.

• Undiscounted life expectancy in the averaged-cohort gap

  Nothing is discounted, as in the article's example. With discounting each stratum total comes from HE-FM-MKCH-001 or HE-CF-MKCH-001 and the gap no longer has the closed form given in HE-FM-COHM-001.

Worked examples

• Article's risk groups under standard care run separately and averaged

  The article's illustrative population is split equally between annual death probabilities of 0.05 and 0.25. Run separately and weighted, life expectancy is 11.5 years, while one cohort at the mean probability of 0.15 gives 6.17 years, so the averaged cohort falls 5.33 years short, 46% of the stratified value, as in the article.

  w = 0.5; q_1 = 0.05; q_2 = 0.25; q_bar = 0.15; L_strat = 11.5; L_avg = 6.1667; gap_L = 5.3333

• Article's risk groups with treatment run separately and averaged

  With the treatment the article's probabilities fall to 0.04 and 0.20, with life expectancies of 24.5 and 4.5 years. The stratified mean is 14.5 years (computed here for illustration) against 7.83 years for one cohort at 0.12, a gap of 6.67 years.

  w = 0.5; q_1 = 0.04; q_2 = 0.20; q_bar = 0.12; L_strat = 14.5; L_avg = 7.8333; gap_L = 6.6667

• Equal stratum death probabilities leave no averaged-cohort gap

  When both strata have an annual death probability of 0.15 the averaged cohort is exact and the gap is zero, a limiting case that checks an implementation.

  w = 0.5; q_1 = 0.15; q_2 = 0.15; q_bar = 0.15; L_strat = 6.1667; L_avg = 6.1667; gap_L = 0

• Averaged-cohort gap with 80% of the cohort at low risk

  With illustrative shares of 80% at 0.05 and 20% at 0.25 the mean probability is 0.09, the stratified life expectancy 16.3 years and the averaged cohort 10.61 years, a gap of about 5.69 years (computed here for illustration).

  w = 0.8; q_1 = 0.05; q_2 = 0.25; q_bar = 0.09; L_strat = 16.3; L_avg = 10.6111; gap_L = 5.6889

Common errors

• Second-order approximation read as the averaged-cohort gap

  Applying HE-FM-EV-002 to 1/q at the mean probability of 0.15, with a second derivative of about 592.6 and a variance of 0.01 across the two equal strata, gives a gap of about 2.96 years against the exact 5.33, 44% too small (computed here for illustration). With strata this far apart the higher-order terms are large, so the closed form or separate stratum runs give the estimate.

• Half cycle subtracted on one side of an averaged-cohort comparison

  Setting a stratified result counted at the start of each cycle with no half-cycle subtraction, 12 years, against an averaged cohort that has it, 6.17 years, overstates the gap by half a cycle, 5.83 against 5.33 years. Both sides need the same counting weights, as in HE-FM-CSIM-001.

• Death rates averaged in place of probabilities for an averaged cohort

  Converting 0.05 and 0.25 to rates, averaging them and converting back gives a probability of about 0.156 and a life expectancy of about 5.91 years, still 49% below the stratified 11.5 years (computed here for illustration). Changing the scale on which the input is averaged leaves the bias in place; running the strata separately removes it.

• Expecting a proportional cut to gain most where death risk is highest

  Run as separate cohorts, the article's strata gain 5.0 life years from a 20% cut at an annual death probability of 0.05 and 1.0 at 0.25, each gain the difference of two values of L(q), although the absolute risk reduction of HE-FM-ARR-002 in one cycle is 0.01 in the first stratum and 0.05 in the second. At a constant probability the gain equals e divided by (1 minus e) q (derived here), so at a fixed proportional cut it falls as baseline risk rises, and ranking strata by the one-cycle absolute reduction reverses their ranking by lifetime life years gained.

• Averaged-cohort ICER taken as confirmed by the mean of stratum ICERs

  With a one-off cost c, a constant utility u and a proportional cut e, each constant-risk stratum has an ICER of c(1 minus e)q divided by ue (derived here), proportional to q, so the ICER of one cohort at the mean probability equals the share-weighted mean of the stratum ICERs. In the article's example the averaged cohort's GBP 15,000 equals the simple mean of GBP 5,000 and GBP 25,000. The agreement is a property of this model and checks nothing, since both differ from the population ICER of GBP 8,333 per QALY that the share-weighted gains of HE-FM-ATE-001 give with HE-FM-ICER-001; HE-ER-CHN-002 shows why subgroup ICERs are not averaged.

Sources

• TSD 15 on biased cohort results from average patient characteristics

  Davis S, Stevenson M, Tappenden P, Wailoo A. NICE DSU Technical Support Document 15: Cost-effectiveness modelling using patient-level simulation. Sheffield, Decision Support Unit, ScHARR, University of Sheffield; April 2014. Section 3.1, model non-linearity with respect to heterogeneous patient characteristics (pages 14 and 15), which states that estimating outcomes for a cohort using only average characteristics gives a biased estimate of the average outcome across the population where a characteristic has a non-linear relationship with the outcomes, and that subgroup analysis may be used where outcomes within subgroups are reasonably homogeneous. The closed form of the gap is derived here and is not printed in TSD 15.

  View source →

• ISPOR-SMDM task force on running a model once for each stratum

  Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. Section Starting Cohort, which states that if outputs are wanted for a population-based starting cohort the model must be run multiple times, one for each stratum, and then aggregated across strata.

  View source →

• Sonnenberg and Beck on the half cycle subtracted from a matrix solution

  Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322-338. The Half-Cycle Correction (pages 329 and 330), which states that the fundamental matrix representation is equivalent to counting state membership at the beginning of each cycle and that the correction to a matrix solution is subtraction of one half cycle from the membership of each starting state.

  View source →

Canonical Identity