Inverse probability weighted ATE estimator

Signature

ATE_IPW = (1/n) * sum_(i=1)^n (T_i * Y_i / e_i) - (1/n) * sum_(i=1)^n ((1 - T_i) * Y_i / (1 - e_i))

Function

Average treatment effect function

Computational function

• Computational function: IPW incremental net monetary benefit from patient-level data

  Takes patient-level data as an analysis file holds it, one treatment flag, propensity score, cost and QALY total per patient, together with a value per QALY, and returns the incremental cost, the incremental QALYs and the incremental net monetary benefit. It applies the estimator HE-FM-ATE-003 twice with the same weights, once to costs and once to QALYs, and then the incremental net monetary benefit formula HE-FM-NMB-002, so three steps run inside one function. The inputs therefore differ from the formula's variables: the formula takes one outcome vector, and the function takes two outcomes and a value per QALY.

  Inputs and outputs: T_i: Treatment flag per patient, 1 if treated and 0 if not; required. Unit: none.; e_i: Propensity score per patient; required, above zero and below one. Unit: probability.; C_i: Total discounted cost per patient; required. Unit: currency, for example £.; Q_i: Total discounted QALYs per patient; required. Unit: QALYs.; lambda: Value placed on one QALY; required, above zero. Unit: currency per QALY.; dC_IPW: Inverse probability weighted incremental cost. Unit: currency per patient.; dQ_IPW: Inverse probability weighted incremental QALYs. Unit: QALYs per patient.; INMB_IPW: Incremental net monetary benefit per patient. Unit: currency per patient.

  Assumption: The same propensity score weights apply to a patient's cost and QALYs, and ignorability and overlap hold as in HE-FM-ATE-003. The vectors are aligned so that row i of each refers to the same patient.

  Worked example (Whole sample at £25,000 per QALY): The 20-patient version of the article's example, with the group costs and QALYs, gives an incremental cost of £3,400, an incremental gain of 0.22 QALYs and an incremental net monetary benefit of £2,100 per patient, the population figures in the article. n = 20; T_i = [1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,0]; e_i = [0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.875,0.875,0.875,0.875,0.875,0.875,0.875,0.875]; C_i = [5000,5000,5000,2000,2000,2000,2000,2000,2000,2000,2000,2000,9000,9000,9000,9000,9000,9000,9000,5000]; Q_i = [6.10,6.10,6.10,6.00,6.00,6.00,6.00,6.00,6.00,6.00,6.00,6.00,4.40,4.40,4.40,4.40,4.40,4.40,4.40,4.00]; lambda = 25000; dC_IPW = 3400; dQ_IPW = 0.22; INMB_IPW = 2100

  Worked example (High-risk patients only at £25,000 per QALY): Run on the 8 high-risk patients alone (7 treated, propensity 0.875), the function recovers the subgroup effects of £4,000 and 0.40 QALYs and an incremental net monetary benefit of £6,000, the high-risk figure in the article. n = 8; T_i = [1,1,1,1,1,1,1,0]; e_i = [0.875,0.875,0.875,0.875,0.875,0.875,0.875,0.875]; C_i = [9000,9000,9000,9000,9000,9000,9000,5000]; Q_i = [4.40,4.40,4.40,4.40,4.40,4.40,4.40,4.00]; lambda = 25000; dC_IPW = 4000; dQ_IPW = 0.40; INMB_IPW = 6000

  Excel: =ValuePerQALY*(SUMPRODUCT(Treated,QALYs/Propensity)-SUMPRODUCT(1-Treated,QALYs/(1-Propensity)))/COUNT(Treated)-(SUMPRODUCT(Treated,Costs/Propensity)-SUMPRODUCT(1-Treated,Costs/(1-Propensity)))/COUNT(Treated) With one row per patient in named ranges Treated, Propensity, Costs and QALYs and the value per QALY in a cell named ValuePerQALY, the formula returns the incremental net monetary benefit; the two bracketed terms divided by COUNT(Treated) are the incremental QALYs and cost.

  R: ipw_inmb <- function(t, e, cost, qaly, lambda) { d_c <- mean(t*cost/e)-mean((1-t)*cost/(1-e)); d_q <- mean(t*qaly/e)-mean((1-t)*qaly/(1-e)); c(d_c = d_c, d_q = d_q, inmb = lambda*d_q-d_c) } Vectorised over patients; mean() divides each weighted sum by n.

  Python: def ipw_inmb(t, e, cost, qaly, lam): t, e, cost, qaly = (np.asarray(v, dtype=float) for v in (t, e, cost, qaly)); d_c = np.mean(t*cost/e)-np.mean((1-t)*cost/(1-e)); d_q = np.mean(t*qaly/e)-np.mean((1-t)*qaly/(1-e)); return d_c, d_q, lam*d_q-d_c Uses numpy imported as np; the function returns the incremental cost, incremental QALYs and incremental net monetary benefit.

  Test (Weighted arms each sum to the sample size): With propensity scores equal to the share treated in each covariate stratum, the treated and untreated weights each sum exactly to n. Expected result: TRUE. Excel check: =AND(ABS(SUMPRODUCT(Treated/Propensity)-COUNT(Treated))<1E-9,ABS(SUMPRODUCT((1-Treated)/(1-Propensity))-COUNT(Treated))<1E-9)

  Test (Whole-sample result equals the share-weighted subgroup results): Because the weighted sums split by subgroup, the whole-sample incremental net monetary benefit equals the subgroup results weighted by subgroup size, 0.6 times minus £500 plus 0.4 times £6,000 in the article's example. Expected result: TRUE. Excel check: =ABS(INMB_All-(Share_Low*INMB_Low+Share_High*INMB_High))<1E-6

  Common error (Weighting QALYs but not costs): Applying the weights to QALYs and taking the raw difference in mean cost, £5,500 in the 20-patient example, turns an incremental net monetary benefit of £2,100 into zero at £25,000 per QALY. Costs and QALYs need the same weights.

  Source: Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research. 2011;46(3):399-424. Section on inverse probability of treatment weighting using the propensity score, which gives the estimator of the ATE applied here to costs and to QALYs.

  dC_IPW = (1/n) * sum_(i=1)^n (T_i * C_i / e_i) - (1/n) * sum_(i=1)^n ((1 - T_i) * C_i / (1 - e_i)); dQ_IPW = (1/n) * sum_(i=1)^n (T_i * Q_i / e_i) - (1/n) * sum_(i=1)^n ((1 - T_i) * Q_i / (1 - e_i)); INMB_IPW = lambda * dQ_IPW - dC_IPW

Try this function

Implementations

• Excel

  IPW ATE estimate from patient rows

  With one row per patient in named ranges Treated (1 or 0), Propensity and Outcome, Excel forms the two weighted sums and divides their difference by the number of patients.

  =(SUMPRODUCT(Treated,Outcome/Propensity)-SUMPRODUCT(1-Treated,Outcome/(1-Propensity)))/COUNT(Outcome)

Assumptions

• Ignorability given the propensity covariates

  Once the chosen covariates are controlled for, the potential outcomes are independent of treatment assignment. TSD 17 states that this cannot be tested and must be justified from the literature or expert opinion, and that variables affected by the treatment are left out of the conditioning set.

• Overlap of treated and untreated patients for IPW

  Patients with any combination of the covariates could be found in either group, so every e_i is above zero and below one. TSD 17 notes that propensity scores close to zero give excessively large weights and make the method unstable.

• Correctly specified propensity score model

  The propensity scores come from a model that predicts treatment well and is flexible enough to capture non-linear associations, for example a logit or probit model with squares and interactions of the covariates.

Worked examples

• IPW estimate of the QALY gain from 20 patients

  The article's example scaled to 20 patients with the same shares: 12 low-risk patients (3 treated, propensity 0.25) and 8 high-risk patients (7 treated, propensity 0.875), each with the group's QALYs. The weighted means are 5.42 and 5.20 QALYs, so the estimate is 0.22, equal to the ATE, while the unweighted arm means differ by minus 0.89.

  n = 20; T_i = [1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,0]; e_i = [0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.875,0.875,0.875,0.875,0.875,0.875,0.875,0.875]; Y_i = [6.10,6.10,6.10,6.00,6.00,6.00,6.00,6.00,6.00,6.00,6.00,6.00,4.40,4.40,4.40,4.40,4.40,4.40,4.40,4.00]; ATE_IPW = 0.22

• IPW estimate of the incremental cost from 20 patients

  With the same 20 patients and the group costs, the weighted mean cost is £6,600 in the treated arm and £3,200 in the untreated arm, an estimate of £3,400 that matches the ATE on cost because the propensity scores are known exactly.

  n = 20; T_i = [1,1,1,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,0]; e_i = [0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.25,0.875,0.875,0.875,0.875,0.875,0.875,0.875,0.875]; Y_i = [5000,5000,5000,2000,2000,2000,2000,2000,2000,2000,2000,2000,9000,9000,9000,9000,9000,9000,9000,5000]; ATE_IPW = 3400

Common errors

• Comparing unweighted arm means under confounding by indication

  In the 20-patient example the treated average 4.91 QALYs and the untreated 5.80, a difference of minus 0.89 QALYs, with mean costs of £7,800 against £2,300. Sicker patients are treated more often, so the raw comparison makes the treatment look harmful when the ATE is a gain of 0.22 QALYs.

• Weighting untreated patients by one over the propensity score

  Using one over e_i for untreated patients, instead of one over one minus e_i, gives an untreated weighted mean of about 11.03 QALYs in the 20-patient example, far above the 6.10 QALYs of the best-off patient, and an estimate of about minus 5.61 QALYs.

• Leaving extreme inverse probability weights unchecked

  A treated patient with a propensity score of 0.01 receives a weight of 100 and counts as 100 patients in the treated mean. TSD 17 warns that scores close to zero make the method unstable, so the distribution of propensity scores and weights is examined before the estimate is used.

Sources

• Austin on the IPTW estimator of the ATE

  Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research. 2011;46(3):399-424. Section on inverse probability of treatment weighting using the propensity score, which defines each patient's weight as the inverse of the probability of the treatment actually received and gives this estimator of the ATE.

  View source →

• TSD 17 on inverse probability weighting

  Faria R, Hernández Alava M, Manca A, Wailoo AJ. NICE DSU Technical Support Document 17: the use of observational data to inform estimates of treatment effectiveness in technology appraisal: methods for comparative individual patient data. Sheffield: Decision Support Unit, ScHARR; 2015. Section 2.3, on ignorability and overlap, and section 2.3.2, which states that the ATE corresponds to the difference in means weighted by the inverse of the propensity score and that scores close to zero make the method unstable.

  View source →

• What If on IP weighting and standardisation

  Hernán MA, Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC; 2020. Section 2.4 and Technical Point 2.3, which show that the inverse probability weighted mean equals the standardised mean under positivity, linking this estimator to HE-FM-ATE-001.

  View source →

Canonical Identity