Signature
v_1 = 1 / (1 / v_0 + 1 / s_1^2); m_1 = v_1 * (m_0 / v_0 + y_1 / s_1^2); v_f = 1 / (1 / v_0 + 1 / s_1^2 + 1 / s_2^2); y2_crit = s_2^2 * (z_c / sqrt(v_f) - m_0 / v_0 - y_1 / s_1^2); z_pp = (y2_crit - m_1) / sqrt(v_1 + s_2^2)
| Inputs | Definition | Unit |
|---|---|---|
v_0 | Variance of the normal prior, above zero | squared effect units |
s_1 | Standard error of y_1 | effect units |
m_0 | Mean of the normal prior | effect units |
y_1 | Interim estimate of the effect | effect units |
s_2 | Standard error that the second stage alone is expected to deliver, from its planned size | effect units |
z_c | Phi^-1 of the final posterior probability required, 1.96 for 0.975 | none |
v_1 | Posterior variance after the first stage | squared effect units |
|---|---|---|
m_1 | Posterior mean after the first stage | effect units |
v_f | Posterior variance if the second stage is run | squared effect units |
y2_crit | Smallest second-stage estimate for which the final posterior probability of a positive effect exceeds Phi(z_c) | effect units |
z_pp | y2_crit minus m_1, divided by the predictive standard deviation; the predictive probability is 1 minus Phi(z_pp) | none |
Function
Bayesian interim decision function for an adaptive trial
Maps a prior distribution for the treatment effect, the data observed so far and pre-specified thresholds to the quantities that interim rules consult: the posterior probability that the effect exceeds a minimum value, which summarises the evidence now, and the predictive probability that the final analysis will succeed, which averages over the results still to come. Response-adaptive rules use the same posterior to set the chance of allocation to each arm. The notation follows the Bayesian Adaptive Design article.
Computational function
Computational function: interim look with posterior and predictive probabilities and the stopping decision
Takes the prior mean and standard deviation, the interim estimate and its standard error, the planned second stage's standard error and the protocol's three thresholds, and returns the posterior summary (HE-FM-BAD-001), the boundary and predictive probability (HE-FM-BAD-002) and the protocol decision. It uses the inverse normal for the final rule instead of a rounded z_c, and the normal distribution function, which the formulas leave to Phi.
Inputs and outputs:
m_0,sd_0: Prior mean and standard deviation; required, sd_0 above zero. Unit: effect units.;y_1,se_1: Interim estimate and its standard error; required. Unit: effect units.;se_2: Standard error of the planned second stage; required. Unit: effect units.;c_eff: Posterior probability for stopping on conclusive benefit; default 0.99.;c_final: Final posterior probability for success; default 0.975.;pp_fut: Predictive probability below which the trial stops for futility; default 0.10.;m_1,sd_1: Interim posterior mean and standard deviation.;P_post: Posterior probability of a positive effect.;y2_crit: Second-stage boundary.;PP: Predictive probability of success.;decision: Stop for conclusive benefit, stop for futility or continue.Assumption: Normal prior and normal estimates with known standard errors, independent stages, a final analysis that pools the prior and both stages, and the efficacy rule checked before the futility rule.
Worked example (Article's interim look): The posterior mean is about 1,034.48 with standard deviation 742.78, the posterior probability 0.9181 is below 0.99, the boundary is about 1,104.44 with the exact inverse normal, and the predictive probability 0.4745 is above 0.10, so the trial continues, as in the article.
m_0 = 0; sd_0 = 2000; y_1 = 1200; se_1 = 800; se_2 = 800; m_1 = 1034.48; sd_1 = 742.78; P_post = 0.9181; y2_crit = 1104.44; PP = 0.4745; decision = continueWorked example (Interim estimate of 2,100 pounds): The posterior probability of about 0.9926 exceeds 0.99, so the protocol stops for conclusive net benefit (computed here for illustration).
m_0 = 0; sd_0 = 2000; y_1 = 2100; se_1 = 800; se_2 = 800; m_1 = 1810.34; P_post = 0.9926; PP = 0.9294; decision = stop: conclusive benefitExcel: The HE-FM-BAD-001 and HE-FM-BAD-002 formulas with ZFinal set to
=NORM.S.INV(0.975), then=IF(PostProb>0.99,"stop: conclusive benefit",IF(PredProb<0.1,"stop: futility","continue"))for the decision.R:
interim_look <- function(m_0, sd_0, y_1, se_1, se_2, c_eff = 0.99, c_final = 0.975, pp_fut = 0.10) { v_0 <- sd_0^2; v_1 <- 1/(1/v_0+1/se_1^2); m_1 <- v_1*(m_0/v_0+y_1/se_1^2); P_post <- pnorm(m_1/sqrt(v_1)); v_f <- 1/(1/v_1+1/se_2^2); y2_crit <- se_2^2*(qnorm(c_final)/sqrt(v_f)-m_1/v_1); PP <- 1-pnorm((y2_crit-m_1)/sqrt(v_1+se_2^2)); decision <- if (P_post > c_eff) "stop: conclusive benefit" else if (PP < pp_fut) "stop: futility" else "continue"; list(m_1 = m_1, sd_1 = sqrt(v_1), P_post = P_post, y2_crit = y2_crit, PP = PP, decision = decision) }Returns 1034.48, 742.78, 0.9181, 1104.44, 0.4745 and continue for the first example.Python:
def interim_look(m_0, sd_0, y_1, se_1, se_2, c_eff=0.99, c_final=0.975, pp_fut=0.10): from statistics import NormalDist; N = NormalDist(); v_0 = sd_0**2; v_1 = 1/(1/v_0+1/se_1**2); m_1 = v_1*(m_0/v_0+y_1/se_1**2); P_post = N.cdf(m_1/v_1**0.5); v_f = 1/(1/v_1+1/se_2**2); y2_crit = se_2**2*(N.inv_cdf(c_final)/v_f**0.5-m_1/v_1); PP = 1-N.cdf((y2_crit-m_1)/(v_1+se_2**2)**0.5); decision = "stop: conclusive benefit" if P_post > c_eff else ("stop: futility" if PP < pp_fut else "continue"); return {"m_1": m_1, "sd_1": v_1**0.5, "P_post": P_post, "y2_crit": y2_crit, "PP": PP, "decision": decision}Returns the same values as the R function.Test (Futility at an interim estimate of zero): With y_1 of 0 the predictive probability is about 0.0174, below 0.10, and the decision is to stop for futility. Expected result: TRUE. Excel check:
=AND(PredProb<0.1,PostProb<0.99)with InterimEst set to 0.Test (Predictive probability rises with the interim estimate): Raising y_1 from 1,200 to 1,600 raises PP from about 0.4745 to about 0.7318. Expected result: TRUE. Excel check, with the two results in PredProbLow and PredProbHigh:
=PredProbHigh>PredProbLowCommon error (Interim standard error reused for the second stage when the stages differ in size): If the second stage had 50 patients per arm, its standard error would be about 800 times the square root of 2, about 1,131, and reusing 800 overstates the information still to come.
Source: Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd ed. Boca Raton: CRC Press; 2013 (electronic edition for non-commercial use). Section 2.5. Lee JJ, Liu DD. A predictive probability design for phase II cancer clinical trials. Clinical Trials. 2008;5(2):93-106. Methods.
v_1 = 1 / (1 / sd_0^2 + 1 / se_1^2); m_1 = v_1 * (m_0 / sd_0^2 + y_1 / se_1^2); P_post = Phi(m_1 / sqrt(v_1)); v_f = 1 / (1 / v_1 + 1 / se_2^2); y2_crit = se_2^2 * (Phi^-1(c_final) / sqrt(v_f) - m_1 / v_1); PP = 1 - Phi((y2_crit - m_1) / sqrt(v_1 + se_2^2))
Try this function
Implementations
Excel
Predictive probability of final success from named cells
With PriorMean, PriorVar, InterimEst, InterimSE, Stage2SE and ZFinal named, and PostMean and PostVar from HE-FM-BAD-001, the formulas return the final variance (FinalVar), the boundary (Stage2Crit) and the predictive probability (PredProb). ZFinal is NORM.S.INV(0.975) for a 0.975 rule.
=1/(1/PriorVar+1/InterimSE^2+1/Stage2SE^2); =Stage2SE^2*(ZFinal/SQRT(FinalVar)-PriorMean/PriorVar-InterimEst/InterimSE^2); =1-NORM.S.DIST((Stage2Crit-PostMean)/SQRT(PostVar+Stage2SE^2),TRUE)
Assumptions
Second-stage estimate normal and independent of the first given the effect
Given the true effect, the second-stage estimate is normal with standard error s_2 and independent of the first, so its predictive distribution has the interim posterior mean and the sum of the posterior and sampling variances.
Final analysis pools the prior and both stages
The final posterior combines the prior with both estimates by precision weighting, and success means a final posterior probability of a positive effect above Phi(z_c). A different final rule changes y2_crit.
Worked examples
Article's interim look: predictive probability about 0.474
With the article's prior and interim data and a second stage of 100 patients per arm with standard error 800, the final standard deviation would be about 544.3, success needs a second-stage estimate above about 1,104.5 pounds, and z_pp is about 0.0641, so the predictive probability of success is about 0.474, above the futility bound of 0.10 (the article, rounding the boundary to 1,104.3, reports 0.4745).
m_0 = 0; v_0 = 4000000; y_1 = 1200; s_1 = 800; s_2 = 800; z_c = 1.96; v_1 = 551724.1; m_1 = 1034.48; v_f = 296296.3; y2_crit = 1104.48; z_pp = 0.0641
Interim estimate of zero triggers the futility rule
If the interim estimate had been 0 pounds, the posterior mean would be 0, success would need a second-stage estimate above about 2,304.5 pounds and the predictive probability would be about 0.017, below 0.10, so the protocol would stop for futility (computed here for illustration).
m_0 = 0; v_0 = 4000000; y_1 = 0; s_1 = 800; s_2 = 800; z_c = 1.96; v_1 = 551724.1; m_1 = 0; v_f = 296296.3; y2_crit = 2304.48; z_pp = 2.111
Interim estimate of 1,600 pounds
An interim estimate of 1,600 pounds gives a posterior mean of about 1,379.31, a boundary of about 704.5 pounds and a predictive probability of about 0.732 (computed here for illustration).
m_0 = 0; v_0 = 4000000; y_1 = 1600; s_1 = 800; s_2 = 800; z_c = 1.96; v_1 = 551724.1; m_1 = 1379.31; v_f = 296296.3; y2_crit = 704.48; z_pp = -0.6182
Common errors
Conditional power at the observed estimate in place of the predictive probability
Treating the interim estimate of 1,200 pounds as the true effect and ignoring posterior uncertainty gives a probability of about 0.548 that the second stage clears the boundary, against a predictive probability of about 0.474 in the article's example (computed here for illustration).
Confusing the predictive probability with the posterior probability
In the article's example the posterior probability of positive net benefit is about 0.918 while the predictive probability of meeting the final 0.975 rule is about 0.474. A high posterior probability now does not mean the final analysis is likely to succeed.
Sources
Posterior predictive distribution of a future normal observation
Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd ed. Boca Raton: CRC Press; 2013 (electronic edition for non-commercial use). Section 2.5, posterior predictive distribution: the predictive distribution of a future observation has mean equal to the posterior mean and variance equal to the sampling variance plus the posterior variance.
Predictive probability of success as an interim stopping scale
Lee JJ, Liu DD. A predictive probability design for phase II cancer clinical trials. Clinical Trials. 2008;5(2):93-106. Methods: the predictive probability is the weighted average, over possible future results, of the indicator of a positive trial at the end of the study; the trial stops for futility when it falls below theta_L.
Posterior or predictive probability as the scale of a stopping rule
US Food and Drug Administration. Adaptive Designs for Clinical Trials of Drugs and Biologics: Guidance for Industry. Silver Spring, MD: FDA; November 2019. Section V: stopping rules can be specified on scales including the Bayesian posterior probability that the drug is effective or the Bayesian predictive probability of trial success; the choice of scale is relatively unimportant as long as the operating characteristics are adequately evaluated.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0