Partially pooled basket response under a normal approximation

Signature

p_hat_j = y_j / n_j; s2_j = mu * (1 - mu) / n_j; w_j = tau2 / (tau2 + s2_j); p_tilde_j = w_j * p_hat_j + (1 - w_j) * mu

Function

Partial pooling of response rates across basket trial baskets

Computational function

• Computational function: partial pooling of a basket trial table and its tumour-type mix response

  Takes a basket trial's results as reported, responders and evaluable patients per basket, together with a common mean response, a between-basket standard deviation and the tumour-type shares expected in practice, and returns the partially pooled response for every basket and the expected response for that mix. It applies HE-FM-BSKT-001 to each basket and then HE-FM-BSKT-002 to the results, so two formulae are chained over a whole table. The inputs therefore differ from the formulae's variables: the function takes count vectors and a standard deviation, squares the standard deviation itself, and returns the mix response without a separate step.

  Inputs and outputs: y_j: Responders per basket; required, whole numbers from 0 up to n_j. Unit: count.; n_j: Evaluable patients per basket, in the same basket order as y_j; required, above zero. Unit: count.; mu: Common mean response on the probability scale; required, above 0 and below 1. Unit: probability.; tau: Between-basket standard deviation of response; required, zero or above. Unit: probability.; pi_j: Share of each basket's tumour type in the population expected to be treated; required, summing to 1. Unit: proportion.; p_tilde_j: Partially pooled response per basket. Unit: probability.; R_pp: Expected response for the tumour-type mix from the partially pooled estimates. Unit: probability.

  Assumption: As in HE-FM-BSKT-001, mu and tau are fixed rather than estimated, the baskets are exchangeable and the sampling variance is evaluated at the common mean. The shares cover the whole population expected to be treated.

  Worked example (Four illustrative baskets with an NHS tumour-type mix): The article's four baskets, with mu of 0.50, a between-basket standard deviation of 0.15 and a mix of 10% A, 10% B, 60% C and 20% D, give partially pooled responses of about 0.5643, 0.5574, 0.3105 and 0.5310 and a mix response of about 0.4047. The article reports 0.404 because it weights the rounded estimates. y_j = [12,9,1,3]; n_j = [20,15,10,5]; mu = 0.50; tau = 0.15; pi_j = [0.1,0.1,0.6,0.2]; p_tilde_j = [0.5643,0.5574,0.3105,0.5310]; R_pp = 0.4047

  Worked example (Zero between-basket standard deviation): With tau set to 0 every basket receives the common mean and the mix response is 0.50, the complete pooling limit. y_j = [12,9,1,3]; n_j = [20,15,10,5]; mu = 0.50; tau = 0; pi_j = [0.1,0.1,0.6,0.2]; p_tilde_j = [0.50,0.50,0.50,0.50]; R_pp = 0.50

  Excel: =SUMPRODUCT(Shares,(BetweenSD^2/(BetweenSD^2+CommonMean*(1-CommonMean)/Patients))*Responders/Patients+(1-BetweenSD^2/(BetweenSD^2+CommonMean*(1-CommonMean)/Patients))*CommonMean) With one row per basket in named ranges Responders, Patients and Shares, and the common mean and between-basket standard deviation in cells named CommonMean and BetweenSD, the formula returns R_pp. The expression after Shares, entered row by row, returns each p_tilde_j.

  R: pool_baskets <- function(y, n, mu, tau, share) { w <- tau^2/(tau^2+mu*(1-mu)/n); p <- w*y/n+(1-w)*mu; list(p_tilde = p, R_pp = sum(share*p)) } Vectorised over baskets; the three vectors must hold the baskets in the same order.

  Python: def pool_baskets(y, n, mu, tau, share): y, n, share = (np.asarray(v, dtype=float) for v in (y, n, share)); w = tau**2/(tau**2+mu*(1-mu)/n); p = w*y/n+(1-w)*mu; return p, float(np.sum(share*p)) Uses numpy imported as np; returns the partially pooled responses and the mix response.

  Test (Very large between-basket standard deviation returns the separate-analysis mix): With tau of 100 every weight is close to 1, so the mix response is close to 0.30, the figure from the separate basket proportions. Expected result: TRUE. Excel check: =ABS(SUMPRODUCT({0.1,0.1,0.6,0.2},(100^2/(100^2+0.25/{20,15,10,5}))*{12,9,1,3}/{20,15,10,5}+(1-100^2/(100^2+0.25/{20,15,10,5}))*0.5)-0.3)<1E-4

  Test (No basket moves further from the common mean than its own proportion): With the partially pooled responses in a range named PooledResp, every estimate is at least as close to the common mean as the basket's observed proportion. Expected result: TRUE. Excel check: =SUMPRODUCT(--(ABS(PooledResp-CommonMean)<=ABS(Responders/Patients-CommonMean)+1E-12))=ROWS(Responders)

  Common error (Weighting the partially pooled estimates by trial enrolment): Applying the enrolment shares of 0.4, 0.3, 0.2 and 0.1 instead of the population mix gives about 0.508 rather than about 0.4047, above even the complete pooling figure of 0.50, because the trial over-represents tumour types that respond well.

  Source: Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd edition. Boca Raton: Chapman & Hall/CRC; 2013. Section 5.4, equation 5.17, which gives the conditional posterior mean of each group parameter in the normal hierarchical model as a precision-weighted average of the group's sample mean and the population mean, with the group's sampling variance equal to the data variance divided by the group size.

  tau2 = tau * tau; p_hat_j = y_j / n_j; s2_j = mu * (1 - mu) / n_j; w_j = tau2 / (tau2 + s2_j); p_tilde_j = w_j * p_hat_j + (1 - w_j) * mu; R_pp = sum_(j=1)^J [pi_j * p_tilde_j]

Try this function

Implementations

• Excel

  Basket weight and partially pooled response in two cells

  With named cells Responders, Patients, CommonMean and BetweenVar, the first formula returns the weight w_j, held in a cell named OwnWeight, and the second returns the partially pooled response.

  =BetweenVar/(BetweenVar+CommonMean*(1-CommonMean)/Patients); =OwnWeight*Responders/Patients+(1-OwnWeight)*CommonMean

Assumptions

• Common mean and between-basket variance treated as known

  The formula fixes mu and tau2 rather than estimating them. In a full hierarchical model both are estimated jointly with the basket responses, and with only a handful of small baskets the data say little about tau2, so its prior largely decides how much borrowing occurs.

• Exchangeable baskets in a basket trial

  Each basket's true response is treated as a draw from one common distribution, so the baskets are similar enough for borrowing to be justified. Where some baskets may differ, the exchangeability-nonexchangeability (EXNEX) model of Neuenschwander and colleagues lets each basket share a distribution with similar baskets or stand apart.

• Normal approximation for basket proportions on the probability scale

  The basket proportions are treated as normally distributed, with the sampling variance evaluated at the common mean. The approximation is weakest for very small baskets and for responses near 0 or 1, where the full model on the log-odds scale is preferred.

Worked examples

• Basket C shrunk from 0.10 towards a common mean of 0.50

  Basket C has 1 responder among 10 evaluable patients. With mu fixed at 0.50 and a between-basket standard deviation of 0.15, so that tau2 is 0.0225, the sampling variance is 0.025 and the weight on basket C's own data is about 0.4737. The partially pooled response is about 0.3105, shown as 0.31 in the article. Basket C moves furthest because its result lies far from the common mean.

  y_j = 1; n_j = 10; mu = 0.50; tau2 = 0.0225; p_hat_j = 0.10; s2_j = 0.025; w_j = 0.4737; p_tilde_j = 0.3105

• Basket D with 5 patients carries the lowest weight

  Basket D has 3 responders among 5 evaluable patients. Its sampling variance of 0.05 gives the lowest weight of the four baskets, about 0.3103, and its response moves from 0.60 to about 0.5310, further than in the larger baskets A and B.

  y_j = 3; n_j = 5; mu = 0.50; tau2 = 0.0225; p_hat_j = 0.60; s2_j = 0.05; w_j = 0.3103; p_tilde_j = 0.5310

• Basket A with 20 patients keeps most weight on its own data

  Basket A has 12 responders among 20 evaluable patients. The sampling variance of 0.0125 gives a weight of about 0.6429, and the response moves only from 0.60 to about 0.5643, shown as 0.56 in the article.

  y_j = 12; n_j = 20; mu = 0.50; tau2 = 0.0225; p_hat_j = 0.60; s2_j = 0.0125; w_j = 0.6429; p_tilde_j = 0.5643

Common errors

• Between-basket standard deviation entered as the variance

  Entering the standard deviation of 0.15 where tau2 belongs raises basket C's weight from about 0.474 to about 0.857 and gives a partially pooled response of about 0.157 instead of 0.3105, so far less borrowing occurs than intended. The variance is the square of the standard deviation, 0.0225.

• Basket's own proportion used in the basket sampling variance

  Evaluating the sampling variance at p_hat_j rather than mu makes extreme baskets look precise. Basket C's variance falls from 0.025 to 0.009, its weight rises to about 0.714 and its estimate falls to about 0.214 rather than 0.3105. A basket with no responders would have zero variance, a weight of 1 and no borrowing at all.

• Reading a borrowed basket estimate as that basket's own evidence

  Basket C rises from 0.10 to about 0.31 purely because of the other baskets. If basket C truly differs, the borrowed estimate overstates its response. In simulations, Berry and colleagues found that when a treatment was promising in only one group, borrowing cut the power for that group to 65%, and that borrowing could inflate the type I error rate when the drug was promising in some groups and not others.

Sources

• Precision-weighted group mean in the normal hierarchical model

  Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. 3rd edition. Boca Raton: Chapman & Hall/CRC; 2013. Section 5.4, equation 5.17, which gives the conditional posterior mean of each group parameter in the normal hierarchical model as a precision-weighted average of the group's sample mean and the population mean, with the group's sampling variance equal to the data variance divided by the group size.

  View source →

• TA630 on hierarchical borrowing across basket trial tumour types

  National Institute for Health and Care Excellence. Larotrectinib for treating NTRK fusion-positive solid tumours (TA630). London: NICE; 2020. Section 3.14, which describes the evidence review group's Bayesian hierarchical model as developed for basket trials, borrowing strength across tumour types to avoid extreme results caused by limited patient numbers, and reports that it reduced the overall response from 72% to 57%.

  View source →

• Berry and colleagues on the costs of borrowing across patient groups

  Berry SM, Broglio KR, Groshen S, Berry DA. Bayesian hierarchical modeling of patient subpopulations: efficient designs of Phase II oncology clinical trials. Clinical Trials. 2013;10(5):720-734. Abstract, which reports gains in power from hierarchical modelling across four patient groups, power reduced to 65% when the treatment is promising in only one group, and a potential for borrowing to inflate the type I error rate.

  View source →

Canonical Identity