Common Random Numbers

Common Random Numbers as Matched Random Draws for Sharper Strategy Comparisons in Health Economic Simulation

In a patient-level simulation every strategy's costs and QALYs carry Monte Carlo error, so when two strategies are run on different random numbers part of the estimated difference between them is chance. Common random numbers remove much of that chance by putting the same simulated patients, facing the same random events, through every strategy, so that the remaining difference mostly reflects the strategies. Economic evaluation turns on incremental results such as incremental net benefit, and a noisy increment needs far more simulated patients to settle. The method applies in both microsimulation and discrete event simulation, provided the random number streams are kept in step.

Why matched draws shrink the variance of an incremental result

The quantity of interest is the expected difference between two strategies, and simulation can estimate it in two ways. Each strategy can be run on its own independent random numbers and the two averages subtracted, or both can be run on the same random inputs and the paired differences averaged. Owen sets out the variances of the two estimators as follows.

$$\operatorname{Var}(\hat{D}_{\text{com}}) = \frac{\sigma_f^2 + \sigma_g^2 - 2\rho\,\sigma_f\sigma_g}{n}, \qquad \operatorname{Var}(\hat{D}_{\text{ind}}) = \frac{\sigma_f^2 + \sigma_g^2}{n}$$

where $\hat{D}_{\text{com}}$ is the average of $n$ paired differences computed with common random numbers, $\hat{D}_{\text{ind}}$ is the difference between two averages of $n$ independent runs each, $\sigma_f^2$ and $\sigma_g^2$ are the variances of a single simulated output under each strategy, and $\rho$ is the correlation between the paired outputs.

When $\rho$ is positive the covariance term is subtracted and common random numbers win; when it is negative they lose. Owen stresses that there is no guarantee that $\rho$ is positive, although it is expected when the two strategies compute similar quantities, as competing treatments in the same disease model often do. Stout and Goldie note the other side of the method, which is that it does not reduce the stochastic variation within a single model run, as simulating a larger cohort would. Its benefit lies entirely in the comparison, since it is a variance reduction technique for increments, with no gain for the mean cost or QALYs of one strategy alone.

Keeping the same patient on the same random path

Sharing a seed between two runs is only the starting point. The NICE Decision Support Unit's Technical Support Document 15 (TSD 15) calls common random numbers, which let treatments be compared "under similar experimental conditions", a popular variance reduction technique that can decrease the number of patients that need to be sampled. It explains that bespoke simulation packages can control the random number streams so that the same patient characteristics and pathways are sampled in each arm, while in a generic programming language the same result needs either parallel processing of patients through both arms or manual control of the streams.

Murphy and colleagues separate two aims, namely identical patient populations across arms, and removal of every difference between arms that is not due to the comparators. They describe a separate random number stream for each source of variability as necessary but not sufficient, because events can fall out of step. If a patient dies during an operation in one arm, the random numbers that would have set later treatment characteristics go unused, while in an arm where the operation succeeds they are sampled. Their full implementation therefore keeps unused random numbers so that the sequence stays synchronised. The ISPOR-SMDM good practice report on discrete event simulation makes the same point. Separate streams for different events, for example one for whether an event occurs and another for length of stay, reduce the chance that the same patient draws different numbers for the same event under different strategies.

Stout and Goldie give practical designs. Stochastic events are grouped by function (in their breast cancer model, tumour characteristics, symptom detection and screening), and each group in each simulated woman draws from its own sequence. A generator built for the purpose, such as L'Ecuyer's RngStream with independent streams and substreams, is the method they describe as least prone to unintended correlations; an alternative is to pre-draw a fixed block of numbers for each group and individual, large enough for any lifetime. They warn that linearly related seeds can produce correlated sequences in some generators, and that the generator's period must not repeat within a run. Random number generation and pseudo-random numbers have their own pages.

Other implementation traps are documented. Owen notes that acceptance-rejection sampling does not fit cleanly because the number of uniforms it consumes is not fixed, so inversion methods keep draws aligned more easily. TSD 15 reports an Excel model in which each patient was simulated under all strategies before the next patient, and adding a further strategy changed the results substantially because of autocorrelation in Excel's random numbers; evaluating all patients for one strategy before the next reduced the problem. In an agent-based model, Chhatwal and He explain that interaction between agents can change the sequence of events across interventions, and describe assigning each agent its own random seed as one solution.

Which random draws should be shared

Not every draw should be common. Stout and Goldie show that synchronising a treatment-effectiveness draw preserves a simulated woman's "tendency" to respond, which is reasonable where it may represent genetic factors or tumour markers, but add that where little clinical evidence supports such a link, synchronising is not desired and could lead to incorrect conclusions. Murphy and colleagues, citing Stout and Goldie, give the case of treatments acting through different mechanisms. There is then no reason to expect a patient's responses to be maximally correlated, and treatment-related draws should use separate random numbers for each treatment. Shared draws therefore belong to the patient and the disease process; a treatment-specific response is shared only when the model's assumptions justify it.

What published health simulation models found

Two published health applications illustrate the size of the gain, which depends on the model. Murphy and colleagues adapted an osteoporosis microsimulation comparing teriparatide with no treatment. For incremental patient costs over 1,024,000 simulated patients, full common random numbers cut the variance by 93.6% and the 95% confidence interval half-width by 74.7%, while a partial version that matched only the initial patient characteristics cut the variance by only 5.6%. The confidence interval reached with 1,000 patients using full common random numbers needed 16,000 patients without them.

The authors attribute the weak partial result to the model's structure, chiefly that death was evaluated cycle by cycle instead of being sampled at entry, and note that other studies found larger gains from partial schemes. In 1,000 replications of 1,000 patients run at base-case parameter values, some replications without common random numbers showed negative incremental QALYs, while all 1,000 were positive with them. Stout and Goldie, calibrating a breast cancer model, found that common random numbers reduced the variance of the differences between two runs by 82%, against 71% from simulating a cohort ten times larger.

Worked example of incremental net monetary benefit with and without matched draws

The numbers in this example are illustrative. A patient-level model compares a new treatment with standard care and reports per-patient net monetary benefit, whose standard deviation across simulated patients is GBP 30,000 under each strategy. The true incremental net benefit is GBP 400 per patient, and 5,000 patients are simulated per strategy. With independent streams $\rho = 0$; suppose common random numbers give a correlation of 0.95 between a patient's two outputs.

1. Variance of one paired difference. Each strategy's variance is 30,000 × 30,000 = 900,000,000. Independent runs give 2 × 900,000,000 = 1,800,000,000; common random numbers give 1,800,000,000 × (1 − 0.95) = 90,000,000.

2. Standard error. Dividing by 5,000 and taking the square root gives the square root of 360,000, or GBP 600, for independent runs and the square root of 18,000, or GBP 134.2, with common random numbers.

3. Confidence intervals. Half-widths are 1.96 × 600 = GBP 1,176 and 1.96 × 134.2 = GBP 263, so the 95% intervals around GBP 400 run from minus GBP 776 to GBP 1,576 without matching and from GBP 137 to GBP 663 with it.

4. Equivalent sample size. Independent streams need 5,000 / (1 − 0.95) = 100,000 patients per strategy to match the smaller standard error, twenty times as many.

5. Interpretation. Without matched draws the simulation cannot say whether the treatment's net benefit is positive at this sample size; with them the sign is clear. The gain comes entirely from $\rho$. At a correlation of 0.5 the factor $(1 - \rho)$ would halve the variance instead of cutting it twentyfold, and a negative correlation would make matters worse.

Analysing and reporting matched simulations

Because the arms are deliberately correlated, the analysis has to treat the outputs as pairs. Stout and Goldie state that tests which account for the correlation, such as the paired t test, should be used, and two-sample tests that assume independence between model runs should not; the standard error comes from the spread of the paired differences, with replications independent of one another. Murphy and colleagues judge the value of the method to be model specific, determinable except in simple cases only by comparing variance estimates with and without it, and recommend such test runs, which also expose a correlation that has turned negative.

The ISPOR-SMDM discrete event simulation report recommends using variance reduction techniques, balancing coding time against gains in run time and accuracy, and notes a debugging use, in that the time to an event not influenced by the strategy should be identical for the same simulated patient across strategies.

In probabilistic sensitivity analysis of a patient-level model, TSD 15 explains that two nested loops are usually needed, an inner loop over simulated patients and an outer loop over parameter values. Within each parameter set the strategies can share patients and streams, so the patient-level noise in each iteration's increment falls. Murphy and colleagues held parameters at base-case values to isolate sampling variation, and found that without matched draws the scatter of incremental costs and QALYs gave a much riskier appraisal of the treatment.

Limits and common misunderstandings

The method reduces simulation noise in a comparison and leaves real uncertainty alone. Parameter uncertainty, structural uncertainty and the spread of incremental results across a probabilistic analysis's parameter draws are unchanged, so the cost-effectiveness acceptability curve should still reflect them; only the part of its spread caused by too few simulated patients shrinks. Reproducible output from a fixed seed is a separate property and says nothing about whether the model is valid.

Several errors recur. A shared seed without per-event streams gives drifting draws once arms diverge, which can leave much of the noise in place, much as matching only the initial patient characteristics did in the osteoporosis model. Murphy and colleagues add that where several variables interact in a model, common random numbers can actually increase variance. Synchronising treatment-related draws without evidence imposes a correlation that becomes a hidden modelling assumption, and it shapes any individual-level comparison of the same patient across arms. Finally, a two-sample standard error applied to positively correlated output is too large, so it understates the precision the matching has bought.

Common random numbers beside antithetic variates and other variance reduction methods

Chhatwal and He list common random numbers alongside antithetic variates, control variates, importance sampling and stratified sampling, and note that because most decision models compare two or more strategies, common random numbers have been widely used in health economic modelling. Antithetic variates aim at the opposite sign. They pair each draw with its mirror image to induce negative correlation within the estimate of one expected output, while common random numbers induce positive correlation between strategies. The two can be combined, but whether the combination helps an incremental result has to be checked. Simulating more patients remains the simplest alternative and the benchmark for any gain.

Sources

• Chhatwal J, He T. Economic evaluations with agent-based modelling: an introduction. PharmacoEconomics. 2015;33(5):423-433. https://doi.org/10.1007/s40273-015-0254-2
• Davis S, Stevenson M, Tappenden P, Wailoo A. NICE DSU Technical Support Document 15: Cost-effectiveness modelling using patient-level simulation. Sheffield: Decision Support Unit, ScHARR, University of Sheffield; April 2014. Sections 3.6, 4.1, 4.6 and 5.2. https://www.sheffield.ac.uk/nice-dsu/tsds/patient-level-simulation
• Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Möller J. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Value in Health. 2012;15(6):821-827. https://doi.org/10.1016/j.jval.2012.04.013
• Murphy DR, Klein RW, Smolen LJ, Klein TM, Roberts SD. Using common random numbers in health care cost-effectiveness simulation modeling. Health Services Research. 2013;48(4):1508-1525. https://doi.org/10.1111/1475-6773.12044
• Owen AB. Monte Carlo Theory, Methods and Examples. 2013. Chapter 8, Variance reduction, section 8.6. https://artowen.su.domains/mc/Ch-var-basic.pdf
• Stout NK, Goldie SJ. Keeping the noise down: common random numbers for disease simulation modeling. Health Care Management Science. 2008;11(4):399-406. https://doi.org/10.1007/s10729-008-9067-6