Stochastic Simulation

What stochastic simulation does

Repeated runs produce different realisations even when the model's fixed inputs are unchanged, allowing analysts to estimate expected outcomes, variability and decision uncertainty. This page explains the simulation process, major model forms, health-economic applications, statistical precision and validation requirements.

The word stochastic describes randomness inside the model. It does not by itself identify the model structure, the source of uncertainty or the decision question being answered.

A stochastic model maps random draws into outcomes

Let (U) denote a vector of pseudo-random numbers and (\theta) the model parameters. A stochastic simulation produces an outcome through a model function:

$$Y=g(U;\theta)$$

Different realisations of (U) generate different values of (Y). If the model is run independently (R) times, the expected outcome can be estimated by the sample mean:

$$\bar{Y}_R=\frac{1}{R}\sum_{r=1}^{R}Y_r$$

The empirical variance across replications is:

$$s_Y^2=\frac{1}{R-1}\sum_{r=1}^{R}(Y_r-\bar{Y}_R)^2$$

and the estimated Monte Carlo standard error of the mean is:

$$\operatorname{MCSE}(\bar{Y}_R)=\frac{s_Y}{\sqrt{R}}$$

More replications reduce simulation error but do not correct biased inputs, an invalid structure or missing sources of uncertainty.

Random-number generation drives the simulation

Most computer simulations use deterministic algorithms that generate sequences designed to behave like independent random numbers. A seed determines the starting point of the sequence. Recording the seed, generator and software version makes a run reproducible, while using well-managed independent streams prevents accidental reuse across replications or parallel processes.

A common starting point is a pseudo-random draw:

$$U\sim\operatorname{Uniform}(0,1)$$

The draw can be transformed into another distribution using its inverse cumulative distribution function (F^{-1}):

$$X=F^{-1}(U)$$

This inverse-transform relationship is exact for a continuous distribution when (U) is uniform and (F^{-1}) is defined appropriately. Software may use other specialised algorithms for speed or numerical stability.

The probability model must match the quantity being simulated

Distributions encode the possible values and relative likelihoods of random quantities. The choice should reflect the data-generating process, support, dependence and intended uncertainty rather than convenience alone. Implausible distributional tails can dominate rare events, long-term costs or survival projections.

Empirical resampling can preserve observed features without imposing a simple parametric form, but it remains limited by the representativeness and size of the source data.

Event probabilities can be simulated with a uniform draw

For a binary event with probability (p), draw (U\sim\operatorname{Uniform}(0,1)) and define:

$$I= \begin{cases} 1, & U<p\\ 0, & U\ge p \end{cases}$$

Then (I) follows a Bernoulli distribution with:

$$E[I]=p$$

and:

$$\operatorname{Var}(I)=p(1-p)$$

For several mutually exclusive outcomes with probabilities (p_1,\ldots,p_K) summing to one, cumulative probability intervals map the same uniform draw to one outcome. The implementation should verify non-negative probabilities and a cumulative total of one within numerical tolerance.

A worked patient-level example

Suppose a model simulates whether each patient experiences a complication during one year. The complication probability is 0.20, annual cost without a complication is £1,000, and a complication adds £4,000.

For patient (i):

$$I_i\sim\operatorname{Bernoulli}(0.20)$$ $$C_i=£1{,}000+£4{,}000I_i$$

The expected cost is:

$$E[C_i] =£1{,}000+£4{,}000(0.20) =£1{,}800$$

If five illustrative uniform draws are 0.08, 0.64, 0.17, 0.42 and 0.91, the first and third patients experience the complication. Their simulated costs are £5,000, £1,000, £5,000, £1,000 and £1,000, giving a small-sample mean of £2,600. The difference from £1,800 is simulation variability, not evidence that the analytical expectation is wrong; many more independent patients or replications are needed for precision.

Different stochastic simulations represent time differently

Stochastic simulation is a broad family rather than a single model type. The treatment of time, individuals and events determines which structures can be represented and how results should be validated.

A cohort Markov model that propagates expected state proportions deterministically is not a stochastic simulation merely because its transition probabilities were estimated with uncertainty. Sampling individual transitions converts it into an individual-level stochastic implementation.

Time-to-event simulation advances through sampled event times

In a continuous-time simulation, an event time can be generated from a survival distribution. If the time to an event follows an exponential distribution with constant rate (\lambda>0), inverse transform sampling gives:

$$T=-\frac{\ln(U)}{\lambda}$$

where (U\sim\operatorname{Uniform}(0,1)). The exponential survival function is:

$$S(t)=P(T>t)=e^{-\lambda t}$$

Competing-event models may sample a time for every enabled event and execute the earliest, or sample from the combined hazard and then select the event type. The chosen algorithm must reproduce the intended joint event process and update the event list when a patient's state changes.

First-order and second-order uncertainty must be separated

Individual-level randomness and parameter uncertainty answer different questions. First-order uncertainty represents stochastic variation between otherwise comparable individuals or trajectories. Second-order uncertainty represents uncertainty about parameter values because the available evidence is incomplete.

A nested simulation may draw parameter set (\theta_b) in outer loop (b) and then simulate individuals or replications (r) within that parameter set:

$$\theta_b\sim p(\theta), \qquad Y_{br}=g(U_{br};\theta_b)$$

The inner loop estimates the conditional expectation (E[Y\mid\theta_b]), while variation across outer-loop parameter draws represents parameter uncertainty. Mixing both layers without identifying them can inflate or misinterpret uncertainty and can distort cost-effectiveness probabilities.

Stochastic simulation is related to but not identical to Monte Carlo simulation

Monte Carlo methods use repeated random sampling to estimate quantities. Stochastic simulation commonly uses Monte Carlo computation, but its primary purpose may be to reproduce a random process, individual pathway or interacting system rather than only to approximate an integral. The terms overlap, yet they should not be treated as exact synonyms in every context.

Probabilistic sensitivity analysis is also distinct. It samples uncertain model parameters to propagate parameter uncertainty. A probabilistic sensitivity analysis can use a deterministic model, while a stochastic patient-level model can be run with fixed parameters; combining them requires an explicit nested or otherwise justified design.

Simulation error should be measured, not assumed away

Finite stochastic simulations contain Monte Carlo error. Reporting only the point estimate can make random fluctuation appear to be a stable model result. Precision should be assessed for the decision-relevant outputs and, when alternatives are compared, for the incremental outcome rather than each arm in isolation.

An approximate confidence interval for an estimated mean is:

$$\bar{Y}_R \pm t_{R-1,1-\alpha/2} \frac{s_Y}{\sqrt{R}}$$

The required replication count for target half-width (h) may be approximated after a pilot estimate of (s_Y):

$$R \approx \left( \frac{z_{1-\alpha/2}s_Y}{h} \right)^2$$

This calculation applies to the precision of a mean under the stated assumptions. Quantiles, tail probabilities, ratios and rare events may need different diagnostics and substantially larger samples.

Warm-up, run length and replication design affect results

Models of ongoing systems may begin in an artificial empty or otherwise unrepresentative state. A warm-up period allows the simulated process to approach its long-run behaviour before observations are collected. The need for warm-up depends on whether the model represents a terminating patient pathway or a non-terminating service system.

Key design choices include:

• A terminating simulation should run until its clinically or operationally defined endpoint.
• A non-terminating simulation needs a justified warm-up period and observation horizon.
• Independent replications allow conventional estimates of between-run simulation error.
• One extremely long run does not automatically replace replication when serial dependence complicates variance estimation.
• Batch means or time-series methods may be required for steady-state output from a single long run.
• Rare outcomes require targeted precision checks because apparently stable averages can coexist with unstable tail estimates.

Discarding an arbitrary initial period without diagnostic evidence can introduce rather than remove bias.

Variance-reduction methods improve efficiency

Variance-reduction methods seek more precise comparisons for a given amount of computation. They change how random draws are organised, not the substantive uncertainty represented by the model. Their validity depends on correct implementation and on preserving the intended marginal and joint distributions.

Common methods include:

• Common random numbers use matched streams for competing strategies so shared stochastic variation cancels in incremental comparisons.
• Antithetic variates pair draws such as (U) and (1-U) to induce negative correlation in suitable settings.
• Control variates use a correlated quantity with known expectation to adjust the estimator.
• Stratified or Latin hypercube sampling spreads draws more evenly across a distribution than simple random sampling.
• Importance sampling over-samples influential or rare regions and applies weights to recover the target expectation.

Common random numbers should align equivalent random processes across strategies. Reusing draws where the pathways no longer correspond can add bias or obscure genuine structural differences.

Correlation and dependence must be represented explicitly

Independent draws are inappropriate when parameters, repeated outcomes, competing risks or patient characteristics are dependent. Dependence may be introduced through a multivariate distribution, Cholesky transformation, copula, shared latent variable, conditional model or resampling of complete correlated records.

If (z\sim\operatorname{Normal}(0,I)) and a covariance matrix satisfies (\Sigma=LL^{\mathsf T}), then:

$$x=\mu+Lz$$

has covariance (\Sigma). This construction applies directly to multivariate normal quantities; non-normal marginals require additional transformation and careful checking of the resulting dependence. Pairwise correlations selected independently may not form a valid joint correlation matrix.

Health-economic outputs require decision-level aggregation

Patient-level simulated costs and outcomes are normally aggregated across individuals, replications and treatment strategies. Discounting, censoring, competing events and time horizons must be applied consistently. Incremental outcomes should be formed within matched replications when common random numbers or other pairing is used.

At willingness-to-pay threshold (\lambda), the simulated incremental net monetary benefit in replication (r) is:

$$\operatorname{INMB}_r(\lambda) =\lambda\bigl(E_{1r}-E_{0r}\bigr) -\bigl(C_{1r}-C_{0r}\bigr)$$

The mean incremental net monetary benefit is:

$$\overline{\operatorname{INMB}}(\lambda) =\frac{1}{R}\sum_{r=1}^{R}\operatorname{INMB}_r(\lambda)$$

A positive mean INMB supports the intervention under the specified decision rule, but the decision still depends on model validity, evidence, represented uncertainty and relevant non-modelled considerations.

Verification checks the code against the intended model

Verification asks whether the simulation was implemented correctly. Because stochastic output changes from run to run, tests should include fixed-seed reproducibility, deterministic limiting cases and distributional checks rather than relying only on exact output matching.

Useful verification tests include:

• Confirm that identical seeds and settings reproduce identical results in the same computing environment.
• Test probabilities of zero and one, zero rates and other boundary cases with known outcomes.
• Compare simulated frequencies, means and variances with analytical expectations in simplified cases.
• Trace individual entities through events, states, resource use, costs and outcomes.
• Check that mutually exclusive events remain exclusive and that competing-event logic selects the correct event.
• Verify that time never moves backward and that obsolete scheduled events are cancelled or updated.
• Reconcile totals across patient, state, event and strategy summaries.
• Test stream independence and parallel execution for accidental seed duplication.

Passing these checks shows that selected implementation properties behave as intended; it does not prove that the conceptual model represents reality adequately.

Validation compares model behaviour with knowledge and evidence

Validation assesses whether the model is suitable for its purpose. It should examine input data, face validity, internal behaviour, external predictions and the consequences of uncertainty. Stochastic variability means comparisons require tolerances or statistical criteria rather than exact agreement.

Validation should consider:

• Whether simulated pathways and event order are clinically and operationally credible.
• Whether event rates, waiting times, costs and outcomes agree with source evidence within expected sampling variation.
• Whether subgroup patterns and long-term projections are plausible.
• Whether alternative implementations reproduce analytical or independently modelled benchmarks.
• Whether experts can explain unexpected tail behaviour, bottlenecks or feedback effects.
• Whether decisions remain stable after simulation error is reduced to an acceptable level.

Calibration to observed data is not independent external validation when the same outcomes were used to estimate parameters.

Common errors can produce convincing but wrong results

Stochastic output often looks realistic even when the underlying logic is flawed. Visual plausibility must therefore be supported by deterministic tests, statistical diagnostics and transparent documentation.

• Failing to distinguish parameter uncertainty from individual variability can misstate decision uncertainty.
• Using too few replications can make random noise look like a strategy difference.
• Resetting or duplicating seeds can create unintended correlation across patients or parallel workers.
• Ignoring clinically required dependence can generate impossible patient histories.
• Allowing events to remain scheduled after they become impossible can corrupt event-based models.
• Selecting distributions only from means and standard deviations can overlook support, skewness and tail behaviour.
• Treating a fixed seed as proof of validity confuses reproducibility with correctness.
• Reporting standard deviation instead of Monte Carlo standard error misstates the precision of the estimated mean.
• Optimising code before establishing a clear reference implementation can make errors harder to detect.

A reproducible stochastic-simulation workflow

A defensible simulation preserves the path from conceptual model to random-number streams, event logic, results and decision interpretation. The workflow should be documented sufficiently for an independent analyst to reproduce both the model and its precision assessment.

1. Define the purpose: State the decision problem, population, strategies, perspective, horizon and required outputs.
2. Specify the process: Describe entities, states, events, timing, interactions, constraints and stopping rules before coding.
3. Classify uncertainty: Separate stochastic variability, parameter uncertainty, heterogeneity and structural uncertainty.
4. Assign distributions: Justify supports, parameters, dependence and tail behaviour from appropriate evidence.
5. Design random streams: Record the generator, seeds, stream allocation and parallelisation method.
6. Build a testable implementation: Keep random-draw, event, state, cost and outcome logic modular and traceable.
7. Verify limiting cases: Compare simplified simulations with exact or independently calculated expectations.
8. Plan precision: Select warm-up, run length, replications and stopping criteria from the outputs that drive the decision.
9. Validate behaviour: Compare pathways and aggregate outputs with empirical evidence, expert knowledge and external benchmarks.
10. Report transparently: Present estimates, simulation error, uncertainty layers, diagnostics, seeds, software and unresolved limitations.

Stochastic simulation is valuable when randomness, timing, individual pathways or interactions matter to the decision. Its credibility depends not on the volume of simulated data but on a valid structure, justified probability model, controlled random-number design and demonstrated numerical precision.