Concept Architecture
Concept
Theoretically, The Law of Large Numbers (LLN) is a fundamental theorem of probability stating that the sample average of independent and identically distributed random variables converges to the corresponding population expected value as the sample size increases. The theorem provides the mathematical justification for statistical estimation by demonstrating that random sampling produces increasingly reliable estimates of population parameters. The Law of Large Numbers exists to establish the long-run stability of sample statistics and underpins nearly all inferential statistical methods used in health economics.
Mathematically, the Law of Large Numbers is expressed as a convergence theorem. The Weak Law of Large Numbers states that the probability of the sample mean differing from the population mean by more than any positive amount approaches zero as the sample size tends to infinity. The Strong Law of Large Numbers strengthens this result by showing that the sample mean converges almost surely to the expected value. These results justify replacing unknown population expectations with empirical averages in statistical estimation and simulation.
In practice, the Law of Large Numbers underlies Monte Carlo simulation, bootstrap methods, maximum likelihood estimation, Bayesian computation, clinical trials, observational studies and health economic modelling. As the number of simulated patients, trial participants or sampled observations increases, estimated costs, utilities, transition probabilities and other model parameters converge towards their true expected values, reducing sampling variability and improving the precision of economic evaluations.
Purpose
Used to justify statistical estimation from sample data, demonstrate convergence of sample averages to population expectations, support simulation methods and provide the theoretical foundation for statistical inference and health economic modelling.
Mathematical Formulae
Primary Formula
For independent and identically distributed random variables X?, X?, ?, X? with expected value ?,
lim??� P(|X?? ? ?| > �) = 0
where:
- X?? = sample mean
- ? = population mean
- � > 0
Supporting Formulae
Sample mean:
X? = (1/n) ? ????� X?
Expected value:
E(X) = ?
Strong Law of Large Numbers:
P(lim??� X?? = ?) = 1
Related Mathematical Methods
- Central Limit Theorem
- Monte Carlo Simulation
- Bayesian Estimation
- Maximum Likelihood Estimation
- Bootstrap Methods
- Markov Chain Monte Carlo
- Stochastic Simulation
Example
A health economist performs a probabilistic sensitivity analysis to estimate the expected lifetime cost of a new treatment.
The estimated mean lifetime cost is:
- 1,000 simulations: �12,450
- 10,000 simulations: �12,218
- 100,000 simulations: �12,204
As the number of simulations increases, the estimated average stabilises near the expected population value. This convergence illustrates the Law of Large Numbers and provides confidence that the simulation has produced a reliable estimate of expected healthcare costs.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| RAND | =RAND() | Generate random observations for simulation studies. |
| AVERAGE | =AVERAGE(A2:A100001) | Estimate the sample mean as sample size increases. |
| COUNT | =COUNT(A2:A100001) | Monitor the number of observations included in the analysis. |
| STDEV.S | =STDEV.S(A2:A100001) | Assess the reduction in sampling variability with increasing sample size. |
| Data Table | Monte Carlo simulation using repeated recalculation. | Demonstrate convergence of simulated costs, utilities or probabilities. |
VBA (Optional)
A VBA routine can automate large-scale Monte Carlo simulations, continuously update running sample means and monitor convergence towards stable expected values.
Sources
- Ross SM. Introduction to Probability Models. Academic Press.
- Casella G, Berger RL. Statistical Inference. Cengage Learning.
- Billingsley P. Probability and Measure. Wiley.
- Mood AM, Graybill FA, Boes DC. Introduction to the Theory of Statistics. McGraw-Hill.
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press.
- ISPOR Good Practice Reports.
Related Concepts (4)
Library
Publications
1
Bayesian Methods in Health Economics — Gianluca Baio, 1st Edition ed., 2012 (Chapman & Hall / CRC Press)
An overview of Bayesian statistical methods for the analysis of health economic data, covering economic evaluation concepts, statistical cost-effectiveness analysis, Bayesian computation and MCMC, and applied health economic evaluation.
BookView source →
Frequently Asked Questions (6)
What is the law of large numbers?
A theorem stating that as sample size increases, the sample average of a variable converges toward the true underlying population average.
Source: Casella G, Berger RL. Statistical Inference. 2nd ed. Duxbury; 2002.
What does the law of large numbers say happens as a sample grows?
The law of large numbers says that as a sample grows, its average settles ever closer to the true average of the population it is drawn from. Early on, a small sample's mean can wander far from the truth, but with more observations the random ups and downs cancel out and the estimate converges. This is why larger studies give more trustworthy estimates, and it underpins the whole idea of learning about a population by sampling from it. Averages converging on the truth with size is what it establishes. Kirkwood and Sterne (2003) describe this.
Source: Kirkwood & Sterne 2003
Why is the law of large numbers important?
The law of large numbers is important because it justifies using sample averages to estimate population quantities, assuring that with enough data the estimate approaches the truth, which underpins estimation, simulation, and the reliial behaviour of many statistical procedures. So the law of large numbers matters as a foundation of statistics, since it explains why larger samples yield more reliable estimates and why methods based on averaging work, giving confidence that quantities estimated from sufficient data converge to their true values, and it underlies techniques such as Monte Carlo simulation, which rely on averages over many draws approaching expected values.
Source: Casella & Berger 2002
How does the law of large numbers work?
The law of large numbers works because, as more independent observations are averaged, the random fluctuations of individual values increasingly cancel out, so the sample mean stabilises around the population mean, with the deviation shrinking as the sample grows. So the law of large numbers works through the averaging of many observations, in which the variability of the individual values averages away, leaving the sample mean close to the true mean, which is why estimates based on large samples are stable and accurate, and this convergence is the reason sample means are trusted as estimators of population means when samples are large.
Source: Casella & Berger 2002
How does the law of large numbers differ from the central limit theorem?
The law of large numbers states that the sample mean converges to the population mean as the sample grows, concerning where the average settles, while the central limit theorem describes the distribution of the sample mean, stating that it becomes approximately normal around the population mean regardless of the data's shape. So the two differ in what they describe, with the law of large numbers concerning the convergence of the mean to its true value and the central limit theorem concerning the shape and spread of the mean's sampling distribution, and together they underpin estimation and inference, one assuring accuracy and the other enabling probability statements about the mean.
Source: Casella & Berger 2002
What are the practical implications of the law of large numbers?
The practical implications of the law of large numbers are that larger samples give more accurate and stable estimates of population quantities, that averages over many observations or simulations approach their expected values, and that reliable estimation requires adequate sample sizes. So the law of large numbers implies that increasing the sample size improves the precision of estimates and that methods relying on averaging, such as Monte Carlo simulation, converge with enough draws, which is why sample size matters for estimation and why sufficient data are collected, since small samples give averages that may deviate substantially from the truth.
Source: Casella & Berger 2002
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 17 Dec 2025
Content version: 1.0.0
Canonical Identity
- Term code
- HE-ES-SA-100
Stable URI · Machine-readable · Resolvable · CC BY 4.0