Big Data

Big Data in Health Economics and HTA

In health care the term big data usually refers to pooled or linked collections of electronic health records, insurance claims, registries, genomic sequences and device readings that are too large or too mixed for ordinary data handling and statistical routines. For health economists such collections promise local costs, event rates and treatment patterns at a scale no trial can match. Size does not remove bias, though, and a large but selective dataset can give a precise answer that is wrong. The value of big data in an economic evaluation or a health technology assessment therefore rests on data quality, relevance to the decision population and governance, not on the number of records.

Volume, velocity and variety in health data

Big data has no single agreed definition. A common framing describes it by volume, velocity and variety, and some health informatics authors add veracity, meaning whether the data can be trusted, which is the concern of data quality. Raghupathi and Raghupathi illustrate velocity with repeated glucose readings and bedside monitors, and variety with the mix of structured, semi-structured and unstructured data, including written notes and medical imaging.

The HMA-EMA Joint Big Data Taskforce of European medicines regulators defined big data as extremely large datasets that may be complex, multi-dimensional, unstructured and heterogeneous, that accumulate rapidly, and that need advanced or specialised methods to give an answer within reliable constraints. It added that a single dataset, such as one clinical trial or the electronic health records of one supplier, does not strictly meet that definition, but that pooling or linking it with others can give it the characteristics of big data.

How big data differs from real-world data

Real-world data are defined by where they come from, routine care rather than a research protocol, whereas big data is defined by scale, structure and the analytical effort needed. The two overlap heavily, since the largest health datasets are usually linked electronic health records, claims data and registries. A small registry for a rare disease holds real-world data without being big data, and pooled genomic or trial datasets can be big data without being routine care records.

Volume does not turn either kind of data into real-world evidence, which still needs a stated design and analysis. Data mining and data analytics are methods applied to data of any size, and artificial intelligence in health supplies many of the tools, while big data describes the data themselves. Collins saw promise for health economics in large databases showing how drugs work in routine practice once adherence, co-morbidities, interactions and side effects are taken into account.

Uses in economic evaluation and technology assessment

The least contested uses of big data in health technology assessment are descriptive model inputs rather than treatment effects. Kent and colleagues, four of them at NICE, report wide acceptance of observational data for safety, for treatment patterns, and for parameters such as incidence, event rates, overall survival, healthcare use, costs and health-related quality of life.

NICE's methods manual (PMG36) keeps a strong preference for high-quality randomised controlled trials when estimating relative treatment effects, while stating that real-world evidence may be preferred for natural history, treatment patterns or patient experiences. The NICE real-world evidence framework (ECD9, 2022) cites a technology appraisal that estimated the economic burden of severe eosinophilic asthma from CPRD GOLD primary care records linked to Hospital Episode Statistics, a typical input to a cost-of-illness study.

Big data also drives prediction and risk stratification. Bates and colleagues set out six use cases where analytics on electronic records could cut costs, namely high-cost patients, readmissions, triage, decompensation, adverse events and treatment optimisation for diseases affecting several organ systems. An ISPOR task force identified five areas where machine learning could help health economics and outcomes research, from cohort selection to causal inference and economic model development, and, noting that opaque methods raise the risk of relying on their results, developed the PALISADE checklist to weigh such uses against the need for transparency.

Linkage, common data models and federated networks

One patient's care is usually spread across primary care, hospital and death records, so linkage is often the step that turns several ordinary datasets into big data. ECD9 notes that linkage extends the information held on each patient while pooling extends sample size or coverage, and that imperfect linkage can exclude records or misclassify data. It asks submissions to report who did the linkage, whether the method was deterministic or probabilistic, the variables used and the linkage performance.

A common data model standardises the structure and coding of different datasets so that one analysis script can run on all of them. With the OMOP common data model, data stay with each custodian in a federated network and only code and aggregate results are shared, which eases part of the privacy problem. Kent and colleagues found gaps for HTA, because the standard tools were built for drug utilisation and safety studies and lack models for costs and parametric survival, and the cost tables reflect US claims, whereas many European datasets need unit costs attached to recorded healthcare use. Mapping cannot repair missing items, misclassification or selection bias in the source data.

Why a bigger dataset can still be biased

Random sampling error falls as the number of records grows, but bias from who enters the dataset shrinks only as coverage approaches the whole population, so a narrow confidence interval around a biased estimate gives false assurance. A decomposition by Meng, applied by Bradley and colleagues, writes the error of a sample mean as the product of data quality, data quantity and the difficulty of the problem.

$$\bar{Y}_n - \bar{Y}_N = \rho_{R,Y} \times \sqrt{\frac{N-n}{n}} \times \sigma_Y$$

where $\bar{Y}_n$ is the mean in the $n$ records held, $\bar{Y}_N$ is the mean in the target population of size $N$, $\rho_{R,Y}$ is the data defect correlation between being recorded ($R$) and the value of the outcome ($Y$), and $\sigma_Y$ is the standard deviation of the outcome. For a random sample the data defect correlation is of the order of $\frac{1}{\sqrt{N}}$, small enough to offset a large population, but selective recording can make it far larger. Matching mean squared error gives a bias-adjusted effective sample size, the size of a simple random sample with the same error.

$$n_{eff} = \frac{n}{N-n} \times \frac{1}{\rho_{R,Y}^2}$$

where $n_{eff}$ is the effective sample size and the other symbols are as above, in the unweighted form with the squared correlation standing in for its expected value.

Bradley and colleagues applied this to US estimates of first-dose COVID-19 vaccine uptake. In May 2021 the Delphi-Facebook survey, with about 250,000 responses a week, overestimated uptake by 17 percentage points against the official benchmark, while a panel of about 1,000 responses a week gave reliable estimates, and in April 2021 the larger survey's effective sample size was below 10. Kaplan and colleagues give a clinical parallel in the Nurses' Health Study, which followed 48,470 postmenopausal women and concluded that hormone replacement therapy nearly halved serious coronary heart disease, whereas the trial arm of the Women's Health Initiative suggested no reduction in risk.

Worked example with a large but selective cost dataset

All numbers in this example are illustrative. A modeller wants the mean annual NHS cost per person for 1,000,000 adults with type 2 diabetes in a region, to use as a health state cost. A linked dataset holds 200,000 of them, and people in frequent contact with services are slightly more likely to be included, giving a data defect correlation of 0.01. The standard deviation of annual cost is GBP 3,000 and the true population mean is GBP 2,500.

$$0.01 \times \sqrt{\frac{800000}{200000}} \times 3000 = 0.01 \times 2 \times 3000 = 60$$

where 0.01 is the data defect correlation, 800,000 is the number of people outside the dataset, 200,000 is the number inside it and 3,000 is the standard deviation in GBP. The dataset mean is therefore GBP 2,560, an overstatement of GBP 60 per person a year.

The usual standard error ignores this defect. With 200,000 records it is $\sigma_Y/\sqrt{n} = 3000/\sqrt{200000} \approx 6.71$, so the 95% confidence interval runs about GBP 13 either side of GBP 2,560 and excludes the true value by a wide margin. The effective sample size is 200,000 divided by 800,000, multiplied by 1 divided by 0.0001, which is 0.25 times 10,000, or 2,500. A simple random sample of that size would have a standard error of $\sigma_Y/\sqrt{2500} = 3000/50 = 60$, the same size as the bias.

Doubling the dataset with the same data defect correlation cuts the bias to GBP 36.7, but the effective sample size rises only to 6,667 while the reported interval narrows further. In a cost-effectiveness analysis, the narrow interval would pass into probabilistic sensitivity analysis and understate parameter uncertainty. A bias shared equally by both arms partly cancels in the incremental cost, but a bias in a state that one treatment keeps patients in for longer does not. A simple random sample of more than about 6,700 people would estimate the mean more accurately than even the 400,000 selectively collected records.

Quality, relevance and governance requirements

ECD9 sums up fitness for purpose as data quality, meaning the completeness and accuracy of key study variables, and data relevance, meaning content, differences from the NHS target population, sample size and length of follow-up. Since its September 2025 update it also asks for validation of methods that derive structured variables from unstructured data such as free-text notes. Its methods chapter allows machine learning to select covariates when the candidates are very many, provided the choice is justified and consistent with causal assumptions.

The HMA-EMA taskforce found that data quality largely determines the validity of evidence derived from a dataset, and recommended minimum quality standards, distributed data networks for sensitive data, and guidance on governance and anonymisation. In England, the Goldacre review of 2022 criticised sending copies of patient records to many sites and relying on pseudonymisation, and recommended a small number of shared trusted research environments for analysis of NHS records. Such rules, part of data governance, decide which datasets an HTA submission can use and how quickly.

Limitations and common misreadings

The commonest error is to treat the number of records as a measure of evidence. A large dataset narrows intervals and raises statistical power for rare outcomes and subgroups, but it leaves confounding, selection and misclassification untouched, and many variables invite multiple testing and overfitting.

A second error is to assume that a large dataset covers the population of interest. Coverage depends on who uses the service, which practices or insurers contribute and whose records link, and Kent and colleagues note that drugs dispensed in secondary care or over the counter, and patient-reported outcomes, are routinely missing from observational databases.

Sources

• Heads of Medicines Agencies and European Medicines Agency. HMA-EMA Joint Big Data Taskforce: summary report (EMA/105321/2019). Amsterdam: EMA; 13 February 2019. https://www.ema.europa.eu/en/documents/minutes/hma-ema-joint-task-force-big-data-summary-report_en.pdf
• Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Health Inf Sci Syst. 2014;2:3. https://doi.org/10.1186/2047-2501-2-3
• Collins B. Big data and health economics: strengths, weaknesses, opportunities and threats. PharmacoEconomics. 2016;34(2):101-106. https://doi.org/10.1007/s40273-015-0306-7
• Kent S, Burn E, Dawoud D, et al. Common problems, common data model solutions: evidence generation for health technology assessment. PharmacoEconomics. 2021;39(3):275-285. https://doi.org/10.1007/s40273-020-00981-9
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36), section 3.3. London: NICE; 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• National Institute for Health and Care Excellence. NICE real-world evidence framework (ECD9). London: NICE; 2022, updated September 2025. https://www.nice.org.uk/corporate/ecd9
• Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff. 2014;33(7):1123-1131. https://doi.org/10.1377/hlthaff.2014.0041
• Padula WV, Kreif N, Vanness DJ, et al. Machine learning methods in health economics and outcomes research: the PALISADE checklist. A good practices report of an ISPOR task force. Value Health. 2022;25(7):1063-1080. https://doi.org/10.1016/j.jval.2022.03.022
• Bradley VC, Kuriwaki S, Isakov M, Sejdinovic D, Meng XL, Flaxman S. Unrepresentative big surveys significantly overestimated US vaccine uptake. Nature. 2021;600(7890):695-700. https://doi.org/10.1038/s41586-021-04198-4
• Kaplan RM, Chambers DA, Glasgow RE. Big data and large sample size: a cautionary note on the potential for bias. Clin Transl Sci. 2014;7(4):342-346. https://doi.org/10.1111/cts.12178
• Goldacre B, Morley J, et al. Better, broader, safer: using health data for research and analysis. Independent report. London: Department of Health and Social Care; 7 April 2022. https://www.gov.uk/government/publications/better-broader-safer-using-health-data-for-research-and-analysis