Real-World Evidence

How real-world evidence is produced

Real-world evidence is generated by analysing data collected through routine healthcare, patient experience or other settings outside conventional controlled trials. This page explains how real-world data become evidence, how study design affects the conclusions that can be drawn, and how the resulting evidence can support health-economic and regulatory decisions.

Real-world evidence is not defined only by the source of the data. The credibility of a finding depends on whether the data are fit for the question and whether the analytical design addresses bias, confounding, missing information and differences between populations.

Distinguishing real-world data from real-world evidence

Real-world data are observations collected from sources such as electronic health records, claims, registries, patient-generated data and routine service systems. Real-world evidence is the clinical or policy-relevant knowledge produced by analysing those data using an explicit study design.

This distinction prevents a dataset from being treated as evidence before its relevance and limitations have been evaluated. The same dataset may support one question well and be unsuitable for another because required variables, follow-up or outcome definitions are missing.

Sources of real-world data

Real-world data sources differ in their purpose, coverage, clinical detail and reliability. A source created for billing may record resource use comprehensively but contain limited clinical information, while a disease registry may contain detailed outcomes for a selected group of patients.

Common sources include:

• Electronic health records.
• Administrative claims and billing records.
• Disease, treatment and product registries.
• Pharmacy dispensing and prescribing data.
• Laboratory and imaging systems.
• Patient-reported outcomes and patient-generated health data.
• Wearable devices and remote-monitoring systems.
• Public health surveillance systems.
• Social care and linked administrative datasets.
• Pragmatic trials embedded in routine care.

Linking sources can improve completeness but may introduce linkage error, inconsistent definitions and governance requirements. Analysts should explain which records were linked, how linkage was performed and which people or events may have been missed.

Defining the question before analysing the data

A real-world study should begin with a clearly specified question, population, treatment strategies, outcomes, follow-up period and analytical approach. Defining these elements before examining results reduces the risk of selecting methods that favour a desired finding.

Important elements include:

• The target population and eligibility criteria.
• The intervention, exposure or policy being evaluated.
• The relevant comparator.
• The start of follow-up.
• The outcomes and how they will be measured.
• The treatment-assignment or exposure strategy.
• The causal or descriptive quantity to be estimated.
• The handling of treatment changes, loss to follow-up and missing data.

For comparative effectiveness questions, framing the analysis as an attempt to emulate a target trial can expose problems such as unclear eligibility, inconsistent treatment assignment and immortal-time bias.

Determining whether data are fit for purpose

Fitness for purpose means that the data are sufficiently relevant and reliable for the intended use. It is a question-specific judgement rather than a permanent label attached to a database.

Relevance includes:

• Adequate representation of the target population.
• Availability of the intervention and comparator.
• Appropriate follow-up duration.
• Capture of the outcomes and important confounders.
• Sufficient sample size and number of events.
• Compatibility between the data period and current clinical practice.

Reliability includes:

• Clear and consistent variable definitions.
• Accurate capture of exposures and outcomes.
• Transparent data cleaning and transformation.
• Stable data provenance.
• Evidence about completeness and missingness.
• Reproducible linkage and analytical procedures.

A large dataset is not necessarily fit for purpose. Increasing the number of observations can improve precision without correcting systematic bias.

Descriptive and causal uses

Real-world evidence can describe treatment patterns, resource use, adherence, disease burden and outcomes in routine practice. These descriptive questions do not necessarily require a causal interpretation.

Causal questions ask what would have happened under an alternative intervention or exposure. Because treatment is not usually assigned randomly in routine data, patients receiving different treatments may differ in ways that also affect their outcomes.

The study should state whether it estimates an association, predicts an outcome or attempts to estimate a causal effect. These purposes require different assumptions and validation methods.

Confounding and treatment selection

Confounding occurs when a factor influences both treatment selection and the outcome. In routine care, clinicians may select treatments according to disease severity, comorbidities, prior response, access, patient preference or prognosis.

Methods used to address measured confounding include:

• Multivariable outcome regression.
• Matching.
• Stratification.
• Standardisation.
• Propensity-score methods.
• Inverse-probability weighting.
• Instrumental-variable analysis when a defensible instrument exists.

These methods cannot automatically remove bias from variables that were unmeasured, poorly recorded or incorrectly specified. A well-balanced set of measured characteristics does not prove that unmeasured confounding has been eliminated.

Time-related biases

The definition of time zero should align eligibility, treatment assignment and the beginning of outcome follow-up. Misalignment can create periods during which an outcome could not have been observed or attributed consistently.

Important time-related problems include:

• Immortal-time bias.
• Time-varying confounding.
• Changes in treatment over time.
• Informative censoring.
• Incomplete follow-up.
• Changes in coding, clinical practice or data capture across calendar periods.

The analysis should explain how treatment switching, discontinuation and loss to follow-up are handled. Different strategies may answer different questions, such as the effect of initiating treatment or the effect of remaining on treatment.

Measuring exposures and outcomes

Routine data may use codes, prescriptions, laboratory values or algorithms as proxies for clinical events. Each definition should be validated or justified for the data source and population.

Measurement error can occur when:

• A prescription is recorded but the medicine is not taken.
• An outcome occurs outside the observed healthcare system.
• A diagnosis code reflects rule-out testing rather than confirmed disease.
• Clinical severity is recorded inconsistently.
• Changes in coding practice alter apparent event rates.

Misclassification can bias results even when it affects both comparison groups. Sensitivity analyses should test plausible alternative definitions when measurement uncertainty could change the conclusion.

Missing data and incomplete observation

Missing data may reflect how care is delivered rather than random record loss. A test result may be missing because a clinician did not consider the test necessary, because the patient could not access care or because care occurred in another system.

Analysts should report the amount and pattern of missingness, explain the assumptions used and examine whether conclusions change under plausible alternatives. Treating every missing value as normal or carrying forward an old observation can create misleading evidence.

Transparency and reproducibility

A credible real-world study should preserve a traceable path from the research question to the final result. Protocols, analysis plans, data definitions and code lists should be specified before outcome-driven analytical choices are made whenever possible.

Transparent reporting includes:

1. Define the study population and show how records were selected.
2. Describe the data source and its original purpose.
3. Specify exposures, comparators and outcomes using reproducible definitions.
4. State the assumptions required for the intended interpretation.
5. Document data transformations and linkage from source records to analysis variables.
6. Report missing data and exclusions for each stage of the analysis.
7. Present sensitivity and negative-control analyses when they can test important biases.
8. Separate prespecified analyses from exploratory findings.

Privacy or licensing restrictions may prevent public release of patient-level data. They do not remove the need to describe methods sufficiently for independent evaluation.

Using real-world evidence in health economics

Real-world evidence can inform economic evaluations by describing routine practice, patient characteristics, treatment sequences, adherence, resource use, costs and longer-term outcomes. It may also help assess whether trial results are transferable to the population facing the decision.

Potential model inputs include:

• Baseline event rates.
• Disease progression.
• Treatment adherence and persistence.
• Treatment switching.
• Resource use.
• Adverse events.
• Healthcare costs.
• Health-related quality of life.
• Long-term survival.
• Variation between patient subgroups.

Observed associations should not be inserted into a model as treatment effects without examining whether a causal interpretation is justified. Bias in an input can propagate through the model and create precise but misleading estimates of cost effectiveness.

Generalisability and transportability

Real-world evidence may describe routine care more directly than a tightly controlled trial, but it is not automatically representative of every patient or setting. Database coverage, insurance status, healthcare access, treatment availability and recording practices affect who appears in the data.

Transporting results to another population or jurisdiction requires examination of:

• Differences in patient characteristics.
• Differences in clinical pathways and comparators.
• Differences in prices and resource use.
• Differences in data capture.
• Differences in treatment availability and adherence.
• Differences in healthcare-system organisation.

Reweighting or standardisation may improve population alignment when the necessary characteristics are measured. It cannot correct for important differences that are absent from the data.

A simplified example

Suppose a registry is used to compare hospitalisation after two treatments. Patients receiving Treatment A are younger and have fewer comorbidities than patients receiving Treatment B.

The unadjusted hospitalisation rate is lower for Treatment A, but the difference may reflect patient selection rather than treatment effectiveness. The analysis should define eligibility and follow-up consistently, adjust for measured prognostic factors, test alternative specifications and examine whether unmeasured confounding could plausibly explain the result.

If the adjusted estimate remains favourable, it provides stronger evidence than the unadjusted comparison but still depends on the adequacy of the data and assumptions. The study should not be described as equivalent to randomisation.

Real-world evidence and controlled trials

Real-world evidence and randomised controlled trials answer overlapping but distinct questions. Randomisation can protect causal comparisons from measured and unmeasured baseline confounding, while routine data may better represent broader populations, longer follow-up and everyday practice.

Neither evidence source is universally superior. Trials may have limited external validity or follow-up, while observational real-world studies may be vulnerable to confounding and measurement error. Decision makers should examine how the sources complement or contradict one another.

Pragmatic trials can combine random treatment assignment with delivery and outcome collection in routine-care settings. They should not be classified solely by the use of routine data.

Common misunderstandings

Real-world evidence does not mean uncontrolled anecdotal experience. It is produced through a defined study design and analysis of systematically collected data.

Common misunderstandings include:

• Real-world data are not automatically real-world evidence.
• A large sample does not remove confounding or measurement bias.
• Statistical adjustment cannot guarantee comparability between treatment groups.
• Routine data are not automatically representative of the target population.
• An association is not automatically a causal treatment effect.
• Data collected outside a conventional trial are not necessarily low quality.
• Real-world evidence does not always conflict with trial evidence.
• Using advanced analytical methods does not compensate for missing essential variables or unsuitable data.

Interpreting real-world evidence

The interpretation should match the question, data and assumptions. Descriptive evidence can be valuable without supporting a causal claim, and a causal estimate should be presented with the conditions required for it to be valid.

High-quality real-world evidence makes limitations visible and tests whether reasonable alternative choices change the result. Its value lies in producing decision-relevant knowledge from routine data without claiming more certainty than the design can support.