Evidence Synthesis

Evidence Synthesis

Evidence synthesis brings findings from multiple studies together to answer a defined question while accounting for differences in their methods, populations and credibility. It may be a structured narrative or a statistical analysis; a pooled number is not required or always appropriate. This page follows a synthesis from question and study selection through comparison, uncertainty and use in health-economic decisions.

Decide what evidence belongs together

Start with the population, interventions or exposures, comparator, outcomes, study designs and time horizon. A planned review should specify how studies will be found, screened, appraised and combined. Without an explicit question and reproducible search, an apparently persuasive summary may reflect selective citation rather than the available evidence.

Step	Main task	Safeguard
Protocol	Define eligibility, outcomes and planned synthesis.	Identify choices before results influence them.
Search and selection	Find studies in appropriate sources and screen against criteria.	Record exclusions and duplicate reports.
Extraction	Capture populations, interventions, outcomes and effect estimates.	Check units, time horizons and direction of benefit.
Appraisal	Examine bias within each study and missing evidence across studies.	Do not treat all estimates as equally credible.
Synthesis	Compare, narrate or pool compatible results.	Explain heterogeneity and assumptions.
Interpretation	Assess certainty and applicability to the decision.	State what is known and what remains uncertain.

A systematic review is a structured way to conduct evidence synthesis with explicit, reproducible methods. Meta-analysis is a statistical method that can be part of such a review, not a synonym for the entire process. A scoping review maps an evidence field and may answer a different question from a comparative-effectiveness synthesis.

Check comparability before pooling

Effect estimates need a common interpretation before they are combined. A six-month admission risk difference and an annual admission rate ratio are not the same quantity. Differences in eligibility, comparator, outcome definitions, intervention versions and follow-up can make an arithmetic average meaningless even when the studies use identical labels.

Some variation is clinical or methodological; some reflects sampling uncertainty. Fixed-effect and random-effects meta-analytic models embody different assumptions about the effects being estimated and the variation among studies. A random-effects model does not repair bias or make incompatible interventions comparable. When pooling is inappropriate, structured tables and a transparent narrative can still be rigorous synthesis.

Worked example: pooling comparable risk differences

Consider two fictional independent studies estimating the same six-month risk difference, intervention minus comparator, in sufficiently similar settings. Study A estimates $-0.04$ with standard error $0.02$; study B estimates $-0.01$ with standard error $0.03$. Under a simple fixed-effect inverse-variance calculation, weights are $w_A=1/(0.02)^2=2{,}500$ and $w_B=1/(0.03)^2\approx1{,}111.11$.

The weighted estimate is $(-0.04\times2{,}500-0.01\times1{,}111.11)/(2{,}500+1{,}111.11)\approx-0.03077$. Its model-based standard error is $\sqrt{1/(2{,}500+1{,}111.11)}\approx0.01664$, giving an illustrative normal 95% interval of approximately $-0.03077\pm1.96(0.01664)$, or $[-0.0634,0.0018]$. It includes no difference and does not account for between-study heterogeneity or systematic bias.

Item	Illustrative spreadsheet formula	Result
Weight for study A	=1/0.02^2	2500.
Weight for study B	=1/0.03^2	About 1111.11.
Weighted risk difference	=(-0.04*2500-0.01*(1/0.03^2))/(2500+1/0.03^2)	About -0.03077.
Model-based standard error	=SQRT(1/(2500+1/0.03^2))	About 0.01664.

The larger study weight here reflects smaller reported standard error, not proof of lower bias. With only two studies, a between-study variance estimate and heterogeneity judgments can be especially unstable. Real reviews use appropriate software and a prespecified method, check dependence among multiple reports of the same participants, and show the separate results alongside any pooled estimate.

Interpret heterogeneity and missing evidence

When effects differ, investigate whether a change in population, dose, delivery, baseline risk or outcome definition plausibly explains it. Statistical heterogeneity measures can help describe dispersion but do not substitute for clinical judgment. Avoid treating a subgroup found after inspecting results as a confirmed explanation without appropriate testing and external support.

Unpublished studies or unreported outcomes can distort the visible body of evidence. Search trial registries and other relevant sources where appropriate, and compare reported outcomes with protocols when possible. A small number of studies may offer little power to detect publication bias; absence of a signal does not prove absence of missing evidence.

Certainty should be judged for each important outcome, considering risk of bias, inconsistency, indirectness, imprecision and publication bias. A review may have credible short-term process outcomes but weak evidence on long-term patient benefit. Report absolute effects where possible and make clear when no reliable estimate is available.

Use synthesised evidence in health economics

Economic models often need comparative treatment effects, baseline risks, adverse-event rates and durations that come from several sources. Record which estimate came from which synthesis, the target population and the assumptions used to translate it into model inputs. Preserve correlations and uncertainty when multiple parameters are estimated from the same evidence.

For example, applying the illustrative pooled risk difference of about $-0.03077$ to 1,000 comparable people suggests about 31 fewer people with an event over six months, conditional on transportability. The interval spans a sizeable reduction and a small increase, so treating 31 as a certain benefit would be misleading. The model also needs costs, harms and any longer-term effect evidence before drawing a value-for-money conclusion.

Network meta-analysis may connect more than two alternatives through direct and indirect comparisons when its assumptions, including transitivity and consistency, are credible. It is not an automatic solution for a disconnected or clinically incoherent evidence network. Scenario analysis should examine alternative effect estimates and structural choices when the synthesis cannot settle them.

Common mistakes to avoid

Evidence synthesis is more than collecting quotations and more than running a pooling command. Its value comes from transparent selection, careful comparison and honest appraisal of uncertainty. These checks make a review useful for teaching and decision making.

• Use an explicit protocol: Document inclusion criteria and planned comparisons before results are selected.
• Check duplicate populations: Multiple publications can describe the same participants or dataset.
• Align estimands and units: Combine effects only when their meanings and follow-up are compatible.
• Keep bias separate from precision: A narrow confidence interval can surround a systematically wrong estimate.
• Explain heterogeneity: A pooled average may conceal a decision-relevant difference across settings.
• Report all important outcomes: Benefits, harms and resource consequences may have different evidence bases.

Sources and further reading

The Cochrane Handbook chapter on preparing for synthesis defines synthesis and study comparison, while its meta-analysis chapter describes statistical pooling. The PRISMA 2020 statement is reporting guidance, not proof that a review's methods are sound. The Cochrane certainty chapter explains outcome-specific appraisal. The two-study numerical example is original teaching material, not a published meta-analysis.