Effectiveness

Effectiveness

Effectiveness describes the benefit and harm an intervention produces when delivered under conditions relevant to actual practice. It depends on the people offered it, the comparator, how care is delivered and which outcome is measured; there is no context-free effectiveness number. This page connects the clinical question to a credible comparison, then explains how to interpret an effect estimate and use it in a health-economic decision.

Effectiveness needs a population, comparator and outcome

“Does it work?” is incomplete until the target population, intervention, alternative, outcome and time horizon are specified. A medicine may reduce events against no treatment but add little against an effective existing medicine; a service may work at one staffing level and not another. Benefits and harms should both be measured, ideally in outcomes patients value.

Question component	Example for a follow-up service	Why it matters
Population	Adults with a defined condition receiving routine care.	Eligibility and baseline risk affect the result.
Intervention	A specified service with actual staffing and uptake.	Delivery changes what patients receive.
Comparator	Current follow-up in the same setting.	The incremental effect is relative to an alternative.
Outcome	Hospital admission, quality of life and harms.	A process count alone may miss health effects.
Horizon	Six months for admissions and longer for durability.	Effects can emerge, wane or cause delayed harm.

Effectiveness is often contrasted with efficacy under more controlled conditions. That contrast is a continuum of study design and delivery, not a rule that randomised trials measure only efficacy or observational studies automatically measure effectiveness. A pragmatic randomised trial can estimate an effect in routine care; an observational dataset still needs a defensible causal design.

Separate the intervention's offer, use and effect

Offering an intervention to eligible people differs from receiving it as intended. Reach, adherence, clinician skill, capacity and patient preferences can determine the outcome achieved in practice. An analysis must state whether it estimates the effect of offering a service, assigning treatment, initiating treatment or sustained use; these are different questions and may have different biases.

For example, comparing only people who faithfully used a service with people who did not may select people with different severity, motivation or access. A low uptake rate may reduce the effect of offering the programme without proving the intervention lacks benefit among suitable recipients. Record both implementation and patient outcomes, rather than silently replacing the latter with the former.

Read an absolute and relative effect together

Consider an original fictional comparison with 1,000 people in each group followed for six months. An admission occurs in 100 people offered a new service and 150 receiving current care. The observed risks are $100/1{,}000=0.10$ and $150/1{,}000=0.15$; the risk difference, new service minus current care, is $0.10-0.15=-0.05$, or five fewer people admitted per 100 over six months.

The risk ratio is $0.10/0.15\approx0.67$, a relative reduction of about 33% if the difference is causal. Under an illustrative independent-binomial calculation, the standard error of the risk difference is $\sqrt{0.10(0.90)/1{,}000+0.15(0.85)/1{,}000}\approx0.01475$. A simple normal 95% interval is approximately $-0.05\pm1.96(0.01475)$, or $[-0.079,-0.021]$; actual analyses must follow their design and account for clustering, censoring or other complications.

Spreadsheet item	Illustrative formula	Result
New-service risk	=100/1000	10% over six months.
Current-care risk	=150/1000	15% over six months.
Risk difference	=100/1000-150/1000	-5 percentage points for service minus care.
Risk ratio	=(100/1000)/(150/1000)	About 0.67.
Approximate standard error	=SQRT(0.10*0.90/1000+0.15*0.85/1000)	About 0.01475 under stated assumptions.

These calculations describe the observed groups. They estimate a causal effectiveness effect only if the comparison and follow-up support that interpretation. If people with lower baseline admission risk preferentially entered the new service, a narrow interval around the observed difference would not correct that bias.

Decide whether the comparison is credible

Random allocation, when feasible and preserved, helps make groups comparable at baseline; pragmatic design can make its delivery and participants more representative of practice. In non-randomised studies, define eligibility, assignment and time zero carefully and address confounding, missing data, outcome ascertainment and competing care. In either design, examine protocol deviations, crossover and loss to follow-up.

Generalisability asks whether the estimate applies to the people and system considering adoption. Baseline risk, severity, co-treatment, workforce, geography and time may all differ. Report effect modification when supported by evidence, without treating every subgroup contrast as a proven distinct effect.

Threat	How it distorts interpretation	Check
Confounding	Groups have different risk before treatment.	Compare baseline factors and design an appropriate analysis.
Immortal time	Exposed people must survive or remain event-free before classification.	Align eligibility, assignment and follow-up start.
Differential capture	Outcomes outside one data source are missed.	Compare observation windows and linked data coverage.
Missing outcomes	Loss to follow-up differs across groups or prognosis.	Report missingness and test plausible assumptions.
Implementation change	A later service version differs from the studied one.	Document delivery, uptake and setting.

No checklist turns weak evidence into certainty. A transparent estimate states what can be inferred, for whom and under which assumptions, along with the remaining uncertainty.

Use effectiveness in an economic evaluation

Effectiveness supplies consequences of the alternative strategies, while economic evaluation also needs resource use, prices, perspective and a relevant horizon. In the fictional groups, five fewer people admitted per 100 is not automatically five fewer admission episodes, nor does it establish a saving without the cost of admissions and the service. Additional quality-of-life or safety effects may change the decision.

A model may transport an effect from a study to a local population, but should justify how baseline risk, relative effect and implementation change. If the service costs £200 per eligible person, its programme cost for 1,000 people is $1{,}000\times £200=£200{,}000$. Even if the observed 50-person admission difference applied locally, the incremental resource result would require appropriate admission costs, other care and the uncertainty in that difference.

Efficiency and cost effectiveness are different judgments: an effective intervention can consume substantial resources, and a cheaper intervention can still harm patients. Report incremental outcomes and costs separately before applying a decision rule.

Common interpretation errors

Effectiveness is a relationship among a specified intervention, comparator, population and outcome. It can vary with implementation and time, so a single headline percentage should not stand in for the evidence. These checks keep the result useful to a reader or decision maker.

• State the comparator: A benefit relative to no care may not persist relative to best current care.
• Label the scale: Absolute and relative changes convey different information and depend on baseline risk.
• Separate association and causation: An adjusted observational estimate still relies on assumptions.
• Report harms: A favourable primary outcome may coexist with adverse effects or burdens.
• Check delivery: Reach and adherence affect the benefit of offering an intervention in practice.
• Use the correct horizon: A six-month estimate does not establish lifetime effectiveness.

Sources and further reading

The PCORI explanation of comparative clinical effectiveness research frames decisions among care alternatives. The NICE real-world evidence framework and its methods for comparative effects discuss study design and credible inference. The Cochrane Handbook chapter on non-randomised intervention studies addresses bias when estimating effects without random allocation. All counts and costs above are original teaching assumptions, not measured service outcomes.