Core Outcome Set

Core Outcome Set in Trials and Evidence Synthesis

Trials of the same treatment often measure different things, so a reviewer who wants to pool them finds that one trial reported pain, another function, a third hospital admissions and only a few reported health-related quality of life. A core outcome set responds to that problem by fixing, for each condition, a short list of outcomes that trialists commit to collecting and publishing alongside anything else they choose to study. For a health economist the value lies downstream, because a meta-analysis or network meta-analysis that feeds a cost-utility model can only use trials that report the outcome the model needs. The COMET Initiative's handbook, published by Williamson and colleagues in 2017 as version 1.0, sets out how such sets are developed, used and updated, and NICE's guideline manual now tells reviewers to use a suitable core outcome set when one exists.

What a core outcome set fixes and what it leaves open

The COMET Handbook describes a core outcome set as the minimum that should be measured and reported in all clinical trials of a specific condition, and adds that such a set could also be suitable for other types of research and for clinical audit. The word minimum carries the weight. The Handbook expects core outcomes always to be collected and reported, and anticipates that researchers will include other outcomes as well, so a trial that measures only the core set is not the aim.

COMET separates two questions. A fully specified outcome, in the Handbook's terms, has a domain, which is what to measure (anxiety, for example), and a specific measurement, which is how to measure it (the Hamilton Anxiety Rating Scale, in the Handbook's example). A core outcome set agrees the first. The choice of instrument is a later step, and the COS-STAP statement on protocols for core outcome set studies says plainly that it does not cover plans for how outcomes should be defined or measured.

The second step has its own guidance. COMET and COSMIN, the group behind the COSMIN standards for the measurement properties of health instruments, published a joint guideline in 2016, reported by Prinsen and colleagues, on how to select outcome measurement instruments for the outcomes in a core outcome set. Its steps begin with conceptual considerations, then a search for existing instruments by systematic review or literature search, then an assessment of their quality.

The distinction matters to health economists because a core outcome set that names health-related quality of life does not by itself say whether trials should use a preference-based instrument. Two trials can both comply with a set that lists quality of life and still be impossible to combine in a utility analysis, if one used a disease-specific questionnaire and the other a generic measure.

Outcome reporting bias and comparable evidence

The case for core outcome sets rests partly on evidence that trials report outcomes selectively. Kirkham and colleagues examined 283 Cochrane reviews in the ORBIT study, published in 2010. They found that 157 reviews (55 per cent) did not have complete data on the review's primary outcome from all eligible trials, and that 96 reviews (34 per cent) included at least one trial with a high suspicion of outcome reporting bias for the primary outcome.

Outcome reporting bias distorts a pooled estimate when the trials that report an outcome are those with favourable results. A core outcome set addresses the problem by fixing in advance a list of outcomes that every trial is expected to report, so that a missing core outcome becomes visible to reviewers and editors. The COMET Handbook's background chapter says selective reporting prevents fully informed decisions about patient care, resource allocation and research priorities, which is the economic version of the same argument.

Comparability is the other half of the argument. Evidence synthesis for a health technology assessment usually needs the same outcome across several trials, and a network meta-analysis needs it across every trial that links the treatments in the network. When trials in a condition measure different outcomes, some comparisons cannot be made at all and others rest on a thin subset of the evidence. A core outcome set cannot remove between-study heterogeneity in populations or settings, but it reduces the heterogeneity created by the choice of outcome.

Resource use and quality of life in core outcome sets

Most core outcome sets are written for clinical effectiveness, and the outcomes chosen reflect what patients and clinicians judge most important. Whether a set includes health-related quality of life depends on that judgement, and including it as a domain still leaves the instrument open. Economic evaluation needs more than effectiveness outcomes, since a cost-utility analysis also needs resource use and preference-based utility values.

Some groups have tried to apply the core outcome set idea to economic data. Hounsome and colleagues published a protocol in 2017 for developing core economic outcome sets for asthma studies, covering economic outcomes such as costs, resource use or quality-adjusted life years. The review that followed, reported by Roukas and colleagues in 2020, screened 2,531 publications, included 224 and identified 65 economic parameters in 11 groups. It proposed that some parameters, including informal care, medical devices and health utility, apply to particular types of study and could be recommended as supplemental parameters, and it named the lack of stakeholder involvement as its main limitation, to be addressed in a later Delphi stage.

A related Delphi study of health economists, reported by Thorn and colleagues in 2018, agreed 10 core items for a standardised resource use measure, which would play the same role for costs that a core outcome set plays for clinical outcomes.

Standards for judging a core outcome set

A core outcome set is only as credible as the process that produced it, and COMET has published three sets of standards for that process. COS-STAD, reported by Kirkham and colleagues in 2017, gives 11 minimum standards for the design of a core outcome set study, grouped under scope, stakeholders and the consensus process. COS-STAR, published in 2016, is an 18-item checklist for reporting a completed study, and COS-STAP, published in 2019, is a 13-item checklist for the study protocol.

These standards let a reviewer or guideline developer judge whether a published set deserves to be used. The stages of a development study, the stakeholder groups involved and the consensus rules applied are covered under core outcome set development.

Uptake by research funders and NICE

A core outcome set changes practice only if trialists search for one and use it. The NIHR Health Technology Assessment programme added a recommendation to search for a core outcome set to its applicant guidance in January 2012. Hughes and colleagues reviewed 95 researcher-led primary research applications submitted between then and December 2015 and found that 9 applicants (10 per cent) stated that they had searched the COMET database, while another 9 referred to searching by another method, such as a literature review. The authors concluded that funders can influence uptake and could go further by asking applicants to be explicit about the search.

NICE's guideline manual, "Developing NICE guidelines: the manual" (PMG20), addresses core outcome sets in chapter 4, in its guidance on outcomes for review questions about the effectiveness of interventions. The manual says core outcome sets should be used if suitable based on quality and validity, names the COMET database as one source, and says that COS-STAD and COS-STAR should be used to assess the suitability of an identified set. The chapter was last updated in October 2025. The guidance applies to the evidence reviews behind NICE guidelines.

Worked example with sixteen trials and a utility outcome (illustrative)

All numbers in this example are illustrative. Sixteen randomised controlled trials of a treatment for a chronic condition each randomise 50 participants per arm. Without a core outcome set, only 4 of them report a utility outcome. A core outcome set that includes health-related quality of life, together with an agreed preference-based instrument, would bring all 16 into the synthesis. The utility difference has a standard deviation of 0.2 in every trial, and pooling is treated as one large trial for simplicity.

$$SE = \sigma \sqrt{\frac{2}{n}}$$

where $SE$ is the standard error of the pooled difference in mean utility, $\sigma$ is the standard deviation of utility in each arm, and $n$ is the number of participants per arm across the pooled trials.

Step 1. Without a core outcome set. The pooled arm size is 4 × 50 = 200, so the square root of 2 divided by 200 is 0.1 and the standard error is 0.2 × 0.1 = 0.02.

Step 2. With a core outcome set. The pooled arm size is 16 × 50 = 800, the square root of 2 divided by 800 is 0.05, and the standard error is 0.2 × 0.05 = 0.01.

Step 3. Confidence intervals. The pooled difference in utility is 0.03 in both cases. The half-width of the 95 per cent confidence interval is 1.96 × 0.02 = 0.0392 with four trials, giving an interval of minus 0.0092 to 0.0692, and 1.96 × 0.01 = 0.0196 with sixteen trials, giving 0.0104 to 0.0496.

Step 4. QALYs. If the difference lasts 2 years and discounting is ignored, the QALY gain is 0.03 × 2 = 0.06. With sixteen trials the interval for the QALY gain is 0.0208 to 0.0992, since 0.0104 × 2 = 0.0208 and 0.0496 × 2 = 0.0992.

Interpretation. With four trials the interval for the utility difference includes zero, and a probabilistic model would give real weight to no benefit at all. With sixteen trials the interval excludes zero and is half as wide, which narrows the uncertainty around the cost per QALY and lowers the expected value of information from further research. The example understates the gain if the four reporting trials were the ones with favourable results, because the pooled 0.03 would then also be biased upwards.

Core outcome sets beside outcome measures, endpoints and standard sets

A core outcome set is often confused with neighbouring ideas. An outcome measure is a specified instrument or data rule for quantifying an outcome, which is the how that COMET deliberately leaves to a second step. A measurement property such as reliability or construct validity describes how well an instrument performs, and COSMIN standards for those properties guide the choice of instrument once the core outcomes are fixed.

A primary endpoint is chosen by one trial to answer its main question and drive its sample size, and it need not be a core outcome. A core outcome set spans every trial in a condition and says nothing about which outcome a given trial should power on. A composite endpoint combines events into one outcome, while a core outcome set lists separate outcomes.

Standard sets for routine care, such as the ICHOM standard sets, apply a similar logic to clinical practice. They are described under ICHOM Standard Set. A core outcome set for research and a standard set for routine care in the same condition can differ, because the purposes differ.

Common misreadings of core outcome sets

The most frequent misreading treats a core outcome set as the full list of outcomes a trial should measure. The COMET Handbook frames it as a minimum, and a trial designed for an economic evaluation will still need resource use and preference-based quality of life data whether or not the set includes them.

A second misreading assumes that compliance with a core outcome set makes results poolable. Two trials can report the same core outcome with different instruments, time points or definitions, and the synthesis problem then reappears at the level of measurement. A third assumes that any published set is fit for use, when NICE's manual asks reviewers to judge suitability on quality and validity, using COS-STAD and COS-STAR. A fourth reads low uptake as a sign that core outcome sets have failed. The NIHR study shows that many trialists did not search for a set at all, which is a problem of awareness and process that funders and journals can act on.

Sources

• Hounsome N, Fitzsimmons D, Phillips C, Patel A. Developing core economic outcome sets for asthma studies: a protocol for a systematic review. BMJ Open. 2017;7(8):e017054. https://doi.org/10.1136/bmjopen-2017-017054
• Hughes KL, Kirkham JJ, Clarke M, Williamson PR. Assessing the impact of a research funder's recommendation to consider core outcome sets. PLoS One. 2019;14(9):e0222418. https://doi.org/10.1371/journal.pone.0222418
• Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365. https://pubmed.ncbi.nlm.nih.gov/20156912/
• Kirkham JJ, et al. Core Outcome Set-STAndards for Reporting: the COS-STAR Statement. PLoS Medicine. 2016;13(10):e1002148. https://doi.org/10.1371/journal.pmed.1002148
• Kirkham JJ, Davis K, Altman DG, et al. Core Outcome Set-STAndards for Development: the COS-STAD recommendations. PLoS Medicine. 2017;14(11):e1002447. https://doi.org/10.1371/journal.pmed.1002447
• Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Tunis S, Williamson PR. Core Outcome Set-STAndardised Protocol Items: the COS-STAP Statement. Trials. 2019;20(1):116. https://pubmed.ncbi.nlm.nih.gov/30744706/
• National Institute for Health and Care Excellence. Developing NICE guidelines: the manual (PMG20). Chapter 4, Developing review questions and planning the evidence review. Last updated 23 October 2025. https://www.nice.org.uk/process/pmg20/chapter/developing-review-questions-and-planning-the-evidence-review
• Prinsen CAC, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, Williamson PR, Terwee CB. How to select outcome measurement instruments for outcomes included in a "Core Outcome Set", a practical guideline. Trials. 2016;17(1):449. https://doi.org/10.1186/s13063-016-1555-2
• Roukas C, Quayyum Z, Patel A, Fitzsimmons D, Phillips C, Hounsome N. Developing core economic parameter sets for asthma studies: a realist review and an analytical framework. BMJ Open. 2020;10(10):e037889. https://doi.org/10.1136/bmjopen-2020-037889
• Thorn JC, et al. Core items for a standardized resource use measure: expert Delphi consensus survey. Value in Health. 2018;21(6):640-649. https://pmc.ncbi.nlm.nih.gov/articles/PMC6021557/
• Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. https://doi.org/10.1186/s13063-017-1978-4