Secondary Research

Secondary Research in Health Technology Assessment

Secondary research works with evidence that already exists instead of recruiting participants or collecting new measurements. In health technology assessment most of the analytical work is of this kind, since a typical appraisal gathers trials in a systematic review, pools them, and then combines the pooled effect with baseline risks, costs and utilities taken from other published or routine sources inside a decision model. The analyst controls none of the data collection, so the result rests on how well the evidence was found and appraised and on whether it applies to the population and health system making the decision.

How medical research is divided into primary and secondary

Röhrig and colleagues classify medical research into primary and secondary research. Secondary research summarises available studies in the form of reviews and meta-analyses, while the studies themselves are carried out in primary research, which they divide into basic, clinical and epidemiological research. The Cochrane Handbook presents systematic reviews as a way of systematically assessing primary research that no individual decision maker could read alone.

The dividing line is the relationship between the analyst and the data. A secondary study answers its question from results that someone else produced, usually for another purpose, and inherits their strengths and weaknesses. The Cochrane Handbook adds that summaries and syntheses of research need methods to avoid bias just as primary studies do.

Where re-analysis of existing data sits

The boundary is less tidy for re-analysis of individual-level data. Cheng and Phillips, following the usage of the United States National Institutes of Health, limit primary data analysis to analysis by the team that collected the data, answering the hypotheses the study was designed to test. Every other analysis of data collected for research or for other purposes, including registry data, counts as secondary analysis of existing data. Cheng and Phillips classify analyses rather than research as a whole, and apply the label even when the analysts helped to collect the data. Röhrig and colleagues list secondary data analyses among non-interventional clinical studies, which places them inside primary research, so whether a registry re-analysis counts as secondary research depends on the classification followed.

The main forms of secondary research in health economics

Several kinds of work in an appraisal re-use existing material, though only reviews and meta-analyses fall within every definition of secondary research. The table lists those most often met in health economics.

The NICE technology appraisal manual (PMG36) asks for evidence on outcomes to come from a systematic review, defined as systematically locating, including, appraising and synthesising the evidence for a clearly formulated question. It adds that the data from included studies can be synthesised but that this is not required, so a review without a meta-analysis is still secondary research. Where technologies have not been compared within a single randomised trial, the manual asks for pairwise trial data to be presented with a network meta-analysis if appropriate. It also asks for a review of published, relevant economic evaluations alongside any new analysis. For outcomes such as natural history and treatment patterns the manual says real-world evidence may be preferred, which usually means re-using real-world data from registries and records.

Why most health technology assessment is secondary research

Sculpher and colleagues argue that an economic analysis for decision making must estimate costs and effects from all the available evidence, for all relevant alternatives and patient groups over an appropriate time horizon. They conclude that a single randomised controlled trial will in most circumstances be an inadequate and partial basis for that decision, and that trial evidence should be treated as one source placed within a broader framework of evidence synthesis and decision analysis. Buxton and colleagues had earlier called modelling an unavoidable fact of life in economic evaluation, while discussing pragmatic trials as the alternative.

The manual states that modelling provides a framework for synthesising available evidence and generating estimates of clinical and cost effectiveness in a form relevant to the committee, and that models are needed for most evaluations. It notes that trial data may not quantify the risk of some events for the population of interest well enough, or may not cover a long enough period, and asks for the choice of each data set used in a model to be justified by its suitability to that population. A health economic model built for an appraisal therefore draws most of its inputs from secondary research.

How a secondary research project in an appraisal is carried out

The steps follow the order in which evidence enters a cost-effectiveness estimate. The NICE manual sets the expectations for a UK appraisal.

1. Define the question. The decision problem fixes the population, comparators, outcomes and setting.
2. Write a protocol and search. PMG36 asks for a systematic review done to a pre-defined protocol, with attempts to find evidence that is not in the public domain, such as unpublished trial data and trials reported only as abstracts.
3. Appraise each study. The manual asks for critical appraisal of each included study and for the risk of bias in non-randomised studies to be assessed with a validated tool, naming ROBINS-I as an example.
4. Synthesise. Pooling should come with an assessment of heterogeneity, which a random-effects model partly accounts for, and its reasons should be explored.
5. Model and test. The manual asks for full documentation and justification of structural assumptions and data inputs, with sensitivity analyses where plausible alternatives exist.

Worked example of combining a pooled effect with a local baseline risk

This example is illustrative, with invented numbers. A meta-analysis of trials gives a pooled relative risk of hospital admission of 0.80 for a new treatment over five years. The control arms of the trials had a five-year event risk of 10%, but a national registry shows that the decision population has a baseline risk of 15%. The model applies the pooled relative effect to the local baseline.

$$ARR = p_0 \times (1 - RR)$$

where $ARR$ is the absolute risk reduction, $p_0$ is the baseline risk without treatment and $RR$ is the pooled relative risk.

With the trial baseline, $ARR = 0.10 \times 0.20 = 0.02$, or 20 events avoided per 1,000 people treated. With the registry baseline, $ARR = 0.15 \times 0.20 = 0.03$, or 30 events avoided per 1,000. If each event costs GBP 8,000 and treatment costs GBP 400 per person, the net cost per 1,000 treated is GBP 400,000 minus 20 times GBP 8,000, or GBP 240,000, with the trial baseline, and GBP 400,000 minus 30 times GBP 8,000, or GBP 160,000, with the registry baseline.

The localised baseline cuts the net cost by GBP 80,000, one third of GBP 240,000. The calculation relies on the relative risk being constant across baseline risks, and PMG36 asks analysts to check whether the relative effect holds when the model's baseline event rate differs substantially from the rates in the meta-analysed studies.

Quality of the inputs and of the process

Two layers of quality apply. The first is the quality of the re-used studies, which PMG36 describes as internal validity, from each trial's design, conduct and analysis, and external validity, whether results apply to wider patient groups and routine practice. The second is the quality of the secondary work itself. The Cochrane Handbook notes that threats to the validity of a systematic review can come from the studies it includes or from the way the review is conducted, and warns that restricting searches by publication status or language risks publication bias or language bias.

Appraisal tools exist for each layer. Shea and colleagues developed AMSTAR 2 to appraise systematic reviews that include randomised or non-randomised studies of healthcare interventions. Philips and colleagues produced a synthesised good practice guideline and checklist for decision-analytic models in health technology assessment, while reporting their expert advisory group's view that the checklist should not substitute for critical appraisal and cannot judge whether a model's structure is appropriate.

Re-used individual-level data carry their own problems. Cheng and Phillips point out that such data were not collected for the new question, so important third variables may be missing, and that variables removed to protect confidentiality can leave residual confounding.

Transferability of published evidence

The ISPOR task force on transferability, led by Drummond, notes that there are many reasons why cost-effectiveness might vary from place to place and recommends strategies based on analysis of individual patient data and on decision-analytic modelling. A review by Barbieri and colleagues for the same task force of 27 sets of national pharmacoeconomic guidelines found that baseline risk and unit costs were uniformly considered of low transferability while treatment effect was classified as highly transferable. Resource use and utilities were judged of low transferability in 63% and 45% of cases respectively.

This pattern explains why HTA models usually take relative effects from international trials and replace baseline risks, resource use and costs with local data, as in the worked example. PMG36 asks for costs relevant to the UK health care system whenever possible, and when only costs from other countries are available, for conversion to pounds sterling at a current exchange rate from a source such as HM Revenue and Customs or the OECD.

Boundaries with nearby concepts

Evidence synthesis, combining results from several studies, is one part of secondary research, but a re-analysis of a single registry involves no synthesis at all. The Cochrane Handbook calls systematic review methodology highly structured, transparent and reproducible. A narrative review is a less structured form of secondary research. Real-world evidence overlaps with secondary research when it comes from records collected for care or billing, but a prospective study that collects new data in routine practice to answer a question is primary research. Primary research is the counterpart, generating the new observations from trials, cohorts and surveys that secondary research reviews and pools.

Misreadings and limits of secondary research

A frequent misreading treats secondary research as second-rate. The term describes where the data come from, and in health technology assessment a systematic review is the expected basis for estimates of effect, not a fallback. A second error assumes that pooling removes bias. Synthesis increases precision, but it cannot correct systematic error shared by the included studies, which is why appraisal of each study comes before pooling.

A third limitation is that secondary research cannot fill a gap that no study has addressed. PMG36 allows expert elicitation, preferably structured, when empirical evidence is absent, while noting that such evidence is subject to risk of bias and high uncertainty. A decision model also adds a layer of its own, since its results depend on structural assumptions about how health states and care pathways are represented, as well as on the data it re-uses.

Sources

• Barbieri M, Drummond M, Rutten F, Cook J, Glick HA, Lis J, et al. What do international pharmacoeconomic guidelines say about economic data transferability? Value in Health. 2010;13(8):1028-1037. https://doi.org/10.1111/j.1524-4733.2010.00771.x
• Buxton MJ, Drummond MF, Van Hout BA, Prince RL, Sheldon TA, Szucs T, Vray M. Modelling in economic evaluation: an unavoidable fact of life. Health Economics. 1997;6(3):217-227. https://pubmed.ncbi.nlm.nih.gov/9226140/
• Cheng HG, Phillips MR. Secondary analysis of existing data: opportunities and implementation. Shanghai Archives of Psychiatry. 2014;26(6):371-375. https://doi.org/10.11919/j.issn.1002-0829.214171
• Drummond M, Barbieri M, Cook J, Glick HA, Lis J, Malik F, et al. Transferability of economic evaluations across jurisdictions: ISPOR Good Research Practices Task Force report. Value in Health. 2009;12(4):409-418. https://doi.org/10.1111/j.1524-4733.2008.00489.x
• Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review (last updated August 2021). In: Higgins JPT, Thomas J, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane; 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-01
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). London: NICE; 2022, last updated 13 October 2026. https://www.nice.org.uk/process/pmg36
• Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technology Assessment. 2004;8(36). https://doi.org/10.3310/hta8360
• Röhrig B, du Prel JB, Wachtlin D, Blettner M. Types of study in medical research. Part 3 of a series on evaluation of scientific publications. Deutsches Ärzteblatt International. 2009;106(15):262-268. https://doi.org/10.3238/arztebl.2009.0262
• Sculpher MJ, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Economics. 2006;15(7):677-687. https://doi.org/10.1002/hec.1093
• Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.1136/bmj.j4008