Large Language Model

Large Language Model in Evidence Review and Economic Modelling

A large language model is a many-layered neural network that has learned, from a very large body of text, to estimate which small piece of text comes next, and it writes answers and computer code by repeating that estimate one piece at a time. Health economists meet these models in the slow manual steps of a health technology assessment, such as screening search results, data extraction from trial reports and coding a decision model. Published case studies show one model rebuilding network meta-analyses and partitioned survival models with few errors, while also inventing software functions and giving different answers to identical requests. The output reads fluently whether or not it is correct, so every number it supplies needs checking against its source.

Next-token prediction and the transformer

Zhao and colleagues, in a survey of the field, say the term typically refers to transformer language models with hundreds of billions of parameters or more, trained on massive text data. It was coined to separate pretrained models of tens or hundreds of billions of parameters from smaller ones such as BERT. The usual training task is to predict each token of a sequence from the tokens before it.

$$\mathcal{L}(x) = \sum_{i=1}^{n} \log P(x_i \mid x_{<i})$$

where $x$ is a training sequence of $n$ tokens, $x_i$ is the token in position $i$, $x_{<i}$ is the set of tokens before it, $P$ is the probability the model gives to the observed token and $\mathcal{L}$ is the log-likelihood that training makes as large as possible. Reason and colleagues, writing for health economists, describe these as mathematical models that work by repeatedly predicting the next word.

The ISPOR working group on generative AI defines a token as a word, subword or character, and notes that the number of tokens affects performance, cost and the handling of long texts. In the network meta-analysis study by Reason and colleagues, GPT-4 had a limit of 8,192 tokens, about 6,000 words, so each trial publication was cut into overlapping chunks before data were extracted.

The transformer, introduced by Vaswani and colleagues, is the architecture behind most of these models. It relies on attention alone, without recurrence, and self-attention relates the positions of one sequence to each other to build a representation of it. Learned embeddings turn tokens into vectors, a final softmax layer gives predicted probabilities for the next token, and a mask makes each prediction depend only on earlier positions, which lets the model generate text left to right.

Prompting and instruction tuning

A model trained only on next-token prediction can already carry out tasks described in text. Brown and colleagues applied GPT-3, with 175 billion parameters, without any further training, giving the task and a few examples as text in the request. The ISPOR working group lists zero-shot, few-shot, chain-of-thought and persona prompts among the forms of prompting.

Size alone does not make a model follow a user's intent. Ouyang and colleagues note that large models can produce untruthful, toxic or unhelpful output, and they fine-tuned GPT-3 on demonstrations written by human labellers and then by reinforcement learning from human rankings of its outputs. Labellers preferred the resulting 1.3 billion parameter model to the 175 billion parameter GPT-3, yet it could still make up facts.

Prompts carry the subject knowledge. In the network meta-analysis study, the team told GPT-4 that it was a statistician and a medical researcher, supplied an example R script for the multinma package, and added rules such as that a result is statistically significant when both limits of the credible interval lie on the same side of 1.

Sampling temperature and repeated runs

At each step the model gives a probability to every token in its vocabulary, and a decoding rule chooses one. Zhao and colleagues give the temperature form of the softmax.

$$P(x_j \mid x_{<i}) = \frac{\exp(l_j / t)}{\sum_{k} \exp(l_k / t)}$$

where $l_j$ is the model's raw score (logit) for candidate token $j$, the sum runs over every token $k$ in the vocabulary, $t$ is the temperature and $P(x_j \mid x_{<i})$ is the probability of choosing token $j$ next. A lower temperature favours high-probability tokens, a temperature near 0 amounts to always taking the top token, and a very high temperature approaches uniform sampling. The survey adds that a higher temperature can lead to more hallucinations, and cites OpenAI's documentation that its interface can return different outputs for the same input and settings, with a slight chance of variation even at a temperature of 0. Reason and colleagues set GPT-4's temperature to 0, still saw outputs vary, and so ran each analysis 20 times.

Published uses in evidence synthesis and economic modelling

The NICE position statement on the use of AI in evidence generation (ECD11), published on 15 August 2024, lists potential uses and says the list is not an endorsement. Large language model prompts may support search strategies, classification of studies and screening of records, while data extraction and code for a network meta-analysis are described as less well established. For cost-effectiveness evidence, such models could write the code for an economic model after human-led conceptualisation and parameter estimation, update a model with new data and support replication of existing models.

Data extraction and network meta-analysis

Reason and colleagues tested GPT-4 on four network meta-analyses that had already been done by hand, one in hidradenitis suppurativa and three in second-line non-small cell lung cancer. A Python script sent each publication to the model, which extracted the data, wrote an R script and wrote a short report. More than 99% of the required data items were extracted correctly across 20 runs per case study, and the model never reported an incorrect value, only failing at times to find the numbers at risk for two trials.

The code was less reliable. In the largest network, with 12 trials and eight treatments, only four of 20 scripts ran without human input, and five could not be repaired without prior knowledge of the network. Early on the model sometimes invented R packages and functions, and it could not read results shown only in images. Where scripts ran, results closely matched the manual analyses, with identical mean hazard ratios in the lung cancer networks.

Programming a partitioned survival model

In a companion study the same group gave GPT-4 text descriptions of two published partitioned survival models, in non-small cell lung cancer and renal cell carcinoma, and had it code each in R 15 times. For the lung cancer model 14 of 15 scripts were completely free of errors. The renal cell model needed a human to simplify one of its fifteen input calculations, after which 9 of 15 scripts were error-free, and scripts without errors reproduced the published incremental cost-effectiveness ratios to within 1%.

These were feasibility studies by one team, both supported by Bristol Myers Squibb, with one model, simple networks and models described in full in advance, and the model was not asked to check the proportional hazards assumption.

What NICE expects when the models are used

The ECD11 statement covers AI used to produce evidence for NICE, and wider regulation is covered under artificial intelligence in health. Submitting organisations should declare the use, report the human input and remain accountable for the content. The statement also names prompt injection, inserting malicious content into prompts, as a cybersecurity risk, and asks organisations to check that copyrighted material used in an AI tool is licensed.

Worked example with extraction rates and sampling temperature

Steps 1 and 2 recompute figures published by Reason and colleagues, and steps 3 and 4 use hypothetical, illustrative scores. In the base case lung cancer analysis the model had to extract 35 data items in each of 20 runs, and missed three items in two runs.

Step 1. The extraction success rate is

$$s = 1 - \frac{m}{q \times r}$$

where $s$ is the share of required items extracted, $m$ is the number of items missed across all runs, $q$ is the number of items needed per run and $r$ is the number of runs. With six missed items,

$$s = 1 - \frac{6}{35 \times 20} = 1 - \frac{6}{700} = 0.9914$$

which is the 99.1% reported. In the 12-trial network, 77 items per run and three missed items in one run give $s = 1 - 3/1540 = 0.998$, the reported 99.8%.

Step 2. Two of 20 runs, or 10%, lacked the numbers at risk for one trial, so a single unchecked run had a real chance of a gap. The failures were omissions, which a completeness check against the list of required items would catch, while a wrong value would be found only by checking the publication.

Step 3. Suppose, for illustration, that after the words "the comparator arm received" a model gives logits of 2.0 to "docetaxel", 1.0 to "placebo" and 0 to "pembrolizumab". At a temperature of 1 the exponentials are 7.389, 2.718 and 1, which sum to 11.107, so

$$P_{\text{doc}} = \frac{7.389}{11.107} = 0.665$$

where $P_{\text{doc}}$ is the probability of choosing "docetaxel", with 0.245 for placebo and 0.090 for pembrolizumab.

Step 4. At a temperature of 0.5 the scaled logits are 4, 2 and 0, with exponentials 54.598, 7.389 and 1 summing to 62.987, so

$$P_{\text{doc}} = \frac{54.598}{62.987} = 0.867$$

where $P_{\text{doc}}$ is again the probability of "docetaxel", with 0.117 for placebo and 0.016 for pembrolizumab. If the trial's comparator was in fact placebo, the cooler model repeats the wrong drug more reliably, because temperature changes how much the output varies and has no bearing on whether it is correct.

Choices that change the output

Several settings sit between a source and the text a model returns. Each belongs in the methods, since a reviewer cannot repeat the work without them.

• Model and version. Reason and colleagues note that a fixed version, or an open model run locally, gives tighter control for reproducibility.
• Prompt and context. Their prompts were refined over several rounds, and an example script and subject rules changed whether the code ran.
• Chunking. Text beyond the token limit has to be split, and the model could reach the end of a paper without finding all required data.
• Repeats. Running each task several times and comparing the answers exposes variation that a single run hides.

Large language models beside related ideas

A large language model is one kind of deep learning model, and the ISPOR working group places it within the family of foundation models trained on large unlabelled datasets. Text embeddings, semantic search, retrieval-augmented generation and AI agents are separate techniques that represent or retrieve text for a language model, or let it call tools.

Pan and colleagues contrast language models with knowledge graphs, which store factual knowledge explicitly, while large language models are black-box models that represent knowledge implicitly in their parameters. An answer drawn from the weights alone therefore carries no record of its source, which a knowledge graph statement can keep.

Misreadings that lead to wrong inputs

A common source of error is treating fluent text as checked evidence. Four recur in health economics work.

1. Reading fluency as accuracy. Ji and colleagues define hallucination as generated content that is nonsensical or unfaithful to the source content, and the ISPOR working group reports fabricated citations and incorrect MeSH terms in literature searches.
2. Assuming current knowledge. Zhao and colleagues note that models struggle with tasks needing knowledge beyond their training data, so a new price or trial may be missing.
3. Treating a temperature of 0 as reproducible. Identical prompts gave different outputs in the network meta-analysis study, so a single run is not a replicable analysis.
4. Generalising feasibility studies. Success on four networks and two fully described models does not give the error rate on a new systematic review or health economic model.

Sources

• Brown TB, Mann B, Ryder N, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). arXiv:2005.14165v4. https://arxiv.org/abs/2005.14165
• Fleurence RL, Wang X, Bian J, Higashi MK, Ayer T, Xu H, Dawoud D, Chhatwal J. A taxonomy of generative artificial intelligence in health economics and outcomes research: an ISPOR working group report. Value in Health. 2025;28(11):1601-1610. https://doi.org/10.1016/j.jval.2025.04.2167 (preprint read, arXiv:2410.20204v2)
• Ji Z, Lee N, Frieske R, et al. Survey of hallucination in natural language generation. ACM Computing Surveys. 2023;55(12):1-38. https://doi.org/10.1145/3571730 (arXiv:2202.03629v7)
• National Institute for Health and Care Excellence. Use of AI in evidence generation: NICE position statement (ECD11). Corporate document, published 15 August 2024. https://www.nice.org.uk/corporate/ecd11
• Ouyang L, Wu J, Jiang X, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2203.02155. https://arxiv.org/abs/2203.02155
• Pan S, Luo L, Wang Y, Chen C, Wang J, Wu X. Unifying large language models and knowledge graphs: a roadmap. IEEE Transactions on Knowledge and Data Engineering. 2024;36(7):3580-3599. https://doi.org/10.1109/TKDE.2024.3352100
• Reason T, Benbow E, Langham J, Gimblett A, Klijn SL, Malcolm B. Artificial intelligence to automate network meta-analyses: four case studies to evaluate the potential application of large language models. PharmacoEconomics Open. 2024;8(2):205-220. https://doi.org/10.1007/s41669-024-00476-9
• Reason T, Rawlinson W, Langham J, Gimblett A, Malcolm B, Klijn S. Artificial intelligence to automate health economic modelling: a case study to evaluate the potential application of large language models. PharmacoEconomics Open. 2024;8(2):191-203. https://doi.org/10.1007/s41669-024-00477-8
• Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Advances in Neural Information Processing Systems 30 (NIPS 2017). arXiv:1706.03762v7. https://arxiv.org/abs/1706.03762
• Zhao WX, Zhou K, Li J, et al. A survey of large language models. arXiv:2303.18223, version 19, 18 March 2026. https://arxiv.org/abs/2303.18223