Concept Architecture
Concept
Theoretically, Model Validation is the systematic process of determining whether a model is sufficiently accurate, credible and appropriate for its intended purpose. It evaluates whether the model adequately represents the real-world system or decision problem that it is designed to simulate. Model validation is founded on the principle that confidence in model outputs depends not only on mathematical correctness but also on agreement with empirical evidence, theoretical expectations and expert knowledge. In health economics, model validation is a fundamental component of good modelling practice and is applied throughout model development and implementation.
Mathematically, model validation does not have a single mathematical representation because it encompasses multiple complementary quantitative assessments. Validation commonly incorporates statistical comparisons between model predictions and observed data using prediction error measures, goodness-of-fit statistics, calibration metrics, discrimination measures and cross-validation procedures. These methods quantify different aspects of model performance while recognising that no individual measure is sufficient to establish overall validity.
In practice, model validation combines internal validation, external validation, cross-validation, calibration assessment, predictive accuracy evaluation and face validity review. Health economists compare model predictions with independent clinical or observational data, examine consistency with established evidence and seek expert review of model assumptions and outputs. International good practice guidance recommends documenting validation activities throughout model development to support transparency, credibility and decision-making.
Purpose
Used to determine whether a model is sufficiently accurate, reliable and credible for informing health economic decision-making and policy evaluation.
Mathematical Formulae
Primary Formula
There is no universally recognised canonical mathematical formula.
Supporting Formulae
Common validation measures include:
Mean Squared Error:
MSE = (1/n) �???� (y? ? ??)�
Coefficient of determination:
R� = 1 ? [�(y? ? ??)�] / [�(y? ? ?)�]
Cross-validation error:
CV = (1/k) �???? L?
where L? denotes the validation loss for fold i.
Related Mathematical Methods
- Calibration
- Cross-validation
- External validation
- Internal validation
- Goodness of fit assessment
- Predictive accuracy assessment
- Likelihood ratio testing
- Residual analysis
- Discrimination assessment
Example
A health economist develops a Markov model estimating long-term cardiovascular outcomes following antihypertensive treatment.
Validation activities include comparison of predicted five-year survival with published clinical trial data, assessment of transition probabilities against registry evidence, cross-validation of the underlying risk equation and review of model structure by clinical experts. Prediction errors are small, calibration is satisfactory and independent experts judge the model clinically plausible. Collectively, these findings support the model's validity for cost-effectiveness analysis.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| RSQ | =RSQ(B2:B501,C2:C501) | Assesses agreement between observed and predicted outcomes. |
| SUMXMY2 | =SUMXMY2(B2:B501,C2:C501) | Calculates prediction error during validation. |
| AVERAGE | =AVERAGE(D2:D11) | Summarises cross-validation or prediction error across validation samples. |
| CHISQ.TEST | =CHISQ.TEST(B2:B6,C2:C6) | Performs selected goodness-of-fit assessments where appropriate. |
| ABS | =ABS(B2-C2) | Calculates absolute prediction error for validation analyses. |
VBA (Optional)
Automate execution of validation procedures, summarise diagnostic statistics and generate a comprehensive model validation report.
Sources
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.
- Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed.
- ISPOR-SMDM Modeling Good Research Practices Task Force. Model Validation. Medical Decision Making. 2012.
- NICE. Health Technology Evaluation Manual.
- CHEERS 2022 Statement: Updated Reporting Guidance for Health Economic Evaluations.
- Eddy DM, Hollingworth W, Caro JJ, et al. Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force.
Related Concepts (2)
Institutional Perspectives (2)
- PBAC
Model Validation and Executable, Respecifiable Model Required
Validation to demonstrate the model generates what it is intended to represent is expected (Section 3A.7). An electronic copy of the model must be provided in which all variables can be changed independently, the base case can be respecified with new sensitivity analyses run, and results are produced within reasonable running times; more complex, less transparent techniques reduce PBAC confidence in the claim.
Pharmaceutical Benefits Advisory Committee, Guidelines for Preparing a Submission to the PBAC, Sections 3A.2 and 3A.7View source → - NICE
Transparency and Validation With an Executable Model
Economic evaluations must be transparently described and provided as a fully executable model, so the committee and review group can interrogate structure, assumptions, and coding and test alternative scenarios; model structures that limit feasibility of probabilistic analysis must be specified and justified.
NICE Health Technology Evaluations: The Manual (PMG36); company evidence submission requirementsView source →
Library
Publications
3
Modeling Good Research Practices — Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1 — Caro, Briggs, Siebert & Kuntz, Task Force Report 1 ed., 2012 (Value in Health / Medical Decision Making)
The overview paper of the seven-part ISPOR-SMDM modelling good-practice series, setting out best-practice recommendations across model design, technique selection, implementation, validation, parameterisation, uncertainty and use in decision making.
Journal ArticleView source →Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7 — Eddy, Hollingworth, Caro, Tsevat, McDonald & Wong, Task Force Report 7 ed., 2012 (Value in Health / Medical Decision Making)
Best-practice guidance on model transparency and validation, defining face, internal, external and predictive validation and setting out how models should be documented for scrutiny.
Journal ArticleView source →A Need for Change! A Coding Framework for Improving Transparency in Decision Modeling — Alarid-Escudero, Krijkamp, Pechlivanoglou, Jalal, Kao, Yang & Enns, Vol. 37 ed., 2019 (PharmacoEconomics)
The DARTH workgroup’s proposed standardised coding framework and naming conventions for decision-analytic models, aimed at making R- (and Excel-) based health economic models more transparent, reproducible and reviewable.
Journal ArticleView source →
Frequently Asked Questions (6)
What is model validation?
The overall process of establishing confidence that a model's structure and results adequately represent the real-world system it is meant to capture.
Source: Eddy DM, Hollingworth W, Caro JJ, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value in Health. 2012;15(6):843-850. doi:10.1016/j.jval.2012.04.012.
What question does model validation try to answer?
Model validation asks whether a model is a good enough representation of reality to be relied on for its purpose. It goes beyond confirming that the calculations are correct and examines whether the structure, assumptions, and outputs match what is known of the disease and what independent data show. The aim is justified confidence that the model's answers can be trusted, not proof that it is exactly right, which is unattainable. Validation builds the case for using the model. Eddy and colleagues (2012) frame it this way.
Source: Eddy et al. 2012
What are the main types of model validation?
The main types of model validation include face validity, whether the model's structure and assumptions seem reasonable to experts; internal validity, or verification, whether the model is built correctly and computes as intended; cross-validity, whether its results agree with other models or studies; and external validity, whether its predictions match independent real-world data. Together these address whether the model is built right and represents reality appropriately, examining its plausibility, correctness, consistency with other analyses, and agreement with observed outcomes.
Source: Eddy et al. 2012
Why is model validation important?
Model validation is important because a model informs decisions, and confidence in its results depends on evidence that it adequately represents reality, which validation provides. Without validation, a model might be built incorrectly, misrepresent the system, or predict poorly, yet still produce plausible-looking results that mislead. Validating a model, by checking its plausibility, correctness, and agreement with data, establishes whether its outputs can be trusted, so that decisions rest on a model shown to reflect the real-world system it is meant to capture.
Source: Philips et al. 2004
How is model validation conducted?
Model validation is conducted through a combination of checks: expert review of the structure and assumptions for plausibility; verification that the model is built correctly, using error checking, testing, and independent methods; comparison of results with other models and studies for consistency; and comparison of predictions with independent real-world data for external validity. The results of these checks are weighed together to judge overall confidence. Validation is thus a multi-faceted process, applying several methods to establish that the model adequately represents reality.
Source: Philips et al. 2004
How does validation differ from verification?
Validation and verification address different questions. Verification checks that a model is built correctly and computes according to its intended logic, that it is built right, while validation more broadly establishes that the model and its results adequately represent the real-world system, that the right model was built. Verification is part of validation, concerning internal correctness, while validation also examines plausibility and agreement with reality. Both are needed: verification ensures the model works as designed, and validation ensures the design reflects the world it represents.
Source: Eddy et al. 2012
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 16 Oct 2025
Content version: 1.0.0
Canonical Identity
- Persistent URI
- https://healtheconomics.wiki/concept/model-validation
- Term code
- HE-EM-MV-055
Stable URI · Machine-readable · Resolvable · CC BY 4.0