Concept Architecture
Concept
Theoretically, P-value is the probability, calculated under the assumption that the null hypothesis is true, of observing a test statistic at least as extreme as the one obtained from the sample data. It is a fundamental measure in frequentist statistical inference and quantifies the compatibility of the observed data with the null hypothesis. The p-value does not represent the probability that the null hypothesis is true, nor does it measure the magnitude or practical importance of an effect.
Mathematically, the p-value is derived from the sampling distribution of a specified test statistic under the null hypothesis. Its calculation depends on the statistical test being performed, such as the t-test, ?� test, F-test, or Wald test. The p-value is obtained by integrating the probability density or mass function beyond the observed value of the test statistic, with one-sided or two-sided calculations depending on the alternative hypothesis.
In practice, p-values are calculated using recognised statistical tests to assess evidence against the null hypothesis. In health economics they are commonly reported when comparing treatment costs, quality-adjusted life years, resource utilisation, clinical outcomes, and regression coefficients. They are typically interpreted alongside confidence intervals, effect estimates, and measures of clinical or economic significance rather than in isolation.
Purpose
Used to quantify the statistical evidence against a null hypothesis, supporting hypothesis testing, statistical inference, regression analysis, clinical evaluation, and health economic decision-making.
Mathematical Formulae
Primary Formula
For a two-sided hypothesis test:
p = P(|T| � |t???| | H?)
where:
- p = p-value
- T = test statistic under the null hypothesis
- t??? = observed test statistic
- H? = null hypothesis
Supporting Formulae
For a one-sided upper-tail test:
p = P(T � t??? | H?)
For a standard normal test:
Z = (x? ? ??) � (� / �n)
For a Wald statistic:
W = ((?? ? ??) � SE(??))�
W ~ ?�?
Related Mathematical Methods
- Null Hypothesis
- Hypothesis Testing
- Confidence Interval
- Wald Test
- Likelihood Ratio Test
- Score Test
- Student's t-Test
- Chi-Square Test
- Analysis of Variance
- Multiple Comparisons
Example
A health economist compares annual healthcare costs between two treatment groups.
Estimated difference in mean cost:
�1,250
Standard error:
�500
The test statistic is:
t = 1,250 � 500
= 2.50
With the appropriate degrees of freedom, the calculated two-sided p-value is:
p = 0.014
Since p < 0.05, the observed cost difference provides statistical evidence against the null hypothesis of equal mean costs.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| T.DIST.2T | =T.DIST.2T(2.5,98) | Calculate a two-sided p-value from a t-statistic. |
| NORM.S.DIST | =2*(1-NORM.S.DIST(ABS(A2),TRUE)) | Calculate a two-sided p-value from a z-statistic. |
| CHISQ.DIST.RT | =CHISQ.DIST.RT(A2,1) | Calculate a p-value for a chi-square statistic. |
| F.DIST.RT | =F.DIST.RT(A2,3,96) | Calculate a p-value for an F-statistic. |
| T.TEST | =T.TEST(B2:B51,C2:C51,2,2) | Perform a two-sample t-test comparing health economic outcomes. |
VBA (Optional)
Automate hypothesis testing across multiple health economic outcomes and generate tables containing test statistics, p-values, and confidence intervals.
Sources
- Fisher RA. Statistical Methods for Research Workers.
- Fisher RA. The Design of Experiments.
- Altman DG. Practical Statistics for Medical Research.
- Wasserman L. All of Statistics.
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.
- Drummond MF, et al. Methods for the Economic Evaluation of Health Care Programmes.
- NICE Health Technology Evaluation Manual.
- CHEERS 2022 Statement.
Related Concepts (2)
Library
Publications
1
Bayesian Methods in Health Economics — Gianluca Baio, 1st Edition ed., 2012 (Chapman & Hall / CRC Press)
An overview of Bayesian statistical methods for the analysis of health economic data, covering economic evaluation concepts, statistical cost-effectiveness analysis, Bayesian computation and MCMC, and applied health economic evaluation.
BookView source →
Frequently Asked Questions (6)
What is a p-value?
A statistic representing the probability of observing a result at least as extreme as the one obtained, assuming the null hypothesis is true.
Source: Fisher 1925
What probability does a p-value actually give?
A p-value gives the probability of observing a result at least as extreme as the one obtained, if the null hypothesis of no effect were true. A small p-value means the data would be surprising under that assumption, which is taken as evidence against it. Crucially, it is not the probability that the null hypothesis is true, nor a measure of how large or important an effect is; a huge sample can make a trivial effect yield a tiny p-value. The chance of such data under the null is what it captures. Kirkwood and Sterne (2003) describe this.
Source: Kirkwood & Sterne 2003
How is a p-value used in hypothesis testing?
A p-value is used in hypothesis testing by comparing it with a chosen significance level, such as 0.05: if the p-value is below the significance level, the result is deemed statistically significant and the null hypothesis is rejected, otherwise it is not. So a p-value is used as the basis for the test decision, quantifying the evidence against the null and being compared with the significance level to decide whether to reject it, which provides a standard rule for significance testing, though the threshold is a convention and the p-value is best understood as a continuous measure of evidence rather than a strict cut-off.
Source: Fisher 1925
What does a p-value not indicate?
A p-value does not indicate the probability that the null hypothesis is true, nor the probability that the results occurred by chance, nor the size or importance of an effect; it is calculated assuming the null is true and concerns only the compatibility of the data with it. So a p-value does not measure the truth of the null hypothesis or the magnitude of an effect, which are common misinterpretations, and a small p-value does not prove an effect is large or important, which is why p-values are interpreted alongside effect sizes and confidence intervals, since the p-value alone conveys only the strength of evidence against the null, not the practical significance.
Source: Fisher 1925
How does sample size affect the p-value?
Sample size affects the p-value because with a large sample even a small, unimportant effect can produce a small p-value and appear statistically significant, while with a small sample a real and important effect may yield a large p-value and be missed. So the p-value depends on sample size as well as effect size, which means statistical significance does not equate to practical importance, since large studies can find trivial effects significant and small studies can miss meaningful ones, and this is why the p-value is interpreted together with the effect size and confidence interval rather than as a standalone indicator of importance.
Source: Fisher 1925
What are common misinterpretations of the p-value?
Common misinterpretations of the p-value include treating it as the probability that the null hypothesis is true, as the probability the result is due to chance, or as a measure of the size or importance of an effect, and treating a result just above or below the threshold as categorically different. So the p-value is frequently misunderstood, which is why care is taken to interpret it correctly as the probability of data as extreme as observed under the null, not as the probability of the hypothesis, and why it is reported with effect sizes and confidence intervals to avoid the misleading conclusions that these misinterpretations produce.
Source: Fisher 1925
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 23 Dec 2025
Content version: 1.0.0
Canonical Identity
- Persistent URI
- https://healtheconomics.wiki/concept/p-value
- Term code
- HE-ES-SA-169
Stable URI · Machine-readable · Resolvable · CC BY 4.0