Critical Value

Critical Value: Concept Architecture

Orientation and learning roadmap

This page follows a preselected error level to a boundary on a test statistic. It shows how the null distribution and tail direction determine the cutoff, works through a treatment-effect example, and explains why statistical significance is not itself a decision rule for healthcare.

Define the test before choosing a cutoff

State the null and alternative hypotheses, test statistic, reference distribution under the null, and desired type I error level. Decide whether departures in one direction or both directions count against the null before looking at the outcome data.

A critical value divides values of the statistic into rejection and nonrejection regions for that specified test. It is expressed on the statistic's scale, such as standard-normal units or a Student $t$ scale, not automatically in the original units of a health outcome. The NIST handbook on critical values and p values explains how the statistic and significance level determine the cutoff.

Locate the reference-distribution quantile

For a continuous standard-normal statistic in a two-sided test at level $\alpha$, place $\alpha/2$ probability in each tail. The positive boundary is the $(1-\alpha/2)$ quantile, and the negative boundary is its mirror image.

Writing $\Phi$ for the standard-normal cumulative distribution function, the rule is:

$$c=\Phi^{-1}(1-\alpha/2), \qquad \text{reject }H_0\text{ when }|Z|>c.$$

At $\alpha=0.05$, this gives approximately $c=1.96$ under the stated standard-normal reference. A one-sided test uses a different quantile; a $t$ test additionally depends on degrees of freedom. For a discrete statistic, exact tail probabilities and attainable rejection levels may not line up with a nominal cutoff in precisely the same way.

Worked treatment-effect test

Suppose an estimated difference in a health outcome is $0.22$ units with a standard error of $0.10$ units, and the null difference is zero. Under the assumed large-sample standard-normal test, the statistic is $Z=0.22/0.10=2.20$.

Because $|2.20|>1.96$, the two-sided 5% rule rejects this null in the illustrative analysis. The corresponding normal-approximation 95% interval is $0.22\pm1.96(0.10)$, or $[0.024,0.416]$, which excludes zero under the same assumptions. This calculation does not establish that the effect is clinically important or that an intervention is cost-effective.

Change the distribution or testing plan

With a small sample and an estimated variance, a Student $t$ reference may be more appropriate than a standard normal, and its critical value depends on the degrees of freedom. Clustered data, paired observations, or model-based tests require a reference distribution and variance estimate consistent with their design.

The NIST table of Student $t$ critical values illustrates how the boundary changes with sample information. Multiple endpoints or repeated interim looks can require a stricter or differently allocated error rule than a single unadjusted test. The FDA guidance on multiple endpoints discusses control of false conclusions across several planned tests.

Distinguish a cutoff from other thresholds

A critical value is a statistical boundary based on a test's reference distribution. A minimum clinically important difference is a judgement about the size of a health change, while a cost-effectiveness threshold values an incremental health outcome relative to costs.

Crossing the statistical cutoff is not a guarantee of clinical relevance, valid study design, or good value for money. Conversely, not crossing it does not prove that no meaningful effect exists.

Validate the reported decision

Check whether the tail direction, alpha level, statistic, degrees of freedom, and multiplicity plan were fixed in advance. Recalculate the statistic and critical value from the reported data, then compare with the reported p value and interval where those use the same test framework.

Report effect size and uncertainty in original units alongside a testing decision. If the analysis uses adaptive rules, complex dependence, or a nonstandard null distribution, document how its critical boundary was derived or simulated. A post hoc cutoff selected after viewing the data does not retain the prespecified error interpretation without further adjustment.