Frequentist Analysis

A statistical inference approach based on the long-run frequency properties of estimators, calculated from observed data alone, unlike a Bayesian approach.

Source: DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. doi:10.1016/0197-2456(86)90046-2.

Frequentist analysis rests on the idea that probability is the long-run frequency of an event over many repetitions, so it treats an unknown parameter as a fixed quantity and asks how its estimate would behave across hypothetical repeated samples. It draws its conclusions from the observed data alone, without formally incorporating prior belief, using tools such as p-values, confidence intervals, and significance tests. This makes it the traditional framework of much medical research, though it is criticised for how easily its results are misread. Reasoning from long-run frequencies is its basis. Kirkwood and Sterne (2003) describe this approach.

Source: Kirkwood & Sterne 2003

Frequentist analysis works by treating the parameters as fixed and the data as random, and evaluating procedures by their behaviour over hypothetical repetitions of the study: estimators are judged by properties such as unbiasedness, tests control the long-run rate of false positives, and confidence intervals have a specified long-run coverage. So frequentist analysis works by basing inference on the sampling distribution of statistics, asking how they would vary across repeated samples, which yields p-values, confidence intervals, and estimates with guaranteed long-run properties, without assigning probabilities to the parameters themselves, since in this framework the parameters are fixed rather than random.

Source: DerSimonian & Laird 1986

Frequentist analysis treats parameters as fixed and bases inference on the frequency properties of procedures over repeated sampling, without prior distributions, while Bayesian analysis treats parameters as having probability distributions and combines a prior with the data to obtain a posterior. Frequentist inference gives p-values and confidence intervals interpreted through long-run behaviour; Bayesian inference gives direct probability statements about parameters. So the two differ in whether parameters are treated probabilistically and whether prior information is used, with frequentist analysis relying on sampling frequency properties and Bayesian analysis on updating prior beliefs, reflecting a fundamental divide in statistical philosophy.

Source: Bayes 1763

The key tools of frequentist analysis include point estimators judged by properties such as unbiasedness and efficiency; hypothesis tests that control the type I error rate and yield p-values; and confidence intervals with a specified long-run coverage. These rest on the sampling distributions of statistics. So the key tools of frequentist analysis are estimators, significance tests, and confidence intervals, all built on the frequency behaviour of procedures over repeated sampling, which is why concepts such as p-values, significance levels, power, and coverage are central to the framework, providing the means by which frequentist inference quantifies evidence and uncertainty.

Source: DerSimonian & Laird 1986

The criticisms of frequentist analysis include that p-values and confidence intervals are often misinterpreted, for example as probabilities about the hypothesis or parameter, which they are not; that it does not incorporate prior knowledge formally; and that significance testing can encourage dichotomous thinking. So frequentist analysis is used with awareness of these criticisms, since its concepts are subtle and easily misused, and it does not provide the direct probability statements about parameters that some find more intuitive, which is part of why Bayesian methods are sometimes preferred, though frequentist analysis remains the dominant framework and is valid when its tools are correctly understood and applied.

Source: DerSimonian & Laird 1986