Simpson Paradox

A statistical phenomenon where a trend seen within subgroups reverses or disappears once those subgroups are combined into a single aggregated analysis.

Source: Simpson 1951

Simpson's paradox occurs when a relationship that holds in every subgroup vanishes or reverses once the subgroups are pooled. It arises when a lurking variable is unevenly distributed across the subgroups and also affects the outcome, so that combining the groups mixes their different compositions. A treatment can then appear better in each subgroup yet worse overall, simply because the harder cases were concentrated where it was used more. Analysing the subgroups separately reveals the true pattern. Rothman and colleagues (2008) describe this.

Source: Rothman et al. 2008

Simpson's paradox arises when a confounding variable, associated with both the factor and the outcome, is distributed unevenly across the subgroups, so that combining them mixes the within-group relationships with the between-group differences. If the subgroups differ in size and in their baseline outcome levels, the pooled association can be dominated by these differences and reverse the direction seen within each subgroup. So the paradox reflects confounding by the grouping variable, where aggregation conflates the genuine within-group effect with the differing composition of the groups.

Source: Simpson 1951

A classic example of Simpson's paradox is a treatment that has a higher success rate than an alternative within each of two patient groups, such as those with mild and severe disease, yet a lower overall success rate when the groups are combined, because the treatment was given more often to the severe group with poorer outcomes generally. The uneven distribution of severity across the treatments reverses the aggregated comparison. This illustrates how the pooled result can contradict the consistent within-group findings, the defining situation of the paradox.

Source: Robinson 1950

Simpson's paradox matters because it shows that aggregated analyses can mislead, reversing relationships that hold within subgroups, so drawing conclusions from combined data without considering how subgroups differ can produce wrong inferences and decisions. It underscores the importance of identifying confounding variables and analysing data at the appropriate level. In comparing treatments or groups, ignoring a variable that differs across the compared groups can invert the true relationship. Recognising the paradox prompts careful attention to subgroup structure and confounding when interpreting associations in aggregated data.

Source: Simpson 1951

Simpson's paradox can be avoided by identifying and accounting for confounding variables that differ across the groups being compared, analysing data with appropriate stratification or adjustment rather than relying on crude aggregated comparisons, and considering whether the within-group relationships differ from the pooled one. Examining the subgroup results alongside the combined result reveals when the paradox is present. Using methods that control for the confounder, such as stratified analysis or regression, gives estimates that reflect the genuine relationship, so the paradox is avoided by not interpreting aggregated associations without accounting for group structure.

Source: Robinson 1950