Concept Architecture
Concept
Theoretically, Pairwise Deletion is a missing-data handling method in which all available observations are used for each individual statistical calculation by excluding only those cases that are missing the specific variables required for that calculation. Unlike Listwise Deletion, which removes an entire observation whenever any required variable is missing, Pairwise Deletion retains partially observed cases wherever possible. The method exists to maximise the use of available information while reducing unnecessary loss of sample size.
Mathematically, Pairwise Deletion computes each covariance, correlation or other statistic using the subset of observations for which the relevant variables are jointly observed. Consequently, different elements of a covariance or correlation matrix may be estimated from different sample sizes. Although this approach increases data utilisation, it may produce covariance matrices that are not positive definite and can yield inconsistent parameter estimates when the missing-data mechanism is not Missing Completely at Random.
In practice, Pairwise Deletion is commonly used for exploratory analyses, descriptive statistics and correlation matrices rather than complex multivariable modelling. In health economics, it may be applied when examining associations between healthcare costs, quality-adjusted life years, clinical outcomes and patient characteristics containing incomplete observations. Because different analyses are based on different subsets of participants, analysts should report the effective sample size for each calculation and consider multiple imputation or likelihood-based methods for confirmatory analyses.
Purpose
Used to maximise the use of available data by calculating each statistical quantity from all observations containing the variables required for that calculation while avoiding unnecessary deletion of partially observed records.
Mathematical Formulae
Primary Formula
For variables X and Y:
rXY = ???S (x? ? x?)(y? ? ?) / �[???S (x? ? x?)� ? ???S (y? ? ?)�]
where:
- S = subset of observations with both X and Y observed
- rXY = pairwise correlation coefficient
Supporting Formulae
Pairwise sample size:
nXY = ?? I(X? observed and Y? observed)
Covariance:
Cov(X,Y) = ???S (x? ? x?)(y? ? ?) / (nXY ? 1)
Related Mathematical Methods
- Listwise Deletion
- Complete-Case Analysis
- Multiple Imputation
- Maximum Likelihood Estimation
- Missing Completely at Random
- Covariance Estimation
- Correlation Analysis
Example
A health economist analyses the relationship between annual healthcare costs, EQ-5D utility scores and age.
The available observations are:
- Cost and utility: 470 patients
- Cost and age: 495 patients
- Utility and age: 482 patients
Using Pairwise Deletion:
- The cost?utility correlation is calculated using 470 observations.
- The cost?age correlation is calculated using 495 observations.
- The utility?age correlation is calculated using 482 observations.
Each statistic therefore uses the largest possible sample for the variables involved, although the differing sample sizes may complicate interpretation and subsequent multivariable analyses.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| FILTER | =FILTER(B2:C501,(B2:B501<>"")*(C2:C501<>"")) | Extract observations with both variables observed in supported Excel versions. |
| CORREL | =CORREL(B2:B471,C2:C471) | Calculate correlations using pairwise-complete observations. |
| COUNTIFS | =COUNTIFS(B2:B501,"<>",C2:C501,"<>") | Calculate the pairwise sample size. |
| COVARIANCE.S | =COVARIANCE.S(B2:B471,C2:C471) | Estimate covariance using complete pairs only. |
| IF | =IF(AND(B2<>"",C2<>""),1,0) | Identify observations contributing to a pairwise calculation. |
VBA (Optional)
A VBA routine can automatically identify complete observation pairs for each variable combination, calculate pairwise statistics and report the effective sample size for every analysis.
Sources
- Little RJA, Rubin DB. Statistical Analysis with Missing Data. Wiley.
- Allison PD. Missing Data. Sage Publications.
- Carpenter JR, Kenward MG. Multiple Imputation and Its Application. Wiley.
- Enders CK. Applied Missing Data Analysis. Guilford Press.
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press.
- NICE. Health Technology Evaluation Manual.
Related Concepts (2)
Library
Publications
1
Bayesian Methods in Health Economics — Gianluca Baio, 1st Edition ed., 2012 (Chapman & Hall / CRC Press)
An overview of Bayesian statistical methods for the analysis of health economic data, covering economic evaluation concepts, statistical cost-effectiveness analysis, Bayesian computation and MCMC, and applied health economic evaluation.
BookView source →
Frequently Asked Questions (6)
What is pairwise deletion?
A missing data method in which each specific calculation uses all observations with complete data for that particular pair of variables.
Source: Rubin 1976
How does pairwise deletion use the available data for each calculation?
Pairwise deletion handles missing data by using, for each particular calculation, every record that has complete data for the variables that calculation involves. So a correlation between two variables is computed from all cases with both values present, even if those cases are missing other variables used elsewhere. This salvages more data than dropping any incomplete record entirely, but because different calculations rest on different subsets, the pieces may not fit together consistently. Using each complete pair where available is its method. Little and Rubin (2002) describe this.
Source: Little & Rubin 2002
How does pairwise deletion differ from listwise deletion?
Pairwise deletion uses, for each calculation, all cases with the data needed for that specific computation, so different calculations use different subsets, while listwise deletion excludes any case with a missing value on any variable in the analysis, using a single consistent set of complete cases. So the two differ in whether the sample varies by calculation or is fixed to fully complete cases, with pairwise deletion retaining more data at the cost of consistency and listwise deletion using a smaller but consistent sample, and both rely on the missing completely at random assumption to avoid bias, being simpler but less sound than principled methods.
Source: Rubin 1976
What are the advantages and disadvantages of pairwise deletion?
The advantages of pairwise deletion include using more of the available data than listwise deletion, since each calculation draws on all cases with the relevant variables, which can improve precision; its disadvantages include that different calculations rest on different subsets, which can produce inconsistencies, such as correlation matrices that are not internally coherent, and bias if data are not missing completely at random. So pairwise deletion trades fuller data use against consistency and validity, and it is used with caution, since its varying samples and its reliance on the missing completely at random assumption can undermine the coherence and validity of the results.
Source: Rubin 1976
When is pairwise deletion valid?
Pairwise deletion is generally valid, giving unbiased results, only when the data are missing completely at random, so that the cases contributing to each calculation are representative; when missingness depends on the data, the subsets can be unrepresentative and the results biased. So pairwise deletion is valid mainly under the strong missing completely at random assumption, which often does not hold, since missingness is frequently related to the variables, meaning the differing subsets for each calculation may be unrepresentative, which is why more principled methods such as multiple imputation are usually preferred when missingness is not completely at random.
Source: Rubin 1976
What are the alternatives to pairwise deletion?
The alternatives to pairwise deletion include multiple imputation, which creates completed datasets with plausible values reflecting uncertainty; maximum likelihood and mixed model methods, which use all available data validly under the missing at random assumption; and listwise deletion where a consistent complete-case sample is acceptable. So rather than using varying subsets across calculations, the alternatives to pairwise deletion make principled use of the data under more reasonable assumptions, which is why multiple imputation and likelihood-based methods are generally preferred, since they retain information, reduce bias when data are missing at random, and avoid the inconsistencies that pairwise deletion can produce across an analysis.
Source: Rubin 1987
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 22 Dec 2025
Content version: 1.0.0
Canonical Identity
- Persistent URI
- https://healtheconomics.wiki/concept/pairwise-deletion
- Term code
- HE-ES-SA-147
Stable URI · Machine-readable · Resolvable · CC BY 4.0