Variance-Covariance Matrix

Variance-Covariance Matrix

A variance-covariance matrix records the variability of several quantities and how they vary together. Its diagonal entries are variances; its off-diagonal entries are covariances. In health economics it helps preserve the joint uncertainty of costs, health outcomes or model parameters, rather than treating each estimate as if it moved independently.

Reading the rows and columns

Let $\boldsymbol{X}=(X_1,\ldots,X_k)^{\mathrm{T}}$ be a vector of random quantities with finite second moments and mean vector $\boldsymbol{\mu}=E(\boldsymbol{X})$. Its covariance matrix $\boldsymbol{\Sigma}$ has entry $\Sigma_{ij}=E[(X_i-\mu_i)(X_j-\mu_j)]$. The same quantity appears when the indices are reversed, so the matrix is symmetric.

$$\boldsymbol{\Sigma}=\operatorname{Cov}(\boldsymbol{X})=\begin{pmatrix}\operatorname{Var}(X_1)&\operatorname{Cov}(X_1,X_2)\\\operatorname{Cov}(X_2,X_1)&\operatorname{Var}(X_2)\end{pmatrix}.$$

For two quantities, a positive covariance means they tend to deviate from their means in the same direction, and a negative covariance means deviations tend to oppose one another. A zero covariance means no linear association in that distribution, but does not generally establish independence. The value of a covariance depends on both variables' units and scales; the dimensionless correlation $\rho_{ij}=\Sigma_{ij}/(\sigma_i\sigma_j)$, when both standard deviations are nonzero, is easier to compare across pairs. Correlation describes association, not causation.

The row and column labels and their order are part of the data. Swapping them without also changing the matrix changes the meaning of every calculation that uses it.

Why a valid matrix matters

For any real vector $\boldsymbol{a}$, the variance of the linear combination $\boldsymbol{a}^{\mathrm{T}}\boldsymbol{X}$ is $\boldsymbol{a}^{\mathrm{T}}\boldsymbol{\Sigma}\boldsymbol{a}$. A variance cannot be negative, so a valid covariance matrix must be positive semidefinite: this quadratic form is nonnegative for every $\boldsymbol{a}$. It must also be symmetric, with nonnegative diagonal entries. A true covariance matrix can have a zero eigenvalue when one variable is an exact linear combination of others; positive definiteness is stronger than validity.

For two variables, the covariance is bounded by $|\Sigma_{12}|\leq\sqrt{\Sigma_{11}\Sigma_{22}}$. Equivalently, the absolute correlation cannot exceed one. A constructed matrix that violates this bound cannot be a covariance matrix, even if each diagonal variance looks plausible. For larger matrices, checking pairwise bounds alone is insufficient; check all eigenvalues within numerical tolerance or use a mathematically valid covariance construction. Rounding may create tiny negative computed eigenvalues, but a materially negative value needs investigation, not an unexplained software repair.

Positive semidefiniteness describes consistency of second moments, not whether a multivariate normal model is appropriate. A covariance matrix plus mean vector does not by itself determine the complete joint distribution, its tail behavior, or the probability that bounded parameters take admissible values.

Worked example: costs and QALYs together

Suppose estimated incremental QALYs have mean $0.12$ and standard error $0.03$, while incremental cost has mean £1,800 and standard error £200. Assume a sampling correlation of $0.50$ between the two estimators, with coordinate order $(\widehat{\Delta Q},\widehat{\Delta C})$. The covariance is $0.50(0.03)(200)=3$ QALY-pounds, giving this illustrative sampling covariance matrix:

$$\boldsymbol{\Sigma}=\begin{pmatrix}0.0009&3\\3&40{,}000\end{pmatrix}.$$

The determinant is $(0.0009)(40{,}000)-3^2=27>0$, so this symmetric two-by-two matrix with positive diagonal is positive definite. At £20,000 per QALY, incremental net monetary benefit is $\widehat{\mathrm{INMB}}=20{,}000\widehat{\Delta Q}-\widehat{\Delta C}=£600$. Its variance follows from the weight vector $(20{,}000,-1)^{\mathrm{T}}$:

$$\operatorname{Var}(\widehat{\mathrm{INMB}})=(20{,}000)^2(0.0009)+40{,}000-2(20{,}000)(3)=280{,}000\ \mathrm{GBP}^2.$$

The resulting standard error is $\sqrt{280{,}000}=£529.15$ (rounded). If the covariance were incorrectly set to zero, the variance would be 400,000 £ squared and the standard error £632.46. Thus positive cost–effect correlation reduces this particular net-benefit variance because cost has a negative weight; for a sum of cost and another positively correlated cost, the corresponding covariance term would increase variance. The net-benefit variance is exact given the stated sampling covariance because the function is linear. A standard error alone does not validate a normal confidence interval or estimate the probability that the intervention is cost-effective.

Which kind of variability is in the matrix?

A matrix of patient-level costs and QALYs describes differences among patients, whereas a matrix of estimated mean costs and effects describes uncertainty in those estimated means. The first is not automatically a standard-error matrix for the second. In a simple independent and identically distributed sample of size $n$, the sample mean's sampling covariance is the individual-level covariance divided by $n$; clustering, weights, missingness, regression adjustment and paired or longitudinal designs require their appropriate estimation methods.

In an economic model, a matrix may instead describe joint uncertainty of model parameters, such as correlated regression coefficients. It is then used to draw coherent parameter sets in probabilistic sensitivity analysis or to propagate parameter uncertainty through a derived result. It must be on the same scale as the parameters being drawn, with transforms and constraints respected. The covariance between outcomes across simulated individuals answers a different question from covariance among uncertain parameter estimates.

Comparisons between arms require preserving shared information. For example, $\operatorname{Var}(A-B)=\operatorname{Var}(A)+\operatorname{Var}(B)-2\operatorname{Cov}(A,B)$; setting the last term to zero is justified only for the specific estimators under consideration when their covariance is zero. Correlated arm estimates can arise from shared controls, matched data or common model parameters. Likewise, costs and QALYs measured on the same patients often have a relevant covariance even if treatment arms are independently randomized.

Using the matrix in analysis

For a linear result $Y=\boldsymbol{a}^{\mathrm{T}}\boldsymbol{X}$, calculate $\operatorname{Var}(Y)=\boldsymbol{a}^{\mathrm{T}}\boldsymbol{\Sigma}\boldsymbol{a}$. For a smooth nonlinear function $g(\boldsymbol{X})$, a local first-order approximation replaces $\boldsymbol{a}$ with the gradient evaluated at the chosen reference point: $\operatorname{Var}\{g(\boldsymbol{X})\}\approx\nabla g(\boldsymbol{\mu})^{\mathrm{T}}\boldsymbol{\Sigma}\nabla g(\boldsymbol{\mu})$. Curvature or a denominator near zero can make this approximation poor; simulation or other inference may be needed.

If a multivariate normal distribution is justified, a suitable matrix factorization can generate draws with the specified covariance. Ordinary Cholesky factorization requires positive definiteness; a singular positive-semidefinite matrix needs a method that can handle its lower rank. For bounded probabilities or skew costs, a normal distribution on the original scale may be inappropriate even when the covariance is valid. Check the joint distribution, constraints and resulting simulated correlations instead of assuming a covariance matrix alone is a complete probabilistic model.

Frequent mistakes and reporting checks

Problems often arise from mixing scales, populations and uncertainty types. A covariance matrix cannot repair biased estimates, omitted variables, structural uncertainty or a poor model of the data-generating process. Its purpose is to keep specified second-moment relationships consistent in a particular calculation.

• Keep the input order visible. Name every row and column and preserve the same order in gradients, coefficient vectors and simulation code.
• Keep units explicit. Variances are in squared units and cross-covariances in products of the respective units; the result's variance must be in squared output units.
• Distinguish covariance from correlation. Convert with $\Sigma_{ij}=\rho_{ij}\sigma_i\sigma_j$ rather than inserting a unitless correlation into a covariance cell.
• Inspect the source. State whether entries are empirical patient covariances, sampling covariances of estimates, or elicited parameter covariances, and how uncertainty was estimated.
• Test matrix validity. Check symmetry, diagonal values and positive semidefiniteness before propagation or joint draws; disclose any material adjustment.
• Preserve dependencies. Explain any independence assumption, especially for linked costs and effects, common controls and parameters fitted from the same data.

Sources and further reading

Penn State's multivariate statistics lesson on covariance defines matrix entries, while its lesson on linear combinations derives the variance of weighted sums. The ISPOR-SMDM uncertainty report discusses joint parameter uncertainty in decision models. A trial-based economic-evaluation methods paper develops variance and covariance expressions for incremental net benefit. The numerical example here is an original teaching calculation, not a published empirical estimate.