Mean Absolute Error

Mean Absolute Error: Concept Architecture

Mean absolute error (MAE) summarizes how far numerical predictions are from their observed values in the original outcome units. The sections below show how the metric is calculated, what its average conceals, and how to evaluate it fairly for forecasts, health outcomes, and costs. An MAE is meaningful only alongside the prediction target, observation horizon, evaluation population, and a relevant comparator.

Calculate the error for each prediction first

Let $y_i$ be the observed value and $\hat y_i$ the prediction for the same case $i$, among $n>0$ evaluable cases. The signed error can be defined as $e_i=\hat y_i-y_i$, and its absolute magnitude is $|e_i|$. Reversing the sign convention changes the signed errors but leaves MAE unchanged.

$$\mathrm{MAE}=\frac{1}{n}\sum_{i=1}^{n}|y_i-\hat y_i|,\qquad n>0.$$

The absolute value prevents overpredictions and underpredictions from canceling. MAE is nonnegative, equals zero only when every evaluated prediction is exact, and has the same units as the target: an MAE of £50 for costs is £50 per evaluated prediction, while an MAE of 2.4 days for a length-of-stay forecast is 2.4 days. It describes the arithmetic mean of absolute errors, not the absolute value of the mean signed error.

Work through a small cost-prediction example

Suppose four patients have observed costs and predictions in pounds, using the same follow-up period and price basis. All numbers below are illustrative. The signed error is prediction minus observation; the absolute column is what enters MAE.

Here $\mathrm{MAE}=(10+10+30+40)/4=22.50$ pounds. The signed errors are $10,-10,30,-40$, so their mean is $-2.50$ pounds; its absolute value is £2.50, which is not MAE. The metric says the average error magnitude is £22.50 for these four cases; it does not say every patient has an error near that amount. The largest individual error in this example is £40.

Compare models on the same cases

A smaller MAE indicates smaller average absolute error when models predict the same target on the same appropriately selected cases. Compare each model with a simple, predeclared baseline, such as a training-set median or a suitable prior-period forecast. Use an evaluation set separated from the data used for model fitting, tuning, target transformation, and choice of preprocessing.

If a second model errs by £20 on each of the four patients above, its MAE is £20 and it performs better by this metric on those cases, although other criteria could favor the first model. The improvement is £22.50 minus £20, or £2.50 per prediction, or about 11.1% of the first MAE. This observed comparison alone is not evidence of a clinically important change, statistically reliable improvement, better calibration, or economic value.

For $K$ validation folds, do not automatically average fold MAEs equally when fold sizes differ. The pooled out-of-fold MAE is $\sum_{k=1}^{K}n_k\mathrm{MAE}_k/\sum_{k=1}^{K}n_k$, provided each relevant case receives one valid out-of-fold prediction. This expression evaluates the collection of out-of-fold predictions; it is not an external test on an untouched future cohort. For time-ordered data, use a temporal evaluation that avoids training on future information, and report which forecast horizon was scored.

State what is being averaged and weighted

The ordinary MAE gives one equal contribution to each evaluable case. A weighted version can target a different population or emphasize a justified subset, but weights change the estimand and must be stated. For nonnegative case weights $w_i$ with positive sum, define:

$$\mathrm{MAE}_w=\frac{\sum_{i=1}^{n}w_i|y_i-\hat y_i|}{\sum_{i=1}^{n}w_i},\qquad w_i\geq0,\quad\sum_{i=1}^{n}w_i>0.$$

Using weights $(1,1,2,2)$ for the worked example yields $(10+10+2\cdot30+2\cdot40)/6=£26.67$ after rounding. The change from £22.50 reflects the chosen weights, not an improvement or deterioration in any individual prediction. Frequency or survey weights, unequal follow-up, and deliberately decision-weighted errors have different interpretations; document the weighting objective and use an uncertainty method consistent with the sampling design. If errors are asymmetrically costly, a symmetric absolute-error loss may not represent the decision loss, even when the arithmetic is correct.

What MAE rewards and what it hides

MAE penalizes an error of size $a$ in proportion to $a$ regardless of direction. A single error of 100 contributes as much as ten errors of 10 to the sum of absolute errors. Squaring errors in mean squared error (MSE), or taking the square root to obtain root mean square error (RMSE), places more relative emphasis on large errors; these metrics answer a different evaluation question. For the worked example, $\mathrm{RMSE}=\sqrt{(10^2+10^2+30^2+40^2)/4}=\sqrt{675}\approx25.98$ pounds, greater than its MAE of £22.50.

The sign of error and the distribution of errors should be shown when systematic overprediction, underprediction, or unequal subgroup performance matters. MAE alone does not reveal whether a model misses rare but consequential high-cost patients, whether errors rise with severity, or whether predictions are calibrated across risk groups. A histogram or quantiles of absolute and signed errors and stratified analyses often convey that missing information.

Why the median appears when MAE is optimized

Absolute-error loss has an important modeling implication. For a random outcome $Y$ conditional on features $X=x$, any conditional median of $Y\mid X=x$ minimizes the expected absolute loss $E[|Y-a|\mid X=x]$ over point predictions $a$, under the usual existence conditions. Squared-error loss instead targets a conditional mean when that mean exists. The point forecast that optimizes MAE can therefore differ from the point forecast required to estimate mean resource use or aggregate expenditure.

For example, a cost distribution with a long right tail can have a median well below its mean. A model trained explicitly to minimize absolute error may generate predictions close to typical costs but systematically understate the average cost needed for a population budget if those predictions are used as mean-cost inputs without adjustment. Assess the intended estimand and decision before choosing a loss function; reporting MAE does not by itself establish that a model predicts an unbiased mean.

Interpretation across scales and health applications

MAE is scale-dependent. A £100 MAE for annual costs and a £100 MAE for five-year costs cannot be interpreted as equivalent accuracy; nor should a £100 cost MAE be compared numerically with a 0.1-QALY MAE. Across different series or populations, compare performance to a meaningful baseline and consider a carefully defined scaled metric rather than assuming that one raw MAE ranks all tasks fairly.

Health economic applications may use MAE to evaluate predicted costs, utilization, duration, or continuous health outcomes, but prediction quality is distinct from causal effect estimation or cost-effectiveness. A model with low MAE can still be unsuitable for treatment decisions if important subgroup errors, uncertainty, selection bias, or consequences of mistakes are ignored. For binary outcomes, coding outcomes as zero and one makes MAE computable for probabilities, yet the score alone does not establish probability calibration or replace an evaluation of discrimination and the decision's loss. Keep the chosen metric aligned with the target and use case.

Missing outcomes and uncertainty in the reported score

An observed MAE uses cases with both an eligible prediction and a valid reference outcome. If outcomes are missing selectively, delayed, censored, or measured with error, a complete-case MAE may misrepresent performance in the target population. Prespecify which cases and time points count and how follow-up or missing data are handled; for survival predictions, use an evaluation method suited to censoring rather than treating unknown event times as observed values.

The sample MAE is an estimate of performance in a target population only if evaluation cases and outcomes support that inference. When comparing two models on the same cases, preserve this pairing in an uncertainty analysis, such as an appropriately designed paired bootstrap; preserve clusters, patients, sites, or time blocks as needed. A confidence interval quantifies sampling variation under its assumptions, not uncertainty from an unsuitable dataset or a mismatched outcome definition. Report sample size and evaluation design alongside the point score.

Checks before using an MAE result

These checks turn a number into a reproducible evaluation. They also prevent a good-looking aggregate score from hiding an unsuitable prediction target. Record decisions before inspecting test performance wherever feasible.

1. Define the target and timing. Specify outcome, unit, price year if relevant, prediction time, forecast horizon, and which information was available then.
2. Protect the evaluation sample. Separate fitting and tuning from testing; use patient, site, and time splits that match deployment and avoid leakage.
3. Match cases and denominators. Verify one prediction against each eligible reference outcome and document missing, excluded, or censored cases.
4. Recalculate and compare. Check absolute errors and denominator, any weights, a baseline, and one or more competing metrics chosen for the decision.
5. Inspect heterogeneity. Report signed errors, tails, and clinically or economically important subgroups when aggregate MAE might conceal harm.
6. State uncertainty and limits. Use an inference method appropriate to the evaluation design and distinguish predictive error from clinical benefit or economic value.

Sources and further study

The software and statistical references below verify the definition, weighting options, and interpretation of absolute residuals. The forecasting paper explains the limitation of comparing unscaled MAE across different series and motivates scaled alternatives.

1. scikit-learn developers. mean_absolute_error API reference. Primary software reference for absolute-error regression loss, sample weighting, and multioutput aggregation.
2. NIST/SEMATECH. Constructing a goodness-of-fit metric. Primary statistical reference for average absolute residuals and comparison with squared-error summaries.
3. Hyndman RJ, Koehler AB. Another look at measures of forecast accuracy. Original forecasting research on scale-dependent measures and mean absolute scaled error.