AIC difference from the best-scoring candidate model

Rescales each model's AIC to its distance from the smallest AIC in the candidate set, so the best model scores 0 and the arbitrary constant cancels. On Burnham and Anderson's rule of thumb, a difference of 2 or less indicates substantial support, 4 to 7 considerably less, and above 10 almost none.

Signature

delta = AIC_i - AIC_min
Inputs
InputsDefinitionUnit
AIC_iAIC of the candidate model being comparedAIC units
AIC_minSmallest AIC among all candidate models fitted to the same dataAIC units
Output
deltaDifference between the AIC of model i and the smallest AIC in the set; 0 for the best-scoring modelAIC units

Function

Penalised-likelihood ranking of candidate survival models

Maps the maximised log-likelihood and the number of estimated parameters of each model fitted to the same data to an information criterion, a fit statistic with a penalty for each extra parameter. Lower values rank higher, and only differences between models fitted to the same observations carry meaning. In health technology assessment the candidates are usually parametric survival curves fitted to trial time-to-event data before extrapolation. The criteria measure fit within follow-up only, so they inform but do not settle the choice of curve.

Try this function

Implementations

  • Excel

    AIC difference in a column of candidate models

    With the AIC values of all candidates in a range named AICRange and the AIC of the current model in a cell named AICValue, the formula returns its difference from the best-scoring model.

    =AICValue-MIN(AICRange)

Assumptions

  • Differences read within one candidate set

    All AIC values come from models fitted to the same observations, and AIC_min is taken over the whole candidate set. The rule-of-thumb bands are rough guides to the strength of evidence, not significance thresholds.

Worked examples

  • Weibull curve 3.8 units behind the log-normal

    In the six-curve example the Weibull has an AIC of 1214.2 and the log-normal the minimum of 1210.4. The Weibull is 3.8 units behind, with less support than the log-normal on the rule of thumb.

    AIC_i = 1214.2; AIC_min = 1210.4; delta = 3.8

  • Generalised gamma curve 1.2 units behind the log-normal

    The generalised gamma, at 1211.6, is 1.2 units behind the log-normal and within the band of substantial support.

    AIC_i = 1211.6; AIC_min = 1210.4; delta = 1.2

  • Exponential curve 16.4 units behind the log-normal

    The exponential, at 1226.8, is 16.4 units behind and has almost no support in the set.

    AIC_i = 1226.8; AIC_min = 1210.4; delta = 16.4

Common errors

  • Settling a near-tie by the smallest AIC

    A difference of 1.2 between the log-normal and the generalised gamma does not separate them. A near-tie signals structural uncertainty, which belongs in scenario analysis rather than being settled by the smallest number.

  • Interpreting a single AIC value on its own

    An AIC of 1210.4 says nothing by itself, because individual AIC values contain an arbitrary constant and change with the sample size. Only differences between models fitted to the same data can be read.

Sources

  • AIC differences and the rule of thumb for support

    Burnham KP, Anderson DR. Multimodel inference: understanding AIC and BIC in model selection. Sociological Methods & Research. 2004;33(2):261-304. Page 271, which defines the difference from the minimum AIC and gives the rule of thumb for substantial, considerably less and almost no support.

    View source →

Canonical Identity

Persistent URI
https://healtheconomics.wiki/formula/aic-difference-from-the-best-scoring-candidate-model

Stable URI · Machine-readable · Resolvable · CC BY 4.0

AIC difference from the best-scoring candidate model | HealthEconomics.wiki