Signature
B_12 = exp((BIC_2 - BIC_1) / 2)
| Inputs | Definition | Unit |
|---|---|---|
BIC_2 | BIC of model 2, computed from the same data and the same penalty count as BIC_1 | none |
BIC_1 | BIC of model 1, the model whose support is measured | none |
B_12 | Approximate Bayes factor in favour of model 1 over model 2, the factor that converts prior odds on the two models into posterior odds | ratio |
|---|
Function
Schwarz criterion for ranking survival curves and approximating Bayes factors
Maps the maximised log-likelihood, the number of estimated parameters and a sample count for each model fitted to the same data to the Bayesian information criterion: minus twice the log-likelihood plus the parameter count times the natural log of the count. Lower values rank higher, and the difference between two models approximates twice the natural log of the Bayes factor between them, which also yields approximate posterior model probabilities. The patient-count form, with its worked examples, is recorded on the Akaike information criterion page; this page adds the event-count penalty for censored survival data and the Bayesian readings of BIC differences. Like AIC, the criterion describes fit within follow-up only.
Try this function
Implementations
Excel
Approximate Bayes factor from two BIC cells
Excel exponentiates half the difference between named cells holding the two BIC values.
=EXP((BIC2-BIC1)/2)
Excel
Kass and Raftery category from a BIC difference
Returns the evidence category for the difference between a model and the best-scoring model, held in a cell named BICDiff. A difference that falls exactly on 2, 6 or 10 is placed in the weaker category by this formula.
=IF(BICDiff<=2,"Not worth more than a bare mention",IF(BICDiff<=6,"Positive",IF(BICDiff<=10,"Strong","Very strong")))
Assumptions
BIC approximates the log Bayes factor, not the Bayes factor itself
The approximation works on the logarithmic scale: its relative error as an estimate of the log Bayes factor tends to zero as the sample grows, but the exponentiated value keeps an error of constant order even in very large samples. The result indicates the strength of evidence without giving the Bayes factor exactly.
Closer approximation under a unit-information prior
The approximation is closer when the prior on the extra parameters carries about as much information as a single observation, the unit-information prior. A full Bayesian analysis with explicit priors computes the Bayes factor directly instead.
Kass and Raftery categories as a descriptive scale
Kass and Raftery present their categories, adapted from Jeffreys, as a rough descriptive statement about standards of evidence rather than a calibration of the Bayes factor, and accept that the reading may depend on context.
Worked examples
Generalised gamma over exponential with 120 deaths
Under the event count the exponential is 7.83 behind the generalised gamma, strong evidence on the Kass and Raftery scale, with an approximate Bayes factor of about 50 in favour of the generalised gamma.
BIC_1 = 1056.96; BIC_2 = 1064.79; B_12 = 50.1
Generalised gamma over Weibull with 120 deaths
The Weibull is 0.61 behind the generalised gamma under the event count, an approximate Bayes factor of about 1.36, which is not worth more than a bare mention.
BIC_1 = 1056.96; BIC_2 = 1057.57; B_12 = 1.36
BIC difference of 10 at the edge of very strong evidence
A BIC difference of 10 gives an approximate Bayes factor of about 148, close to the Bayes factor of 150 that separates strong from very strong evidence in the Kass and Raftery table.
BIC_1 = 1000; BIC_2 = 1010; B_12 = 148.41
Common errors
Exponentiating the whole BIC difference without halving it
Taking exp of the whole difference squares the approximate Bayes factor. For the exponential against the generalised gamma, a difference of 7.83 would give about 2,515 instead of about 50, far into the range the table calls very strong.
Treating BIC differences either side of a category edge as different verdicts
A BIC difference only approximates twice the log Bayes factor, so differences of 1.9 and 2.1, which fall either side of the first category edge, carry practically the same evidence. Treating them as different verdicts reads more precision into the scale than it holds.
Sources
Kass and Raftery on the Schwarz approximation and evidence categories
Kass RE, Raftery AE. Bayes factors. Journal of the American Statistical Association. 1995;90(430):773-795. Section 4.1.3, which shows that the Schwarz criterion is a rough approximation to the log Bayes factor and that minus twice it is BIC, and section 3.2, which gives the categories for twice the natural log of the Bayes factor.
Schwarz large-sample Bayes solution for choosing model dimension
Schwarz G. Estimating the dimension of a model. Annals of Statistics. 1978;6(2):461-464. Abstract and section 1, which obtain the criterion from the leading terms of the large-sample expansion of the Bayes solution, terms that do not depend on the prior.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0