Functions & Formulae

Each applied formula has its own function page, with a signature, implementations, and tests.

Bayesian optimisation of a model calibration loss with a Gaussian process surrogate

theta_star = argmin L(theta); theta_next = argmax EI_n(theta)

Searches a bounded parameter region for the values that minimise a goodness-of-fit loss between model outputs and calibration targets when each model run is expensive. A Gaussian process fitted to the losses already observed gives a normal posterior for the loss at any untried value, and an acquisition function, here expected improvement, chooses the next run. The notation follows the Bayesian Optimisation article and its one-parameter incidence calibration.

  • Weighted absolute-difference calibration loss for two targets

    L = w_1 * abs(g_1 - T_1) + w_2 * abs(g_2 - T_2)

    Folds the distances between model predictions and calibration targets into one weighted score for the search to minimise. The article's general form sums over J targets with any distance d; two targets with the absolute difference are written out here, and a single target is the case w_2 = 0. Each evaluation needs a full model run, so the loss has no formula or derivatives the search can use.

  • Squared exponential kernel covariance between losses at two parameter values

    c_ab = exp(-(theta_a - theta_b)^2 / (2 * ell^2)); k_ab = sf2 * c_ab

    Gives the prior covariance between the calibration losses at two parameter values: the prior variance times a correlation that falls with the squared distance between them, scaled by the length-scale. Nearby values are expected to have similar losses. For several parameters the squared difference becomes the squared Euclidean distance.

  • Gaussian process posterior mean and variance of the loss after two noise-free runs

    a_1 = (c_1 - c_12 * c_2) / (1 - c_12^2); a_2 = (c_2 - c_12 * c_1) / (1 - c_12^2); mu_n = mu_0 + a_1 * (y_1 - mu_0) + a_2 * (y_2 - mu_0); var_n = sf2 * (1 - (c_1 * a_1 + c_2 * a_2))

    Writes the posterior mean and variance at an untried value in closed form when two runs have been made without noise. The weights a_1 and a_2 solve the two-by-two correlation system of the runs; the posterior mean moves the prior mean towards the observed losses by those weights, and the posterior variance is the prior variance less the part the runs explain. Correlations come from the kernel (HE-FM-BOPT-002); for more runs or noisy runs the matrix form is HE-CF-BOPT-001.

  • Expected improvement of a candidate run for a loss that is minimised

    z = (L_best - mu_n) / s_n; EI = (L_best - mu_n) * Phi_z + s_n * exp(-z^2 / 2) / 2.5066283

    Values a candidate by how far its loss is expected to fall below the best loss so far, given a normal posterior with mean mu_n and standard deviation s_n. The first term rewards candidates predicted to fit well and the second rewards uncertainty. The site's calculator has no normal distribution function, so Phi_z, the standard normal distribution function at z, is entered as an input (NORM.S.DIST(z, TRUE) in Excel, pnorm(z) in R); the normal density is computed as exp(minus z^2 / 2) / 2.5066283, the constant being the square root of 2 times pi.