Gradient Descent

Concept

Theoretically, Gradient Descent is an iterative optimisation algorithm that identifies a local minimum of an objective function by repeatedly moving in the direction of the negative gradient. The method is founded on differential calculus and numerical optimisation and exists to estimate model parameters that minimise a specified loss or objective function. In health economics, Gradient Descent is used primarily for parameter estimation, machine learning applications, model calibration and optimisation of predictive models rather than for routine economic evaluation itself.

Mathematically, Gradient Descent updates model parameters iteratively by subtracting the product of the learning rate and the gradient of the objective function. The gradient indicates the direction of the steepest increase in the objective function; therefore, movement in the opposite direction reduces the objective value. The optimisation continues until convergence criteria are met, such as negligible parameter changes or sufficiently small gradients.

In practice, Gradient Descent is implemented when fitting statistical or machine learning models to health economic data, calibrating simulation models and optimising prediction algorithms. Analysts select an appropriate learning rate and stopping criterion before estimation begins. Variants including batch, stochastic and mini-batch Gradient Descent are chosen according to dataset size and computational efficiency requirements.

---

Purpose

Used to estimate model parameters by iteratively minimising objective functions in statistical models, machine learning algorithms and health economic model calibration.

---

Mathematical Formulae

Primary Formula

Parameter update rule:

???? = ?? ? �?J(??)

where:

• ?? = parameter vector at iteration t
• � = learning rate
• ?J(??) = gradient of the objective function
• J(?) = objective (loss) function

Supporting Formulae

Objective function:

? = arg min??? J(?)*

Gradient vector:

?J(?) = (?J/???, ?J/???, ?, ?J/???)?

Related Mathematical Methods

• Numerical optimisation
• Convex optimisation
• Maximum likelihood estimation
• Nonlinear regression
• Machine learning
• Stochastic Gradient Descent
• Newton-Raphson optimisation
• Quasi-Newton methods

---

Example

A researcher calibrates a disease progression model by minimising the sum of squared differences between observed and predicted survival probabilities.

The objective function is initially 240.6. Using a learning rate of 0.01, Gradient Descent updates the model parameters iteratively. After 350 iterations, the objective function decreases to 18.4, indicating substantially improved agreement between model predictions and observed epidemiological data.

---

Excel Implementation

Function	Example Formula	Health Economics Application
Solver	Minimise objective cell by changing parameters	Estimate model parameters through optimisation.
SUMXMY2	=SUMXMY2(B2:B101,C2:C101)	Calculate the sum of squared prediction errors.
SUMSQ	=SUMSQ(D2:D20)	Evaluate objective functions involving squared residuals.
ABS	=ABS(NewValue-OldValue)	Assess convergence between successive parameter estimates.

---

VBA (Optional)

Automate iterative parameter updates, convergence testing and optimisation reporting for statistical and simulation models.

---

Sources

• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.
• Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed.
• Nocedal J, Wright SJ. Numerical Optimization. 2nd ed.
• Bishop CM. Pattern Recognition and Machine Learning.
• Goodfellow I, Bengio Y, Courville A. Deep Learning.