Gradient Descent

Gradient Descent: Concept Architecture

Orientation and learning roadmap

Gradient descent connects a clearly specified objective function to a disciplined sequence of parameter updates. The architecture below develops that connection from the core update rule through step-size choice, scaling, convergence, algorithmic variants, implementation, and validation in health-economic modelling and analysis.

What gradient descent does

Gradient descent searches for parameter values that minimize a differentiable objective function. At the current point, the gradient identifies the direction of steepest local increase under the chosen coordinate scale, so moving in the negative-gradient direction gives a local descent direction when the gradient is non-zero. Repeating this calculation can approach a local or global minimum depending on the objective and algorithm settings.

For objective (J(\boldsymbol{\theta})), parameter vector (\boldsymbol{\theta}_k), and positive step size (\alpha_k):

[ \boldsymbol{\theta}_{k+1}=\boldsymbol{\theta}_k-\alpha_k\nabla J(\boldsymbol{\theta}_k) ]

The gradient contains one partial derivative for each parameter:

[ \nabla J(\boldsymbol{\theta})= \begin{bmatrix} \partial J/\partial\theta_1\ \vdots\ \partial J/\partial\theta_p \end{bmatrix} ]

The update uses only first-derivative information. It does not directly account for curvature, constraints, parameter uncertainty, or whether the objective represents the correct health-economic question.

Define the optimization problem first

An optimizer can only minimize the objective it is given. The parameter domain, data targets, weights, penalties, and transformations determine what a low objective value means. In health-economic calibration, these choices can influence both fitted parameters and downstream cost-effectiveness results.

A weighted least-squares calibration objective may be written as:

[ J(\boldsymbol{\theta})=\sum_{j=1}^{m}w_j\left[g_j(\boldsymbol{\theta})-y_j\right]^2 ]

where (g_j(\boldsymbol{\theta})) is model output (j), (y_j) is its calibration target, and (w_j) controls its contribution. Weights may reflect variance, sample size, clinical importance, correlation, or another justified criterion. Arbitrary weights can cause an apparently successful fit to prioritise some targets while effectively ignoring others.

Understand the geometry of the update

The gradient is perpendicular to a contour of equal objective value and points toward the greatest local increase in ordinary Euclidean coordinates. A sufficiently small step in the opposite direction lowers a smooth objective. The path can nevertheless zigzag or advance slowly when contours are elongated or parameters have very different scales.

For a first-order approximation:

[ J(\boldsymbol{\theta}+\mathbf{s})\approx J(\boldsymbol{\theta})+\nabla J(\boldsymbol{\theta})^T\mathbf{s} ]

Choosing (\mathbf{s}=-\alpha\nabla J(\boldsymbol{\theta})) gives:

[ J(\boldsymbol{\theta}+\mathbf{s})\approx J(\boldsymbol{\theta})-\alpha|\nabla J(\boldsymbol{\theta})|^2 ]

for positive (\alpha). This is a local argument; a step that is too large can leave the neighbourhood where the approximation is useful and increase the objective.

Choose the step size carefully

The step size, often called the learning rate, controls how far the algorithm moves at each iteration. A very small value produces slow progress, while a large value can overshoot a minimum, oscillate, or diverge. The appropriate scale depends on curvature, parameter scaling, and the region of the objective surface.

Common strategies include:

• A fixed step size selected through theory, pilot runs, or sensitivity testing.
• A decreasing schedule that allows larger early moves and smaller later refinements.
• Backtracking line search that reduces the step until a sufficient decrease condition is met.
• Exact line search that minimizes the objective along the descent direction when that one-dimensional problem is tractable.
• Adaptive methods that scale steps using recent gradient information, with additional assumptions and tuning requirements.

For a smooth convex objective with (L)-Lipschitz gradient, step sizes below a relevant bound such as (2/L) can support convergence under standard conditions. In applied work (L) is often unknown, making line search and empirical diagnostics important.

A worked quadratic example

Consider the one-parameter objective (J(\theta)=(\theta-3)^2). Its unique minimum occurs at (\theta=3), and its derivative is (J'(\theta)=2(\theta-3)). Starting at (\theta_0=0) with step size (\alpha=0.25), the update is:

[ \theta_{k+1}=\theta_k-0.25[2(\theta_k-3)] ]

Each iteration halves the remaining distance to the minimum. With (alpha=1), the update would alternate between 0 and 6 and would not converge; with a still larger step, the iterates would diverge. The example shows that even a simple convex objective requires an appropriate step size.

Convex and non-convex objectives

When a differentiable objective is convex, every local minimum is global, and gradient descent can converge to a global solution under suitable smoothness and step-size conditions. Strong convexity can provide faster and more specific convergence guarantees. Many health models, likelihoods, neural networks, and calibration objectives are non-convex, so those guarantees do not apply.

In a non-convex problem, gradient descent may reach a local minimum, a saddle region, a flat plateau, or a boundary determined by transformations. Different initial values can produce different solutions with similar objective values but different policy implications. Multiple starts and alternative optimization methods are therefore substantive validation tools, not merely computational extras.

Parameter scaling changes the path

Ordinary gradient descent depends on the coordinate system. If one parameter is measured in thousands and another in fractions, the gradient components can differ greatly because of units rather than genuine importance. Poor scaling can produce slow zigzagging and make one global step size unsuitable.

Useful responses include standardising parameters, optimizing on log or logit scales, dividing by credible reference values, or using a preconditioner. Transformations should also preserve valid domains: probabilities can be represented with logits, positive rates with logarithms, and simplex parameters with suitable constrained transformations. The results must be transformed back and interpreted on the original health-economic scale.

Calculate gradients accurately

Analytic derivatives can be efficient and accurate when they are derived and implemented correctly. Automatic differentiation applies the chain rule through program operations and is often preferable for complex differentiable models. Finite differences are simple but introduce truncation and rounding error and can be unreliable for noisy or discontinuous objectives.

For parameter (i), a central finite-difference approximation is:

[ \frac{\partial J}{\partial\theta_i}\approx \frac{J(\boldsymbol{\theta}+h\mathbf{e}_i)-J(\boldsymbol{\theta}-h\mathbf{e}_i)}{2h} ]

The perturbation (h) must be large enough to avoid cancellation and small enough to approximate the local derivative. A gradient check compares analytic or automatic derivatives with finite differences at several valid parameter points, including points away from the optimum.

Batch, stochastic, and mini-batch variants

Batch gradient descent computes the objective and gradient using the complete dataset or full deterministic model at every iteration. Stochastic gradient descent uses one observation or simulation contribution at a time, while mini-batch methods use a subset. These variants reduce computation per update but introduce gradient noise and require step-size schedules and reproducibility controls.

In stochastic health simulations, objective noise may also arise from random model runs rather than data subsampling. Common random numbers, larger simulation samples, smoothing, or simulation-specific optimization methods may be needed so that the algorithm follows a meaningful signal rather than Monte Carlo fluctuation.

Apply gradient descent to model calibration

Health-economic model calibration estimates uncertain inputs so selected model outputs match observed targets. Gradient descent can minimize a calibration loss when the model is differentiable or when useful gradients can be approximated. A low loss indicates agreement with the chosen targets and weights, not proof that the calibrated model is valid for every outcome or decision.

Suppose a model produces incidence (g_1(\boldsymbol{\theta})) and mortality (g_2(\boldsymbol{\theta})), with observed targets (y_1) and (y_2). A scaled objective might be:

[ J(\boldsymbol{\theta})= \left(\frac{g_1(\boldsymbol{\theta})-y_1}{s_1}\right)^2+ \left(\frac{g_2(\boldsymbol{\theta})-y_2}{s_2}\right)^2 ]

where (s_1) and (s_2) reflect sampling uncertainty or justified target scales. Calibration should retain target-specific residuals, not report only the aggregate loss, because opposing or uneven errors can be hidden within the sum.

Constraints need explicit handling

The basic update is unconstrained and can propose invalid values. Health-economic parameters often require positivity, probability bounds, ordering, or sum-to-one constraints. Ignoring those constraints can make the model fail or create mathematically converged but impossible estimates.

Projected gradient descent applies an unconstrained step and projects it back to the feasible set (\mathcal{C}):

[ \boldsymbol{\theta}{k+1}=\Pi{\mathcal{C}}\left(\boldsymbol{\theta}_k-\alpha_k\nabla J(\boldsymbol{\theta}_k)\right) ]

Reparameterisation can enforce constraints smoothly, while penalty and barrier methods incorporate constraint violations into the objective. Each approach changes the optimization geometry and should be documented; silently clipping values can trap the algorithm at boundaries and invalidate derivative interpretation.

Choose defensible stopping rules

Stopping because the maximum iteration count was reached is not evidence of convergence. A credible stopping decision uses several diagnostics, such as small gradient norm, small parameter change, small objective change, and satisfaction of constraints. Tolerances should be linked to numerical and decision relevance rather than copied without context.

Possible criteria include:

[ |\nabla J(\boldsymbol{\theta}_k)|\leq\varepsilon_g ]

[ \frac{|J_k-J_{k-1}|}{1+|J_{k-1}|}\leq\varepsilon_J ]

[ \frac{|\boldsymbol{\theta}k-\boldsymbol{\theta}{k-1}|}{1+|\boldsymbol{\theta}{k-1}|}\leq\varepsilon\theta ]

A small gradient can occur at a saddle point or broad plateau, and a small step can result from an excessively small learning rate. Diagnostics must therefore be interpreted together with objective history, multiple starts, curvature or perturbation checks, and comparison with alternative solutions.

Implement the algorithm transparently

A reproducible implementation records the initial point, transformations, objective definition, gradient method, step rule, stopping criteria, iteration limit, random seed, software version, and final diagnostics. The iteration history should be retained so reviewers can see whether the objective decreased, stalled, oscillated, or diverged. Health-economic outputs should be recalculated at the final parameter set and reconciled with the calibration targets.

A basic algorithm is:

1. Initialise parameters. Select a valid starting vector and apply the declared transformation.
2. Evaluate the objective. Store the total loss and target-specific components.
3. Calculate the gradient. Use verified analytic, automatic, or numerical derivatives.
4. Choose the step size. Apply the fixed, scheduled, or line-search rule.
5. Update parameters. Move in the negative-gradient direction and enforce constraints.
6. Re-evaluate the model. Confirm that the candidate step produces the intended descent or satisfies the acceptance rule.
7. Check stopping criteria. Evaluate gradient, objective, parameter, constraint, and iteration diagnostics.
8. Repeat and retain history. Continue until justified convergence or a documented failure condition.

Validate the optimization

Optimization validation asks both whether the algorithm solved its numerical problem and whether that problem supports the intended decision model. A single successful run cannot separate a robust solution from a fortunate starting point. Validation should examine gradients, starts, tolerances, targets, alternative algorithms, and downstream decisions.

1. Check gradients. Compare implemented derivatives with finite differences at multiple points.
2. Check descent. Confirm that accepted steps satisfy the declared decrease condition.
3. Use multiple starts. Compare solutions from dispersed valid initial values.
4. Tighten tolerances. Determine whether parameters and decision outputs remain stable.
5. Change scaling. Confirm that equivalent parameterisations lead to compatible solutions.
6. Compare algorithms. Use a second credible optimizer or direct solution for a simplified case.
7. Inspect residuals. Review every calibration target, not only the total objective.
8. Validate externally. Compare outputs not used in fitting with independent evidence where possible.
9. Test decision impact. Determine whether alternative well-fitting solutions change costs, QALYs, net benefit, or the preferred strategy.

Distinguish optimization from uncertainty analysis

Gradient descent produces a point estimate or local optimum; it does not by itself quantify parameter uncertainty. The path of the optimizer is not a posterior distribution, and repeated starts are not probabilistic sensitivity analysis. Uncertainty requires a justified statistical or decision-analytic framework.

After calibration, uncertainty may be assessed through likelihood-based methods, Bayesian posterior sampling, bootstrap methods, profile analysis, or other approaches suited to the calibration problem. Propagating only the single best-fitting vector through the economic model can materially understate uncertainty.

Compare gradient descent with other methods

Gradient descent is attractive when gradients are available and the parameter dimension is large. It can be slow on ill-conditioned objectives and unsuitable for discontinuous, noisy, or derivative-free problems. Alternative methods use different information and have different computational and robustness properties.

Algorithm choice should follow the objective's smoothness, dimension, noise, constraints, available derivatives, and need for uncertainty. Selecting gradient descent merely because it is familiar can be inefficient or misleading.

Common errors and safeguards

Gradient descent can report success even when the objective, derivatives, or stopping logic are wrong. The clearest safeguards are explicit problem definition, gradient checks, iteration histories, multiple starts, and decision-level validation. Numerical convergence should never be treated as a substitute for substantive model validation.

• Do not use raw parameter scales that differ greatly without checking conditioning.
• Do not treat a lower objective as evidence that every calibration target improved appropriately.
• Do not assume convergence to a global minimum for a non-convex model.
• Do not use an excessively small learning rate and mistake stalled progress for convergence.
• Do not use an excessively large learning rate that causes oscillation or divergence.
• Do not clip invalid parameters silently after each step.
• Do not compute finite-difference gradients with one arbitrary perturbation for every parameter scale.
• Do not ignore randomness in a simulation-based objective.
• Do not report only optimized parameters without initial values, bounds, objective, and diagnostics.
• Do not interpret point optimization as a complete analysis of parameter uncertainty.

Interpreting gradient-descent results responsibly

Gradient descent returns a solution conditional on the objective, data targets, weights, transformations, constraints, starting values, derivative calculation, step rule, and stopping tolerance. A strong analysis reports all of these elements and shows that alternative credible implementations do not materially alter the decision. If several parameter sets fit similarly but produce different policy conclusions, that uncertainty is a substantive finding rather than an inconvenience to conceal.

The method is most useful when the objective is differentiable and the gradient can be calculated reliably. In health-economic applications, the ultimate standard is not merely a low loss but a calibrated, externally credible, transparent model whose uncertainty and decision consequences have been evaluated.