Concept Architecture
Concept
Theoretically, Least Squares Approximation is a mathematical optimisation method that determines the function or model that best fits a set of observed data by minimising the sum of squared residuals between observed and predicted values. It is founded on optimisation theory and linear algebra and forms the basis of regression analysis, curve fitting and statistical estimation. The method provides an objective criterion for estimating unknown model parameters when observations contain random error.
Mathematically, Least Squares Approximation estimates model parameters by minimising the residual sum of squares (RSS). The method may be applied to linear and non-linear models, although linear least squares possesses a closed-form solution expressed using matrix algebra. Under the assumptions of the Gauss?Markov theorem, least squares estimators are the Best Linear Unbiased Estimators (BLUE).
In practice, Least Squares Approximation is widely used in health economics to estimate regression models, fit cost and outcome relationships, calibrate decision models, predict healthcare expenditure and analyse longitudinal and observational data. It underpins numerous econometric and statistical techniques used throughout health economic evaluation.
Purpose
Used to estimate model parameters by minimising prediction error, fit mathematical models to observed data, support statistical inference and enable health economic modelling and econometric analysis.
Mathematical Formulae
Primary Formula
Residual Sum of Squares
RSS = �(y? ? ??)�
Least Squares Estimator
?? = (X?X)??X?y
where:
- y = observed response vector
- ? = predicted response vector
- X = design matrix
- ?? = estimated parameter vector
Supporting Formulae
Residual
e? = y? ? ??
Mean Squared Error
MSE = RSS � (n ? p)
Linear Prediction
? = X??
Related Mathematical Methods
- Ordinary least squares
- Linear regression
- Multiple regression
- Matrix algebra
- Maximum likelihood estimation
- Gauss?Markov theorem
Example
Suppose observed healthcare costs are modelled using patient age.
Observed costs:
�1,000, �1,200, �1,500
Predicted costs:
�1,050, �1,180, �1,460
Residual Sum of Squares
RSS
= (1000 ? 1050)� + (1200 ? 1180)� + (1500 ? 1460)�
= 2,500 + 400 + 1,600
= 4,500
The least squares procedure selects the parameter estimates that minimise this Residual Sum of Squares, producing the best-fitting regression model under the least squares criterion.
Excel Implementation
| Function | Example Formula | Health Economics Application |
|---|---|---|
| LINEST | =LINEST(B2:B101,A2:A101,TRUE,TRUE) | Estimate least squares regression coefficients. |
| TREND | =TREND(B2:B101,A2:A101,A102) | Predict values from the fitted model. |
| SUMXMY2 | =SUMXMY2(B2:B101,C2:C101) | Calculate the residual sum of squares. |
| MMULT | =MMULT(MINVERSE(MMULT(TRANSPOSE(X),X)),MMULT(TRANSPOSE(X),Y)) | Compute matrix least squares estimates. |
| RSQ | =RSQ(B2:B101,C2:C101) | Assess goodness of fit of the least squares model. |
VBA (Optional)
Automate least squares estimation for regression models, curve fitting and calibration of health economic models while generating parameter estimates, residual diagnostics and model performance reports.
Sources
- Golub GH, Van Loan CF. Matrix Computations.
- Bj�rck �. Numerical Methods for Least Squares Problems.
- Seber GAF, Lee AJ. Linear Regression Analysis.
- Draper NR, Smith H. Applied Regression Analysis.
- Greene WH. Econometric Analysis.
- Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.
Related Concepts (3)
Library
Publications
1
Accuracy and Stability of Numerical Algorithms — Nicholas J. Higham, 2nd Edition ed., 2002 (Society for Industrial and Applied Mathematics (SIAM))
The standard reference on floating-point error, conditioning, stability and the numerical reliability of matrix, polynomial and linear-algebra algorithms.
BookView source →
Frequently Asked Questions (6)
What is least squares approximation?
A method of estimating a model or function by minimising the sum of the squared differences between observed and predicted values.
Source: Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
How does least squares approximation fit a model?
Least squares approximation fits a model or function by minimising the sum of the squared differences between observed and predicted values. It chooses the model that makes those squared discrepancies as small as possible in total, so the fit sits as close to the data as the model allows. Minimising this sum of squared differences is the defining criterion of least squares approximation By penalising larger gaps more heavily and preventing positive and negative errors from cancelling, the squared-difference criterion gives a stable, widely used basis for fitting a model to data.
Source: Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
What quantity does least squares approximation minimise?
Least squares approximation minimises the sum of the squared differences between observed and predicted values, the total of the squared gaps between the data and the model's estimates. Squaring the differences penalises larger discrepancies more heavily and keeps the total non-negative. Making this sum of squared differences as small as possible is what least squares approximation does By penalising larger gaps more heavily and preventing positive and negative errors from cancelling, the squared-difference criterion gives a stable, widely used basis for fitting a model to data.
Source: Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
Why does least squares approximation square the differences?
Least squares approximation squares the differences so that positive and negative discrepancies do not cancel and so that larger errors weigh more in the total being minimised. Because it estimates a model by minimising the sum of these squared differences between observed and predicted values, squaring gives a well-behaved quantity to minimise. This squaring is intrinsic to how least squares approximation measures fit By penalising larger gaps more heavily and preventing positive and negative errors from cancelling, the squared-difference criterion gives a stable, widely used basis for fitting a model to data.
Source: Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
What does least squares approximation produce?
Least squares approximation produces the model or function whose predicted values minimise the sum of the squared differences from the observed values. The result is the best fit within the chosen model form according to that squared-difference criterion. Yielding this best-fitting model is the outcome of applying least squares approximation By penalising larger gaps more heavily and preventing positive and negative errors from cancelling, the squared-difference criterion gives a stable, widely used basis for fitting a model to data.
Source: Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
How does least squares approximation relate to polynomial approximation?
Least squares approximation estimates a model by minimising the sum of squared differences between observed and predicted values, while polynomial approximation represents a function or dataset using a polynomial that gives a sufficiently accurate estimate. When the model chosen is a polynomial, least squares approximation provides one way to fit it to data. The two are connected, since least squares supplies a fitting criterion that a polynomial approximation can be built on By penalising larger gaps more heavily and preventing positive and negative errors from cancelling, the squared-difference criterion gives a stable, widely used basis for fitting a model to data.
Source: Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
Trust Record
Verified by Dr Darrin Baines
British health economist
Professional identity: darrinbaines.org
Verification date: 3 Apr 2026
Content version: 1.0.0
Canonical Identity
- Term code
- CS-NA-AP-001
Stable URI · Machine-readable · Resolvable · CC BY 4.0