ROC Curve

Concept

---

Theoretically, Receiver Operating Characteristic (ROC) Curve is a graphical method for evaluating the diagnostic or predictive performance of a binary classification model across all possible decision thresholds. Originally developed in signal detection theory, the ROC curve illustrates the trade-off between sensitivity and specificity by plotting the true positive rate against the false positive rate. It provides a threshold-independent assessment of a model's ability to discriminate between individuals with and without the outcome of interest.

Mathematically, the ROC curve is constructed by calculating sensitivity and false positive rate for every possible classification threshold. The overall discriminative performance of the model is commonly summarised by the Area Under the ROC Curve (AUROC or AUC), which represents the probability that a randomly selected positive case receives a higher predicted score than a randomly selected negative case. Values range from 0.5, indicating no discrimination beyond chance, to 1.0, indicating perfect discrimination.

In practice, ROC curves are generated from predicted probabilities or diagnostic test scores and are widely used to evaluate clinical prediction models, diagnostic tests, risk scores, and machine learning classifiers. In health economics they are used when assessing prognostic models, disease screening strategies, treatment selection algorithms, and health technology assessments in which diagnostic accuracy influences costs, health outcomes, or resource allocation.

Purpose

---

Used to evaluate and compare the discriminatory performance of diagnostic tests and prediction models across all classification thresholds, supporting model selection, threshold determination, and decision-making in health economics.

---

Mathematical Formulae

Primary Formula

ROC = {(FPR(t), TPR(t)) : t ? Thresholds}

where:

TPR(t) = TP � (TP + FN)

FPR(t) = FP � (FP + TN)

Supporting Formulae

Sensitivity:

Sensitivity = TP � (TP + FN)

Specificity:

Specificity = TN � (TN + FP)

False Positive Rate:

FPR = 1 ? Specificity

Area Under the Curve:

AUC = ??? TPR(FPR) d(FPR)

Related Mathematical Methods

• Area Under the Curve (AUC)
• Sensitivity
• Specificity
• Confusion Matrix
• Diagnostic Accuracy
• Binary Classification
• Logistic Regression
• Classification Threshold Analysis

---

Example

A health economist evaluates a prediction model identifying patients at high risk of hospital readmission.

At one decision threshold:

• True Positives = 90
• False Positives = 30
• True Negatives = 170
• False Negatives = 10

Sensitivity:

90 � (90 + 10) = 0.90

False Positive Rate:

30 � (30 + 170) = 0.15

Repeating these calculations across all possible thresholds produces the ROC curve. The calculated AUROC is 0.91, indicating excellent discrimination between patients who are and are not subsequently readmitted.

---

Excel Implementation

Function	Example Formula	Health Economics Application
IF	=IF(B2>=Threshold,1,0)	Classify observations at a specified decision threshold.
COUNTIFS	=COUNTIFS(PredictedRange,1,ObservedRange,1)	Calculate true positives, false positives, true negatives, and false negatives.
SORT	=SORT(ProbabilityRange,-1)	Order predicted probabilities to construct ROC coordinates.
SUMPRODUCT	=SUMPRODUCT((PredictedRange=1)*(ObservedRange=1))	Calculate classification counts used in ROC analysis.
TRAPZ*	=((B3-B2)*(C3+C2))/2	Approximate the area under the ROC curve using the trapezoidal rule.

*Implemented by copying the trapezoidal calculation across adjacent ROC coordinates.

VBA (Optional)

Automate ROC curve construction, AUROC calculation using the trapezoidal rule, and comparison of multiple prediction models.

---

Sources

• Hanley JA, McNeil BJ. The Meaning and Use of the Area Under a Receiver Operating Characteristic (ROC) Curve. Radiology. 1982.
• Fawcett T. An Introduction to ROC Analysis. Pattern Recognition Letters. 2006.
• Pepe MS. The Statistical Evaluation of Medical Tests for Classification and Prediction.
• Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression.
• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation.
• Drummond MF, et al. Methods for the Economic Evaluation of Health Care Programmes.
• NICE Health Technology Evaluation Manual.
• ISPOR Good Practice Reports.