Signature
S_diff = sqrt(2) * SEM; MDC = 1.96 * S_diff; RC = (x_2 - x_1) / S_diff
| Inputs | Definition | Unit |
|---|---|---|
SEM | Standard error of measurement of one score (HE-FM-DMID-002) | instrument units |
x_2 | Person's score after the change | instrument units |
x_1 | Person's score before the change | instrument units |
S_diff | Standard error of a change score under measurement error alone | instrument units |
|---|---|---|
MDC | Smallest change in one person's score that exceeds measurement error at the 95 per cent level | instrument units |
RC | Change divided by the standard error of the difference; beyond 1.96 in size is reliable change at the 5 per cent level | standard errors |
Function
Distribution-based yardsticks for a minimal important difference in patient-reported and utility scores
Expresses an important or detectable change in a questionnaire or utility score as a multiple of the spread of scores or of the instrument's measurement error: half a baseline standard deviation, a benchmark effect size, one standard error of measurement, the minimally detectable change and the reliable change index for one person. These yardsticks describe precision, not what patients value, and methodological and regulatory guidance ranks them behind anchor-based estimates. Standardised mean differences themselves are HE-FM-CD-001 on the Cohen's d page. Notation follows the Distribution-Based MID article.
Computational function
Computational function: distribution-based yardsticks and reliable change classification for a cohort of paired scores
Computes the five yardsticks of the article from a sample's baseline standard deviation and reliability, then classifies every patient's change as a reliable gain, a reliable loss or neither and returns the share in each class. The inputs differ from the formulas': vectors of paired scores for a cohort, the baseline standard deviation and the reliability, with the 1.96 multiplier as an option.
Inputs and outputs:
x1,x2: Each patient's scores at the two measurements. Unit: instrument units.;sd0: Baseline standard deviation (HE-FM-DMID-001). Unit: instrument units.;rel: Test-retest reliability. Unit: coefficient.;z: Normal multiplier, default 1.96. Unit: none.;half_sd,es_small: Half a standard deviation and 0.2 standard deviations. Unit: instrument units.;sem,sdiff,mdc: SEM, standard error of the difference and MDC (HE-FM-DMID-002, HE-FM-DMID-003). Unit: instrument units.;change,rc: Each patient's change and reliable change index. Unit: instrument units; standard errors.;reliable_gain,reliable_loss: Whether RC is above z or below minus z. Unit: logical.;share_gain,share_loss: Share of patients with reliable gain and with reliable loss. Unit: proportion.Assumption: One SEM applies to every patient and both measurements, with independent normal errors; the shares describe individual change, not a difference between arms.
Worked example (Five patients against the article's yardsticks): With a baseline standard deviation of 0.20 and reliability of 0.80, the yardsticks are 0.10, 0.04, 0.0894, 0.1265 and 0.2479 as in the article (MDC carried unrounded). The first patient is the article's, rising from 0.60 to 0.75 (RC 1.19); four more, rising from 0.50 to 0.80, falling from 0.70 to 0.65, rising from 0.40 to 0.70 and rising from 0.55 to 0.75, have RC of 2.37, minus 0.40, 2.37 and 1.58, so 2 of 5 patients, a share of 0.4, show reliable gain (computed here for illustration).
x1 = 0.60, 0.50, 0.70, 0.40, 0.55; x2 = 0.75, 0.80, 0.65, 0.70, 0.75; sd0 = 0.2; rel = 0.8; mdc = 0.2479; share_gain = 0.4Excel: With the scores in ScoreT1Vec and ScoreT2Vec, SEMVal and SDiffVal from HE-FM-DMID-002 and HE-FM-DMID-003, the array formula
=(ScoreT2Vec-ScoreT1Vec)/SDiffValreturns the indices into RCVec,=AVERAGE(--(RCVec>1.96))the share with reliable gain and=AVERAGE(--(RCVec<-1.96))the share with reliable loss.R:
rel_change <- function(x1, x2, sd0, rel, z = 1.96) { sem <- sd0*sqrt(1-rel); sdiff <- sqrt(2)*sem; rc <- (x2-x1)/sdiff; list(half_sd = 0.5*sd0, es_small = 0.2*sd0, sem = sem, sdiff = sdiff, mdc = z*sdiff, table = data.frame(x1 = x1, x2 = x2, change = x2-x1, rc = rc, reliable_gain = rc > z, reliable_loss = rc < -z), share_gain = mean(rc > z), share_loss = mean(rc < -z)) }Base R only;rel_change(c(0.60, 0.50, 0.70, 0.40, 0.55), c(0.75, 0.80, 0.65, 0.70, 0.75), 0.20, 0.80)returns the example in input order.Python:
def rel_change(x1, x2, sd0, rel, z=1.96): sem = sd0*math.sqrt(1-rel); sdiff = math.sqrt(2)*sem; rc = [(b-a)/sdiff for a, b in zip(x1, x2)]; return {"half_sd": 0.5*sd0, "es_small": 0.2*sd0, "sem": sem, "sdiff": sdiff, "mdc": z*sdiff, "change": [b-a for a, b in zip(x1, x2)], "rc": rc, "reliable_gain": [v > z for v in rc], "reliable_loss": [v < -z for v in rc], "share_gain": sum(v > z for v in rc)/len(rc), "share_loss": sum(v < -z for v in rc)/len(rc)}Needsimport math; returns the same values as the R function.Test (Reliable patients are those whose change exceeds the MDC): The number of patients with an index beyond 1.96 in size equals the number whose absolute change exceeds MDCVal. Expected result: TRUE. FALSE shows the indices divided by the SEM instead of the standard error of the difference, which classes the fifth patient (change 0.20, index 2.24) as reliable and gives 3 against 2. Excel check:
=SUMPRODUCT(--(ABS(RCVec)>1.96))=SUMPRODUCT(--(ABS(ScoreT2Vec-ScoreT1Vec)>MDCVal))Common error (Reading the share with reliable gain as a responder rate for an MID): The share counts change beyond measurement error, which is larger than a patient-perceived important change in the article's example (0.247 against 0.05), so it is not the proportion of patients reaching an anchor-based MID.
Source: de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM. Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health and Quality of Life Outcomes. 2006;4:54. doi:10.1186/1477-7525-4-54 (full text read). Comparison of SEM with anchor-based approaches, Interpretation and applicability; Blampied NM. Reliable change and the reliable change index: still useful after all these years? The Cognitive Behaviour Therapist. 2022;15:e50. doi:10.1017/S1754470X22000484 (full text read). Section on the standardised change score.
sem = sd0 * sqrt(1 - rel); sdiff = sqrt(2) * sem; mdc = z * sdiff; rc_i = (x2_i - x1_i) / sdiff; share_gain = mean(rc_i > z); share_loss = mean(rc_i < -z)
Try this function
Implementations
Excel
Minimally detectable change and reliable change index from named cells
With SEMVal (HE-FM-DMID-002), ScoreT1 and ScoreT2 named, the formulas return the standard error of the difference, the MDC and the reliable change index, held in SDiffVal, MDCVal and RelChange.
=SQRT(2)*SEMVal; =1.96*SDiffVal; =(ScoreT2-ScoreT1)/SDiffVal
Assumptions
Independent equal measurement errors for the MDC and reliable change index
The two scores carry independent errors with the same SEM, so the variance of their difference is 2 SEM squared; the 1.96 multiplier assumes normally distributed errors.
Worked examples
Detectable and reliable change for an EQ-5D-5L index with the article's rounded SEM
With the article's rounded SEM of 0.089, the standard error of the difference is 0.126, the MDC 1.96 x 0.126 = 0.247 and a patient whose index rises from 0.60 to 0.75 has RC = 0.15 / 0.126 = 1.19, below 1.96, as in the article.
SEM = 0.089; x_1 = 0.6; x_2 = 0.75; S_diff = 0.126; MDC = 0.247; RC = 1.19
Detectable and reliable change from the unrounded SEM
Carrying the SEM unrounded (0.08944) gives a standard error of the difference of 0.1265, an MDC of 0.2479 and the same RC of 1.19; the article's 0.247 comes from multiplying the rounded 0.126 (computed here for illustration).
SEM = 0.08944; x_1 = 0.6; x_2 = 0.75; S_diff = 0.1265; MDC = 0.2479; RC = 1.19
Reliable improvement of 0.30 index points
A patient whose index rises from 0.50 to 0.80 has RC = 0.30 / 0.1265 = 2.37, beyond 1.96, and a change larger than the MDC (computed here for illustration).
SEM = 0.08944; x_1 = 0.5; x_2 = 0.8; S_diff = 0.1265; MDC = 0.2479; RC = 2.37
Common errors
Applying the minimally detectable change to a difference between trial arms
The MDC and RC describe change in one person beyond measurement error; in the article's example the MDC of 0.247 is about five times the anchor-based estimate of 0.05, so using it as a threshold for a difference in arm means would dismiss changes patients notice.
Leaving out the square root of 2 for a change score
De Vet and colleagues include the square root of 2 because two measurements are involved in measuring change; without it the MDC in the article's example falls from 0.247 to 0.174 (1.96 x 0.089, computed here for illustration) and more patients are classed as changed.
Using distribution-based change as the sole basis for a responder definition
The FDA's 2009 guidance on patient-reported outcome measures says the empirical evidence for any responder definition is derived using anchor-based methods, and that distribution-based methods are supportive and not appropriate as the sole basis for one.
Sources
De Vet and colleagues on the minimally detectable change formula
de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM. Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health and Quality of Life Outcomes. 2006;4:54. doi:10.1186/1477-7525-4-54 (full text read). Interpretation and applicability: MDC = 1.96 x sqrt(2) x SEM, where 1.96 derives from the 95 per cent confidence interval of no change and the square root of 2 is included because two measurements are involved in measuring change.
Blampied on the standard error of the difference and the reliable change index
Blampied NM. Reliable change and the reliable change index: still useful after all these years? The Cognitive Behaviour Therapist. 2022;15:e50. doi:10.1017/S1754470X22000484 (full text read). Section on the standardised change score: the standard error of the difference is the square root of 2 SEM squared; Jacobson and Truax defined RC by dividing the difference score by it, and if the standardised change lies beyond 1.96 or minus 1.96 the probability of an error of measurement this large is 0.05 or less.
Jacobson and Truax reliable change index for one client
Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology. 1991;59(1):12-19. doi:10.1037/0022-006X.59.1.12 (abstract read). Abstract: a reliable change index is proposed to determine whether the magnitude of change for a given client is statistically reliable.
FDA 2009 on distribution-based methods and responder definitions
US Food and Drug Administration. Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring, MD: FDA; December 2009 (full text read). Section IV.E: the empiric evidence for any responder definition is derived using anchor-based methods; distribution-based methods should be considered supportive and are not appropriate as the sole basis for determining a responder definition; distribution-based methods can categorise changes as small, moderate and large.
Canonical Identity
Stable URI · Machine-readable · Resolvable · CC BY 4.0