Standardised Response Mean

Standardised Response Mean in Responsiveness Testing

The standardised response mean (SRM) expresses how far a group's scores on an outcome instrument moved between two assessments, in units of how much that movement varied between patients. Health economists meet it when choosing or defending an instrument, because a measure that registers little change cannot show much health gain, and an instrument that misses real change can understate the QALYs an intervention produces. It is one of several standardised change statistics, and its value depends on the correlation between baseline and follow-up scores as well as on the size of the change. That dependence is the main reason SRMs are often misread against benchmarks that were built for a different index. Whether an instrument shows responsiveness in the wider sense depends on more evidence than any single statistic supplies.

Calculating the SRM from paired scores

The SRM needs each patient's score at two time points on the same instrument, scored in the same direction. Each patient's change is computed first, and the statistic is then built from the mean and spread of those individual changes, so only patients with both assessments contribute.

$$\Delta_i = S_{i,1} - S_{i,0}, \qquad \mathrm{SRM} = \frac{\bar{\Delta}}{s_{\Delta}}, \qquad s_{\Delta} = \sqrt{\frac{\sum_{i=1}^{n} (\Delta_i - \bar{\Delta})^2}{n - 1}}$$

where $S_{i,0}$ and $S_{i,1}$ are the scores of patient $i$ at baseline and at follow-up, $\Delta_i$ is that patient's change score, $\bar{\Delta}$ is the mean change across the $n$ patients, and $s_{\Delta}$ is the sample standard deviation of the change scores.

Middel and van Sonderen describe the SRM as the ratio between the mean change score and the standard deviation of that change score within the same group, and note that it is applied on a wide scale when health outcome questionnaires are used. They trace the index to Cohen, who introduced it as the effect size for matched pairs, and report that Liang and colleagues later renamed it the standardised response mean to avoid confusion with other effect size indices. Liang, Fossel and Larson used SRMs to compare the sensitivity to change of five health status questionnaires in 38 patients having hip or knee arthroplasty.

Because the SRM uses the same ingredients as the paired t statistic, the two are linked by sample size alone.

$$t = \frac{\bar{\Delta}}{s_{\Delta} / \sqrt{n}} = \mathrm{SRM} \times \sqrt{n}$$

where $t$ is the paired t statistic for the null hypothesis of no mean change and the other symbols are as above. The SRM is therefore the paired t statistic with the sample size taken out, which is why a large study can show a highly significant change with a modest SRM.

How the SRM relates to other standardised change indices

Every standardised change index divides the same mean change by a standard deviation, and the choice of standard deviation decides the value. Kazis, Anderson and Meenan defined effect sizes for health status change as the mean change in a variable divided by the standard deviation of that variable, and Middel and van Sonderen identify this as the version standardised by the baseline standard deviation. Cohen's d for two independent groups uses a pooled standard deviation, and the standardised mean difference used to pool trials in meta-analysis is a between-group index of the same family. The SRM is the only one of these whose denominator is built from the change scores themselves, so it reflects how consistently patients changed as well as how much.

The variance of a change score depends on the spread at each time point and on how strongly the two assessments are correlated.

$$s_{\Delta}^2 = s_0^2 + s_1^2 - 2\,r\,s_0\,s_1$$

where $s_0$ and $s_1$ are the standard deviations of the scores at baseline and at follow-up and $r$ is the correlation between the two sets of scores. When $s_0$ and $s_1$ are both equal to a common value $s$, this reduces to $s_{\Delta} = s\sqrt{2(1 - r)}$, and so

$$\mathrm{SRM} = \frac{\mathrm{ES}}{\sqrt{2(1 - r)}}$$

where $\mathrm{ES}$ is the mean change divided by the common standard deviation $s$. Middel and van Sonderen quote Cohen's observation that for dependent samples the standardising value is $\sqrt{2(1-r)}$ times as large as it would be were they independent, and give the same equivalence. The SRM equals the effect size when $r$ is 0.5, exceeds it when the scores are more strongly correlated and falls below it when they are less so. Norman, Wyrwich and Patrick showed that the differences between the commonly used responsiveness coefficients are completely predictable once their analytical form is set out in this way.

Benchmarks and why Cohen's thresholds mislead

Cohen's labels of 0.20 for a small, 0.50 for a moderate and 0.80 for a large effect are widely applied to SRMs. Middel and van Sonderen point out that these thresholds were set for an effect size standardised by the pooled standard deviation of independent samples, not for the matched pairs index, so applying them to an SRM without allowing for the correlation can misclassify the magnitude of change. In their worked conversion, a pooled effect size of 0.42 corresponds to an SRM of 0.33 when $r$ is 0.20, 0.42 when $r$ is 0.50 and 0.79 when $r$ is 0.86, so the same change can be labelled small or large depending only on how stable the patients' relative positions were.

They also reanalysed 148 SRMs that authors had interpreted with Cohen's thresholds and found that 23% fell into a different magnitude category once the same data were expressed as a pooled effect size. Their remedy is to adjust the SRM for the observed correlation before applying Cohen's labels, and they ask studies that standardise by the baseline standard deviation to report the pooled effect size as well.

Worked example with an EQ-5D index and a condition-specific score

All numbers in this example are illustrative. A single-arm study collects EQ-5D index values from eight patients before a rehabilitation programme and twelve weeks later, together with a condition-specific function score on a 0 to 100 scale. The analyst wants to know how responsive each instrument was in this group before choosing which outcome to carry into an economic model.

1. Mean change. The eight changes sum to 0.72, so the mean change is 0.72 / 8 = 0.09 on the index scale.

2. Spread of change. The deviations of the changes from 0.09 are 0.06, minus 0.03, 0.09, minus 0.07, 0.03, minus 0.11, 0.04 and minus 0.01. Their squares sum to 0.0036 + 0.0009 + 0.0081 + 0.0049 + 0.0009 + 0.0121 + 0.0016 + 0.0001 = 0.0322. Dividing by $n - 1 = 7$ gives a variance of 0.0046, and the standard deviation of change is the square root, 0.0678.

3. SRM. The SRM is 0.09 / 0.0678 = 1.33.

4. Baseline effect size for comparison. The baseline values sum to 4.50, with mean 0.5625. Their squared deviations sum to 0.06815, so the baseline standard deviation is the square root of 0.06815 / 7 = 0.009736, which is 0.0987. The effect size standardised by baseline is 0.09 / 0.0987 = 0.91. The SRM is larger because the standard deviation of change (0.0678) is smaller than the baseline standard deviation (0.0987), which follows from the correlation of about 0.73 between baseline and week 12 values.

5. Link to the t test. The paired t statistic is the unrounded SRM of 1.327 multiplied by the square root of 8, which is 1.327 × 2.828 = 3.75.

6. The condition-specific score. For the same eight patients the function score has a mean change of 6.0 points with a standard deviation of change of 4.0 points, so its SRM is 6.0 / 4.0 = 1.50.

7. Interpretation. Both instruments moved clearly in this group, and the condition-specific score moved slightly more consistently. That does not make it the better outcome for a cost-utility model, because it yields no utility weights for a QALY calculation without a mapping or valuation step. A mean gain of 0.09 sustained for half a year would be worth 0.09 × 0.5 = 0.045 QALYs per patient, but a single-arm pre and post comparison cannot rule out natural recovery or regression to the mean as the cause of the gain, so the SRM speaks to the instrument and not to the intervention's effectiveness.

Where the SRM enters health economic work

In health economics the SRM is most relevant to whether a generic preference-based measure can register the health changes that matter in a condition. The NICE manual (PMG36, section 4.3.10) states that a case that the EQ-5D is inappropriate needs qualitative empirical evidence on its lack of content validity, supported by evidence that it performs poorly on tests of construct validity and responsiveness in a particular patient population, derived from a synthesis of peer-reviewed literature. The SRM is one of the statistics that can contribute to that responsiveness evidence.

Longworth and colleagues, reviewing generic measures for NICE decision-making, found the EQ-5D valid and responsive for skin conditions and most cancers, with performance varying by aetiology in vision and poor for hearing impairments. Findings of this kind shape whether a health-state utility value is taken from the EQ-5D directly, mapped from a condition-specific measure or estimated with an alternative instrument. A low SRM for a generic measure in a condition where patients clearly improve is one signal that the measure may understate benefit. It is weaker evidence when the sample or interval differs from the decision population.

Design choices that move the SRM

An SRM belongs to a sample, an interval and an instrument together, so a value from one study does not transfer automatically to another. Several features of the study design change it without any change in the instrument.

• Who is included. A group in which every patient improved gives a much higher SRM than a mixed group in which some improved, some stayed stable and some declined.
• Timing. The interval between assessments sets how much change has had time to happen, and a longer interval can also lower the correlation between scores.
• Correlation between assessments. As the formulae above show, a highly stable ordering of patients shrinks the standard deviation of change and raises the SRM.
• Heterogeneous response. Genuine variation in how patients respond adds to the standard deviation of change, so an effective treatment with uneven benefit can produce a modest SRM.
• Scale limits. A ceiling effect caps the change that patients near the top of the scale can show, which lowers the mean change for a generic measure in mildly affected populations.

Limits and the COSMIN position

The SRM has been criticised as a measure of responsiveness itself. Husted and colleagues distinguish internal responsiveness, the ability of a measure to change over a prespecified time frame, from external responsiveness, the extent to which change in a measure relates to change in a reference measure of clinical or health status, and they consider the external statistics particularly attractive for interpretation. The SRM uses the instrument's own change scores alone, so it describes the first of these.

The COSMIN Delphi panel, as reported by Mokkink and colleagues, reached consensus that effect sizes and related measures such as the SRM are inappropriate measures of responsiveness, treating them as measures of the magnitude of change due to an intervention or other event rather than of the quality of the instrument. Norman, Wyrwich and Patrick took a different view, judging Cohen's effect size and the SRM the two most appropriate indices, while warning that any measure based on variability in change scores can mislead in some circumstances and recommending the Cohen effect size for comparability.

The SRM is also silent on importance. A large SRM shows that change was consistent relative to its spread, not that patients regard it as worthwhile, which is the question addressed by the minimal clinically important difference. Distribution-based estimates of a minimal important difference rest on similar standard deviation units, and their limits are discussed under distribution-based MID. Nor does the SRM separate true change from measurement error, since random error in either assessment widens the standard deviation of change and lowers the index.

Sources

• Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. Journal of Clinical Epidemiology. 2000;53(5):459-468. https://doi.org/10.1016/s0895-4356(99)00206-1
• Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Medical Care. 1989;27(3 Suppl):S178-S189. https://doi.org/10.1097/00005650-198903001-00015
• Liang MH, Fossel AH, Larson MG. Comparisons of five health status instruments for orthopedic evaluation. Medical Care. 1990;28(7):632-642. https://doi.org/10.1097/00005650-199007000-00008
• Longworth L, Yang Y, Young T, et al. Use of generic and condition-specific measures of health-related quality of life in NICE decision-making: a systematic review, statistical modelling and survey. Health Technology Assessment. 2014;18(9):1-224. https://doi.org/10.3310/hta18090
• Middel B, van Sonderen E. Statistical significant change versus relevant or important change in (quasi) experimental design: some conceptual and methodological problems in estimating magnitude of intervention-related change in health services research. International Journal of Integrated Care. 2002;2:e15. https://doi.org/10.5334/ijic.65
• Mokkink LB, Terwee CB, Knol DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Medical Research Methodology. 2010;10:22. https://doi.org/10.1186/1471-2288-10-22
• National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (PMG36). Published 31 January 2022, last updated 13 October 2026. Section 4.3.10. https://www.nice.org.uk/process/pmg36/chapter/economic-evaluation-2
• Norman GR, Wyrwich KW, Patrick DL. The mathematical relationship among different forms of responsiveness coefficients. Quality of Life Research. 2007;16(5):815-822. https://doi.org/10.1007/s11136-007-9180-x