Effective Sample Size

The sample size a simple random sample would need to match the precision of an actual sample from a more complex design, such as clustered.

Source: Kish 1965

Effective sample size is the number of participants a simple random sample would need to carry the same precision as an actual sample drawn from a more complex design, such as a clustered survey. It tells us how much usable information the design really holds, which is often less than the raw headcount because clustering makes observations partly redundant. A study of a thousand people spread across a few clusters may carry the precision of far fewer independent ones. The true informational size of a sample is what it captures. Kirkwood and Sterne (2003) describe this.

Source: Kirkwood & Sterne 2003

Effective sample size is calculated by dividing the actual sample size by the design effect, the ratio of the design's variance to that of simple random sampling. A design effect above one, common with clustering, therefore gives an effective sample size smaller than the actual number. So effective sample size is calculated as the actual size divided by the design effect, which translates the complex design's reduced precision into the size of an equivalent independent sample, showing how much usable information the complex sample provides, and it depends on factors such as the intraclass correlation and cluster size that determine the design effect.

Source: Kish 1965

Effective sample size matters because it reflects the true precision of estimates from a complex design, which can be considerably less than the actual number of observations suggests, so using the nominal sample size would overstate the precision. It is important for both analysis and planning. So effective sample size matters for correctly gauging the information in a complex sample, since clustering and weighting reduce precision, and recognising the smaller effective sample prevents overconfidence in the estimates and informs how large a study must be, which is why it is considered when designing and interpreting surveys and clustered studies.

Source: Kish 1965

Effective sample size relates to the design effect as its inverse scaling of the actual sample size: the effective sample size equals the actual sample size divided by the design effect, so a larger design effect, indicating greater loss of precision, gives a smaller effective sample size. So the design effect and effective sample size are directly linked, with the design effect quantifying how much the design inflates variance relative to simple random sampling and the effective sample size expressing the equivalent independent sample, and together they convey the impact of the design on precision, which is important for the analysis and planning of complex or clustered studies.

Source: Kish 1965

Effective sample size is used in study design to ensure that a complex or clustered study collects enough observations to achieve the desired precision, since the design effect reduces the information per observation; the actual sample size is set larger than the target effective size by the design effect. So effective sample size is used, together with the design effect, to plan adequately powered complex studies, because achieving a required precision demands more observations under clustering than under simple random sampling, and calculating the effective sample size ensures the study is large enough to yield estimates as precise as intended despite the loss of efficiency from the design.

Source: Kish 1965