Bootstrapping Method

The specific procedure of repeatedly resampling, with replacement, from a dataset to empirically approximate the sampling distribution of a statistic of interest.

Source: Efron 1979

The bootstrapping method repeatedly draws new samples from an observed dataset, each formed by sampling with replacement to the original size, and computes the statistic of interest on every one. Doing this thousands of times builds an empirical picture of how the statistic varies from sample to sample, which stands in for its true sampling distribution. From that spread it derives standard errors and confidence intervals without any formula, treating the data themselves as the population. Learning variability by resampling the data is what it does. Efron and Tibshirani (1993) describe this procedure.

Source: Efron & Tibshirani 1993

The steps of the bootstrapping method are to draw a resample of the same size as the original data by sampling with replacement; to compute the statistic of interest on that resample; to repeat this many times, producing many values of the statistic; and to summarise the resulting distribution, for example by its standard deviation for a standard error or its percentiles for a confidence interval. So the bootstrapping method proceeds by repeated resampling, recomputation, and summarisation, building an empirical sampling distribution from the data, which is then used to quantify the uncertainty of the statistic without relying on analytical formulas or distributional assumptions.

Source: Efron 1979

The bootstrapping method estimates uncertainty by using the spread of a statistic across many resamples as an approximation of its sampling variability: the standard deviation of the statistic over the resamples gives a standard error, and the percentiles of the resampled values give a confidence interval. So the bootstrapping method estimates uncertainty empirically from the variability of the statistic under resampling, treating the observed data as representative of the population, which allows standard errors and confidence intervals to be obtained for statistics whose theoretical sampling distributions are unknown or hard to derive, making it a flexible tool for inference.

Source: Efron 1979

The bootstrapping method estimates a statistic's sampling distribution empirically by resampling the data, while traditional inference derives it analytically from assumed distributions and formulas, such as assuming normality to compute a standard error. Bootstrapping relies on computation and few assumptions, whereas traditional methods rely on mathematical derivation and distributional assumptions. So the bootstrapping method differs from traditional inference in obtaining uncertainty from resampling rather than from theory, which makes it applicable where analytical results are unavailable or their assumptions doubtful, though it is more computationally intensive and still depends on the sample being representative of the population.

Source: Efron 1979

The bootstrapping method is appropriate when a statistic's sampling distribution cannot be derived easily or when the assumptions of formula-based inference are questionable, and when the sample is large enough and representative enough for resampling to reflect the population's variability. So the bootstrapping method is appropriate for quantifying uncertainty in complex or non-standard statistics and where distributional assumptions are doubtful, provided the sample is adequate, since its validity rests on the data representing the population, which means it is applied with attention to sample size and representativeness and is less reliable for very small samples or certain extreme statistics.

Source: Efron 1979