Sampling Methods Exercises in R: 20 Real Practice Problems

Explanation: sample(n, size) is the standard idiom for drawing row positions: it returns a permutation of 1:n truncated to size. Integer indexing then preserves row names and column types, which a slice() or [].sample() clone in another language would not always do. Default is without replacement, so each row appears at most once. If you needed the same draw twice (a paired bootstrap, for example) you would set the seed before each call.

Explanation: replace = TRUE is what makes this a sampling problem instead of a permutation: each draw is independent, so the same face can come up many times. Without it you would just be shuffling the integers 1:6. For a biased die you would add prob = c(0.1, 0.1, 0.1, 0.1, 0.1, 0.5) so the probabilities apply per draw. The expected count under uniformity is 1000/6 ~ 166.7 per face; the observed counts here are within the normal Monte Carlo spread.

Explanation: set.seed() rewinds R's RNG state. Any function downstream that consumes random numbers (sample, rnorm, kmeans with random init, ranger's bootstrap, mclust's EM start) will produce the same draw. A common bug is calling set.seed() once at the top of a script and then expecting two later calls to match; they will not, because intermediate code between them consumed RNG state. The fix is to re-seed immediately before each block whose reproducibility you care about.

Explanation: The bootstrap treats the observed sample as the population and resamples from it with replacement at the same size n. The standard deviation of the resulting replicate statistics is a direct estimate of the standard error of that statistic. Notice that the bootstrap mean (20.10) is essentially the sample mean (20.09); the bootstrap does not change point estimates, only their uncertainty. replicate() is the most concise base-R driver for this; the boot package is preferred when you also want bias correction and confidence intervals.

Explanation: The percentile method takes the empirical quantiles of the replicate statistics directly. It works well when the bootstrap distribution is roughly symmetric and the statistic has no strong bias. For skewed distributions the bias-corrected and accelerated (BCa) interval, demonstrated in exercise 2.4, has better coverage. Two thousand replicates is a reasonable default for a CI; standard errors converge faster (around 500 reps is enough) but CIs need more because the tails are noisier.

Explanation: A paired (or case) bootstrap resamples whole rows, which keeps the bivariate structure intact. Resampling mpg and wt independently would destroy the correlation. The replicate correlations stay tightly negative, which gives the manager a usable confidence statement: the correlation is reliably below -0.75 even under heavy resampling. For small n like 32 the Fisher z-transform plus normal interval is the parametric alternative; bootstrap is preferred when you cannot defend the normality of the transformed correlations.

Explanation: boot::boot() expects a statistic function with signature (data, idx) so it can pass in the resampled row indices. The bias column is the difference between the mean of the bootstrap replicates and the original estimate; if it is large relative to the standard error you have a bias problem that the BCa interval partially corrects. BCa is the default recommendation in Davison and Hinkley because it has second-order accuracy under regular conditions. The plain percentile and basic intervals are good fallbacks when the BCa acceleration computation fails (it can with very small samples).

Explanation: Residual resampling assumes the regression model is correctly specified (linear, homoscedastic errors) and only randomizes the noise, so it produces a tighter SE. Case resampling makes no functional-form assumption and absorbs heteroscedasticity and model misspecification into wider intervals. When you cannot defend the linearity assumption (which is most real datasets) the case bootstrap is the safer default. The gap between the two SEs here, about 35 percent, is a useful diagnostic: it suggests the homoscedasticity assumption is overstating precision.

Explanation: slice_sample() is the modern dplyr replacement for sample_n() and works gracefully with grouped data: the n = 5 applies per group, not in total. Use prop = 0.1 instead of n for a 10 percent sample per stratum, which is what proportional allocation calls for. Stratifying is what makes a sample defensible when some classes are rare (you still see every class) and is the first reflex of any biostatistician dealing with imbalanced treatment arms.

Explanation: Cluster sampling is what you do when units of observation come in natural groups and visiting one group costs the same whether you measure 1 unit or all of them: schools, hospitals, city blocks, gene families. The downside is design effect: observations within a cluster are correlated, so the effective sample size is smaller than the raw row count. Variance estimates that ignore the cluster structure (a plain mean() and sd()/sqrt(n)) will be too narrow. The survey package in exercise 4.3 handles this properly.

Explanation: Systematic sampling spreads draws evenly through the frame, which is useful when the frame is ordered by something correlated with the outcome (price, time of arrival, position on a conveyor) and you want coverage rather than clustering. The hidden risk is periodicity: if the frame has a hidden cycle of length k or a multiple of k, the sample is biased. With diamonds, the rows are ordered by carat and there is no built-in period of 4, so systematic sampling here behaves almost identically to a simple random sample of the same size.

Explanation: Equal allocation gives every stratum the same precision, which is what you want when you plan to compare strata against each other (treatment versus control, men versus women). Proportional allocation gives every stratum a share matching its population weight, which is what you want when the headline statistic is the overall mean. With iris the two designs happen to land on identical counts because the three Species are perfectly balanced at 50 rows each; for an imbalanced frame they diverge sharply. Neyman allocation, a third option, weighs by stratum standard deviation and is optimal when strata variances differ.

Explanation: When prob is supplied to sample(), R normalizes it internally so the values sum to 1, which is why raw revenue dollars work as weights with no manual division. The empirical probabilities track the theoretical p_i = revenue_i / sum(revenue) closely after 10,000 draws; the gap is Monte Carlo noise. The same prob argument drives any weighted draw in base R, including the bootstrap weighted variant where the resampling weights are not all equal. For survey work, the design weights live on the design object rather than each draw.

Explanation: PPS sampling is the standard for multistage surveys where larger units (cities, schools, hospitals) deserve a higher chance of selection because they hold more units of the second stage (people, students, patients). Done correctly, the unequal inclusion probabilities are offset by Horvitz-Thompson weights at the estimation stage so the overall mean is still unbiased. Note that base R's sample(..., replace = FALSE, prob = ...) uses sequential PPS without replacement, which is fine for small samples but biases inclusion probabilities once you sample a substantial fraction of the frame; for that case look at pps::ppswr or sampling::UPpoisson.

Explanation: Design-based inference (the Horvitz-Thompson framework) builds the standard error from the sampling design itself, not from an assumed superpopulation model. With stratified weights, the urban respondents each represent 5000/12 ~ 417 people and the rural ones each represent 375; ignoring those weights would give the unweighted mean and an underestimated SE. The survey package is the standard R interface for NHANES, BRFSS, and any other complex survey with strata, clusters, replicate weights, or finite-population corrections. For modeling under the same design, use svyglm() instead of glm().

Explanation: A permutation test asks: if there were truly no association between am and mpg, how often would I see a mean difference at least as extreme as the one I observed, just by shuffling labels? It needs no normality assumption and no model: only that observations are exchangeable under the null. The p-value here (~0.0002) is essentially zero, agreeing with what a Welch t-test would conclude. Permutation tests shine when the test statistic has no analytic null distribution (a complicated ratio, a custom score function, a paired difference under heteroscedasticity).

Explanation: The jackknife is the bootstrap's older deterministic cousin: instead of resampling B times, it produces n replicates by leaving each observation out once. For smooth statistics like the mean it gives an SE essentially identical to the parametric sd(x)/sqrt(n), which is what you see here (1.065 vs 1.065). The jackknife breaks down for non-smooth statistics like the median or a quantile, where small leave-one-out changes can flip the estimate by a discrete jump. The bootstrap handles those cases more gracefully.

Explanation: Jackknife slope estimates are a quick robustness check: if any single row swings the coefficient noticeably, you have a high-leverage observation worth investigating. Chrysler Imperial is heavy (5.345 thousand lbs) with high mpg relative to its weight, which is exactly the leverage pattern Cook's distance also flags. The classical diagnostic cooks.distance(fit) is closely related to this leave-one-out construction. Jackknifing coefficients is computationally cheaper than bootstrapping when n is small and you want a quick influence audit before a full bootstrap analysis.

Explanation: A random 70/30 split is the simplest holdout design and is fine when the response is roughly balanced and rows are exchangeable. It fails for small datasets like mtcars (n = 32), where a single bad split can move the test RMSE by 10-20 percent; in that regime cross-validation is the proper tool. The split also fails for time series (you would leak the future into training) and for grouped data (rows from the same patient or customer should stay together); both call for rsample::initial_time_split() and rsample::group_initial_split() respectively.

Explanation: Stratified splits matter most for classification with imbalanced classes (fraud, disease, churn): a random split can put zero positives in test, which makes any held-out metric meaningless. The row_id trick is a cheap way to express "everything not in train" with anti_join() because dplyr does not have a native split-then-complement function for grouped tibbles. The rsample::initial_split(strata = Species) wrapper does the same thing in one line and is what most tidymodels code uses in production.