Hypothesis Testing Exercises in R: 25 Real-World Practice Problems

Explanation: t.test(x, mu = ...) runs a one-sample t-test against the hypothesised mean. The sample mean of 3.057 sits only 0.057 above 3.0, the t statistic is a modest 1.61, and p = 0.109 fails to reject H0 at the usual 0.05 cutoff. The 95% CI \[2.987, 3.128\] contains 3.0, which is the same decision expressed as an interval. Keeping the full object (not just the p-value) lets you pull every component later.

Explanation: Every htest object is a list, so $p.value, $statistic, $conf.int, and $parameter give you the raw numbers. unname() strips the "t" label R puts on the statistic element so the output vector looks clean. This pattern is the foundation of every automated reporting pipeline: run a test, pluck the numbers, drop them in a table. broom::tidy(ex_1_1) is the tidyverse equivalent and returns a 1-row tibble with the same columns.

Explanation: The one-sided alternative "greater" only counts deviations above 40, so the t distribution's upper tail supplies the p-value. The sample mean is 42.13 (above 40) yet noise is large, so p = 0.28 fails to reject. The CI lower bound becomes finite (35.91) while the upper is +Inf: a one-sided CI is a half-line, not an interval. A two-sided test here would give roughly double the p-value but a symmetric finite interval.

Explanation: The two-sided CI and the two-sided p-value are mathematically the same procedure expressed differently: the 95% CI is the set of mu values that would NOT be rejected at alpha = 0.05. If your hypothesised mu sits inside the CI, p > 0.05; if outside, p < 0.05. Always. When students see the two disagree, it is almost always a one-sided test with a half-line CI being compared against a two-sided cutoff.

Explanation: The formula interface y ~ group is the cleanest way to pass a two-sample test: R splits Sepal.Length by the factor levels. droplevels() removes the unused virginica level so the test labels print cleanly. Welch is the default because it does NOT assume equal variances; it adjusts the degrees of freedom (here 86.5, not 98) to compensate. The negative CI tells you setosa (the first level alphabetically) has the smaller mean.

Explanation: Paired t-tests treat the 20 differences as a one-sample test against zero, so within-subject correlation is removed and statistical power jumps compared to a two-sample t-test on the same data. The mean difference of -1.97 kg is small in absolute terms but consistent enough across subjects that the test rejects at p well below 0.001. Forgetting paired = TRUE here would still give a directionally correct estimate but with a much larger p-value because between-subject baseline variance is reintroduced.

Explanation: Welch's correction lowers degrees of freedom whenever the sample variances differ, which widens the t distribution's tails and raises the p-value. With equal sample sizes and similar variances the two tests are nearly identical, but with unequal sizes AND unequal variances they diverge sharply. Welch should be your default: it only loses a tiny bit of power when variances really are equal but stays valid when they are not. The Student version is a relic worth knowing only because legacy code uses it.

Explanation: var.test() runs an F-test on the ratio of two sample variances against H0: ratio = 1. Here the variance of 4-cyl mpg is roughly 4.4 times that of 8-cyl mpg, and p = 0.014 rejects equal variances. That is exactly the situation where Welch matters: forcing var.equal = TRUE on a Student t-test would inflate Type I error. F-tests are highly sensitive to non-normality though, so for serious work use Levene's test from car::leveneTest() (or Brown-Forsythe) which is more robust to skewed data.

Explanation: Welch t = (m1 - m2) / sqrt(s1^2/n1 + s2^2/n2), and the Welch-Satterthwaite degrees of freedom blend the two within-group variance estimates into a single non-integer df. The two-sided p-value is 2 * pt(-abs(t), df). This pattern is essential for meta-analysis, replication checks, and recomputing tests when only summary tables exist. The BSDA::tsum.test() function wraps the same math if you want a one-line version.

Explanation: prop.test() runs a score test based on the normal approximation. The X-squared statistic equals the square of the z statistic; 6.4 corresponds to z = 2.53 and a two-sided p-value of 0.011. Setting correct = FALSE skips Yates' continuity correction so the output lines up with the closed-form formula z = (p_hat - p0) / sqrt(p0 * (1 - p0) / n). For small n use binom.test() (exact binomial); for very small n the Wilson interval from prop.test() still works well.

Explanation: Both x and n accept length-2 vectors and prop.test() computes the absolute risk difference (p1 - p2) along with its 95% CI. p = 0.0065 rejects equal proportions, and the CI \[-0.012, -0.002\] indicates B beats A by 0.2 to 1.2 percentage points. For relative effects (lift), divide the rates yourself; prop.test() always works on the absolute difference. Tools like rstatix::pairwise_prop_test() or infer::prop_test() make this composable inside a tidy pipeline.

Explanation: chisq.test() on a matrix computes the Pearson chi-squared statistic comparing observed counts to those expected under independence. Degrees of freedom equal (rows - 1) * (cols - 1), here 1 * 2 = 2. With p = 0.003 you reject independence and conclude smoking status varies with gender in this sample. When any expected cell drops below 5, R warns and you should switch to fisher.test(). Inspect ex_3_3$residuals to see WHICH cells drive the rejection: the largest absolute standardised residuals are the cells most out of line with the independence model.

Explanation: Goodness-of-fit uses a single vector of observed counts and a vector of expected probabilities. df = number of categories minus 1 (here 6 - 1 = 5). With p = 0.376 you fail to reject fairness: the observed counts are well within sampling noise for a fair die. Note that "fail to reject" is NOT "the die is fair"; you simply lack evidence to claim it is biased. A sample of 6,000 rolls instead of 600 would give a much sharper test if you wanted to rule out a 1-2 percent bias.

Explanation: A paired Wilcoxon ranks the absolute differences and tests whether positive and negative ranks balance under H0. It is the right choice for ordinal data like a pain scale where the interval between 5 and 6 is not guaranteed to equal the interval between 8 and 9. Here V = 0 means every difference moved the same direction (pain dropped for every patient), which yields p = 0.0007 and a clear rejection. A paired t-test on the same data would also reject, but its assumption of normally distributed differences is shaky on ordinal scales.

Explanation: The independent-samples Wilcoxon (also called Mann-Whitney U) ranks all observations together and asks whether ranks in group A tend to be larger or smaller than ranks in group B. It is invariant to monotone transforms of the response, so log-transforming ozone would give the same p-value. With p = 0.00012 you reject equal distributions: August ozone is reliably higher than May ozone. Note that "location shift" in R's output is a special case; the test rejects more broadly any shift in stochastic dominance.

Explanation: Kruskal-Wallis generalises Mann-Whitney to k > 2 groups by ranking all observations and comparing rank sums. df = k - 1 = 2 here. With p = 0.018 the test rejects "all groups identical" but does NOT tell you which pair is different; follow up with pairwise.wilcox.test(PlantGrowth$weight, PlantGrowth$group, p.adjust.method = "BH") to get adjusted pairwise comparisons. For parametric data prefer one-way ANOVA via aov(); Kruskal-Wallis is the safe fallback when the residual diagnostics fail.

Explanation: The percentile bootstrap constructs a confidence interval by resampling the data, recomputing the statistic 10,000 times, and taking the 2.5/97.5 percentiles of the resulting distribution. It makes no assumption that the sampling distribution of the mean is symmetric, so on skewed data it produces an asymmetric CI that better reflects reality. t.test(rt)$conf.int on the same data forces symmetry around the sample mean and can over- or undercover. For more rigorous bootstrap CIs (BCa, studentized) use boot::boot() plus boot::boot.ci().

Explanation: When H0 is true the p-value is uniformly distributed on \[0, 1\], so the fraction below alpha = 0.05 should equal alpha. With 5,000 simulations the Monte Carlo standard error is sqrt(0.05 * 0.95 / 5000) ~= 0.003, so any value in roughly \[0.045, 0.055\] is consistent with calibrated nominal error. Running this once gives intuition for why "p < 0.05" is a statement about long-run frequency, not about any individual experiment. Bump n_sim to 100,000 to nail down the third decimal.

Explanation: Cohen's d expresses the group mean difference in units of pooled standard deviation, so it is comparable across studies regardless of the original measurement scale. |d| = 2.10 is huge: setosa sepals are about two standard deviations shorter than versicolor sepals on average. A p-value tells you the difference is unlikely to be zero; d tells you HOW BIG the difference is. Always report both. For paired designs use the SD of the differences instead of the pooled SD (Cohen's dz).

Explanation: Power analysis trades off five quantities: effect size d, alpha, power (1 - beta), sample size n, and the test type. Fix any four and pwr.t.test() solves for the fifth (use NULL for the unknown). Detecting d = 0.5 needs about 64 per group; halving the effect to d = 0.25 quadruples the required n to roughly 252 per group, because power scales with n through d * sqrt(n). Always run power analysis BEFORE collecting data; running it after non-significant results to "explain" the null is post-hoc power and is statistically meaningless.

Explanation: A power curve makes the n/d/power trade-off visible: with only 30 per group, you need a Cohen's d above ~0.74 to hit the conventional 80% power target. Below d = 0.4 you have less than a coin-flip chance of detecting a real effect, so a non-significant result tells you nothing. Power curves are essential when negotiating study scope with stakeholders ("we can detect d >= 0.5 with 80% power, anything smaller you need a bigger sample"). For three-arm trials use pwr::pwr.anova.test() instead.

Explanation: Bonferroni controls the family-wise error rate by multiplying each p-value by the number of tests; it is the strictest method and rarely calls anything significant in high-dimensional settings. Benjamini-Hochberg controls the false discovery rate (the expected fraction of false positives AMONG rejected hypotheses), which gives more power at the cost of allowing some false positives. For genome-wide screens BH (or Storey's q-value) is the standard; for confirmatory clinical analyses Bonferroni or Holm is the default.

Explanation: broom::tidy() converts the wall-of-text htest print output into a clean 1-row tibble per test, which makes group-wise testing composable. With group_by() + reframe() you fan out one t-test per cyl group and stack the results vertically. This pattern is the foundation of every "tests by subgroup" report in production analytics. Beware multiple testing here: three p-values means you should bump alpha down via p.adjust(ex_6_2$p.value, method = "BH") if you want to interpret each independently.

Explanation: A permutation test simulates the exact null distribution by reassigning group labels at random: under H0 the labels are exchangeable, so any reassignment is equally likely. The two-sided p-value is the fraction of permuted differences as extreme as or more extreme than the observed one. It makes NO distributional assumption on the data; the cost is computational. For the 20-observation sleep dataset the permutation p-value lines up with the paired t-test p-value of about 0.076, but for skewed or heavy-tailed data they diverge.

Explanation: The two one-sided tests (TOST) procedure flips the classical question. Instead of asking "is the difference different from zero?", you ask "is the difference inside the practically-equivalent interval \[-delta, +delta\]?". You reject non-equivalence only if BOTH one-sided tests reject, so the relevant p-value is the larger of the two. A non-significant standard t-test does NOT establish equivalence; only TOST does. Packages like TOSTER::tsum_TOST() automate this for both raw and standardised effect sizes.