Top 25 Regression Problems to Solve in R

Explanation: The formula mpg ~ wt reads as "model mpg using wt". The slope of -5.344 says each additional 1000 lb of weight is associated with about 5.3 fewer miles per gallon, holding nothing else. Printing an lm object shows only the call and coefficients; the richer report (standard errors, R-squared, F-test) lives in summary(), which you reach next.

Explanation: summary.lm() returns a list with named components, so $r.squared gives the value programmatically instead of copying it by eye. Here weight alone explains about 75% of the variance in mpg. Prefer this over screen-scraping the printout: it keeps reports reproducible and lets you compare many models in a loop. The paired $adj.r.squared penalises extra predictors.

Explanation: coef() on a summary returns the coefficient table as a numeric matrix, which is far easier to subset than the printed block. Notice the wt slope shrank from -5.34 to -3.88 once hp entered: the two predictors share information, so the simple-regression slope was partly absorbing horsepower. Both p values are tiny, so each predictor adds signal beyond the other.

Explanation: sigma() returns $\hat{\sigma} = \sqrt{\text{RSS}/(n - p)}$, the residual standard error, without parsing the printed summary. Roughly speaking, actual mpg values sit about 2.6 units away from the fitted line on average. Unlike R-squared, sigma keeps the original units, which makes it the more honest yardstick when you care about prediction accuracy rather than variance explained.

Explanation: The slope went from -5.344 per 1000 lb to -0.005344 per single pound, exactly one thousandth, because $\beta$ carries the inverse units of its predictor. The intercept, the R-squared and every fitted value are identical: rescaling a predictor is cosmetic, not statistical. The lesson is that a coefficient's magnitude is meaningless until you name its units, which is why "per 1000 lb" belongs in every write-up.

Explanation: With a logged response the slope is a proportional effect: each extra 1000 lb multiplies mpg by $e^{-0.2718} \approx 0.762$, roughly a 24% fall. For small slopes the raw coefficient itself approximates the percentage change (here about -27% versus the exact -24%), and the gap widens as the slope grows. Reaching for exp() rather than reading the coefficient literally is the habit that keeps semi-log interpretation honest.

Explanation: In a log-log model the slope is an elasticity: a 1% rise in girth is associated with about a 2.2% rise in volume. That is more than proportional, which matches geometry, since volume grows roughly with the square of a linear dimension. Elasticities are unit-free, so they travel across datasets and are the natural currency in economics and allometry. The intercept here is only the log-volume at unit girth and is rarely interpreted on its own.

Explanation: After standardizing, each slope is the mpg change per one standard deviation of the predictor, so wt (-3.79) outranks hp (-2.18) in influence, which the raw units hid. The intercept becomes the mean mpg (20.09) because both predictors are now centered at zero. A caution: standardized coefficients rank importance only within this sample and model, and they can shift when predictors are correlated, so treat them as a guide, not a verdict.

Explanation: With default treatment coding the 4-cylinder group is the baseline, so the intercept 26.66 is the mean mpg for 4-cylinder cars. The other coefficients are differences from that baseline: 6-cylinder cars average 6.92 mpg less, 8-cylinder cars 11.56 mpg less. A dummy coefficient is never an absolute mean for its own group; forgetting the baseline is the single most common misread of categorical regression output.

Explanation: relevel() moves "8" to the front, so the intercept 15.10 is now the 8-cylinder mean and the other slopes are gains relative to it: 4-cylinder cars average 11.56 mpg more, 6-cylinder cars 4.64 more. The model fit is unchanged, only the framing of the contrasts. Choosing a meaningful baseline (often a control group or the most common level) makes the coefficient table tell the story you actually want to report.

Explanation: Sepal.Length * Species fits a separate line per species. The Sepal.Length coefficient 0.799 is the slope for the baseline species (setosa); the interaction terms bend it, so versicolor's slope is $0.799 - 0.479 = 0.320$ and virginica's is $0.799 - 0.567 = 0.232$. Pooling all species into one slope would mask that every group actually shows a positive width-length link, the classic Simpson's paradox that interactions exist to expose.

Explanation: Row-indexing the coefficient matrix by term name returns the four summary numbers for just that interaction. The p value of 0.00046 is well below 0.05, so versicolor's slope is reliably shallower than setosa's; the interaction is not noise. When a factor has several levels, prefer anova() or car::Anova() to test the whole interaction at once, since scanning individual dummy rows inflates the chance of a false positive.

Explanation: residuals() returns observed minus fitted for every car, keeping the row names, so sorting exposes the biggest positive misses: the Fiat 128 gets about 6.9 mpg more than its weight predicts. Large residuals are not errors to delete; they are prompts to ask what the single-predictor model omits, such as engine size or aerodynamics. Inspecting the extremes by name is often the fastest route to the next useful variable.

Explanation: Hat values sum to $p$ (here 3, counting the intercept), so average leverage is $p/n = 3/32 \approx 0.094$ and points above $2p/n = 0.1875$ deserve a look. The Maserati Bora at 0.394 sits far out in weight-horsepower space. High leverage is only a risk, not a problem by itself: a high-leverage point that lies on the trend barely moves the fit. To see which ones actually matter, combine leverage with the residual, which is exactly what Cook's distance does next.

Explanation: Cook's distance measures the shift in all fitted values when a point is removed, so it catches cars that are both unusual and off-trend. The Chrysler Imperial tops the list even though the Maserati Bora had higher leverage, because the Imperial pairs moderate leverage with a large residual. A common flag is $D_i > 4/n = 0.125$ here, but treat it as a prompt to investigate rather than an automatic delete: influential points are frequently the most informative rows in the data.

Explanation: The null hypothesis is that residuals are normal; a p value of 0.10 gives no reason to reject it, so the normality assumption is defensible here. Two cautions matter: in small samples the test has little power to detect non-normality, while in large samples it flags trivial departures as significant. Pair it with a Q-Q plot, and remember that mild non-normality mostly threatens prediction intervals rather than the coefficient estimates themselves.

Explanation: A well-specified linear model shows residuals scattered flat around zero with no shape. Here the points bow, sitting above the line at both ends of the fitted range and below it in the middle, a signature of missing curvature that a term like I(wt^2) or a log transform can absorb. Reading this plot before trusting p values is essential, because curvature and funnel shapes quietly break the constant-variance and linearity assumptions that the standard errors depend on.

Explanation: predict() plugs the new waiting value into the fitted line $\hat{y} = -1.874 + 0.0756 \times \text{waiting}$, returning about 4.18 minutes for an 80-minute wait. The newdata column name must match the formula exactly, or predict() silently returns fitted values for the training rows instead. A bare point estimate hides all uncertainty, which is why the next problems attach intervals to this same number.

Explanation: A confidence interval answers "where is the mean eruption length for 80-minute waits?" and spans only about 4.10 to 4.25, because it reflects uncertainty in the estimated line alone. It narrows as the sample grows and is tightest near the mean of waiting. This interval is the right one for statements about a group average, but it badly understates the spread you would see for any single future eruption, which is the job of the prediction interval.

Explanation: The point estimate 4.18 is identical, but the prediction interval spans 3.20 to 5.16, far wider than the confidence interval's 4.10 to 4.25. The reason is variance: a prediction interval adds the residual variance $\sigma^2$ for an individual case to the variance of the fitted mean, so its half-width is built on $\sqrt{\sigma^2 + \text{Var}(\hat{y})}$. Confusing the two is a classic error; use confidence intervals for averages and prediction intervals for single new observations.

Explanation: One predict() call scores every row of newdata, returning a matrix with a row per input, which is how you build a whole prediction table without looping. Notice the intervals are tightest for the 75-minute wait, closest to the mean of waiting, and widen toward the extremes: confidence bands flare as you move away from the center of the data. Extrapolating far past the observed wait range would widen them further and should be treated with suspicion.

Explanation: Adjusted R-squared rose from 0.745 to 0.815 when hp entered, so horsepower explains genuinely new variance rather than padding the fit. Unlike ordinary R-squared, the adjusted version can fall when a useless predictor is added, because its penalty for parameters outweighs a trivial gain in fit. That makes it a safe quick screen, though it only ranks models on the same response and is not a formal significance test, which the F-test provides.

Explanation: AIC estimates out-of-sample prediction error up to a constant, so only differences matter; the two-predictor model wins by about 9.4 AIC units, a gap large enough (a difference above ~10 is decisive) to prefer it clearly. The df column counts estimated parameters, including the error variance. Unlike the F-test, AIC can compare non-nested models, which is why it is the workhorse for stepwise selection, though it still assumes the same response and dataset across candidates.

Explanation: The F-test asks whether adding hp reduces the residual sum of squares by more than random noise would. Here $F = 12.38$ on 1 and 29 degrees of freedom gives $p = 0.0015$, so horsepower's contribution is real. This test is exact for nested linear models fitted on identical rows, and for a single added term its p value matches that predictor's t-test p value. Use it when one model strictly contains the other; reach for AIC when they do not.

Explanation: The two cylinder dummies use 2 degrees of freedom, and the F-test judges them jointly: $F = 2.88$ with $p = 0.074$ falls just short of the 0.05 line, so cylinders add only weak evidence once weight and horsepower are present. Testing the block together is the correct move, because scanning the two individual dummy p values separately would multiply the chance of a false positive and could not answer "does cylinder count matter at all?" in one number.