Machine Learning Exercises in R: 50 Real Practice Problems

Explanation: Setting the seed before sample() is the only guarantee that two runs (or two analysts) produce the same split. sample(n, k) returns k unique row indices, so mtcars[idx, ] and mtcars[-idx, ] partition the rows without overlap. A common mistake is to call set.seed() once at the top of a long script and then run interactive snippets out of order, breaking reproducibility.

Explanation: A plain random split can over-sample one class by chance, especially when the minority class is rare. Splitting within each species, then re-combining, preserves the marginal distribution and is the right default for classification. split() returns a list keyed by factor level; do.call(rbind, ...) glues the pieces back into a data frame.

Explanation: A validation set lets you peek at model selection without spending your test budget. The single-shuffle trick (sample(n) once, slice three ways) avoids the bug where independent random draws can put the same row in two splits. Holding out a true test set is the practical defence against tuning yourself into an optimistic accuracy number.

Explanation: vapply() returns a typed logical vector, so raw[chr_cols] selects exactly the character columns. lapply() walks each one and replaces it in place via the assignment. Doing the conversion in one pass beats a column-by-column copy: it keeps the data frame intact and avoids subtle bugs where you forget to convert one column.

Explanation: is.na() returns a logical matrix the same shape as the data; colSums() then counts the TRUE cells per column because logicals coerce to 0/1. Sorting descending puts the problem columns first. If a column is more than half missing, that's usually a deletion candidate; small gaps are imputed.

Explanation: A naive baseline anchors your interpretation of every later RMSE. If the linear model lands at 3 and the baseline is 6, that's a 50 percent improvement. Without the anchor, "RMSE 3" is meaningless. The mean predictor minimises squared error among constant predictors, which is why it's the default null model for regression.

Explanation: For balanced multi-class problems the majority-class baseline is 1 / k, here 1/3. On an imbalanced dataset (say 95 percent class A), the baseline jumps to 0.95 and headline accuracy stops being informative. That's the moment to switch to precision, recall, or balanced accuracy, all of which we explore in Section 6.

Explanation: Passing the formula in as an argument is what makes the helper general: all.vars(formula)[1] pulls the response name so the function can extract y_test without hardcoding mpg. The same skeleton extends to other model families by swapping lm for rpart or randomForest. Once you have this helper, comparing five models is five one-liners.

Explanation: Each coefficient is the change in mpg for a one-unit increase in that predictor while holding the other fixed. Negative slopes for wt and hp match the physical intuition: heavier and more powerful cars use more fuel. lm() solves the normal equations under the hood; for problems with multicollinearity or huge p, ridge or lasso are safer.

Explanation: The base slope -5.647 is the wt effect for the reference level (4 cylinders). The interaction terms wt:factor(cyl)6 and wt:factor(cyl)8 add to that base slope, so 8-cylinder cars actually have a less steep wt-mpg relationship (-5.647 + 3.455 = -2.192). The factor() wrapper is what stops R from treating cyl as a number and forces it to be a categorical predictor.

Explanation: poly(wt, 2) builds orthogonal polynomial basis columns, which is numerically safer than raw wt + I(wt^2) when degrees climb. The R-squared jumps from about 0.75 (linear) to 0.82, a real improvement. The risk: high-degree polynomials overfit, so always validate on a holdout or via cross-validation before deploying.

Explanation: With default complexity parameter the tree often stops at a single split, here on cyl. That's expected: rpart prunes aggressively by default to avoid overfitting on the small 32-row mtcars dataset. Drop the complexity parameter via control = rpart.control(cp = 0.01) to grow a bushier tree. Trees split on the variable that most reduces residual variance at each node.

Explanation: xerror is the cross-validated relative error and xstd is its standard deviation. The "1-SE rule" says pick the smallest tree within one xstd of the minimum xerror; that's typically more robust than picking the absolute minimum. Pruning back to that point reduces variance without giving up much accuracy.

Explanation: Each tree sees a bootstrap sample of rows and a random subset of mtry predictors at each split, which decorrelates the trees and lets averaging cut variance. The reported "Mean of squared residuals" is the out-of-bag error: each row is scored only by trees that didn't see it during training, so OOB stands in for a free validation set.

Explanation: With default settings randomForest reports node-purity decrease, which is the total drop in sum-of-squares across all splits using that variable. Pass importance = TRUE at fit time to also get the permutation-based %IncMSE, which is generally the more honest metric because it isn't biased toward high-cardinality predictors.

Explanation: On this tiny 32-row dataset the random forest wins, but the gap shrinks (and lm often wins) when predictors are mostly linear. The general rule: pick the simplest model that meets your accuracy bar. A one-line lm is easier to debug, ship, and explain than a forest, and the bar to switch models should be a clear improvement on a real holdout.

Explanation: Coefficients are log-odds, not probabilities. exp(coef) turns them into odds ratios: a one-unit drop in wt multiplies the odds of being a manual transmission by exp(8.838), which is enormous. Big coefficients on a 32-row dataset are a warning sign of near-perfect separation; on real data you'd want regularisation or more rows before trusting the standard errors.

Explanation: type = "response" returns probabilities in [0, 1]. Without it you get log-odds. The 0.5 threshold is arbitrary and tuned later in Exercise 3.10. For imbalanced classes or asymmetric costs (a missed fraud is worse than a false alarm), shift the threshold to trade precision against recall.

Explanation: kNN votes among the k closest training points by Euclidean distance, so feature scale matters. iris is already on a comparable scale; on real data you'd standardise first with scale(). The trade-off in k: small k overfits (noisy boundary), large k underfits (smooths over class boundaries). Tune k via cross-validation in the next exercise.

Explanation: Iris is easy enough that many k values tie. On a harder dataset you'd see a U-shape: error falls then rises as k grows past the optimum. Picking k from a holdout is fine for one shot, but cross-validation (Section 4) gives a less noisy estimate. Re-running the search on a different seed quickly tells you whether the chosen k is robust.

Explanation: Naive Bayes estimates P(feature | class) separately per class. For numeric features it uses a Gaussian by default, parameterised by per-class mean and variance. The independence assumption is almost always wrong, yet the classifier still ranks classes well because mis-specification cancels out across features. Great for text and high-cardinality categorical data.

Explanation: Trees pick the split that maximises information gain (Gini by default) at each node. On iris two splits separate the three species with only 6 mistakes. The misclassification loss 100 at the root drops to 0 + 5 + 1 = 6 total across leaves, which is the empirical training error. Test accuracy is usually lower; always score on a holdout.

Explanation: The confusion matrix here is built from out-of-bag predictions, so it estimates generalisation error without a separate test set. About 5 percent of versicolor and virginica rows are confused with each other, which matches the known overlap between those species on petal measurements. Class error per row is a quick way to spot which class is hardest.

Explanation: For random forests, the class probability is the fraction of trees that voted for each class. Rows in the easy setosa region get unanimous 1.000 estimates; rows near the versicolor / virginica boundary spread their probability mass across both. A practitioner uses these probabilities for threshold tuning, cost-sensitive decisions, or building a stacked ensemble downstream.

Explanation: A confusion matrix lays the foundation for accuracy, precision, recall, and F1. The diagonal counts correct predictions; off-diagonal cells are the two error types. Naming the dimensions (predicted, actual) keeps the orientation clear so you don't accidentally swap rows and columns and report transposed metrics. Many production bugs trace back to this exact mix-up.

Explanation: F1 is the harmonic mean of precision and recall: it punishes the case where one metric is high and the other low. Sweeping the threshold and picking the F1-maximising point is a standard tuning move for imbalanced binary problems. For asymmetric cost matrices (say, missing a fraud is 10x worse than a false alarm), use weighted F1 or expected cost instead.

Explanation: Writing the CV loop yourself is the most reliable way to understand exactly which rows the model sees at each step. Many bugs come from accidentally leaking the response into preprocessing. sample(rep(1:k, length.out = n)) is the idiomatic way to assign each row a fold label uniformly at random. The mean of the fold RMSEs is your CV-RMSE estimate.

Explanation: Wrapping CV into a function is the move that lets you compare many models on the same folds. With small datasets like mtcars, 10-fold or even leave-one-out CV gives a less noisy RMSE estimate than 5-fold, at the cost of more compute. For larger data, 5-fold is usually enough and runs in a fifth of the time.

Explanation: Each repetition gives one CV estimate; averaging across repetitions cuts the variance contributed by the random fold assignment. You'll see a tighter confidence interval around the mean. Don't confuse repeated CV with nested CV: repeated CV smooths the same level, nested CV avoids leakage when you also tune hyperparameters.

Explanation: LOOCV has very low bias because each fit sees almost all the training data, but it has higher variance because the n estimates are highly correlated. For OLS there's a closed-form shortcut using leverage h_ii that avoids n separate refits: PRESS = sum of (resid_i / (1 - h_ii))^2. For other model families, you do the n refits.

Explanation: The bootstrap simulates draws from the population by sampling rows with replacement. The empirical 2.5 to 97.5 percentile range is the percentile confidence interval. This works without assuming a particular sampling distribution for the estimator, which is the whole point: when residuals aren't normal or the model isn't standard, the bootstrap is still valid.

Explanation: With only 32 rows, rpart keeps prefering a stump regardless of the cp knob, so all grid points tie. On bigger datasets you'd see a U-shape: tiny cp overfits, huge cp underfits, and the minimum CV RMSE is the practical choice. Always pair grid search with CV, not a single holdout, when the dataset is small.

Explanation: Larger mtry lets trees be greedier on each split, which can lower bias but increase variance because trees end up more correlated. The default mtry = floor(p/3) for regression is usually a strong starting point, and tuning it rarely moves OOB error by more than a few percent. Tune only if the default is clearly off.

Explanation: Two error curves on one plot tell you a lot. A large train-val gap that doesn't close with more data is high variance: regularise, simplify, or get more training rows. A train error that's already high is high bias: the model is underfit, swap in a more flexible family. The shape of the curves is the diagnostic, not any one number.

Explanation: Dropping the intercept via - 1 produces k columns instead of k - 1, which is what you want when you feed the matrix to a tree-based model or kNN. For lm you usually keep the intercept and accept the reference-level encoding, since collinear columns wreck OLS. The choice depends on what consumes the matrix downstream.

Explanation: scale(x) subtracts the column mean and divides by the column SD. The result is unitless, so kNN and clustering algorithms treat all features fairly. The catch in real pipelines: compute mean and SD on the training set only, then apply the same shift and scale to the test set. Otherwise you leak test information through the standardisation step.

Explanation: Median imputation is robust to outliers and preserves the central tendency; mean imputation distorts the distribution when there are skewed values. For more sophisticated work, use multivariate imputation (mice) or model-based imputation (missForest). Always compute the imputation statistic on training data and apply it to test, or you'll leak.

Explanation: table() counts the non-NA levels; sorting by count and taking the top name gives the mode. Mode imputation is the categorical equivalent of median imputation, but it concentrates mass on the majority class. If the missingness isn't random (say, the questionnaire skipped a follow-up for low-engagement users), mode imputation can bias downstream models toward that pattern.

Explanation: Coefficient of variation (sd / mean) is a scale-free way to spot uninformative columns. Pure constant columns make lm refuse to estimate a coefficient (NA) and break some algorithms outright. Near-zero variance columns rarely help and often inflate variance estimates because the small denominator amplifies noise.

Explanation: The iterative greedy prune drops the predictor with the highest average correlation to remaining columns at each step. It's the same approach caret::findCorrelation uses. An alternative is to keep all predictors but use a regularised model (ridge), which handles collinearity without losing variables. Pruning is preferred when interpretability or runtime matters.

Explanation: log1p(x) = log(x + 1) is numerically safer than log(x) when zeros are possible. After the transform, a linear model fits the geometric mean rather than the arithmetic mean of price, and the residuals are usually closer to homoscedastic. Predict on the log scale, then back-transform with expm1() to report dollar predictions.

Explanation: Pre-computing interactions makes the resulting frame portable across model families: trees, kNN, and any model that doesn't understand R formulae can use them directly. The downside is feature-set sprawl: three columns becomes nine for a 4-way frame, and 21 columns for a 7-way one. Tools like model.matrix(~ .^2) automate two-way interactions when you want all of them at once.

Explanation: RMSE > MAE always, and the gap widens when errors are heavy-tailed. If your business cares more about avoiding big misses than small ones, optimise RMSE; if all errors cost the same regardless of size, use MAE. The choice should match the cost function in the real world, not be picked because one is more familiar.

Explanation: R-squared is the fraction of variance the model explains relative to the mean-only baseline. It's bounded by 1 from above but unbounded below: on a holdout, you can easily get negative R-squared if your model is worse than predicting the train mean. That's the standard sanity check for whether a model is actually adding value.

Explanation: OOB accuracy is the random-forest specific equivalent of CV accuracy. Each tree votes only for rows it didn't see during training. On balanced data accuracy is fine; on a 95/5 split, a constant majority predictor scores 0.95 and accuracy stops being informative. Then you switch to precision-recall (Exercise 6.4) or balanced accuracy.

Explanation: Precision answers: of the rows I called positive, how many really are? Recall answers: of all the real positives, how many did I catch? F1 mashes them into one number via the harmonic mean. For a fraud team, recall matters most. For a marketing team that pays per outreach, precision matters most. Pick the metric your stakeholder actually cares about.

Explanation: The formula uses the link between AUC and the Mann-Whitney U statistic: rank the probabilities, sum the ranks of positives, subtract the expected sum if positives were tied at the bottom, then normalise by the maximum possible. AUC = 0.5 means no skill, 1.0 means perfect separation. It doesn't depend on the threshold, which is why ROC AUC is preferred over accuracy for tuning probabilistic classifiers.

Explanation: With only 5 positives, simple random sampling routinely lands a test set with zero positives, making downstream metrics meaningless or undefined (recall divided by zero positives). Stratified sampling guarantees at least one positive in each split as long as the test fraction times the positive count is at least one. This is non-negotiable on real fraud, churn, or rare-disease datasets.

Explanation: If a model is well-calibrated, mean_pred should track mean_obs along the identity line. Deviations tell you where the model lies: a row predicting 0.95 when the bin observed rate is 0.6 is over-confident. Calibration is fixed post-hoc via isotonic or Platt scaling, which doesn't change rankings (so AUC stays) but realigns probabilities to observed rates.

Explanation: OLS forces mean(residuals) = 0 and cor(fitted, residuals) = 0 algebraically, so deviations are signs of numerical issues, not modelling problems. The sd of residuals tells you the typical error scale; the max absolute residual flags outliers. To diagnose non-linearity or heteroscedasticity, plot residuals against each predictor and look for patterns, not just summary statistics.