caret Exercises in R: 21 Real-World Practice Problems

Explanation: train() is caret's universal front door: pass a formula, a data frame, and a method string, and the same call works for over 200 models. With no trainControl specified, caret defaults to 25 bootstrap resamples, which is why you see resampling output even though you only asked for lm. The bootstrap RMSE is honest out-of-bag error, not the in-sample residual SD that summary(lm(...)) prints.

Explanation: method = "rpart" hands the formula to the rpart package and asks caret to tune the complexity parameter cp, which controls when splits are pruned. The default tune grid picks three cp values from the unpruned tree's pruning sequence, which is why the readout shows exactly three rows. For an actual production tree, prefer rpart2 (tuned by maxdepth) when you need a depth ceiling rather than a complexity ceiling.

Explanation: The x and y interface skips formula parsing, which preserves factor encodings and is friendlier when predictor columns are matrices or sparse objects (the formula interface would densify them). tuneLength = 2 narrows caret's auto-generated mtry grid to two candidate values, cutting fit time on small datasets. For real binary classification with many predictors, switch to a tuneGrid expand.grid(mtry = c(2, 4, 8, 16)) so the search is explicit and reproducible.

Explanation: trainControl is the single object that holds every resampling decision: method, number, repeats, summaryFunction, classProbs, sampling, and seeds. Building it once and reusing it across competing models guarantees identical fold assignments so resamples() can make fair pairwise comparisons later. Bootstrap (the default) tends to bias estimates downward on small training sets and is harder to interpret than honest k-fold accuracy.

Explanation: Repeated k-fold runs the entire k-fold scheme multiple times with different random fold assignments, then averages across the repeats. This is the standard remedy when 5-fold variance is high relative to model-to-model differences (commonly under 1000 rows). With 5 folds and 3 repeats you get 15 held-out accuracy estimates per hyperparameter row, so the readout has tighter standard deviations than a single 5-fold run.

Explanation: createFolds() with the response as input does stratified splitting: each fold contains roughly the same proportion of each class as the full dataset. Passing returnTrain = TRUE flips its default behaviour (otherwise it returns held-out indices). Stratification is mandatory whenever a class is rare enough that a random split could put zero positives in a fold, which collapses metrics like sensitivity and ROC. The createDataPartition() helper does the same job for a single train/test split.

Explanation: Passing preProcess inside train() (not as a separate preProcess() call upstream) is the only way to keep the scaling parameters honest: caret fits the mean and SD on each training fold and applies them to the held-out fold, so no test-time information leaks into the means used at training time. Centering and scaling are required for any distance- or kernel-based model (knn, SVM, kernel ridge) and they are harmless for tree-based models.

Explanation: caret's pca preprocessor keeps enough components to retain 95 percent of variance by default (override with trainControl(preProcOptions = list(thresh = 0.99))). The center and scale steps run first (PCA assumes unit variance) and are not optional when preProcess includes pca. For trees this rotation usually hurts interpretability without helping accuracy (tree splits already partition arbitrary directions), but for linear and distance models on correlated predictors PCA acts as a poor-man's regularizer.

Explanation: Box-Cox only applies to strictly positive numeric columns, which is why the readout often shows fewer transformed columns than total predictors (it silently skips columns with zeros or negatives). The estimated lambda per column is fit on each training fold and applied to the held-out fold, just like center and scale. For columns with zeros, swap to "YeoJohnson", which handles non-positive values; for log-only transformations the plain log1p is simpler if you have a domain reason.

Explanation: Order matters inside preProcess: caret runs imputation first (so NAs do not break variance calculations), then nzv (so constant columns are dropped before scaling produces NaN), then center, then scale. The medianImpute step computes a per-column median on each training fold, never peeking at the held-out fold. If you need richer imputation switch to "knnImpute" or "bagImpute" (slower but multivariate). Always drop rows where the response is NA before train(), because caret cannot impute targets.

Explanation: tuneLength is caret's "give me N candidate values per tunable parameter" knob: it asks the method's underlying grid generator for that many values, spread sensibly across the legal range. It is the fastest way to explore beyond the default three rows when you do not have a strong prior on the parameter. For method = "rf" the grid contains only mtry (ntree is treated as a fixed hyperparameter and overridden via the ntree dot argument).

Explanation: The columns of tuneGrid must exactly match the method's tunable parameters, which you can look up with modelLookup("rpart") (returns cp, parameter type, and label). Any typo in the column name produces "The tuning parameter grid should have columns ..." which is the most common caret error after silent failures from a misnamed method. Use expand.grid() when you have more than one tunable parameter, which produces the cartesian product automatically.

Explanation: alpha = 0 is pure ridge regression (every coefficient shrinks toward zero but stays non-zero), alpha = 1 is pure lasso (some coefficients become exactly zero), and intermediate values blend the two. lambda controls the overall penalty strength: zero is OLS, infinity zeros everything out. Sweep both because the surface is two-dimensional and tuning lambda at a fixed alpha can land in a poor neighbourhood. glmnet internally fits along a full lambda path per alpha for free, so the joint grid is cheap.

Explanation: twoClassSummary computes area under the ROC curve, sensitivity, and specificity (in that order) and only works when classProbs = TRUE so caret has probabilities to threshold. The factor levels matter: caret treats the FIRST level as the "positive" class for sensitivity (so "auto" here is positive). Flip with relevel() if your convention is the opposite. For multi-class problems, replace twoClassSummary with multiClassSummary (from the caret extras) or build a custom summary function.

Explanation: createDataPartition() does stratified train/test splits, preserving class proportions in both halves, which matters more than people expect on three-class iris because a naive sample.int() can put unequal counts in the test slice and warp the accuracy estimate. confusionMatrix() prints both the counts table and a battery of derived metrics (PPV, NPV, balanced accuracy, kappa, exact binomial CI), so it is the natural one-line audit report for a classifier.

Explanation: sampling = "down" tells caret to randomly drop majority-class rows inside each training fold until the classes are balanced, then fit the model on that balanced slice; the held-out fold is left untouched so metrics still reflect the real prevalence. Alternatives are "up" (oversample minority with replacement), "smote" (synthetic minority oversampling, needs the themis or DMwR helper), and "rose" (a different synthetic method). Set the sampling on trainControl rather than upstream of train() so the rebalancing happens INSIDE the resampling loop, otherwise CV scores are optimistic.

Explanation: Two train() calls with the same seed BEFORE each call produce identical fold assignments, which is the prerequisite for a paired comparison. resamples() collates the per-fold metrics into a long table you can summarize, diff(), and plot with bwplot() or dotplot(). The cleaner alternative when you have many models is to use trainControl(seeds = ...) to pre-generate seeds for every resample, which decouples reproducibility from the order of model calls.

Explanation: summary.resamples() returns a list with one table per metric, each row a model, each column a quantile or mean. For visual comparison reach for bwplot(res) (box-and-whisker per model) or dotplot(res, metric = "Accuracy") which also draws Tukey confidence intervals. The mean alone hides skew and outliers, so always check the full quartile table before declaring a winner; a higher-mean model with a long left tail is often worse in practice than a slightly-lower-mean model with a tight distribution.

Explanation: diff() on a resamples object computes per-fold pairwise differences (knn fold 1 minus rpart fold 1, and so on), then summary() runs a paired t-test on each pair with Bonferroni-adjusted p-values. The upper triangle holds the mean difference (positive favours the row), the lower triangle holds the adjusted p-value. This is the cleanest defensible answer to "is model A really better than model B" when both share folds, and it is what to put in a report when stakeholders ask for statistical significance, not just a higher mean.

Explanation: varImp() returns a model-specific importance score scaled to 0 through 100 by default, with the most important predictor pinned at 100. For random forest the underlying number is mean decrease in Gini impurity, summed across trees; for glm it is the absolute value of the t-statistic; for rpart it is the surrogate-split score. Always read varImp values relative to the others in the same model, never compare raw values across model types (the scales are not commensurable).

Explanation: predict() on a caret train object defaults to type = "raw" (class label or numeric prediction), but type = "prob" returns one column per class for classification models, which is what most downstream calibration, threshold tuning, and lift analysis needs. Class probability output requires that the model was trained with classProbs = TRUE in trainControl, otherwise caret will refuse and raise an error. model$bestTune holds the single hyperparameter row that won on the resampling metric, which is what predict() uses by default.