Tune with the tune package

In Lesson 5 the loan-default model left us with an uncomfortable number: a cross-validated recall of about 0.26, meaning it caught barely a quarter of the applicants who actually defaulted. One honest response is to reach for a more flexible model. A decision tree can carve the loan book into regions of high and low risk that a straight-line logistic regression cannot.

But a tree comes with settings you have to choose: how deep it may grow, how hard it gets pruned back. Set them wrong and the tree either memorizes the training rows (and stumbles on new applicants) or collapses to a useless stump. The picture below is that whole problem in one shape. As a model grows more complex, error on the training data keeps falling, but error on fresh data traces a U: too simple on the left, too flexible on the right, with a sweet spot in between. Drag the slider and watch the two curves separate.

Tuning is how you find that sweet spot on purpose instead of guessing. The tune package searches a menu of candidate settings, scores each one honestly with the resampling from Lesson 4, and hands you the best.

By the end of this lesson you will be able to:

• Tell a parameter the model learns from a hyperparameter you set, and name a tree's two knobs
• Mark hyperparameters with tune() and lay out a grid of candidate settings
• Run a grid search across cross-validation folds with tune_grid(), scored by a metric set
• Read the results, select the best settings, finalize the workflow, and confirm it once on a sealed test set

Prerequisites: you can run R and use the |> pipe, and from earlier lessons you can bundle a recipe and model into a workflow (Lesson 3), score across cross-validation folds (Lesson 4), and [measure with roc_auc and a metric set](Measure-with-yardstick.html) (Lesson 5). We work from that same lender's loan book here.

A model with knobs to turn

Logistic regression had nothing to tune: you hand it the data, it estimates one slope per predictor, and that is that. A decision tree is different. It learns where to split (is income under 45k? is the job under two years old?), but you must tell it in advance how far it is allowed to go. Here is a small loan tree so the picture is concrete: each box asks one yes-or-no question, and each leaf at the bottom is a final verdict, default or repay.

Those settings you fix in advance are the tree's hyperparameters, and a tree has two that matter most:

• **tree_depth**: the most questions the tree may ask along any path from top to bottom. A depth of 2 asks at most two questions before deciding; a depth of 12 can ask twelve, carving the loan book into far finer regions.
• **cost_complexity**: how much the tree is punished for having many leaves, which prunes it back. rpart grows a tree by minimizing

\[ R_\alpha(T) = R(T) + \alpha \lvert T \rvert \]

where \(R(T)\) is how often tree \(T\) misclassifies a training applicant, \(\lvert T \rvert\) is the number of leaves (the final risk buckets), and \(\alpha\) is cost_complexity. A big \(\alpha\) makes every extra leaf expensive, so the tree stays small; an \(\alpha\) near zero lets it grow almost without limit.

The distinction is not academic; the knob genuinely changes the tree. Rebuild the lender's loan book here so the page runs on its own. Each row is one applicant, and defaulted (yes or no) is what we predict, with yes as the first factor level so it stays the class we care about catching. We also split off a quarter of the book as a test set right now, and do not touch it again until the final step.

Of 500 applicants, 202 defaulted and 298 repaid. Now grow two trees on the training applicants, one with pruning switched off and one with it turned up, and count how many leaves each ends up with:

Same data, same algorithm, one setting changed. The overgrown tree splits the 374 training applicants into 93 tiny buckets (it has all but memorized them); the pruned tree keeps just 2. Neither is likely the right tree for judging a brand-new applicant. That single hyperparameter, cost_complexity, swung the model from one extreme to the other. Choosing well between those extremes is exactly what tuning does.