Lesson 3 of 3
Train, tune and read a forest in R
You understand why a forest works. Now you will build one for real, tune it without a test set, and learn to read what it tells you.
The story so far: a single tree overfits (Lesson 1); bootstrap, random features and averaging fix it (Lesson 2). This lesson turns that understanding into working R and the few knobs that actually matter.
- Get a free test set from the forest itself (OOB error)
- Tune mtry and trees on a live model and see the sweet spot
- Read variable importance, and know the limits
Prerequisites: Lessons 1 and 2 (trees, and how bootstrap + random features build a forest).
A free test set
Out-of-bag error
Remember the ~37% of rows each tree never saw (its bootstrap left them out)? Here is the payoff. To score the forest, run every row through only the trees that did not train on it, and average. No data held back, no separate test split needed.
Key Insight
OOB error is an honest, almost-free estimate of test performance, computed during training. On most problems it lands very close to a proper cross-validation, for a fraction of the work.