Navigate
Roadmap Tutorials Exercises Tools Get certified
Lesson 6 of 6

Quantile Regression Forests and Prediction Intervals

In Lesson 5 you forced a model's predictions to move in a sensible DIRECTION. This last lesson fixes a different gap, one that every model in this course has left open: they all hand back a single number.

Sam, who runs the bike-rental kiosks, asks his forecast one honest question the morning before a warm day: "How many bikes will go out tomorrow?" The model answers "about 210." But Sam does not run on a point estimate. He has to decide how many bikes to have serviced and how many staff to call in. "About 210" gives him no way to plan for a slow day or a rush. What he actually needs is a RANGE he can trust: "very likely somewhere between 130 and 279, and here is how sure I am."

That range is a prediction interval, and by the end of this lesson you will be able to:

  • Say what a prediction interval is, and why it is not the same as a confidence interval
  • Read a quantile of past outcomes in R, and turn "a 90% interval" into the right pair of quantiles
  • Explain the quantile (pinball) loss and see, on real numbers, why its minimizer is a quantile
  • Build a prediction interval from a random forest in R, and check that it actually covers

Prerequisites: you can run R, and you know what a random forest is: an ensemble that averages many decision trees (the Random Forests course builds one from scratch). Lesson 1 of this course (Gradient Boosting from Scratch) helps but is not required. The band below is the shape of the answer we are after: not a single line, but a range around it.

The building block

A quantile is a cut point in your outcomes

Before we touch a forest, let us nail down the one word the whole lesson rests on: quantile.

Sam pulls the 21 most similar warm days from his records, the rentals on each:

RInteractive R
past <- c(150, 162, 170, 176, 182, 188, 193, 197, 200, 203, 206, 210, 214, 219, 225, 231, 238, 246, 252, 258, 270)

  

  





A quantile is just a value that a given FRACTION of these outcomes fall below. The median is the 0.5 quantile: half the days came in under it. The 0.9 quantile is the value that 90% of days stayed below, and 10% beat. In R, quantile() reads them straight off the data:




  

    
RInteractive R

    
quantile(past, c(0.05, 0.50, 0.95))
#>  5% 50% 95%
#> 162 206 258

    

      
      
    

    

  

  





So on a day like these, half the time Sam rents fewer than 206 bikes; only the busiest 5% of days top 258; and only the slowest 5% fall below 162.



Key Insight
Write \(q_\tau\) for the quantile at level \(\tau\) (a fraction between 0 and 1). It is defined by \(P(Y \le q_\tau) = \tau\): the outcome \(Y\) lands at or below \(q_\tau\) a fraction \(\tau\) of the time. The median is \(q_{0.5}\); the two ends of a 90% interval are \(q_{0.05}\) and \(q_{0.95}\).



Those three numbers, 162 / 206 / 258, already ARE a forecast with a range attached. The rest of the lesson is about getting them from a model, for any day, without hand-picking "similar days."