Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 6 of 13

GAMs, Splines and Smooths

Priya runs an ice-cream cart outside a train station. She has logged 120 days: the day's high temperature and how many ice creams she sold. She fits the obvious model, sales ~ temp, and it tells her something absurd: temperature has almost no effect (correlation -0.08, not significant). Yet she knows temperature is the single biggest thing that moves her sales.

The temperature is not the problem. The straight line is. Sales climb as it warms, peak on a pleasant 24 degree afternoon, then fall again when it is too hot to stand on the platform. That is a hill, and a straight line cannot climb a hill and come back down. This lesson gives your regression the ability to bend.

Lesson 5 handed Maya a short list of the channels that matter. Every model in this course so far, ordinary least squares, robust, quantile, ridge and lasso, has fit the same fundamental shape: a straight line (or a flat plane). Today the truth bends, and we let the model bend with it.

By the end of this lesson you will be able to:

  • Diagnose why a straight-line model fails on a relationship that bends
  • Explain a smooth as a spline: a flexible curve built by adding up simple building-block curves
  • Understand how a wiggliness penalty lets the data choose the right flexibility, so you never guess a polynomial degree
  • Fit a GAM in R, and read the effective degrees of freedom (edf) that tell you how much the curve bent

Prerequisites: you can fit and read an [lm()](OLS-Regression-from-Scratch.html) (coefficients, R-squared, significance), you know the bias-variance tradeoff (underfit vs overfit), and you have met a penalty that trades fit for simplicity in ridge regression.

Drag the smoothness dial below. Watch the fit go from a stiff line that misses the bend, to an honest curve that tracks the true shape (dashed), to a wild overfit that chases every noisy point. Finding that middle, automatically, is what a GAM does.

The problem

A straight line cannot bend

Let us reproduce Priya's frustration in code. We build her 120 days inline (each lesson runs in a fresh R session, so all the data lives right here), then fit the straight line she tried.

RInteractive R
library(ggplot2) set.seed(7) n <- 120 ice <- data.frame( temp = round(runif(n, 12, 36), 1), # daily high temperature, Celsius humidity = round(runif(n, 30, 90)), # percent relative humidity weekend = rbinom(n, 1, 0.3) # 1 on Saturday and Sunday ) # true sales: a hill that peaks near 24 C, plus a weekend bump and day-to-day noise base <- 135 - 0.22 * (ice$temp - 24)^2 ice$sales <- round(pmax(0, base + 10 * ice$weekend - 0.12 * ice$humidity + rnorm(n, 0, 8))) fit_lin <- lm(sales ~ temp, data = ice) round(coef(summary(fit_lin)), 3) #> Estimate Std. Error t value Pr(>|t|) #> (Intercept) 123.815 4.179 29.631 0.000 #> temp -0.140 0.166 -0.841 0.402

  

  





The slope on temp is -0.14 with a p-value of 0.40. Read literally, the model says: for every extra degree, sales change by essentially nothing, and we cannot even be sure of the direction. The overall fit is just as bleak.




  

    
RInteractive R

    
round(c(correlation = cor(ice$temp, ice$sales),
        r_squared   = summary(fit_lin)$r.squared), 3)
#> correlation   r_squared
#>      -0.077       0.006

    

      
      
    

    

  

  





An R-squared of 0.006 means the line explains six-tenths of one percent of the variation in sales. Now look at why. Plot the days and lay the best straight line over them.




  

    
RInteractive R

    
ggplot(ice, aes(temp, sales)) +
  geom_point(alpha = 0.5) +
  geom_smooth(method = "lm", se = FALSE) +      # the best straight line
  labs(x = "temperature (C)", y = "ice creams sold")

    

      
      
    

    

  

  





The points make a clear hill: low at both cold and sweltering days, high in the comfortable middle. The straight line runs flat through the crest, because the rising left half and the falling right half cancel out to a slope of nearly zero. The relationship is strong; the line is just the wrong shape to see it. We need a model that can follow the bend.