Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 1 of 11

Matching and the Propensity Score

A hospital offers an optional cardiac-rehab program to patients recovering from a heart attack. Three months later, the patients who enrolled walk 52 metres further in six minutes than the patients who did not. A press release writes itself: rehab adds 52 metres. Except it did not. The patients who signed up were fitter and more motivated to begin with, so they would have recovered better even with no program at all. Part of that 52-metre gap is the rehab; part is just who chose to enroll.

This is the central problem of learning from data you did not get to randomize. When people pick their own treatment, the treated and untreated groups differ for reasons that have nothing to do with the treatment. This lesson gives you the first and most intuitive tool for fixing it: estimate how likely each person was to be treated (their propensity score), pair each treated person with an untreated look-alike, and compare only fair matches. Toggle the panel below to feel the whole idea in miniature: two groups that start out mismatched, snapped into alignment.

By the end of this lesson you will be able to:

  • Explain why a naive difference between a treated and an untreated group is biased when people self-select
  • State what a causal effect actually is, using potential outcomes, and split the naive gap into a real effect plus selection bias
  • Estimate a propensity score with glm, match on it, and recover an effect a raw comparison gets badly wrong
  • Read the standardized mean difference to check that matching worked, and name what matching still cannot fix

Prerequisites: you can fit a logistic regression with glm(..., family = binomial) and read its predicted probabilities, and you know why a randomized experiment removes confounding.

The trap

You cannot just compare the two groups

Let us build the hospital's data so the trap is impossible to wave away. Four hundred patients; each has a baseline fitness at discharge (metres walked in six minutes) and either enrolled in rehab or did not. We plant a true rehab effect of exactly 25 metres, so we know the honest answer and can watch the naive comparison miss it. Crucially, fitter patients (higher baseline) are both more likely to enroll AND recover more on their own, which is the confounding.

RInteractive R
set.seed(2024) n <- 400 baseline <- round(rnorm(n, 350, 70)) # 6-min walk at discharge, metres rehab <- rbinom(n, 1, plogis(0.15 + 1.1 * (baseline - 350) / 70)) # 1 = enrolled in rehab improve <- round(40 + 25 * rehab + 0.45 * (baseline - 350) + rnorm(n, 0, 25)) # 3-month gain, metres study <- data.frame(baseline, rehab, improve) table(study$rehab) # how many enrolled #> #> 0 1 #> 189 211 round(tapply(study$improve, study$rehab, mean), 1) # mean gain in each group #> 0 1 #> 26.4 78.5

  

  





The enrolled group gained 78.5 m on average, the others 26.4 m. Subtract, and you get the headline.




  

    
RInteractive R

    
naive <- mean(study$improve[study$rehab == 1]) - mean(study$improve[study$rehab == 0])
round(naive, 1)
#> [1] 52.1

    

      
      
    

    

  

  





A 52.1-metre gap, more than double the 25 metres we actually built in. Why is it so inflated? Because we are not comparing rehab against no-rehab. We are comparing the kind of patient who enrolls against the kind who does not, and those two kinds of patient were different before rehab entered the picture. The next step draws exactly why that ruins the comparison.