Conjugacy and Choosing Priors
In Lesson 1, Asha updated her belief about her redesigned checkout page the brute-force way: 1,001 candidate conversion rates on a grid, multiply prior by likelihood at every single one, rescale. It worked, and it ended on a loose thread. Her prior was dbeta(theta, 8, 72), and the posterior came out as another curve of the exact same family. That is not luck. It is a property called conjugacy, and it collapses the whole grid ritual into two additions you can do in your head.
This lesson is about that shortcut, and about the responsibility it exposes: the prior is the one ingredient of a Bayesian answer that you choose. By the end you will be able to:
- Update a Beta prior with new counts in closed form, two additions, no grid
- Read any prior as pseudo-data and predict, before computing anything, how hard it will pull the answer
- Run the matching closed-form update for an average (the Normal-Normal pair) in base R
- Choose between flat, weakly informative and informative priors, and measure with a sensitivity analysis how much the choice mattered
Prerequisites: Lesson 1 of this course (prior, likelihood, posterior, the grid update, the 95 percent credible interval, and Asha's numbers: a Beta(8, 72) prior, then 8 purchases in 40 visits), plus base R vectors and plots.
The machine you drove in Lesson 1 sits below. Every posterior it draws, this lesson teaches you to compute with one line of arithmetic.
Conjugacy: why Beta in gives Beta out
To see why the grid was never necessary, write down what it actually multiplied. Lesson 1 introduced the Beta family through dbeta(): a curve over rates between 0 and 1 with two shape numbers, which we will now call \(a\) and \(b\) (Asha used \(a = 8\), \(b = 72\)). Its recipe, up to the usual rescaling constant, is
\[ p(\theta) \;\propto\; \theta^{\,a-1}\,(1-\theta)^{\,b-1}, \]
where \(\theta\) is the candidate conversion rate and \(\propto\) means "proportional to", equal up to a constant. The likelihood of \(k\) purchases in \(n\) visits is the binomial from Lesson 1, and its \(\theta\)-dependent part is \(\theta^{k}(1-\theta)^{\,n-k}\) (the counting term \(\binom{n}{k}\) is a constant, and constants wash out in the rescale). Now multiply prior by likelihood, the whole Bayesian update, and just add exponents:
\[ \underbrace{\theta^{\,a-1}(1-\theta)^{\,b-1}}_{\text{prior}} \;\times\; \underbrace{\theta^{k}(1-\theta)^{\,n-k}}_{\text{likelihood}} \;=\; \theta^{\,(a+k)-1}\,(1-\theta)^{\,(b+n-k)-1}. \]
Look at the right-hand side: it is the Beta recipe again, with new shape numbers. That gives the entire update in one line:
\[ \text{Beta}(a,\; b) \;+\; \{k \text{ successes in } n \text{ trials}\} \;\longrightarrow\; \text{Beta}(a+k,\; b+n-k). \]
Successes land on \(a\), failures land on \(b\). For Asha: Beta(8, 72) plus 8 buys in 40 visits is Beta(8 + 8, 72 + 32) = Beta(16, 104). No grid, no loop, two additions. Check it against the grid you built last lesson:
The navy grid answer disappears inside the thick grey closed form: same peak, and the exact interval (0.079, 0.199) matches the one you read off the grid last lesson. A prior family that the update cannot kick you out of is called conjugate to the likelihood; the Beta family is conjugate to the binomial.