Top 20 Bayesian Problems to Solve in R

Explanation: This is the classic base-rate lesson. Even a 99% sensitive test flags only about 1 in 6 positives as genuine, because the 5% false-positive rate acts on the 99% of patients who are healthy and swamps the tiny pool of true positives. The common mistake is to read the test's accuracy as the answer; Bayes' theorem forces the prevalence back into the calculation. Lower the prevalence and the posterior collapses further.

Explanation: The binomial coefficient cancels in the normalization, so dropping it costs nothing. The prior favored 0.5 heavily, but the data pulled weight toward 0.7 because 3 heads in 4 is more likely under a biased coin. The posterior is a compromise: still peaked at 0.5, yet with the 0.7 hypothesis now the second strongest. Dividing by sum(prior * lik) is the discrete version of the normalizing constant.

Explanation: Coherent Bayesian updating is order independent and stream independent, so the running posterior after four single steps lands exactly on the one-shot batch posterior. The 1e-16 gap is floating-point rounding, not a real disagreement. One trap worth naming: use if (x == 1) theta else 1 - theta, not ifelse(), because ifelse() with a scalar test returns a single element and silently breaks the vectorized update.

Explanation: The Bayes factor is the ratio of how well each hypothesis predicted the data, here about 2.1 in favor of the biased coin. Because the prior odds were 1 to 1, the posterior odds equal the Bayes factor, and converting odds to a probability gives 0.68. A Bayes factor near 2 is only weak evidence, so 7 heads in 10 is far from conclusive. Prior odds other than 1 to 1 would multiply through before the final conversion.

Explanation: Conjugacy is what makes this a one-line update: the Beta prior and binomial likelihood combine into a Beta posterior with alpha incremented by successes and beta by failures. You can read the prior parameters as pseudo-counts, so Beta(2, 2) behaves like two prior heads and two prior tails before any real data arrives. This same rule powers every later problem in the section.

Explanation: The posterior mean 0.714 sits below the MLE of 0.8 because the Beta(2, 2) prior shrinks the estimate toward its own mean of 0.5. Mean and mode differ (0.714 versus 0.75) because the Beta is skewed for unequal parameters; the mode matches what a penalized maximum would report. Reporting the variance alongside the mean is what turns a point estimate into an honest statement of uncertainty, which a bare MLE cannot give.

Explanation: Prior parameters act like data you have already seen, so Beta(20, 20) enters the update with 40 pseudo-observations centered at 0.5 and barely budges after only 10 real trials, landing at 0.56. The flat Beta(1, 1) prior adds almost nothing and lets the data speak, reaching 0.75. The lesson for practice is to size the prior to your actual confidence: a strong prior needs strong justification, because with small samples it dominates the answer.

Explanation: The Beta-Binomial posterior mean is a shrinkage estimator: it is the sample proportion pulled a fraction of the way back to the prior mean, with the fraction set by how many pseudo-counts the prior carries relative to the sample size. Here the prior weight is 4/14, so 29% prior and 71% data. As n grows, the prior weight shrinks toward zero and the posterior mean converges to the MLE, which is exactly why priors matter most when data are scarce.

Explanation: Because the posterior is a genuine probability distribution over theta, its quantiles carve out an interval that contains 95% of the posterior mass, and you can say plainly that there is a 95% probability theta lies inside it. That interpretation is exactly the one students wrongly attach to a confidence interval. The equal-tailed choice is the simplest; a highest-density interval would be slightly narrower for this skewed posterior but needs an optimizer.

Explanation: The Wald interval runs past 1.0, an impossible value for a probability, which exposes how badly the normal approximation behaves near a boundary with only 10 trials. The credible interval stays inside [0, 1] by construction because the Beta posterior lives there. Beyond the numbers, the interpretations differ: the credible interval is a statement about theta given this data set, whereas the confidence interval is a statement about the procedure across hypothetical repetitions.

Explanation: Setting lower.tail = FALSE returns the upper-tail area, so these are direct answers to "how probable is it that theta beats this bar." There is a 95% posterior probability the coin favors heads and only a 3% probability it is extreme past 0.9. A frequentist cannot make either statement, because in that framework theta is a fixed unknown, not a random quantity. This is the machinery decisions are actually built on: probabilities of hypotheses, not just intervals.

Explanation: The uniform and Jeffreys priors are both weak, so with 10 observations they land within two percentage points of each other, which is the reassuring case where the prior barely matters. The informative Beta(20, 20) tells a different story: it adds 40 pseudo-counts at 0.5 and drags the mean down to 0.56 with a much tighter interval. A sensitivity table like this is standard practice; when weak priors agree you report robustness, and when a strong prior disagrees you must defend it.

Explanation: The posterior predictive integrates the outcome model over every value of theta weighted by the posterior, and for a single Bernoulli trial that integral is just the posterior mean. It happens to coincide with the plug-in estimate here only because the success probability is linear in theta. Any nonlinear summary, such as the odds in Exercise 6.3 or the count in the next problem, would separate the fully-averaged predictive from the naive plug-in, which is the whole reason prediction and estimation are distinct steps.

Explanation: The Beta-Binomial is what you get when you refuse to freeze theta at a point estimate and instead carry its full posterior into the forecast. The resulting distribution has heavier tails than a binomial evaluated at theta-hat = 0.714, because it folds in the uncertainty about theta on top of the sampling variability. Predicting with dbinom(y, 5, 0.714) is the common shortcut that understates how uncertain a five-trial forecast really is.

Explanation: The marginal likelihood is the probability of the data averaged over a model's prior, and comparing two of them yields a Bayes factor that automatically penalizes needless flexibility, an embodiment of Occam's razor. The value near 2 says the free-parameter model explains 8 in 10 only about twice as well as the fair-coin model, which counts as weak evidence. Marginal likelihoods are notoriously sensitive to prior width, so a Bayes factor should always be reported with the prior it used.

Explanation: Grid approximation replaces calculus with arithmetic: lay down closely spaced values of theta, weight each by prior times likelihood, normalize so the weights sum to one, and any posterior summary becomes a weighted sum. With 1001 points it recovers the exact conjugate mean to the printed precision. The catch is dimensionality: a grid is fine for one or two parameters but the number of points explodes with each extra dimension, which is why real models reach for sampling instead.

Explanation: The triangular prior has no conjugate partner, so there is no Beta shortcut, yet the grid method handles it with the same three steps as before. The prior's pull toward 0.5 lowers the posterior mean to 0.697, below the conjugate 0.75, and the maximum a posteriori estimate at 0.727 marks the grid point of highest posterior density. The gap between the MAP and the mean signals a skewed posterior, a diagnostic you get for free once the whole curve is in hand.

Explanation: Monte Carlo trades algebra for samples: with 10000 draws the sample mean matches the analytic posterior mean to three decimals, and the sample quantiles reproduce the credible interval from Exercise 3.1. The approximation error shrinks like one over the square root of the sample count, so more draws buy more precision. Fixing the seed is what makes the result reproducible, and it is the reason the same three problems in this section share identical draws.

Explanation: The comparison draws > 0.5 produces a logical vector, and taking its mean averages ones and zeros into the sampled probability, which lands within 0.002 of the exact tail area. This indicator-average trick estimates any event probability the same way, including events with no closed form where pbeta is unavailable. Its Monte Carlo error scales with the square root of p times one minus p over the sample size, so rare events need many more draws for the same relative accuracy.

Explanation: The change-of-variables calculus that a nonlinear transform normally demands disappears once you have draws: apply the function elementwise and summarise the result. The odds distribution is strongly right-skewed, so its mean of 3.35 sits well above its median and the upper credible bound stretches past 10. Plugging the posterior mean odds from a single point estimate would badly understate that upper tail, which is precisely why Monte Carlo is the default for derived quantities in modern Bayesian work.