Poisson Regression Exercises in R: 20 Practice Problems

Explanation: family = poisson chooses the log link by default, so each coefficient is on the log-rate scale. The negative woolB, tensionM, and tensionH coefficients mean wool B and higher tension both reduce expected breaks compared with the reference categories (wool A, tension L). The residual deviance (210.4) being well above the residual degrees of freedom (50) is an early signal of overdispersion: you will return to this in Section 4.

Explanation: Because the log link means the linear predictor is the log of the expected count, exponentiating each coefficient gives an incidence rate ratio. tensionH has the smallest IRR (0.595), so switching from low to high tension multiplies the expected breaks by about 0.6, the strongest reduction in the model. The intercept exponential is the expected count when every predictor is at its reference level (wool A, tension L).

Explanation: With one categorical predictor and identical replicates per level, the Poisson MLE for each region equals the log of that region's mean count: log(7) = 1.9459 (intercept = East), log(14/7) = 0.6931 (North vs East), and so on. Residual deviance is 0 here only because the within-region variance is artificially zero in the constructed data; on real data you would never see a perfect fit.

Explanation: All three non-intercept terms have p-values well below 0.05, so wool type and tension level both affect the expected break count. The z values use the asymptotic normal approximation to the Wald statistic. Be cautious about reading these p-values when overdispersion is present: in Section 4 you will see that quasi-Poisson inflates the standard errors and can flip borderline terms to non-significant.

Explanation: The Poisson GLM predicts on the log-rate scale by default; passing type = "response" exponentiates the linear predictor for you and returns the expected count. Make sure the factor levels in newdata match the training data so the design matrix is built consistently: forgetting to set levels for tension would silently shift the reference category and corrupt the prediction.

Explanation: Always build the interval on the link (log) scale where the normal approximation is reasonable, then exponentiate to land on the count scale. Building the interval directly on the count scale (fit +/- 1.96 * se) is wrong because it ignores the asymmetry that the exponential introduces. The interval here is wider on the upside than the downside, exactly as you would expect for a log-linear model.

Explanation: Pearson residuals divide the raw residual by the square root of the fitted variance (sqrt(mu) for Poisson), while deviance residuals are signed square roots of the per-observation contribution to the deviance. They give very similar pictures for well-fit Poisson models but diverge in the tails; if a few deviance residuals exceed 2 in absolute value, treat them as a fit warning. The sum of squared Pearson residuals is what feeds the standard dispersion check in Section 4.

Explanation: Under the null that the Poisson model captures the variance correctly, the residual deviance is asymptotically chi-squared with the residual degrees of freedom. Here the test rejects spectacularly: the model leaves far more residual variation than Poisson allows. This is your second consistent signal of overdispersion and motivates the quasi-Poisson and negative binomial extensions later. Note that this test breaks down when fitted values are very small (under 1), so always pair it with a residual plot.

Explanation: Counts alone are uncomparable across regions with different exposure, so the offset rescales the linear predictor: log(mu) = log(exposure) + X beta, which is the same as modelling log(rate) = X beta. Always use offset(log(exposure)), never raw exposure as a covariate, because raw exposure would estimate an extra coefficient instead of fixing it at 1. The intercept exponential exp(-2.996) = 0.05 is the East baseline rate of 5 claims per 100 policy-years.

Explanation: With an offset, varying policy_years linearly scales the predicted count: 100 policy-years yields the expected claim count per 100 policy-years, exactly what stakeholders typically read. North has the highest rate (7.5 claims per 100 years) and South the lowest (3.75). If you set policy_years = 1 you would get the rate per policy-year, which is small and harder to read; rescale the offset value to the unit that communicates best.

Explanation: Both models fit the observed counts perfectly because each has one parameter per region, but they encode very different stories. The offset version says East and South share the same rate of 5 claims per 100 policy-years; the no-offset version implies East has nearly double South's risk simply because East's exposure happens to be larger. As soon as you predict for a new region with a different exposure, the no-offset model misleads.

Explanation: When you let R estimate the exposure coefficient instead of fixing it at 1 via offset, you let the data tell you whether the proportional-exposure assumption holds. Here the estimate is 1.000 to three decimals, confirming the assumption. In real practice, a coefficient noticeably below 1 (sub-linear exposure scaling) often signals concentration risk or capacity limits; above 1 suggests acceleration that may need a non-Poisson model.

Explanation: A dispersion of 3.83 means the data have roughly four times the variance that pure Poisson would allow, so standard errors from the Poisson fit are underestimated by sqrt(3.83) approximately 2x. Coefficient point estimates remain unbiased under quasi-likelihood, but every Wald p-value from summary(ex_1_1) is too small. This single number, together with the deviance test from 2.4, almost always tells you whether overdispersion is real.

Explanation: Quasi-Poisson inflates every standard error by sqrt(phi), where phi is estimated from the data. Coefficients are identical to the Poisson fit; only inference changes. The Wald statistic for woolB drops from 4.0 to 2.0 and its p-value jumps from 6e-5 to 0.046; on a stricter threshold (0.01) you would now treat wool type as marginal. Quasi-Poisson is the simplest, lightest-touch fix and works whenever variance grows linearly with the mean.

Explanation: The negative binomial adds a multiplicative gamma-distributed random effect that lets the variance be mu + mu^2 / theta. Coefficients are very close to the Poisson fit because the design and mean structure are unchanged, but standard errors now properly reflect overdispersion. Whenever quasi-Poisson and negative binomial disagree on which terms matter, prefer the negative binomial: it is a proper likelihood and supports AIC, likelihood ratio tests, and model comparison out of the box.

Explanation: The test on the boundary of the parameter space (theta -> infinity for Poisson) follows a half chi-squared distribution, so divide the standard p-value by 2. Here the negative binomial fits dramatically better than Poisson, which matches the dispersion of 3.83 you saw in 4.1. When the p-value is borderline (above about 0.01) and dispersion is below 2, prefer the simpler Poisson; otherwise, stick with the negative binomial.

Explanation: count() alone drops (day, campaign) pairs with zero observations, which is fatal for a Poisson model: zero counts carry real information and dropping them biases the rate upward. tidyr::complete() re-introduces missing combinations and fills the count with zero. Always complete the grid before fitting any count regression on aggregated events.

Explanation: Under Poisson, the expected count of zeros is sum(exp(-mu_i)) across observations. Warpbreaks has no zeros and only 0.026 expected zeros, so zero inflation is not the issue here (overdispersion comes from somewhere else, likely loom-to-loom variation). When you see observed zeros several times higher than expected, switch to a zero-inflated or hurdle model (pscl::zeroinfl, pscl::hurdle) instead of plain Poisson or negative binomial.

Explanation: The deviance drop of 71.6 on 2 degrees of freedom has a p-value below 2.2e-16, so tension clearly improves the fit even on top of wool. For non-nested models or when comparing Poisson with quasi-Poisson, prefer AIC or BIC instead. Note that on a quasi-Poisson model you would pass test = "F" because the dispersion parameter changes the test statistic.

Explanation: This stitches together every idea from the hub: rate modelling via offset, prediction on the link scale, exponentiating bounds back to counts, and reporting in the unit the stakeholder cares about (visits per 1000 staffed hours). In a real ER staffing review you would add a dispersion check (4.1) and probably refit as negative binomial if phi exceeds 1.5; the structure of the pipeline does not change.