Logistic Regression Exercises in R: 25 Practice Problems

Explanation: glm() with family = binomial fits a logistic regression by maximum likelihood. The default link is logit, so coefficients are on the log-odds scale: a one-MPG increase raises the log-odds of a manual transmission by 0.307. The Wald z-value tests whether each coefficient differs from zero. Use family = binomial(link = "probit") if you ever need the probit alternative.

Explanation: The default type = "link" returns log-odds; type = "response" applies the inverse logit so values sit in (0, 1). For a thin-tailed sigmoid like this one, the probability jumps fastest near the decision boundary (mpg around 21.5 where log-odds = 0). Always pass a data frame with the same column name as the predictor, not a bare vector.

Explanation: plogis(x) is 1/(1 + exp(-x)), the inverse logit. It maps any real number to (0, 1). Useful when you have raw linear predictors from predict(type = "link"), hand-computed log-odds, or coefficients from a paper. Its companion qlogis() goes the other way (probability to log-odds), handy for plotting on the logit scale.

Explanation: Null deviance measures the fit of an intercept-only model; residual deviance measures the fit with predictors. The drop (43.2 to 29.7, a difference of 13.5) is the model's improvement. Compared to a chi-squared distribution with df = 1, that drop is highly significant. AIC penalizes deviance by 2 * p, so it can rank non-nested models with different predictor sets.

Explanation: Logistic coefficients are log-odds, so exp() returns the multiplicative effect on the odds. An OR of 1.36 for mpg means the odds rise about 36% for every extra MPG. ORs above 1 mean positive association, below 1 negative, and exactly 1 no effect. The intercept's OR (0.00136) is the baseline odds when mpg = 0, which is rarely interesting on its own.

Explanation: confint() uses a profile-likelihood CI by default (slower but more accurate than the Wald CI from confint.default()). Because the mpg CI [1.13, 1.79] excludes 1, the effect is significant at the 5% level: every extra MPG multiplies the odds of a manual by between 13% and 79%. CIs that span 1 mean the predictor is not statistically distinguishable from no effect.

Explanation: Reporting "odds increase 35.9% per MPG" is more readable than an OR of 1.36 for non-technical audiences. The conversion is (OR - 1) * 100. If the OR were below 1 (say 0.80), you would report "odds decrease 20%". For very small effect sizes, log-odds and the (OR - 1) * 100 value are approximately equal because log(1 + x) ≈ x near zero.

Explanation: The LRT statistic equals the deviance drop (10.5) and follows a chi-squared distribution with df equal to the parameter difference (1). A p-value of 0.0012 rejects the null that the smaller model is adequate. Use test = "LRT" (or "Chisq") for nested logistic models. For comparing non-nested or different-family models, use AIC instead.

Explanation: Multi-predictor logistic models are fit the same way as univariate ones, just with more terms on the right side of the formula. Each coefficient is the partial log-odds effect holding the others constant. The "fitted probabilities 0 or 1" warning means the classes are (nearly) perfectly separable: log-likelihood can be increased indefinitely by inflating coefficients. For real classification you would penalize this with glmnet::glmnet() or arm::bayesglm().

Explanation: A factor predictor with K levels becomes K-1 dummies. The intercept's OR (2.67) is the baseline odds of a manual for 4-cylinder cars. The OR for factor(cyl)6 of 0.50 means 6-cylinder cars have half the odds of a manual versus 4-cylinder; 8-cylinder cars drop to 16%. Change the reference category with relevel(factor(cyl), ref = "8") if you want 8-cyl as the baseline.

Explanation: discount * new_customer is shorthand for discount + new_customer + discount:new_customer. The interaction coefficient (0.071 on the log-odds scale) means each extra discount point lifts the log-odds of conversion an additional 0.071 for new customers, on top of the main effect of 0.052. Always interpret interactions before main effects; main-effect coefficients are conditional when one variable is zero.

Explanation: After standardizing, each coefficient is the log-odds change per one-standard-deviation move in that predictor, so magnitudes are comparable. Here wt has the largest absolute effect (-9.57), then hp (+7.33), then mpg (+3.21). Standardization does NOT change predictions or p-values, only the coefficient scale and interpretation; use it for ranking importance, not for changing model fit.

Explanation: A confusion matrix is the foundation of every classification metric. Diagonal entries (17 + 10 = 27 correct) divided by total (32) gives accuracy of 0.844. Off-diagonals split into 3 false negatives (predicted 0, actually 1) and 2 false positives. Pick the cutoff to match business cost: lower it to catch more positives, raise it to be more confident before flagging. The choice is decision-theoretic, not statistical.

Explanation: Sensitivity and specificity are conditional rates, not unconditional. Sensitivity asks "of the actual positives, what share did we catch?"; specificity asks "of the actual negatives, what share did we correctly rule out?". They trade off as you slide the threshold. Accuracy alone is misleading under class imbalance: a 99% no-event base rate makes "always predict no" hit 99% accuracy while catching zero positives.

Explanation: AUC integrates the ROC across all thresholds, giving one threshold-free summary of how well the model ranks positives above negatives. AUC of 0.5 is random; 1.0 is perfect ranking; 0.892 is strong. Equivalent interpretation: pick a random positive and a random negative; AUC is the probability the model assigns the positive a higher score. For severely imbalanced data, prefer area under the precision-recall curve instead.

Explanation: McFadden's pseudo-R-squared compares the log-likelihood of the fitted model to that of an intercept-only model. Unlike OLS R-squared, it is bounded by 1 only in the limit and McFadden suggested values of 0.2 to 0.4 indicate "excellent" fit. Other variants exist (Cox-Snell, Nagelkerke, Tjur), and they disagree by design, so always name the one you report. Computed here without packages, but pscl::pR2() returns five of them at once.

Explanation: The "best" threshold is not always 0.5: it depends on class balance and cost asymmetry. Tuning by accuracy on training data risks overfitting and ignores the cost of false positives versus false negatives. In production, optimize on a holdout set and target a business metric (expected dollar value, F1, recall at fixed precision) rather than raw accuracy. Youden's J (sensitivity + specificity - 1) is also a popular threshold criterion.

Explanation: Logistic regression trained on highly imbalanced data tends to under-predict the rare class because the intercept is pulled toward the prevalence. Downsampling the majority class restores a balanced training set but throws away information; upsampling the minority (e.g., slice_sample(replace = TRUE)) or ROSE::ovun.sample() are alternatives. Always evaluate on the original imbalanced test set, not on balanced data, to get realistic performance.

Explanation: F1 is the harmonic mean of precision and recall, giving low values whenever either is low. It is preferred to accuracy under class imbalance because it ignores true negatives, which dominate accuracy on rare-event problems. The MLmetrics::F1_Score() and yardstick::f_meas() helpers compute it in one call, but writing it from scratch reinforces the formula. F-beta lets you weight recall versus precision asymmetrically.

Explanation: A precision-recall table at a handful of thresholds is the simplest way to expose the trade-off without a chart. Increasing the threshold raises precision (fewer false alarms) at the cost of recall (more missed positives). For ranking workflows (rank-and-review fraud queues), pick the threshold that hits the team's daily review capacity. For automated blocking, pick a precision floor and read off the matching recall.

Explanation: Case weights tell glm() to count each rare-class observation as if it appeared multiple times, raising the intercept toward what it would be under a balanced sample. Standard errors from weighted GLMs treat the weights as known frequencies, so confidence intervals shrink artificially compared to actually having that many samples. For correct inference, prefer a real resampling scheme or a robust sandwich estimator via sandwich::vcovHC().

Explanation: The training AUC tells you how well the model fits the data it has seen; the test AUC tells you how well it generalizes. The gap between them quantifies overfitting. For small samples, repeat the split many times or use caret::createFolds() or rsample::vfold_cv() for k-fold cross-validation. Hand-coded splits with sample() are fine for one-off teaching but lack stratification by outcome.

Explanation: A complete classification report needs more than accuracy. AUC is threshold-free; sensitivity/specificity expose where errors are happening at the chosen cutoff. For a churn use case, sensitivity matters more (catching at-risk customers), so the team might pick a lower threshold to trade specificity for recall. Wrap the whole pipeline in a function (fit_eval(train, test)) for repeatable benchmarking across model families.

Explanation: broom::tidy() returns one row per coefficient as a tibble, which composes naturally with dplyr, ggplot2, and gt. exponentiate = TRUE converts log-odds to odds ratios automatically, and conf.int = TRUE adds profile-likelihood CIs. The accompanying broom::glance() returns one row per model (deviance, AIC, df) and broom::augment() returns one row per observation (predicted probabilities, residuals).

Explanation: The cost-aware threshold solves "email if expected profit > 0", i.e. p * 10 - 0.5 > 0, which closed-form gives p > 0.05. The empirical sweep recovers the same answer and generalizes to non-linear cost functions (e.g. fatigue limits, daily budgets). Always tie classification thresholds to a business objective, not an off-the-shelf default like 0.5; the right number for a high-volume low-cost campaign is very different from a one-shot expensive intervention.