Type I / II Error Visualizer
Every hypothesis test can go wrong two ways: it can raise a false alarm when nothing is happening (a Type I error, rate alpha), or miss a real effect (a Type II error, rate beta). Pick a test, set the true effect size, sample size and significance level, and watch the null and alternative distributions overlap: the shaded regions are exactly your two error rates, and power = 1 - beta is your chance of catching a real effect. Everything runs in your browser.
-
The four outcomes, and how power responds
Reality is either "no effect" or "a real effect"; your test either rejects or retains the null. That is a 2x2 of outcomes, two of them errors. The bars fill to each outcome's probability. The sparklines trace how power moves as you push one input at a time.
How this is computed
✓ No data leaves your browser
✓ Verified against R's pwr package to 1e-6
✓ Free, no sign-up
Six tests, one picture
The overlap picture is the same for every test: a null distribution centered at "no effect" and an alternative shifted by the effect size. Only the reference distribution and the effect measure change.
| Test | What it models | Effect measure | Reference distribution |
|---|---|---|---|
| One-sample t | One group's mean vs a known value | Cohen's d | central vs noncentral t |
| Two-sample t | Two independent group means (equal n) | Cohen's d | central vs noncentral t |
| One proportion | A rate vs a fixed reference p0 | Cohen's h | normal, shifted by h√n |
| Two proportions | Two rates (A/B test), per-arm n | Cohen's h | normal, shifted by h√(n/2) |
| One-way ANOVA | k group means at once | Cohen's f | central vs noncentral F |
| Correlation | Pearson r vs zero | Pearson r | Fisher-z, normal on the √(n-3) scale |
Two details worth stating plainly, because tools disagree here. Two-proportion power uses the noncentrality h√(n/2) with n the per-arm size, matching R's pwr.2p.test. Correlation power uses the Fisher-z transform, matching pwr.r.test, not a noncentral-t approximation, so the number you see is the number that R prints.
Reading the plot
| Region | Meaning |
|---|---|
| Red tail on the null curve | Type I error rate alpha: the chance of rejecting H0 when it is actually true. |
| Amber area on the alt curve | Type II error rate beta: the chance of retaining H0 when the effect is real. |
| Alt curve beyond the critical line | Power = 1 - beta: the chance of correctly detecting the real effect. |
| Gap between the two peaks | The noncentrality: how far the effect and sample size push the alternative away from the null. |
Push n up and the two curves separate, so beta (amber) shrinks and power grows. Tighten alpha (0.05 to 0.01) and the critical line moves outward, so the red tail shrinks but the amber area grows: the classic alpha-beta trade-off.
Frequently asked questions
What is the difference between Type I and Type II error?
Type I error (alpha) is rejecting a true null hypothesis, a false positive. Type II error (beta) is failing to reject a false null, a false negative. Power is 1 - beta. They trade off: lowering alpha (a stricter test) increases beta and lowers power. The visualizer shows both distributions and how they shift as you change alpha, n or effect size.
How does sample size affect Type II error?
Larger n shrinks the standard error of the test statistic, separating the null and alternative distributions and reducing the overlap region. Type II error falls as n grows: doubling n roughly halves beta when other parameters are fixed. Drag the n slider and watch beta drop.
Why is alpha conventionally set to 0.05?
It is a historical convention from R.A. Fisher, not a theoretical optimum. Choose alpha from the cost of a false positive in your domain: 0.001 for high-stakes drug trials, 0.05 for most science, 0.1 for exploratory or pilot work. Pre-register the choice; do not shop for an alpha that flatters your result.
What exactly is statistical power?
Power is the probability that your test rejects the null hypothesis when a real effect of the size you specified is present. It is 1 minus the Type II error rate. A common target is 0.80, meaning you would catch a real effect four times out of five. Power depends on the effect size, the sample size and alpha, all of which you control here.
How do I raise power for a study?
Three levers: collect a larger sample, target a larger true effect (or measure it more precisely to reduce noise), or accept a larger alpha. Sample size is usually the practical one. Use this visualizer to find the n where power crosses 0.80 for your effect, then plan for at least that many observations per group.