Statistical Significance Calculator
Drop in the numbers from your test and find out whether the difference you are seeing is real or just noise. Compare two conversion rates, compare two averages, or test a single rate against a target. You get the p-value, a confidence interval, and a plain-English verdict you can paste straight into a report.
I want to compare two conversion rates
Statistically significant
How this is computed
prop.test() and t.test()
✓ Free, no sign-up
Which test does what
The right test depends on the shape of your data. This tool picks it for you from the mode you choose.
| You have | Question | Test used | Reproduces |
|---|---|---|---|
| Visitors and conversions for two variants | Do the two conversion rates differ? | Two-proportion z-test | prop.test(correct = FALSE) |
| Sample size, mean and SD for two groups | Do the two averages differ? | Welch t-test | t.test(var.equal = FALSE) |
| Visitors and conversions for one group plus a target | Does the rate differ from a target? | One-proportion z-test | prop.test(x, n, p = target) |
All three compare an observed difference to the spread you would expect from chance, then report a p-value. A small p-value means the difference is hard to explain as noise. The confidence interval shows the range of true differences consistent with your data, which is what tells you whether a significant effect is also a useful one.
Questions people ask
What does statistically significant actually mean?
It means the difference you observed is unlikely to have happened by random chance alone, given your sample size. The p-value is the probability of seeing a difference at least this large if there were truly no difference. When it falls below your threshold (commonly 0.05) the result is called significant. It does not mean the difference is large or important, only that it is unlikely to be pure noise.
How do I know if my A/B test result is significant?
Enter the visitors and conversions for each variant. The calculator runs a two-proportion z-test, computes the p-value, and compares it to your significance level. If the p-value is below the level, variant B's rate is significantly different from A's. Read the confidence interval for the lift alongside it: it shows the plausible range of the true difference.
Is statistical significance the same as a meaningful result?
No. With a very large sample even a tiny, useless lift can be significant; with a small sample a large and valuable lift can miss significance. Always read the confidence interval next to the p-value: it shows how big the true effect could plausibly be, which is what decides whether the change is worth shipping.
Should I use a one-sided or two-sided test?
Use a two-sided test when you care whether the groups differ in either direction. This is the safe default and the one to report. Use a one-sided test only when the direction was fixed in advance for a real reason, because it puts the whole significance budget in one tail and will flag smaller effects as significant in that direction.
Does this significance calculator match R?
Yes. The conversion test reproduces prop.test(correct = FALSE), the two-mean test reproduces t.test(var.equal = FALSE), and the one-proportion test reproduces prop.test against a target, matching to at least nine decimals across two-sided and one-sided alternatives and edge cases such as zero conversions.
Keep going
Related calculators and tutorials on r-statistics.co.