Sample Size Calculator for Proportions
Before you launch a test on rates, this answers the question you actually have: how many people do I need? Give it the rate you expect now and the rate you want to be able to detect, or hand it Cohen's h directly, and it returns the sample size that gives you a real chance of seeing that difference if it is there.
Part of the sample-size family: all sample size calculators.
I want to size an A/B test on two rates▾
Enter rates as decimals: 0.05 means 5%. Power 0.80 and alpha 0.05 are the usual defaults. Nothing you type leaves your browser.
Recruit 8,143 per group
16,286 subjects in total.
How this number is computed
✓ No data leaves your browser ✓ Verified against R's pwr.p.test(), pwr.2p.test(), pwr.2p2n.test() and power.prop.test() ✓ Free, no sign-up
Why the same lift costs more in the middle
Hold the absolute difference fixed and slide the baseline across the range. The sample size you need is not flat: it peaks when your two rates straddle 0.5 and falls away towards either end.
A proportion carries its own noise, and the amount of it is p(1 − p), which is largest at 0.5 and shrinks to nothing at 0 and 1. Near the middle you are trying to see a small signal through the most noise the scale can produce. Near the ends the same signal stands against much less noise, so far fewer subjects settle the question. This is why a conversion test moving 2% to 3% is a very different job from a satisfaction survey moving 50% to 51%, even though both are one point.
It also explains a result that looks wrong at first glance: 0.05 to 0.10 and 0.90 to 0.95 need exactly the same sample size. They are mirror images, and p(1 − p) is symmetric about 0.5.
What Cohen's h is, and why not just p₁ − p₂
The raw gap between two rates is a bad effect size. Cohen's h fixes it with one trick.
The problem with p₁ − p₂ is that it means different things in different places. A gap of 0.05 is a large, easily detected effect down at 0.02, and a small, expensive one at 0.50. One number, two completely different jobs, so it cannot be used to compare studies or to look up a sample size.
Cohen's h applies the arcsine transform to each rate first, then takes the difference:
| Step | What happens |
|---|---|
φ = 2 × arcsin(√p) | Stretches the crowded ends of the 0 to 1 scale and compresses the middle, so that a step of the same size is equally detectable wherever you take it. This is what "variance-stabilising" means. |
h = φ₁ − φ₂ | The distance between the two rates measured on that fairer scale. Now a given h means the same amount of detectability at 0.02 as at 0.50, so it can be looked up in a table. |
Cohen proposed 0.2, 0.5 and 0.8 as small, medium and large for h, mirroring his benchmarks for d. Treat them as a fallback for when you have nothing better, not as a law. The rates below are worked at 80% power and alpha 0.05, two-tailed.
| Cohen's h | Label | An example pair of rates | n per group at 80% power |
|---|---|---|---|
| 0.2 | Small | 0.05 vs 0.102 | 393 |
| 0.5 | Medium | 0.25 vs 0.488 | 63 |
| 0.8 | Large | 0.20 vs 0.578 | 25 |
The same 1/h² scaling as every other power calculation applies here: halving the effect you want to catch does not double the sample, it roughly quadruples it.
Why pwr and power.prop.test disagree
Two R functions, two answers, both defensible. This table is live: it recomputes for your current power, alpha and test direction.
| Rates | Cohen's h | Arcsine, pwr.2p.test | Normal, power.prop.test | Gap |
|---|
The pattern is not random. The two routines agree almost exactly near 0.5 and drift apart as the rates approach 0 or 1. That is the arcsine transform earning its keep: the normal approximation assumes the variance is roughly constant across the interval you are testing, which is fine in the middle and progressively worse at the ends, where p(1 − p) is changing fastest relative to its own size.
Neither answer is the truth. The truth is a binomial problem with no tidy closed form, and both of these are approximations to it. Use the arcsine route if you want the answer a power table or pwr would give you and you are thinking in effect sizes. Use the normal route if you are thinking in raw rates and want base R with no packages. When they disagree, take the larger one: being slightly over-powered costs money, being under-powered costs the whole study.
There is a second, separate reason they can differ, and it is a convention rather than mathematics. pwr's two-sided test counts the chance of rejecting in either tail. power.prop.test defaults to strict = FALSE, which ignores the chance of a significant result in the wrong direction and so reports a slightly smaller n. This page always uses strict = TRUE, and the R code it emits passes it explicitly, so the only difference you see in the table above is the real one.
What each design assumes
| Design | What n counts | Noncentrality | R function |
|---|---|---|---|
| One proportion | Observations | h√n | pwr.p.test |
| Two proportions, equal groups | Per group | h√(n/2) | pwr.2p.test |
| Two proportions, unequal groups | Each group | h√(n₁n₂/(n₁+n₂)) | pwr.2p2n.test |
| Two proportions, raw scale | Per group | none: works on p(1−p) directly | power.prop.test |
The unequal-groups formula is the general one. Set n₁ = n₂ = n and h√(n₁n₂/(n₁+n₂)) collapses to h√(n/2), which is why the balanced and unbalanced rows always agree at a 1:1 ratio. This calculator uses the general form for every two-proportion case.
All four assume you will analyse the result with a plain unpaired test of proportions, that the observations are independent, and that your assumed rates are roughly right. That last one is the weak link: a sample size is a forecast, and it inherits every error in the rates you fed it. If your baseline is a guess, run the numbers again with a pessimistic guess and see how much the answer moves.
Questions people ask
How many users do I need for an A/B test?
It depends on your baseline rate and the lift you want to catch, not on how much traffic you happen to have. To detect a move from 5% to 6% with 80% power at alpha = 0.05 two-tailed you need 8,143 per group, so 16,286 users in total. Conversion tests need big samples because a one point lift on a small base is a genuinely tiny effect: in Cohen's h it is 0.044, a fifth of what he called "small".
What is Cohen's h and why not just use p₁ − p₂?
Cohen's h is 2·arcsin(√p₁) − 2·arcsin(√p₂). The raw difference is not a stable effect size, because the noise in a proportion depends on where the proportion sits: p(1−p) is largest at 0.5 and shrinks towards 0 and 1. A gap of 0.05 is easy to detect at 0.02 and hard at 0.50, though the number is identical. The arcsine transform stretches the ends and flattens the middle, so one value of h means the same detectability everywhere and can be looked up in a table.
Why do pwr.2p.test and power.prop.test give different sample sizes?
They approximate the same binomial problem in two legitimate ways. pwr::pwr.2p.test works on the arcsine scale through Cohen's h, which stabilises the variance. stats::power.prop.test works on the raw proportion scale with p(1−p) variance. They agree closely near 0.5 and drift apart towards the ends, exactly where the raw-scale assumption is weakest: detecting 0.05 versus 0.10 at 80% power needs 424 per group by the arcsine route and 435 by the normal route. Neither is wrong. When they disagree, take the larger.
Why does the same lift need more users at 50% than at 5%?
Because a proportion is noisiest in the middle. The binomial variance p(1−p) peaks at 0.5 and falls away towards either end. A five point lift from 0.50 to 0.55 needs 1,565 per group at 80% power. The same five point lift from 0.05 to 0.10 needs 424, and from 0.90 to 0.95 also 424. The absolute difference is identical; the noise you are trying to see it through is not.
Should I use a one-tailed or two-tailed test?
Use two-tailed unless you have a strong, pre-registered reason to look in only one direction and would genuinely treat a move the other way as no effect at all. One-tailed needs a smaller sample for the same power, which makes it tempting, but choosing the direction after seeing the data inflates your false positive rate. In an A/B test the variant can always lose, and you would want to know.
What if my two groups will not be the same size?
Set the allocation ratio: 2 means the second group is twice the first. The calculator switches to the pwr.2p2n.test formula, whose noncentrality is h√(n₁n₂/(n₁+n₂)). Unequal allocation always costs more total subjects for the same power, so a balanced design is the most efficient use of a fixed total. Ratios still help when one arm is cheaper or more plentiful, or when you want most users on the safer variant.
Is n per group or the total?
For a two-proportion test this page reports n per group, matching pwr.2p.test, and shows the total next to it. For a one-proportion test n is simply the number of observations in your single sample, compared against a fixed reference value.
My calculated n is fractional. Do I round up or down?
Always up. pwr returns the exact point where power equals your target, for example 8142.19. Rounding down to 8142 leaves you a hair below the power you asked for; rounding up puts you just above it. This page shows both: the exact root, and the power the rounded-up integer actually buys.