Equivalence & Non-Inferiority Calculator
A normal significance test can only fail to find a difference; it can never show two things are the same. This tool does the opposite: pick a margin you consider negligible, then use two one-sided tests (TOST) to decide whether two means or proportions are equivalent, non-inferior or superior, or plan the sample size you would need. Everything runs in your browser.
-
How this is computed
✓ No data leaves your browser
✓ Verified against R and the TOSTER package to 7+ decimals
✓ Free, no sign-up
Equivalence, non-inferiority or superiority?
All three flip the usual null hypothesis. Instead of "is there a difference?", they ask "is the difference small enough (or large enough) to matter?" You choose the margin.
Two bounds. You show the difference is neither meaningfully worse nor meaningfully better, so the two options are interchangeable. Classic use: generic vs brand-name drug.
One lower margin. You show the new option is not meaningfully worse than the control, and it may be cheaper, safer or simpler. One-sided.
One upper margin. You show the new option beats the control by more than a margin you decide is worth it, not just by a trivial amount.
The method behind the number
Equivalence and non-inferiority both run two one-sided tests (TOST). Each one-sided test uses the ordinary two-sample statistic, so nothing exotic is going on. The only new idea is the band: a lower and upper bound you set in advance to mark the largest difference you would still call negligible.
| Decision | Rule used here |
|---|---|
| Equivalent | Both one-sided p-values below α. Equivalently, the (1−2α) confidence interval lies entirely inside the band. |
| Non-inferior | The lower one-sided p-value below α, so the lower confidence bound sits above the (negative) margin. |
| Superior | The upper one-sided p-value below α, so the lower confidence bound sits above the (positive) margin. |
| Confidence interval | A (1−2α) two-sided interval, because each of the two one-sided tests spends α on its own side. |
| Two means | Pooled (Schuirmann) or Welch two-sample t; degrees of freedom follow the choice. |
| Two proportions | Two one-sided z-tests on the risk difference with an unpooled (Wald) standard error. |
| Sample size | Normal approximation. Equivalence splits power across both one-sided tests (z at 1−β/2); non-inferiority and superiority use the one-sided z at 1−β. |
Superiority here is the mirror image of non-inferiority: the same one-sided "difference beats the margin" test, only the margin is positive. That is a deliberate fix. Treating it as the upper equivalence test (a common mistake) would reject when the difference is below the margin, the exact opposite of what superiority means.
Frequently asked questions
What is a TOST equivalence test?
TOST stands for two one-sided tests. You pick an equivalence band, a lower and an upper bound that you consider practically negligible, then run two one-sided tests: one that the difference is above the lower bound and one that it is below the upper bound. If both reject, the difference lies inside the band and the groups are declared equivalent.
How is non-inferiority different from equivalence?
Equivalence uses two bounds and asks whether the difference is neither meaningfully worse nor meaningfully better. Non-inferiority uses a single lower margin and asks only whether the new option is not meaningfully worse than the control. It is a one-sided version of the same test.
Why a 90% confidence interval for a 5% test?
Each of the two one-sided tests runs at α, so the matching confidence interval is the two-sided 1−2α interval. For α = 0.05 that is a 90% interval. Equivalence holds exactly when that interval falls entirely inside your bounds.
Should I use pooled or Welch for two means?
Pooled (equal-variance) is the classic Schuirmann TOST and is fine when the two group standard deviations are similar. Welch does not assume equal variances and is safer when they differ. Both are offered; the verdict updates instantly when you switch.
Does this match the TOSTER package in R?
Yes. Every analysis result and sample size is verified against R, and against TOSTER's tsum_TOST, TOSTtwo.prop and powerTOSTtwo functions to seven or more decimal places across all modes.
Why does the calculator show a 90% confidence interval for a 5% test?
Each of the two one-sided tests runs at alpha, so the equivalent confidence interval is the two-sided 1 minus 2 times alpha interval. For alpha = 0.05 that is a 90% interval. Equivalence holds exactly when that interval falls entirely inside the bounds.