Home / Tools / Multiple Testing Correction

P-Value Adjustment Calculator

Run many tests at once and some look significant by pure luck. This tool adjusts your p-values to keep that under control. Paste a vector of raw p-values, choose a method (Bonferroni, Holm, Hochberg, Hommel, BH or BY), and see how many results survive, with the working shown. Everything runs in your browser.

Try:

Significance α

When to use this method

Use when

Inputs

REJECTED AFTER CORRECTION

- / -

The same correction in R

✓ No data leaves your browser ✓ Verified against R's p.adjust() to machine precision ✓ Free, no sign-up

Every p-value, adjusted

Each raw p-value and what it becomes under all six corrections. Cells that survive at the current α are marked; the active method's column is highlighted.

Why correct at all?

Run one test and you accept a 5% chance of a false alarm. Run twenty and you would expect about one false positive by luck alone; run a thousand and you would expect roughly fifty. Multiple testing correction tightens the bar for "significant" so the chance of being fooled stays under control no matter how big the family of tests is.

There are two error rates to choose between, and picking one is the whole game:

Family-wise error rate

The probability of even one false positive across the entire family. Strict. Controlled by Bonferroni, Holm, Hochberg and Hommel.

any false positive is costly

False discovery rate

The expected fraction of false positives among the results you call significant. More powerful for large screens. Controlled by BH and BY.

a few false hits are tolerable

Pick before you peek

Choose the method first, then run the data through it. Choosing the method that maximises survivors after seeing the results is p-hacking.

decide up front

Which correction do I need?

Six methods from R's p.adjust, in order of increasing power within each family.

Method	Controls	Assumption	Use when
Bonferroni	FWER	none	A few pre-registered confirmatory tests where any false positive is unacceptable. Simplest and strictest.
Holm	FWER	none	Anywhere you would use Bonferroni: same guarantee, uniformly more power. There is rarely a reason to prefer Bonferroni.
Hochberg	FWER	independence / positive dependence	Endpoints from a well-behaved model. Step-up, at least as powerful as Holm when its assumption holds.
Hommel	FWER	independence / positive dependence	The most powerful FWER method (a closed test on Simes). Dominates Hochberg; heavier to compute.
BH	FDR	weak dependence	Genomics, brain imaging, large A/B grids. The default when many tests run and a few percent of false hits is fine.
BY	FDR	any dependence	FDR control when tests are correlated (pathway genes, voxels, time points). Safer than BH, a little less powerful.

Adjusted p-values match R's p.adjust() to machine precision (worst difference under 1e-15) across all six methods, cross-checked over 29 input vectors including ties, zeros, single-test families and family-size overrides.

Anatomy of each correction

Bonferroni: adj_p_(i) = min(1, m · p_(i)) reject if adj_p ≤ α

Bonferroni. Multiply every p-value by the family size m, cap at 1. Controls the family-wise error rate strictly. The simplest correction; also the most conservative, so it leaves real findings on the table when m is large.

Holm (step-down): sort p-values p_(1) ≤ … ≤ p_(m) adj_p_(i) = max over j≤i of (m − j + 1) · p_(j) capped at 1, monotone

Holm. Walk up the sorted p-values, scaling each by a smaller factor than Bonferroni. Same FWER guarantee, uniformly more powerful: it rejects every test Bonferroni rejects, and sometimes more.

Hochberg (step-up): sort p-values p_(1) ≤ … ≤ p_(m) adj_p_(i) = min over j≥i of (m − j + 1) · p_(j) capped at 1, monotone

Hochberg. Same critical constants as Holm, but applied step-up (from the largest p down). Uniformly at least as powerful as Holm, at the price of one assumption: the tests must be independent or positively dependent.

Hommel: closed test over every intersection hypothesis, each judged by Simes' rule; adj_p = sharpest Simes bound (see R source)

Hommel. A full closed-testing procedure built on Simes' test. The most powerful of the four FWER methods: it rejects everything Hochberg does and occasionally more, under the same independence / positive-dependence assumption. The computation is heavier, which is why it is used less than it deserves.

BH (Benjamini–Hochberg, FDR): adj_p_(i) = min over j≥i of (m / j) · p_(j) reject if adj_p ≤ α

BH. The default for genomics, brain imaging and screens. Controls the expected false discovery rate (the fraction of significant calls that are false). Far more powerful than FWER methods when m is large and many true alternatives exist.

BY (Benjamini–Yekutieli): c(m) = Σ 1/k for k = 1..m adj_p_(i) = min over j≥i of (m · c(m) / j) · p_(j)

BY. The robust cousin of BH. The harmonic factor c(m) bakes in worst-case dependence between tests, useful when statistics are correlated (genes in the same pathway, voxels in the same region). Slightly less powerful than BH; valid under arbitrary dependence.

When this is the wrong tool

If you have…: Use instead
One primary endpoint and many secondary tests: A fixed-sequence or hierarchical procedure (Maurer–Bretz). A blanket correction over the whole family wastes power.
Hierarchical or structured tests (genomic regions, brain ROIs): Group-FDR or hierarchical FDR (Yekutieli, Heller): better power for structured families than flat BH.
Highly dependent tests (linkage, time series, repeated measures): BY (FDR under dependence) or a permutation-based correction. BH assumes weak dependence.
Sequential or interim looks at a trial: Group-sequential or alpha-spending designs (O'Brien–Fleming, Pocock). Single-look correction is the wrong model.
Storey q-values with bootstrap π̂₀ smoothing: This tool ships the six p.adjust methods. For full smoothing or bootstrap π₀, use qvalue::qvalue() in R.
You are picking the method after seeing the corrected p-values: Stop. Pick the method first; the data go through it. The other order is p-hacking.

Frequently asked questions

When should I use Bonferroni versus Benjamini-Hochberg?

Bonferroni controls the family-wise error rate (FWER), the probability of ANY false positive. Use it when even one false positive is costly (clinical trials, regulatory submissions). BH controls the false discovery rate (FDR), the expected proportion of false positives among your hits. Use it for genomics, A/B test screens and other discovery contexts where some false positives are acceptable.

What is the difference between Holm and Bonferroni correction?

Holm is uniformly more powerful than Bonferroni and controls the same family-wise error rate. It applies the Bonferroni cutoff only to the smallest p-value, then a slightly less strict cutoff to the next, and so on. There is essentially no reason to use Bonferroni instead of Holm when controlling FWER.

How does FDR control work?

Benjamini-Hochberg (BH) sorts p-values smallest to largest and finds the largest k where p_k is at or below k times alpha over m. All tests with p-values at or below that threshold are rejected. The expected proportion of false discoveries among the rejections is at most alpha. With m = 1000 tests and alpha = 0.05, you expect about 5% of "discoveries" to be false.

Why do Hochberg and Hommel need an assumption Holm does not?

Holm is a step-down procedure valid under any dependence between tests. Hochberg and Hommel are step-up procedures: they can reject more, but that extra power relies on the tests being independent or positively dependent. When that assumption is plausible, Hommel dominates Hochberg, which dominates Holm.

What does the family size m do?

By default m is the number of p-values you paste. If you ran a larger family of tests but only kept a shortlist, set m to the true family size. It feeds R's n argument, so a larger m genuinely tightens the adjustment, not just the "out of m" denominator.

Go deeper

Significance testing in R Hypothesis testing, explained P-value calculator Confidence interval calculator All calculators