Statistical Test Chooser
Not sure which statistical test fits your study? Answer a few plain-English questions about your goal and your data, and this chooser walks the standard decision tree to name the right test. For each recommendation you get the assumptions to check, a distribution-free alternative, runnable R code, and a link straight to our matching calculator. Everything runs in your browser.
-
✓ No data leaves your browser ✓ Every R snippet run-verified in R 4.6.0 ✓ Free, no sign-up
How this recommendation is made
The full decision tree
Every path the chooser can take, grouped by goal. Each row links to the tutorial for that test. This is the same logic the wizard above runs.
Compare groups or conditions
You have a variable measured in different groups (or the same subjects more than once) and want to know whether they differ.
- A continuous outcome, one group vs a known value, roughly normal One-sample t-test
- A continuous outcome, one group vs a value, skewed Wilcoxon signed-rank test
- Two independent groups, normal, equal spread Two-sample t-test (pooled)
- Two independent groups, normal, unequal spread Welch’s t-test
- Two independent groups, skewed Mann-Whitney U test
- Two paired measurements, normal differences Paired t-test
- Two paired measurements, skewed differences Wilcoxon signed-rank test
- Paired ordinal data (only the direction is meaningful) Sign test
- Three or more independent groups, normal, equal spread One-way ANOVA
- Three or more independent groups, normal, unequal spread Welch’s ANOVA
- Three or more independent groups, skewed Kruskal-Wallis test
- Three or more repeated measures, normal Repeated-measures ANOVA
- Three or more repeated measures, skewed or ordinal Friedman test
- A single yes/no proportion vs a value, small sample Exact binomial test
- A single yes/no proportion vs a value, large sample One-sample z-test for a proportion
- Two independent yes/no rates, large sample Two-sample z-test for proportions
- Two independent yes/no rates, small or sparse Fisher’s exact test
- A paired yes/no outcome (before vs after) McNemar’s test
- A yes/no outcome across 3+ repeated conditions Cochran’s Q test
- One categorical variable vs an expected distribution Chi-square goodness-of-fit test
- Two categorical variables in a table, large sample Chi-square test of independence
- A count outcome compared across groups Poisson regression
Measure a relationship
You have two variables and want to know whether, and how strongly, they move together.
- Two continuous variables, linear and roughly normal Pearson correlation
- Two continuous variables, monotonic but not normal Spearman correlation
- Two variables, small sample with ties Kendall’s tau
- Two ordinal (ranked) variables Kendall’s tau
- Two categorical variables, large sample Chi-square test of independence
- Two categorical variables, small 2x2 table Fisher’s exact test
Predict or model an outcome
You want to model an outcome from one or more predictors.
- A continuous outcome from one predictor Simple linear regression
- A continuous outcome from several predictors Multiple linear regression
- A yes/no outcome Logistic regression
- A count outcome Poisson regression
- An ordered categorical outcome Ordinal logistic regression
- An unordered categorical outcome (3+ classes) Multinomial logistic regression
Time-to-event (survival)
Your outcome is the time until an event happens, with some observations censored.
- Compare survival curves between groups Log-rank test
- Model the effect of covariates on survival Cox proportional-hazards regression
Rater agreement / reliability
You want to quantify how well raters or methods agree.
- Two raters, categorical labels Cohen’s kappa
- Three or more raters, categorical labels Cohen’s kappa
- Continuous ratings from two or more raters Intraclass correlation (ICC)
Describe your data
You want to summarise a single variable, not run a test.
- A continuous variable Summary (descriptive) statistics
- A categorical variable Frequency table and proportions
Test coverage matrix
The 33 tests this chooser can recommend, with our matching calculator and tutorial for each. A few advanced tests have a tutorial only for now.
| Test | Use it when | Our calculator | Tutorial |
|---|---|---|---|
| One-sample t-test | Compares one group’s mean against a known or claimed value. | Open the t-test calculator | t-tests in R |
| Two-sample t-test (pooled) | Compares the means of two independent groups when their spreads are similar. | Open the t-test calculator | t-tests in R |
| Welch’s t-test | Compares two independent group means without assuming equal variances (R’s default). | Open the t-test calculator | t-tests in R |
| Paired t-test | Compares two measurements taken on the same subjects (before vs after). | Open the t-test calculator | t-tests in R |
| Mann-Whitney U test | Compares two independent groups by ranks when normality is doubtful. | Open the nonparametric test calculator | Mann-Whitney U in R |
| Wilcoxon signed-rank test | Rank-based test for paired data (or one sample vs a value) when data are skewed. | Open the nonparametric test calculator | Wilcoxon signed-rank in R |
| Sign test | Tests whether paired differences tend to be positive or negative; needs only the direction. | Open the nonparametric test calculator | Sign test in R |
| Kruskal-Wallis test | Compares 3+ independent groups by ranks (the nonparametric one-way ANOVA). | Open the nonparametric test calculator | Kruskal-Wallis in R |
| Friedman test | Compares 3+ repeated measures by ranks (the nonparametric repeated-measures ANOVA). | Open the nonparametric test calculator | Friedman test in R |
| One-way ANOVA | Compares the means of 3+ independent groups in a single test. | Open the ANOVA output interpreter | One-way ANOVA in R |
| Welch’s ANOVA | Compares 3+ group means when the variances are unequal. | Open the ANOVA output interpreter | Welch’s ANOVA in R |
| Repeated-measures ANOVA | Compares 3+ conditions measured on the same subjects. | Open the ANOVA output interpreter | Repeated-measures ANOVA in R |
| Chi-square goodness-of-fit test | Checks whether one categorical variable’s frequencies match an expected distribution. | Open the chi-square calculator | Goodness-of-fit in R |
| Chi-square test of independence | Tests whether two categorical variables are associated. | Open the chi-square calculator | Independence test in R |
| Fisher’s exact test | Exact test of association for small 2x2 tables (when expected counts fall below 5). | Open the chi-square calculator | Fisher’s exact test in R |
| McNemar’s test | Tests a change in a paired binary outcome (same subjects, before vs after). | Open the chi-square calculator | McNemar’s test in R |
| Cochran’s Q test | Extends McNemar to 3+ repeated binary measures on the same subjects. | none yet | McNemar’s test (Q generalises it) |
| One-sample z-test for a proportion | Tests whether one proportion differs from a hypothesised value, with a large sample. | none yet | One-sample proportion z-test |
| Two-sample z-test for proportions | Compares a yes/no rate between two independent groups. | Open the A/B (two-proportion) calculator | Proportion tests in R |
| Exact binomial test | Exact test for one proportion; the right choice when the sample is small. | none yet | Exact binomial test in R |
| Pearson correlation | Measures the strength of a straight-line relationship between two continuous variables. | none yet | Pearson correlation in R |
| Spearman correlation | Rank-based correlation for monotonic relationships that need not be linear. | none yet | Spearman & Kendall in R |
| Kendall’s tau | Rank correlation that stays reliable with small samples and many ties. | none yet | Spearman & Kendall in R |
| Simple linear regression | Models a continuous outcome from a single predictor. | Open the linear-model interpreter | Simple linear regression in R |
| Multiple linear regression | Models a continuous outcome from two or more predictors at once. | Open the linear-model interpreter | Multiple regression in R |
| Logistic regression | Models a binary (yes/no) outcome from one or more predictors. | Open the GLM output interpreter | Logistic regression in R |
| Poisson regression | Models a count outcome (events per unit) from predictors. | Open the GLM output interpreter | Poisson regression in R |
| Ordinal logistic regression | Models an ordered categorical outcome (the proportional-odds model). | none yet | Ordinal logistic regression in R |
| Multinomial logistic regression | Models an unordered categorical outcome with 3+ categories. | none yet | Multinomial logistic regression in R |
| Log-rank test | Compares survival curves between two or more groups. | Open the survival calculator | Kaplan-Meier & log-rank |
| Cox proportional-hazards regression | Models time-to-event from covariates without assuming a survival distribution. | Open the survival calculator | Cox proportional hazards |
| Cohen’s kappa | Measures agreement between two raters on categorical labels, correcting for chance. | Open the confusion-matrix interpreter | none yet |
| Intraclass correlation (ICC) | Measures how consistently raters agree on continuous scores. | none yet | none yet |
Assumptions worth checking first
Whatever the chooser recommends, three questions decide whether the test is valid. Check these before you trust the p-value.
| Question | Why it matters | What to do |
|---|---|---|
| Are observations independent? | Almost every test assumes it; clustering or repeated measures break it. | Use a paired or mixed model when the same subjects appear more than once. |
| Is the data roughly normal? | Decides parametric versus rank-based for continuous outcomes. | Check with the normality test picker; the t-test and ANOVA are robust once n is large. |
| Are the group spreads similar? | Unequal variances bias the pooled t-test and classic ANOVA. | Prefer Welch's t-test or Welch's ANOVA, which do not assume equal variances. |
Frequently asked questions
How do I choose the right statistical test?
Start from your goal: comparing groups, measuring a relationship, predicting an outcome, describing data, analysing time to an event, or measuring rater agreement. Then answer follow-up questions about your outcome type, the number of groups, whether the groups are paired, and whether continuous data look normal. The chooser walks that decision tree and names the matching test, its assumptions and a runnable example.
What is the difference between a parametric and a nonparametric test?
Parametric tests such as the t-test and ANOVA assume the data follow a distribution, usually normal. Nonparametric tests such as Mann-Whitney and Kruskal-Wallis work on ranks and make weaker assumptions, so they are safer for skewed or ordinal data at a small cost in statistical power. When normality is doubtful and the sample is small, the rank-based test is the safe pick.
When should I use Welch's t-test instead of Student's?
Use Welch's t-test whenever the two groups may have different variances, which is most real situations. It is R's default and stays accurate when spreads differ, while the pooled Student's t-test can mislead. Choose the pooled version only when you have a good reason to believe the variances are equal.
What test should I use for a yes/no outcome?
For a single proportion against a target, use an exact binomial test with a small sample or a one-sample z-test with a large one. For two independent groups, use a chi-square test of independence or a two-proportion z-test, or Fisher's exact test when the counts are small. For a paired yes/no outcome measured before and after, use McNemar's test, and Cochran's Q for three or more repeated conditions.
Does the R code on each card actually run?
Yes. Every R snippet is self-contained and was run in R 4.6.0 before publishing, so you can paste it straight into R or RStudio and reproduce the example output. The chooser itself does no computation on your numbers; it points you to the correct test and the calculator that runs it.