Choose the right statistical test
Answer a few questions about your question and your data. The chooser suggests a test, the assumptions to check, what to report, and what to use if the assumptions fail.
Test chooser
Quick reference table
Common choices for an outcome measured in one or two groups. "Parametric" means the outcome is roughly normal or the groups are large.
| Question | Continuous, parametric | Ordinal or non-normal | Categorical |
|---|---|---|---|
| Compare 2 independent groups | Independent t-test | Mann-Whitney U | Chi-square or Fisher's exact |
| Compare 2 paired measurements | Paired t-test | Wilcoxon signed-rank | McNemar's test |
| Compare 3+ independent groups | One-way ANOVA | Kruskal-Wallis | Chi-square |
| Compare 3+ repeated measurements | Repeated-measures ANOVA | Friedman | Cochran's Q |
| Relationship between two variables | Pearson correlation | Spearman correlation | Chi-square, Cramér's V |
| One group versus a known value | One-sample t-test | One-sample Wilcoxon | Binomial or proportion test |
Checking normality
Many tests assume the outcome, or the model residuals, are roughly normal. Look at the data before you choose.
- Plots first: a histogram and a Q-Q plot show skew and outliers better than any single number.
- Shapiro-Wilk test: a small p-value (often below 0.05) suggests non-normality. In large samples it flags trivial departures, and in small samples it has little power, so use it together with plots.
- Sample size helps: with roughly 30 or more per group, t-tests and ANOVA are fairly robust to moderate non-normality. Strong skew, outliers or ordinal data still call for rank-based tests.
- For paired tests, check the differences, not each measurement separately.
- For regression, check the residuals.
Before you run the test
Decide the test in advanceChoose your primary test in the protocol. Trying several tests and reporting the one that gives p < 0.05 inflates false positives.
Report effect sizesA p-value shows whether an effect is detectable, not whether it matters. Give an effect size and a 95% confidence interval.
Adjust for multiple comparisonsAfter an ANOVA or Kruskal-Wallis, use post hoc tests with adjusted p-values (Tukey, Bonferroni, Holm or Dunn).
Check independenceObservations from the same person, ward or family are not independent. Use paired, repeated-measures or mixed models.