Question 1

How do I choose the right statistical test?

Accepted Answer

Four things determine it: how many groups or variables you are comparing, whether your outcome is continuous or categorical, whether the observations are independent or repeated on the same people, and whether the distributional assumptions hold. Answer those and the test follows almost automatically.

Question 2

What test compares two groups?

Accepted Answer

For a continuous outcome measured on two separate groups, use an independent samples t test, or the Mann-Whitney U test if the data are heavily skewed or ordinal. If the same people were measured twice, use the paired samples t test instead.

Question 3

What test do I use for three or more groups?

Accepted Answer

A one-way ANOVA, when the outcome is continuous and the groups are independent. If the overall test is significant, follow it with post hoc comparisons to identify which pairs of groups differ. For repeated measurements on the same people, use a repeated measures ANOVA.

Question 4

When should I use a nonparametric test?

Accepted Answer

When the outcome is ordinal rather than truly continuous, when the sample is small and clearly non-normal, or when outliers dominate the result. Nonparametric tests give up some power in exchange for weaker assumptions. With large samples, parametric tests are usually robust enough that the switch is unnecessary.

Which Statistical Test Should I Use?

How the guide decides