Step 1 of 5
Variables and Normality
Statistical selection depends on data scales and distribution shapes.
Research 101 Modules
Statistical Analysis Catalogue
Choosing the right test is critical to draw valid conclusions. Learn the parameters of normality, independence, and variable scales that determine statistical matching.
Module complete. It is checked off in your course progress.
Statistical Test Selector
- Goal
- Sample
- Data type
- Groups
- Test
Goal
What is your primary research goal?
The nature of your research question determines the class of statistical test you need. This is the first and most important branch in any test selection framework (Bahr & Zacks, 2010).
Data scale
What is the scale of your variables?
For correlation, the scale of measurement and whether normality assumptions are met determine whether to use Pearson's r (parametric) or Spearman's rho (non-parametric). Pearson's r exclusively tests linear relationships between continuous variables; Spearman's rho tests monotonic relationships and is appropriate for ordinal data (Bahr & Zacks, 2010).
Outcome type
What type of variable is your outcome?
The type of outcome (dependent) variable determines which regression model is appropriate. Linear regression requires a continuous outcome with normally distributed residuals; logistic regression is used when the outcome is binary (Field, 2018).
Sample relationship
Are your groups independent or are the same participants measured more than once?
Whether participants appear in one group only (independent) or are measured at multiple time points or matched with another participant (paired/repeated) is critical: these structures require different tests, and using the wrong one inflates or deflates the p-value (Field, 2018).
Data type
What is the type and distribution of your outcome variable?
Parametric tests (t-test, ANOVA) assume approximately normal distributions and are more powerful when assumptions hold. Non-parametric alternatives (Mann-Whitney, Kruskal-Wallis) make no normality assumption and should be used for ordinal data or when normality cannot be assumed, especially with small samples (Dwivedi et al., 2021).
Data type
What is the type and distribution of your outcome variable?
As with independent designs, the choice between parametric (paired t-test, repeated measures ANOVA) and non-parametric tests (Wilcoxon, Friedman) depends on whether the assumptions of normality are met. For paired designs, the normality of the differences between pairs (not raw scores) is the key assumption (Field, 2018).
Groups
How many groups are you comparing?
The number of groups determines whether a t-test (two groups) or ANOVA (three or more groups) is required. Running multiple t-tests instead of ANOVA inflates the Type I error rate (the probability of a false positive), which is why ANOVA exists (Field, 2018).
Groups
How many groups are you comparing?
The non-parametric equivalents follow the same group-count logic as parametric tests: Mann-Whitney U for two groups, Kruskal-Wallis for three or more (Bahr & Zacks, 2010).
Time points
How many time points or conditions are there?
For paired/repeated designs, two time points call for a paired t-test, while three or more require repeated measures ANOVA. This mirrors the independent group logic but within the same participants (Field, 2018).
Time points
How many time points or conditions are there?
For non-parametric paired designs: the Wilcoxon signed-rank test is used for two time points, and the Friedman test is used for three or more — the non-parametric counterparts of the paired t-test and repeated measures ANOVA respectively (Bahr & Zacks, 2010).
Sample size
Are any expected cell frequencies below 5?
The chi-square test relies on a large-sample approximation and becomes unreliable when expected cell frequencies fall below 5. In those cases, Fisher's Exact Test provides an exact p-value and should be used instead (Bahr & Zacks, 2010).
Correlation
Pearson Correlation
Also known as: Pearson's r
Measures the strength and direction of a linear relationship between two continuous variables. Produces a coefficient (r) ranging from -1 (perfect negative) to +1 (perfect positive), with 0 indicating no linear relationship.
Data type
Continuous (interval/ratio)
Sample relationship
Independent
Normality required
Yes
Key assumptions
- Both variables are continuous (interval or ratio scale)
- Both variables are approximately normally distributed
- The relationship between variables is linear
- No significant outliers in either variable
Strengths
- Widely used and well understood
- Produces an intuitive effect size (r)
- Sensitive to the true strength of linear relationships when assumptions are met
Limitations
- Detects linear relationships only — misses non-linear patterns
- Sensitive to outliers, which can inflate or deflate r
- Assumes normality — may be inappropriate for small or skewed samples
Example in practice
A researcher measures hours of sleep and exam scores in 80 students and uses Pearson's r to test whether more sleep is linearly associated with higher scores.
Sources: Bahr, S. & Zacks, J. (2010). Choosing Statistical Tests. Deutsches Arzteblatt International, 107(19), 343–348. PMC2881615. | Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics (5th ed.). SAGE.
Correlation
Spearman Rank Correlation
Also known as: Spearman's rho (ρ)
A non-parametric measure of the monotonic relationship between two variables. Works by ranking the data and computing correlation on ranks. Suitable when data are ordinal or when the normality assumption is violated.
Data type
Ordinal or non-normal continuous
Sample relationship
Independent
Normality required
No
Key assumptions
- Variables are at least ordinal (ranked)
- The relationship is monotonic (consistently increasing or decreasing, not necessarily linear)
- No assumption of normality required
Strengths
- Robust to outliers compared to Pearson's r
- Applicable to ordinal data (e.g. Likert scales)
- Does not require normally distributed data
- Detects any monotonic relationship, not just linear ones
Limitations
- Less statistically powerful than Pearson's r when normality assumptions are met
- Discards information by converting to ranks
- Cannot detect non-monotonic relationships
Example in practice
A researcher asks 60 students to rate their research confidence on a 1-5 Likert scale and records their number of publications. Spearman's rho is used because confidence ratings are ordinal.
Sources: Bahr & Zacks (2010). PMC2881615. | Dwivedi, A.K. et al. (2021). Comparing statistical tests. Journal of Family Medicine and Primary Care. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE.
Regression
Simple Linear Regression
Also known as: Ordinary Least Squares (OLS) regression
Models the relationship between one continuous predictor (independent variable) and one continuous outcome (dependent variable). Estimates how much the outcome changes per unit increase in the predictor, and allows prediction.
Data type
Continuous outcome
Sample relationship
Independent
Normality required
Yes (residuals)
Key assumptions
- Linear relationship between predictor and outcome
- Residuals (errors) are normally distributed
- Residuals have constant variance (homoscedasticity)
- Observations are independent of each other
Strengths
- Allows prediction of outcome values
- Produces interpretable coefficients (slope, intercept)
- Quantifies effect size and direction
- Can be extended to multiple predictors (multiple regression)
Limitations
- Assumes linearity — non-linear relationships require transformation or different models
- Sensitive to outliers and influential data points
- Requires normally distributed residuals, not always achievable
- Overfitting is a risk with small samples
Example in practice
A researcher uses simple linear regression to predict a student's research output score from the number of hours per week spent in lab, producing a slope estimate and R² value.
Sources: Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics (5th ed.). SAGE. | Bahr & Zacks (2010). Deutsches Arzteblatt International. PMC2881615.
Regression
Logistic Regression
Also known as: Binary logistic regression
Models the probability of a binary outcome (yes/no, success/failure) as a function of one or more predictor variables. Produces odds ratios indicating how much each predictor changes the odds of the outcome.
Data type
Binary categorical outcome
Sample relationship
Independent
Normality required
No
Key assumptions
- Outcome variable is binary (two categories)
- Observations are independent
- Little to no multicollinearity among predictors
- Large enough sample size — typically ≥10 events per predictor variable
Strengths
- Appropriate for binary outcomes that are common in health and social research
- Produces odds ratios, which are clinically interpretable
- Can handle multiple predictors simultaneously
- Does not require normally distributed predictors
Limitations
- Requires large sample sizes for stable estimates
- Susceptible to complete separation (perfect prediction)
- Odds ratios are often misinterpreted as risk ratios
- Cannot be used directly when the outcome is not binary
Example in practice
A researcher uses logistic regression to model whether a student completes their research project (yes/no) based on number of mentorship hours, year of study, and prior research experience.
Sources: Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Hosmer, D.W. & Lemeshow, S. (2000). Applied Logistic Regression (2nd ed.). Wiley.
Categorical comparison
Chi-Square Test
Also known as: Pearson’s χ² test of independence
Tests whether two categorical variables are statistically independent (i.e. whether the distribution of one variable differs across categories of another). Compares observed frequencies in a contingency table to expected frequencies under independence.
Data type
Categorical (nominal)
Sample relationship
Independent
Normality required
No
Key assumptions
- Both variables are categorical (nominal or ordinal)
- Observations are independent
- Expected frequency in each cell ≥5 (use Fisher’s Exact Test if this is violated)
- Random sampling from the population
Strengths
- Simple and widely applicable to categorical data
- Non-parametric — no normality assumption
- Can be used with larger contingency tables (not just 2x2)
- Well-established, easily interpreted
Limitations
- Unreliable when expected cell counts are below 5
- Does not measure strength of association — only tests independence
- Sensitive to large sample sizes (may flag trivial differences as significant)
- Cannot be used for paired/matched data (use McNemar’s test instead)
Example in practice
A researcher tests whether the proportion of students who complete a research project differs between those who received mentoring and those who did not, using a 2×2 contingency table.
Sources: Bahr & Zacks (2010). PMC2881615. | Dwivedi, A.K. et al. (2021). Journal of Family Medicine and Primary Care. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE.
Categorical comparison
Fisher’s Exact Test
Alternative to chi-square for small samples
An exact test of independence for 2×2 contingency tables. Calculates the exact probability of observing a table as extreme as the one obtained, given fixed marginal totals. Preferred over chi-square when expected cell counts are below 5.
Data type
Categorical (nominal), small samples
Sample relationship
Independent
Normality required
No
Key assumptions
- Both variables are categorical
- 2×2 contingency table (can be extended with software for larger tables)
- Observations are independent
- At least one expected cell frequency is below 5
Strengths
- Exact p-value — does not rely on chi-square approximation
- Valid for small sample sizes where chi-square fails
- More conservative and statistically rigorous for small samples
Limitations
- Computationally intensive for large samples (chi-square preferred then)
- Traditionally limited to 2×2 tables
- Assumes fixed marginal totals, which may not always hold
Example in practice
A researcher has only 18 participants and tests whether those who attended a workshop differ from those who did not in whether they submitted a manuscript. Expected cells are below 5, so Fisher’s Exact Test is used.
Sources: Bahr & Zacks (2010). PMC2881615. | Mehta, C.R. & Patel, N.R. (1983). A network algorithm for performing Fisher's exact test. Journal of the American Statistical Association, 78, 427–434.
Group comparison
Independent Samples t-Test
Also known as: Student’s t-test, unpaired t-test
Compares the means of a continuous outcome variable between two independent (unrelated) groups to determine whether the difference is statistically significant.
Data type
Continuous
Sample relationship
Independent
Normality required
Yes
Key assumptions
- Outcome variable is continuous
- The two groups are independent of each other
- Outcome is approximately normally distributed in each group
- Variances in the two groups are roughly equal (homogeneity of variance; check with Levene’s test)
Strengths
- Simple, well-understood, and widely taught
- Appropriate for the most common two-group comparison
- Robust to mild violations of normality with larger samples (n ≥30 per group)
Limitations
- Assumes normality — use Mann-Whitney U for non-normal data
- Assumes equal variances (Welch’s t-test is preferred when variances differ)
- Only compares two groups — use ANOVA for three or more
- Sensitive to outliers
Example in practice
A researcher compares mean research anxiety scores between students in a mentorship programme and those not in one, using an independent samples t-test with Levene’s test for equality of variances.
Sources: Bahr & Zacks (2010). PMC2881615. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Dwivedi, A.K. et al. (2021). Journal of Family Medicine and Primary Care.
Group comparison
Paired Samples t-Test
Also known as: Dependent t-test, repeated measures t-test
Compares the means of a continuous variable measured twice on the same participants (e.g. pre- and post-intervention), or on matched pairs. Tests whether the mean difference between paired measurements is significantly different from zero.
Data type
Continuous
Sample relationship
Paired/repeated
Normality required
Yes
Key assumptions
- Outcome variable is continuous
- Measurements are paired (same participant measured twice, or matched pairs)
- The differences between paired observations are approximately normally distributed
Strengths
- More statistically powerful than independent t-test because it controls for between-subject variability
- Appropriate for pre-post intervention designs
- Requires fewer participants than independent group designs for the same power
Limitations
- Only applicable when data are truly paired
- Assumes normality of the differences — use Wilcoxon signed-rank test if violated
- Cannot be used if pairing is broken by missing data
Example in practice
A researcher measures students’ self-reported research confidence before and after a 10-week mentorship programme, then applies a paired t-test to determine whether the intervention produced a significant change.
Sources: Bahr & Zacks (2010). PMC2881615. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE.
Group comparison
Mann-Whitney U Test
Also known as: Wilcoxon rank-sum test
A non-parametric test that compares the distributions (medians) of a continuous or ordinal variable between two independent groups by ranking all observations and comparing rank sums. Used when the independent t-test assumptions are not met.
Data type
Ordinal or non-normal continuous
Sample relationship
Independent
Normality required
No
Key assumptions
- Outcome variable is continuous or ordinal
- The two groups are independent
- Observations within each group are independent
- The distributions of the two groups have a similar shape (for median comparison interpretation)
Strengths
- No normality assumption required
- Robust to outliers
- Appropriate for ordinal data (e.g. Likert scales)
- Valid for small sample sizes
Limitations
- Less statistically powerful than the t-test when normality holds
- Tests distributional differences, not strictly mean differences
- Cannot be extended straightforwardly to multiple groups (use Kruskal-Wallis)
Example in practice
A researcher compares student satisfaction scores (1–10 scale) between two lab environments. Because scores are unlikely to be normally distributed, the Mann-Whitney U test is used instead of a t-test.
Sources: Bahr & Zacks (2010). PMC2881615. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Dwivedi, A.K. et al. (2021). Journal of Family Medicine and Primary Care.
Group comparison
Wilcoxon Signed-Rank Test
Non-parametric alternative to the paired t-test
A non-parametric test that compares two related samples or repeated measurements. Ranks the absolute differences between paired observations and tests whether the distribution of positive and negative ranks is symmetric around zero.
Data type
Ordinal or non-normal continuous
Sample relationship
Paired/repeated
Normality required
No
Key assumptions
- Measurements are paired (same participant, two time points, or matched pairs)
- The outcome is continuous or ordinal
- The differences between pairs are approximately symmetrically distributed (not necessarily normal)
Strengths
- No normality assumption for the differences
- Appropriate for pre-post designs with non-normal or ordinal data
- Robust to outliers in the difference scores
Limitations
- Less powerful than the paired t-test when normality holds
- Requires symmetry of the differences (weaker assumption than normality, but still an assumption)
- Only for two time points or paired conditions
Example in practice
A researcher measures lab skill ratings (ordinal scale) before and after training. Because the differences are unlikely to be normally distributed with a small sample, the Wilcoxon signed-rank test is used.
Sources: Bahr & Zacks (2010). PMC2881615. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE.
Group comparison
One-Way ANOVA
Also known as: Analysis of Variance
Compares the means of a continuous outcome variable across three or more independent groups simultaneously. Tests whether at least one group mean differs significantly from the others. Post-hoc tests (e.g. Tukey’s HSD) are needed to identify which specific groups differ.
Data type
Continuous
Sample relationship
Independent
Normality required
Yes
Key assumptions
- Outcome variable is continuous
- Groups are independent
- Outcome is approximately normally distributed within each group
- Variances are approximately equal across groups (homogeneity of variance)
Strengths
- Controls Type I error rate better than running multiple t-tests
- Can compare three or more groups in a single test
- Post-hoc tests allow specific pairwise comparisons
- Extensible to factorial designs (two-way ANOVA)
Limitations
- Only tells you that at least one group differs — post-hoc tests are required
- Assumes normality and equal variances
- Sensitive to outliers
- With very unequal group sizes, assumptions become more important
Example in practice
A researcher compares mean publication counts across three groups of students: those with no mentor, a near-peer mentor, or a faculty mentor. One-way ANOVA tests whether group means differ, followed by Tukey’s HSD.
Sources: Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Bahr & Zacks (2010). PMC2881615.
Group comparison
Kruskal-Wallis Test
Non-parametric alternative to one-way ANOVA
A non-parametric test that compares the distributions of a continuous or ordinal variable across three or more independent groups using ranked data. The non-parametric counterpart of one-way ANOVA.
Data type
Ordinal or non-normal continuous
Sample relationship
Independent
Normality required
No
Key assumptions
- Outcome variable is continuous or ordinal
- Groups are independent
- Observations within groups are independent
- Groups have similar distributional shapes (for median interpretation)
Strengths
- No normality assumption
- Appropriate for ordinal data or skewed continuous data
- Valid for small or unequal group sizes
- Extends Mann-Whitney logic to three or more groups
Limitations
- Less powerful than ANOVA when normality holds
- Post-hoc pairwise comparisons (e.g. Dunn’s test) are less straightforward
- Tests distributional differences, not strictly mean differences
Example in practice
A researcher compares research anxiety scores across four different lab environments. Scores are skewed and sample sizes are small, so the Kruskal-Wallis test is used, followed by Dunn’s post-hoc test.
Sources: Bahr & Zacks (2010). PMC2881615. | Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Dwivedi, A.K. et al. (2021). Journal of Family Medicine and Primary Care.
Group comparison
Repeated Measures ANOVA
Also known as: Within-subjects ANOVA
Compares the means of a continuous outcome measured at three or more time points (or conditions) on the same participants. Accounts for the correlation between repeated measurements on the same individual.
Data type
Continuous
Sample relationship
Paired/repeated
Normality required
Yes
Key assumptions
- Outcome is continuous
- Same participants measured at each time point
- Normality within each measurement occasion
- Sphericity — equal variances of differences between all pairs of time points (test with Mauchly’s test; apply Greenhouse-Geisser correction if violated)
Strengths
- Statistically powerful because each participant serves as their own control
- Controls for individual differences
- Appropriate for longitudinal or multi-condition experimental designs
Limitations
- Assumes sphericity — violated frequently in practice
- Requires complete data across all time points (missing data is problematic)
- More complex to interpret than one-way ANOVA
Example in practice
A researcher measures student research self-efficacy at baseline, 4 weeks, and 8 weeks during a training programme. Repeated measures ANOVA tests whether scores change significantly across the three time points.
Sources: Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Girden, E.R. (1992). ANOVA: Repeated Measures. SAGE.
Group comparison
Friedman Test
Non-parametric alternative to repeated measures ANOVA
A non-parametric test for comparing distributions of an ordinal or continuous variable across three or more related conditions or time points. The non-parametric counterpart of repeated measures ANOVA.
Data type
Ordinal or non-normal continuous
Sample relationship
Paired/repeated
Normality required
No
Key assumptions
- Same participants measured at three or more time points or conditions
- Outcome is at least ordinal
- Observations across conditions are related (same participants)
Strengths
- No normality assumption required
- Appropriate for ordinal outcomes (e.g. ranked preferences, Likert scales)
- Robust to outliers
Limitations
- Less powerful than repeated measures ANOVA when normality holds
- Post-hoc pairwise comparisons require additional tests (e.g. Wilcoxon with Bonferroni correction)
- Does not account for magnitude of differences, only ranks
Example in practice
A researcher asks 20 students to rate three different data analysis software packages on a 1–5 scale. The Friedman test compares whether median ratings differ significantly across the three packages.
Sources: Field, A. (2018). Discovering Statistics (5th ed.). SAGE. | Bahr & Zacks (2010). PMC2881615.