Different Types of Statistical Tests#
Meet Maya, a data scientist who studies patterns in real-world data, from student performance to customer behavior. As she analyzes data, she realizes that different research questions require different statistical tools.
Sometimes she compares averages.
Sometimes she studies relationships between variables.
Sometimes she examines patterns in categories.
Because research questions vary, statistical tests are designed for different purposes.
Broadly, statistical tests fall into two main categories:
Parametric tests
Non-parametric tests
Understanding this distinction helps researchers choose the right method for their data.
Parametric vs Non-Parametric Tests#
Parametric Tests#
Parametric tests make assumptions about the data’s distribution; most commonly that it follows a normal distribution. They typically require:
numerical (continuous) data
approximate normality
independent observations
sufficient sample size
When these assumptions are satisfied, parametric tests are powerful and precise. They are most often used to compare means.
Common parametric test statistics:
z → comparing means with known population variability
t → comparing means with estimated variability
F → comparing multiple group means (ANOVA)
Non-Parametric Tests#
Non-parametric tests make fewer assumptions about the data. They are useful when:
data are not normally distributed
sample sizes are small
variables are categorical or ordinal
parametric assumptions are violated
Instead of comparing means, they often compare medians, ranks, or frequencies.
Tests for Comparing Means#
After learning how hypothesis testing works, Maya — our data scientist — faces a practical question:
Which statistical test should I use for my data?
The answer depends on what kind of averages she wants to compare and what assumptions the data satisfy.
Many real-world questions involve comparing numerical averages; test scores, income, blood pressure, user engagement, or sales. When the data are continuous and follow certain assumptions, we use parametric tests.
Parametric Tests for Means#
Parametric tests compare numerical averages under specific assumptions.
They generally require:
data that are approximately normally distributed
independent observations
a sufficient sample size
When these conditions hold, parametric tests are powerful and precise tools for detecting real differences.
Let’s follow Maya as she chooses the right test for different situations.
(1) Z-Test#
Maya rarely uses this test, but it is important conceptually.
A z-test compares a sample mean to a known population mean when:
the population standard deviation is known
the sample size is large
Example:
Is the average height of a sample different from the known national average?
In practice, population variability is rarely known, so z-tests are less common.
(2) T-Test#
Most of Maya’s work relies on t-tests, because population variability is usually unknown.
Different research questions require different types of t-tests.
One-Sample T-Test#
Compare one group’s mean to a known value.
Independent Two-Sample T-Test#
Compare means of two separate groups.
Paired Sample T-Test#
Compare two measurements from the same individuals.
T-tests compare means when the population standard deviation is unknown, which is almost always the case.
Different research questions require different types of t-tests.
(3) ANOVA (Analysis of Variance)#
Eventually Maya studies more than two groups. She tests three different teaching methods and wants to compare all their average scores. Running many t-tests would increase the chance of false conclusions. Instead, she uses ANOVA, which tests whether at least one group mean differs from the others.
ANOVA tests whether at least one group mean differs from the others using an F-statistic, which compares variability between groups to variability within groups.
However, ANOVA does not identify which specific groups differ; only that a difference exists somewhere.
Post-Hoc Analysis#
If ANOVA finds a significant difference, Maya performs post-hoc tests to determine which specific groups differ.
She performs post-hoc tests, which compare group pairs while controlling for increased error from multiple comparisons. These tests reveal exactly where the differences occur.
Some common types of Post Hoc Test: -Tukey’s Honest Significant Difference (HSD) -Bonferroni Correction -Duncan’s New Multiple Range Test
All of these tests identify which group(s) differ significantly from others
Non-Parametric Test: When Parametric Assumptions Do Not Hold#
Sometimes Maya’s data do not meet parametric assumptions.
Maybe the data are not normally distributed.
Maybe the sample is small.
Maybe the variables are categorical instead of numerical.
In these cases, she uses non-parametric tests that rely on ranks or frequencies instead of raw numerical values.
Rank-Based Non-Parametric Tests#
When comparing groups without normality assumptions, Maya uses rank-based methods.
Mann–Whitney U test → alternative to independent t-test
Wilcoxon signed-rank test → alternative to paired t-test
Kruskal–Wallis test → alternative to ANOVA
Spearman rank correlation → association between ranked variables
These tests compare ordered data rather than numerical means.
Chi-Square Test (Categorical Data)#
The chi-square test analyzes categorical data by comparing observed counts to expected counts. It tests whether distributions differ from what we would expect by chance.
Example:
Is transportation choice associated with income level?
Hypotheses:
H₀: variables are independent
H₁: variables are related
The chi-square statistic measures how far observed frequencies deviate from expected frequencies.
Choosing the Right Test#
To select a statistical test, Maya asks:
Is the data numerical or categorical?
How many groups are being compared?
Are observations independent or paired?
Are parametric assumptions satisfied?
Are we comparing means, ranks, or frequencies?