Analysis of Variance (ANOVA)

Overview And Definition

Analysis of Variance (ANOVA) is a parametric statistical model utilized to compare the means of three or more independent groups.

It provides a methodology for partitioning the total variance computed from a data set into specific components.
Each component represents the amount of the total variance attributed to a specific source of variation.
The primary goal is to test hypotheses about population means by comparing the variance between groups against the variance within groups.
Null Hypothesis ( $H_{0}$ ): All population means are absolutely equal ( $H_{0} : μ_{1} = μ_{2} = \dots = μ_{g}$ ).
Alternative Hypothesis ( $H_{1}$ ): At least one population mean is significantly different from the others.

Three crucial assumptions must be satisfied prior to applying an ANOVA model.

Independence: The observed values must be strictly independent of each other.
Normality: The dependent variable must be normally distributed within each individual group.
- Normality can be investigated using histograms, Q-Q plots, or the Shapiro-Wilk test.
Homogeneity Of Variance: The population variance of each group must be nearly equal.
- This assumption is typically assessed using Bartlett’s test or Levene’s test.
- If variances are unequal, Welch's adjusted ANOVA test must be used.

The fundamental principle of ANOVA relies on disassembling the total variation into distinct parts.

Total Variation ( $S S T$ ): The overall variation among all individual values and the overall mean.
- It represents the total sum of squares.
- It is mathematically defined as $S S T = S S B + S S E$ .
Variation Between Groups ( $S S B$ ): The variation between the individual group means and the overall mean.
- It may be caused by the specific treatment effect or by random error.
Variation Within Groups ( $S S E$ or $S S W$ ): The variation of individual values from their specific group mean.
- It is caused purely by random error or individual differences.

Sums of squares are converted into Mean Squares ( $M S$ ) by dividing them by their respective degrees of freedom ( $d f$ ).
The test statistic generated is the F-statistic.
It is calculated as the ratio of the Mean Square Between groups ( $M S B$ ) to the Mean Square Within groups ( $M S E$ ).

F = \frac{M S B}{M S E}

If the null hypothesis is true, the F-statistic should be approximately 1.
A larger F-statistic indicates that the between-group variance is significantly larger than the within-group variance.

Source of Variation	Sum of Squares ( $S S$ )	Degrees of Freedom ( $D F$ )	Mean Square ( $M S$ )	F-statistic
Between Groups	$S S B$	$g - 1$	$M S B = \frac{S S B}{g - 1}$	$\frac{M S B}{M S E}$
Within Groups (Error)	$S S E$	$N - g$	$M S E = \frac{S S E}{N - g}$
Total	$S S T$	$N - 1$

(Note: $g$ is the number of groups, and $N$ is the total sample size)

The specific type of ANOVA applied depends entirely on the study design and the number of independent variables.

ANOVA Type	Description	Key Features	Non-Parametric Equivalent
One-Way ANOVA	Tests one primary factor with 3 or more independent levels.	Uses a completely randomized design.	Kruskal-Wallis test.
Two-Way ANOVA	Assesses the effect of two independent variables simultaneously.	Allows the assessment of potential interactions between factors.	None directly mapped; requires complex modeling.
Repeated Measures ANOVA	Tests paired or matched data with 3 or more measurements.	The same subjects are measured across multiple time points or conditions.	Friedman test.

An ANOVA test yields a single overarching p-value.

A significant F-value merely indicates that differences exist among the means.
It does not identify exactly which specific groups differ from each other.
To identify pairwise differences, researchers must perform post-hoc multiple comparison tests.
These tests strictly adjust the significance level to prevent the inflation of Type I errors.
Common Post-Hoc Tests:
- Bonferroni Correction: Divides the significance level ( $α$ ) by the number of tests performed.
- Tukey's HSD: Commonly used for all possible pairwise comparisons.
- Dunnett-t Test: Specifically compares multiple treatment groups against a single control group.
- SNK-q Test: Adjusts the significance level using a studentized range statistic (q).