In the crucible of thesis writing, even the most diligent graduate students might overlook the significance of reporting effect sizes alongside statistical significance in their analyses. While it's universally acknowledged that p values are indispensable for confirming the statistical significance of your results beyond descriptive statistics, the role of effect size, particularly in the context of ANOVA (Analysis of Variance), warrants equal attention. In this blog, we look at ANOVA effect sizes using practical R examples and scenarios commonly encountered in graduate theses, advanced undergraduate projects, and PhD research.
Consider a scenario where you're exploring the impact of different study environments on the test performance of students diagnosed with ADHD. You categorize the environments into three groups: Listening to synthwave music, classical music, and no music (silence) before a test. To assess the effectiveness of these conditions, you conduct an ANOVA with 60 students, randomly assigned to the three groups, and measure their test performance on a scale of 0 to 100. Here’s how you can simulate this study and analyze the data in R:
# Load necessary library
library(ggplot2)
# Set seed for reproducibility
set.seed(123)
# Generate mock data for ANOVA
n_group <- 20 # Number of participants in each group
performance_synth <- rnorm(n_group, mean = 80, sd = 10) # Synthwave group
performance_classical <- rnorm(n_group, mean = 75, sd = 10) # Classical music group
performance_silence <- rnorm(n_group, mean = 70, sd = 10) # Silence group
data <- data.frame(
environment = factor(rep(c("Synthwave", "Classical", "Silence"), each = n_group)),
performance = c(performance_synth, performance_classical, performance_silence)
)
# Conduct ANOVA to compare group means
anova_result <- aov(performance ~ environment, data = data)
# Display ANOVA summary
anova_summary <- summary(anova_result)
print(anova_summary)
# Calculate Eta-squared (η²) for effect size
model_ss <- anova_summary[[1]]$"Sum Sq"[1] # Sum of squares for the model (effect)
residual_ss <- anova_summary[[1]]$"Sum Sq"[2] # Sum of squares for residuals (error)
total_ss <- model_ss + residual_ss # Total sum of squares
eta_squared <- model_ss / total_ss # Calculate Eta-squared
print(paste("Eta-squared (η²):", eta_squared))
# Visualize data
ggplot(data, aes(x = environment, y = performance, fill = environment)) +
geom_boxplot() + # Create boxplots for each group
labs(title = "Test Performance by Study Environment",
x = "Study Environment",
y = "Test Performance") +
theme_minimal() # Use a minimal theme for a clean look
Upon analyzing the data, suppose the ANOVA indicates a statistically significant difference among the study environments in influencing test performance.
Great, your model is significant, but what does it mean practically? If someone inquires about the real-world impact of your findings, how would you respond? It's entirely feasible for results to be statistically significant yet possess a negligible practical effect.
Enter effect size for ANOVA, commonly measured by Eta-squared (η²) or Partial Eta-squared (η²), offering a remedy to this predicament. Unlike Cohen’s d for t-tests, which measures the effect size between two means, η² in the context of ANOVA quantifies the proportion of the total variance in the dependent variable that is attributable to the factor (the independent variable).
Interpret η² as follows:
- Small effect: η² = 0.01.
- Medium effect: η² = 0.06.
- Large effect: η² = 0.14.
Here was the Eta-squared from your model:
Therefore, you can describe the effect of synthwave on performance as being large. Isn’t that a lot more informative than just pointing out that it was statistically significant?
BridgeText can help you with all of your statistics needs.
