Understanding Heteroskedasticity in OLS Regression

Feb 26

One key assumption underpinning Ordinary Least Squares (OLS) regression is that of homoscedasticity—this term refers to the assumption that the variance of the error terms (residuals) is constant across all levels of the independent variables. However, in real-world data, this assumption is often violated, giving rise to what we term heteroskedasticity.

Heteroskedasticity occurs when the size or variance of the error terms differs across the range of values of an independent variable. This variance inequality can lead to inefficiency in estimates and biased inference, making standard errors unreliable and leading to incorrect conclusions about the significance of variables.

Why Heteroskedasticity Matters
The presence of heteroskedasticity in a regression model affects the reliability of some key outputs of regression analysis, particularly:

Standard Errors: The estimated standard errors of the regression coefficients become biased, which directly impacts the confidence intervals and hypothesis tests, potentially leading to misleading inferences about the importance or insignificance of predictors.
Efficiency: OLS estimators remain unbiased in the presence of heteroskedasticity, but they lose their BLUE (Best Linear Unbiased Estimator) property, meaning they are no longer the most efficient estimators. This inefficiency can lead to a higher likelihood of making Type I or Type II errors.
Model Specification: Heteroskedasticity can sometimes be a symptom of a misspecified model—for example, missing variables, incorrect functional form, or the need for transformation of variables.

Identifying and Diagnosing Heteroskedasticity
A common initial approach to identifying heteroskedasticity is through visual inspection of the residuals. Plotting the residuals against fitted values or one of the independent variables can reveal patterns that suggest variance changes with the level of the predictor. For a more formal diagnosis, several tests exist, including the Breusch-Pagan and White tests. These tests compare the observed level of heteroskedasticity against what would be expected under homoscedasticity, providing a p-value to help decide if heteroskedasticity is statistically significant.

Addressing Heteroskedasticity
Once identified, there are several approaches to mitigate the effects of heteroskedasticity.

Robust Standard Errors (RSE) regression adjusts the standard errors of the regression coefficients to account for heteroskedasticity without altering the estimated coefficients themselves. This method allows for valid hypothesis testing even in the presence of heteroskedastic errors. Next, applying transformations (e.g., logarithmic, square root) to the dependent variable or to the model itself can sometimes equalize error variances across the range of values. Finally, Weighted Least Squares (WLS) involves weighting observations differently based on the variance of their errors, aiming to give less weight to observations with higher variance.

Complete R Example

Let's create, visualize, and analyze heteroskedasticity in an OLS regression in R:

# Check and install ggplot2 if not installed
if (!requireNamespace("ggplot2", quietly = TRUE)) {
install.packages("ggplot2")
}

# Check and install lmtest if not installed
if (!requireNamespace("lmtest", quietly = TRUE)) {
install.packages("lmtest")
}

# Check and install sandwich if not installed
if (!requireNamespace("sandwich", quietly = TRUE)) {
install.packages("sandwich")
}

# Load necessary packages
library(ggplot2)
library(lmtest) # For Breusch-Pagan test
library(sandwich) # For robust standard errors

# Set seed for reproducibility
set.seed(123)

# Generate independent variable
x <- 1:100

# Generate error term with increasing variance
errors <- rnorm(100, mean = 0, sd = 0.1*x)

# Generate dependent variable
y <- 2 + 3*x + errors

# Plot data to visualize heteroskedasticity
ggplot(data = data.frame(x, y), aes(x = x, y = y)) +
geom_point() +
geom_smooth(method = lm, se = FALSE) +
theme_minimal() +
ggtitle("Heteroskedasticity Illustration")

# Perform OLS regression
model <- lm(y ~ x)

# Summary of model to see coefficients
summary(model)

# Plot residuals to visually check for heteroskedasticity
plot(model$residuals ~ model$fitted.values,
main = "Residuals vs Fitted Values",
xlab = "Fitted Values",
ylab = "Residuals")
abline(h = 0, col = "red")

# Conduct the Breusch-Pagan test to statistically test for heteroskedasticity
bp_test_result <- bptest(model)
print(bp_test_result)

# Use robust standard errors to address heteroskedasticity
coeftest(model, vcov = vcovHC(model, type = "HC1"))

Since the p value of the Breusch-Pagan test is below .05, heteroskedasticity exists in this regression:

You can see the characteristic heteroskedastic fanning pattern in the residuals-vs-fitted plot:

However, you should report the results of the Breusch-Pagan test as your proof of heteroskedasticity. Now let's run an RSE regression instead:

# Load the sandwich package for robust standard errors
library(sandwich)

# Adjust standard errors
coeftest(model, vcov = vcovHC(model, type = "HC1"))

The RSE is significant:

These are the results to report instead of the OLS results.

Conclusion
Understanding, diagnosing, and addressing heteroskedasticity is crucial for conducting reliable regression analysis. By ensuring the assumptions behind OLS regression are met or appropriately dealt with, researchers can maintain the integrity of their findings and make accurate inferences. This guide has provided a foundational understanding of heteroskedasticity, with practical steps to identify and correct it in your analyses, ensuring your research stands on solid statistical ground.

BridgeText can help you with all of your statistics needs.

Have any questions?

Our support team is ready to answer your questions.

Help Center FAQ