Likelihood Ratio Tests in R

Jan 20

Introduction

One of the questions that arises when conducting regression is whether to attempt to reduce models so that they include fewer predictors. In this blog, you’ll learn how the likelihood ratio test in R can help you compare full and reduced regression models.

Example of Likelihood Ratio Test

First, install and load the following packages, if you have not already done so:

install.packages("lmtest")
install.packages("dplyr")
library(lmtest)
library(dplyr)

Next, let’s look at the mtcars dataset:

glimpse(mtcars)

Let’s regress mpg on several of the other variables in the dataset as follows and save the model as follows:

full.model <- lm(mpg ~ wt + hp + cyl + disp + qsec, data = mtcars)
summary(full.model)

The results of this model appear below:

Now let’s create a reduced model as follows:

reduced.model <- lm(mpg ~ wt + cyl, data = mtcars)
summary(reduced.model)

Here are the results of the reduced model:

Now let’s compare these models using the likelihood ratio test in R. Try entering the following code:

lrtest(full.model, reduced.model)

Because the p value of the second model, p = 0.261, is > .05, the models are equal in explanatory power. Had the p value of model 2 been < .05, then model 1, the full model, would have been preferable.

Model Selection and Table Generation

In examining the full and reduced models more closely, you can surmise why they’re equal in explanatory power. The adjusted R2 of the full model was .8214. The adjusted R2 of the reduced model was .8185. Note, however, that cyl, which is a significant predictor in the reduced model, is no longer significant when hp, disp, and qsec are included. Presenting the reduced model on its own could therefore be misleading. Both of these models could be presented alongside each other with some discussion of the changing significance of cyl.

We can use Stargazer to generate a nice table at this point. Try installing Stargazer (if you have not already) and loading it:

install.packages("stargazer")
library(stargazer)

You could try this code to create Latex code:

stargazer(full.model, reduced.model, title="Results", align=TRUE)

Or the following code to create the table in the RStudio environment:

stargazer(full.model, reduced.model, title="Results", type = "text", align=TRUE)