For graduate students engaged in thesis writing or researchers examining binary outcomes, logistic regression is a statistical tool of choice. It’s particularly useful when the dependent variable is dichotomous, such as passing versus not passing a test. While the significance of p values in denoting statistical significance is well-known, the importance of effect sizes, specifically odds ratios in logistic regression, cannot be overstated. This blog post discusses effect sizes through practical logistic regression examples using R, highlighting scenarios prevalent in graduate theses, advanced undergraduate work, and beyond.
Imagine you’re studying the effect of different study environments on the likelihood of ADHD-diagnosed students passing a test. The environments are divided into three groups: listening to synthwave music, classical music, and no music (silence) before a test. You’re interested in whether these environments significantly influence the odds of passing the test. Logistic regression comes into play here, allowing you to model the probability of passing versus not passing based on study environment. Let’s simulate this scenario and analyze the data in R:
In the output of a logistic regression in R, when you have a categorical predictor variable with multiple levels, one level is automatically chosen as the reference category. The coefficients (and thus the odds ratios) for the other levels are reported relative to this reference category.
In this case, the environment variable has three levels: Classical, Silence, and Synthwave. By default, R selects the first level alphabetically as the reference level unless specified otherwise. Therefore, Classical has been chosen as the reference group because it comes first alphabetically.
The coefficients reported in the output:
- (Intercept) represents the log odds of passing for the reference group (Classical), when all other predictors are held at zero (which, for categorical variables, means being in the reference category).
- environmentSilence is the difference in log odds between the Silence group and the Classical group.
- environmentSynthwave is the difference in log odds between the Synthwave group and the Classical group.
Because Classical is the reference category, it does not appear separately in the output; its effect is captured in the intercept. This setup allows you to understand how the likelihood of passing the test differs for students in the Silence and Synthwave groups compared to those in the Classical group.
A negative coefficient for environmentSilence indicates lower odds of passing the test for students in the Silence group compared to the Classical group, as suggested by the negative log odds difference. A positive coefficient for environmentSynthwave suggests higher odds of passing for students who listened to Synthwave music compared to those in the Classical group. This approach helps quantify the impact of being in either the Silence or Synthwave group relative to the baseline (Classical) on the likelihood of passing the test.
As you can see from the output above, your model is significant, but what does it mean practically? Start by noting that the OR for synthwave is well over 1, and the OR for silence is below 1. So you know that synthwave students did significantly better than classic students, while silence students did significantly worse than classical students. How about effect size interpretations, though?
- An OR > 1 suggests that the condition or treatment increases the odds of the outcome happening. For example, given that the OR for students listening to synthwave versus classical is 3.73, it means those students had 3.73 times the odds of passing the test.
- An OR < 1 indicates a decrease in the odds of the outcome with the predictor. It signifies a protective factor or a negative association, depending on the context. Here, the OR for silence means that those students were about half as likely as classical students to pass.
- An OR = 1 implies no effect of the predictor on the outcome odds.
And here is how you would interpret those values as effect sizes:
- Small effect: While there's no strict cutoff like Cohen’s d, an OR close to 1 (e.g., 1.2) might be considered a small effect.
- Medium effect: ORs around 2 could be seen as having a moderate effect.
- Large effect: ORs of 3 or higher are typically indicative of a strong effect, which is definitely true for synthwave.
BridgeText can help you with all of your statistics needs.
