Are we comparing apples or squared apples? The proportion of explained variance exaggerates differences between effects
OpenPsych , June 15, 2021, ISSN: 2597-324X
This brief note addresses a known problem whose implications are still not widely appreciated: using the proportion of explained variance as an index of effect size does not just distort the real-world magnitude of individual effects, but also exaggerates the differences between effects, which may lead to strikingly incorrect judgements of relative importance. Luckily, a meaningful and interpretable “effect ratio” can be easily calculated as the square root of the ratio between proportions of explained variance. In a variety of practical examples, effect ratios tell a different story than variance components, and suggest a different perspective on certain canonical results (e.g., regarding the role of the shared environment in the development of psychological traits). This simple but highly consequential point should be understood more widely, to help researchers avoid fallacious interpretations of empirical findings.