PreprintPDF Available

Understanding, choosing, and unifying multilevel and fixed effect approaches

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

When estimating the effect of a covariate in grouped data, investigators may choose "fixed effects" (FE) approaches with specialized standard errors (e.g., cluster-robust standard errors, CRSE), or multilevel models employing "random effects" (MLM/RE). From over 100 articles in political science, sociology, and education, we find that MLM/RE is often chosen on grounds of greater efficiency and predictive accuracy, simultaneously allowing the inclusion of group-level variables and varying intercepts or slopes, and claims that it ensures correct standard errors for grouped data. Yet, other traditions have favored the FE approach because, unlike MLM/RE, it fully accounts for group-level confounding. We first dissect how these models compare in three analytical pieces: (i) random effects are simply "regularized" fixed effects; (ii) bias is often problematic in MLM/RE, but has a longstanding solution; and (iii) MLM standard errors rely upon narrow assumptions, which can be relaxed. These lessons are known, but we find they are neglected in practice. We then describe how to address the bias and standard error issues within MLM. Fortunately, the resulting approach produces coefficients and standard error estimates exactly equal to the analogous FE estimates with CRSEs, resolving debate about which framework is appropriate.
Content may be subject to copyright.
A preview of the PDF is not available

Supplementary resource (1)

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Although basing instruction on learning trajectories (LTs) is often recommended, there is little direct evidence regarding the premise of a LT approach—that instruction should be presented (only) one LT level beyond a child’s present level. We evaluated this hypothesis in the domain of early shape composition. One group of preschoolers, who were at least two levels below the target instructional LT level, received instruction based on an empirically validated LT. The counterfactual (skip-levels) group received an equal amount of instruction focused only on the target level. At posttest, children in the LT condition exhibited significantly greater learning than children in the skip-levels condition, mainly on near-transfer items; no child-level variables were significant moderators. Implications for theory and practice are discussed.
Article
Full-text available
Clustered data are common in many fields. Some prominent examples of clustering are employees clustered within supervisors, students within classrooms, and clients within therapists. Many methods exist that explicitly consider the dependency introduced by a clustered data structure, but the multitude of available options has resulted in rigid disciplinary preferences. For example, those working in the psychological, organizational behavior, medical, and educational fields generally prefer mixed-effects models, whereas those working in economics, behavioral finance, and strategic management generally prefer fixed-effects models. However, increasingly interdisciplinary research has caused lines that separate the fields grounded in psychology and those grounded in economics to blur, leading to researchers encountering unfamiliar statistical methods commonly found in other disciplines. Persistent discipline-specific preferences can be particularly problematic because (a) each approach has certain limitations that can restrict the types of research questions that can be appropriately addressed and (b) analyses based on the statistical modeling decisions common in one discipline can be difficult to understand for researchers trained in alternative disciplines. This can impede cross-disciplinary collaboration and limit the ability of scientists to make appropriate use of research from adjacent fields. This article discusses the differences between mixed-effects and fixed-effects models for clustered data, reviews each approach, and helps to identify when each approach is optimal. We then discuss the Within-Between specification which blends advantageous properties of each framework into a single model.
Article
Full-text available
Altruism is an important omitted variable in much of the Political Economy literature. While material self-interest is the base of most approaches to redistribution (first affecting preferences and then politics and policy), there is a paucity of research on inequality aversion. I propose that other-regarding concerns influence redistribution preferences and that: (1) they matter most to those in less material need and (2) they are conditional on the identity of the poor. Altruism is most relevant to the rich, and it is most influential when the recipients of benefits are similar to those financing them. Using data from the European Social Survey from 2002 to 2012, I will show that group homogeneity magnifies (or limits) the importance of altruism for the rich. In making these distinctions between the poor and the rich, the arguments in this paper challenge some influential approaches to inequality, immigration and voting. Short title for running header: Food Comes First, Then Morals
Article
Many researchers use unit fixed effects regression models as their default methods for causal inference with longitudinal data. We show that the ability of these models to adjust for unobserved time‐invariant confounders comes at the expense of dynamic causal relationships, which are permitted under an alternative selection‐on‐observables approach. Using the nonparametric directed acyclic graph, we highlight two key causal identification assumptions of unit fixed effects models: Past treatments do not directly influence current outcome, and past outcomes do not affect current treatment. Furthermore, we introduce a new nonparametric matching framework that elucidates how various unit fixed effects models implicitly compare treated and control observations to draw causal inference. By establishing the equivalence between matching and weighted unit fixed effects estimators, this framework enables a diverse set of identification strategies to adjust for unobservables in the absence of dynamic causal relationships between treatment and outcome variables. We illustrate the proposed methodology through its application to the estimation of GATT membership effects on dyadic trade volume.
Book
Statistical Rethinking: A Bayesian Course with Examples in R and Stan builds readers’ knowledge of and confidence in statistical modeling. Reflecting the need for even minor programming in today’s model-based statistics, the book pushes readers to perform step-by-step calculations that are usually automated. This unique computational approach ensures that readers understand enough of the details to make reasonable choices and interpretations in their own modeling work. The text presents generalized linear multilevel models from a Bayesian perspective, relying on a simple logical interpretation of Bayesian probability and maximum entropy. It covers from the basics of regression to multilevel models. The author also discusses measurement error, missing data, and Gaussian process models for spatial and network autocorrelation. By using complete R code examples throughout, this book provides a practical foundation for performing statistical inference. Designed for both PhD students and seasoned professionals in the natural and social sciences, it prepares them for more advanced or specialized statistical modeling. Web Resource The book is accompanied by an R package (rethinking) that is available on the author’s website and GitHub. The two core functions (map and map2stan) of this package allow a variety of statistical models to be constructed from standard model formulas.
Article
Labor market competition theories explaining anti‐immigrant attitudes have received limited or no empirical validation in recent literature. This has led researchers to highlight education and cultural values as the main, if not the sole, drivers of attitudes toward immigration. We present a new labor market competition theory focusing on job availability rather than foreign labor supply. This theory predicts that individuals with low transferable skills in the labor market will articulate a subjective sense of job insecurity and higher hostility toward migrants. Our cross‐classified, longitudinal, and difference‐in‐differences models reveal that skill specificity is a strong driver of anti‐immigrant attitudes, and they suggest that economic competition theories cannot be dismissed. By shifting the attention from supply to demand in the labor market, and from actual to potential competition with migrants, we show that the highly educated are far from immune to anti‐immigrant attitudes.
Article
Our secondary analysis of Measures of Effective Teaching data contributes to growing evidence that observation ratings, used as part of comprehensive teacher evaluation systems across the nation, may measure factors outside of a teacher’s performance or control. Specifically, men and teachers in classrooms with high concentrations of Black, Hispanic, male, and low-performing students receive significantly lower observation ratings. By using various methodological approaches and a subsample of teachers randomly assigned to classrooms, we demonstrate that these differences are unlikely due to actual differences in teacher quality. These results suggest that policymakers consider the unintended consequences of using observational ratings to evaluate teachers and consider ways to adjust ratings to ensure they are fair.
Article
What determines whether countries' institutions attract or deter investment? Although existing theories predict that multinational enterprises (MNEs) avoid locations where institutions cannot constrain public and private actors' opportunistic behavior, we argue host institutions' attractiveness depends on firms' home environment. Home country institutions shape firms' practices and capabilities, thus helping to determine the environments that firms are best prepared to face abroad. We test our predictions using multiple data sets at different levels of analysis: firm-level data on MNEs' foreign subsidiaries, data on bilateral foreign direct investment (FDI) positions, and longitudinal data on bilateral FDI flows. We find that states with independent judiciaries are particularly attractive to investment from countries also possessing independent courts. Similarly, countries with low judicial independence disproportionately send FDI to countries lacking independent judiciaries. These findings' implications challenge conventional wisdom: “Good” institutions may not attract all investors, and “bad” institutions may not always deter, as current research suggests.