Content uploaded by Chad Hazlett
Author content
All content in this area was uploaded by Chad Hazlett on Aug 28, 2018
Content may be subject to copyright.
A preview of the PDF is not available
Trajectory Balancing: A General Reweighting Approach to Causal Inference With Time-Series Cross-Sectional Data
Abstract and Figures
We introduce trajectory balancing, a general reweighting approach to causal inference withtime-series cross-sectional (TSCS) data. We focus on settings where one or more units is exposed to treatment at a given time, while a set of control units remain untreated. First, we show that many commonly used TSCS methods imply an assumption that each unit’s non-treatment potential outcomes in the post-treatment period are linear in that unit’s pre-treatment outcomes and its time-invariant covariates. Under this assumption, we introduce the mean balancing method that reweights control units such that the averages of the pre-treatment outcomes and covariates are approximately equal between the treatment and (reweighted) control groups. Second, we relax the linearity assumption and propose the kernel balancing to seek approximate balance on a kernel-based feature expansion of the pre-treatment outcomes and covariates. The resulting approach inherits the ability of synthetic control and latent factor models to tolerate time-varying unobserved confounders, but (1) improves feasibility and stability with reduced user discretion; (2) accommodates both short and long pre-treatment time periods with many or fewtreated units; and (3) balances on the high-order “trajectory” of pre-treatment outcomes rather than their period-wise average. We illustrate this method with simulations and two empirical examples.
(1) (PDF) Trajectory Balancing: A General Reweighting Approach to Causal Inference With Time-Series Cross-Sectional Data. Available from: https://www.researchgate.net/publication/327121784_Trajectory_Balancing_A_General_Reweighting_Approach_to_Causal_Inference_With_Time-Series_Cross-Sectional_Data [accessed Apr 12 2019].
Figures - uploaded by Chad Hazlett
Author content
All figure content in this area was uploaded by Chad Hazlett
Content may be subject to copyright.
... To estimate the effect of joining the euro for the Eastern European Area on the GDP per capita, we use the KB estimator proposed by Hazlett and Xu (2018). This is a general reweighing approach to the causal inference which builds upon the SCM, developed by Abadie and Gardeazabal (2003) and Abadie et al. (2010), enabling us to estimate the treatment effect in the presence of few treated units. ...
... We then restrict the donor pool of each treated region to regions having the same level of urbanization 2. The algorithm to assess weights: We use Mean Balancing (MB), a procedure developed by Hazlett and Xu (2018) that seeks balance on the first P principal components of the characteristics, where P is chosen automatically by a method that minimizes the worst-case bias, and the SCM (for more details see Abadie et al., 2010;Abadie et al., 2015). The findings shown in Appendix E.2 and E.3 largely support the main analysis, except for Eastern Slovenia, where the SCM showed a more negative effect. ...
What is the economic impact of joining a currency union? Is this impact heterogeneous across regions? And how does it change in case of a recession? We answer these questions by investigating the economic impact of joining the euro area for the latecomers, i.e., the eastern European countries that adopted the euro after 2002. Differently from previous literature, we use NUTS‐2 regions as units of analysis. This novelty allows us to investigate the theoretical predictions of a currency union impact at a more appropriate geographical level. Using a recently developed counterfactual approach, we estimate the overall as well as the disaggregated impact of joining the euro area. We find that the adoption of the euro brought about a small positive effect, which was, however, dampened by the Great Recession. Individual regional estimates suggest heterogeneous returns with benefits accruing mostly to core regions. This article is protected by copyright. All rights reserved.
... Specifically, conditional on hyperparameters, the MTGP estimate can be written as the solution to a constrained optimization problem that minimizes the worst-case mean-square error across the "noiseless" latent functions, f . This representation exploits the connection between Gaussian Processes and kernel ridge regression (Kanagawa et al., 2018); see Hazlett and Xu (2018) for additional discussion of kernel weighting methods for panel data. ...
Gun violence is a critical public safety concern in the United States. In 2006 California implemented a unique firearm monitoring program, the Armed and Prohibited Persons System (APPS), to address gun violence in the state by identifying those firearm owners who become prohibited from owning a firearm due to federal and state law and confiscating their firearms. Our goal is to assess the effect of APPS on California murder rates using annual, state-level crime data across the US for the years before and after the introduction of the program. To do so, we adapt a non-parametric Bayesian approach, multitask Gaussian Processes (MTGPs), to the panel data setting. MTGPs allow for flexible and parsimonious panel data models that nest many existing approaches and allow for direct control over both dependence across time and dependence across units, as well as natural uncertainty quantification. We extend this approach to incorporate non-Normal outcomes, auxiliary covariates, and multiple outcome series, which are all important in our application. We also show that this approach has attractive Frequentist properties, including a representation as a weighting estimator with separate weights over units and time periods. Applying this approach, we find that the increased monitoring and enforcement from the APPS program substantially decreased gun-related homicides in California.
This study examines an initiative by a large multinational garment retailer (H&M Group) to increase wages at its supplier factories by intervening in their wage‐related management practices. Difference‐in‐differences estimates based on eight years of data from over 1,800 factories show that the interventions were associated with an average wage increase of approximately 5 percent by the third year of implementation. Our estimates suggest that the intervention‐associated wage increase was many times greater than if the retailer's cost for the program was instead paid directly to affected workers. We find that the wage effects were driven by factories with relatively poorer supplier ratings and do not find significantly different wage effects depending on the presence of trade unions. We also examine several non‐wage outcomes such as factory orders, supplier price competitiveness, overtime pay, and total employment to probe the mechanisms underlying the wage increases. Overall, these findings have implications for assessing corporate social impact in global supply chains. This article is protected by copyright. All rights reserved
Are place-based policies capable of taking lagging areas to a higher growth trajectory permanently? We answer this crucial question by investigating what happens when strongly subsidized regions experience a substantial reduction in funding. By analysing an extensive database via the mean balancing approach, we estimate the causal impact of exiting the convergence region status of the European Union regional policy. We find that only regions that experienced a considerable reduction in funding during a recession suffered a negative impact on economic growth. However, the impact varies with the features of the regions and the local economic context.
Background:
Traditional epidemiologic approaches such as time-series or case-crossover designs are often used to estimate the effects of extreme weather events but can be limited by unmeasured confounding. Quasi-experimental methods are a family of methods that leverage natural experiments to adjust for unmeasured confounding indirectly. The recently developed generalized synthetic control method that exploits the timing of an exposure is well suited to estimate the impact of acute environmental events on health outcomes. To demonstrate how this method can be used to study extreme weather events, we examined the impact of the October 20th - 26th 2007 Southern California wildfire storm on respiratory hospitalizations.
Methods:
We used generalized synthetic control to compare the average number of ZIP code-level respiratory hospitalizations during the wildfire storm between ZIP codes that were classified as exposed versus unexposed to wildfire smoke. We considered wildfire exposure eligibility for each ZIP code using fire perimeters and satellite-based smoke plume data. We retrieved respiratory hospitalization discharge data from the Office of Statewide Health Planning and Development. R code to implement the generalized synthetic control method is included for reproducibility.
Results:
The analysis included 172 exposed and 578 unexposed ZIP codes. We estimated that the average effect of the wildfire storm among the exposed ZIP codes was an 18% (95% CI: 10%-29%) increase in respiratory hospitalizations.
Conclusions:
We illustrate the use of generalized synthetic control to leverage natural experiments to quantify the health impacts of extreme weather events when traditional approaches are unavailable or limited by assumptions.
This paper introduces a simple framework of counterfactual estimation for causal inference with time‐series cross‐sectional data, in which we estimate the average treatment effect on the treated by directly imputing counterfactual outcomes for treated observations. We discuss several novel estimators under this framework, including the fixed effects counterfactual estimator, interactive fixed effects counterfactual estimator and matrix completion estimator. They provide more reliable causal estimates than conventional two‐way fixed effects models when treatment effects are heterogeneous or unobserved time‐varying confounders exist. Moreover, we propose a new dynamic treatment effects plot, along with several diagnostic tests, to help researchers gauge the validity of the identifying assumptions. We illustrate these methods with two political economy examples and develop an open‐source package, fect, in both R and Stata to facilitate implementation.
Does exposure to a mass migration event cause citizens to vote against incumbents? I offer an answer to this question by studying one of the largest acute periods of migration in the US, the case of the 1980 Mariel Boatlift during which roughly 125,000 Cubans fled to South Florida. I estimate the change in support for Republican presidential candidates in Miami using the synthetic control method and fixed effects regressions with a panel of county-level and archival precinct-level election results. I find that, while Miami voters dramatically increased their support of the Republican candidate in 1980, this shift was not a local consequence of the Boatlift. Instead, the evidence suggests that Cuban support for Reagan was not a local Miami response to the Boatlift—it happened in Cuban communities throughout the US—but it was most noticeable in Miami because Miami had the largest Cuban population in the US even before the Boatlift. I also present evidence that this change in Cuban voting may have been specific to Reagan and not a broader shift against incumbents or toward Republicans. These findings suggest that, in this case, direct exposure to migration did not lead citizens to dramatically change their voting behavior.
Sexual crimes against women are severely underreported to the police, allowing for impunity of perpetrators. Observers suggest that a stimulus towards reporting the crime comes from nearby support services for victims of sexual offences - like refuges and advisors. Still, the evidence about the effects of nearby support on the reporting of sexual crimes remains scattered and mainly qualitative. This paper provides quantitative evidence on this effect, by exploiting the uneven geography of local support services which resulted in the UK after the introduction of the austerity program. Findings highlight a positive net effect of the provision of local support services on the victims’ propensity to report. The positive effect holds also in the aftermath of a space-neutral high-impact media campaign empowering women to report sexual violence. This evidence relates to relevant policy implications, given that in some countries the austerity-driven cuts to public budgets have reduced and dispersed the local availability of support services, making digital support and/or helpline the only available options in many places.
Difference-in-differences (DID) is commonly used for causal inference in time-series cross-sectional data. It requires the assumption that the average outcomes of treated and control units would have followed parallel paths in the absence of treatment. In this paper, we propose a method that not only relaxes this often-violated assumption, but also unifies the synthetic control method (Abadie, Diamond, and Hainmueller 2010) with linear fixed effects models under a simple framework, of which DID is a special case. It imputes counterfactuals for each treated unit using control group information based on a linear interactive fixed effects model that incorporates unit-specific intercepts interacted with time-varying coefficients. This method has several advantages. First, it allows the treatment to be correlated with unobserved unit and time heterogeneities under reasonable modeling assumptions. Second, it generalizes the synthetic control method to the case of multiple treated units and variable treatment periods, and improves efficiency and interpretability. Third, with a built-in cross-validation procedure, it avoids specification searches and thus is easy to implement. An empirical example of Election Day Registration and voter turnout in the United States is provided.
Propensity score methods are used to reduce the effects of observed confounding when using observational data to estimate the effects of treatments or exposures. A popular method of using the propensity score is inverse probability of treatment weighting (IPTW). When using this method, a weight is calculated for each subject that is equal to the inverse of the probability of receiving the treatment that was actually received. These weights are then incorporated into the analyses to minimize the effects of observed confounding. Previous research has found that these methods result in unbiased estimation when estimating the effect of treatment on survival outcomes. However, conventional methods of variance estimation were shown to result in biased estimates of standard error. In this study, we conducted an extensive set of Monte Carlo simulations to examine different methods of variance estimation when using a weighted Cox proportional hazards model to estimate the effect of treatment. We considered three variance estimation methods: (i) a naïve model-based variance estimator; (ii) a robust sandwich-type variance estimator; and (iii) a bootstrap variance estimator. We considered estimation of both the average treatment effect and the average treatment effect in the treated. We found that the use of a bootstrap estimator resulted in approximately correct estimates of standard errors and confidence intervals with the correct coverage rates. The other estimators resulted in biased estimates of standard errors and confidence intervals with incorrect coverage rates. Our simulations were informed by a case study examining the effect of statin prescribing on mortality. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.
An important problem in econometrics and marketing is to infer the causal
impact that a designed market intervention has exerted on an outcome metric
over time. This paper proposes to infer causal impact on the basis of a
diffusion-regression state-space model that predicts the counterfactual market
response in a synthetic control that would have occurred had no intervention
taken place. In contrast to classical difference-in-differences schemes,
state-space models make it possible to (i) infer the temporal evolution of
attributable impact, (ii) incorporate empirical priors on the parameters in a
fully Bayesian treatment, and (iii) flexibly accommodate multiple sources of
variation, including local trends, seasonality and the time-varying influence
of contemporaneous covariates. Using a Markov chain Monte Carlo algorithm for
posterior inference, we illustrate the statistical properties of our approach
on simulated data. We then demonstrate its practical utility by estimating the
causal effect of an online advertising campaign on search-related site visits.
We discuss the strengths and limitations of state-space models in enabling
causal attribution in those settings where a randomised experiment is
unavailable. The CausalImpact R package provides an implementation of our
approach.
We propose the use of Kernel Regularized Least Squares (KRLS) for social science modeling and inference problems. KRLS borrows
from machine learning methods designed to solve regression and classification problems without relying on linearity or additivity
assumptions. The method constructs a flexible hypothesis space that uses kernels as radial basis functions and finds the best-fitting
surface in this space by minimizing a complexity-penalized least squares problem. We argue that the method is well-suited
for social science inquiry because it avoids strong parametric assumptions, yet allows interpretation in ways analogous to
generalized linear models while also permitting more complex interpretation to examine nonlinearities, interactions, and heterogeneous
effects. We also extend the method in several directions to make it more effective for social inquiry, by (1) deriving estimators
for the pointwise marginal effects and their variances, (2) establishing unbiasedness, consistency, and asymptotic normality
of the KRLS estimator under fairly general conditions, (3) proposing a simple automated rule for choosing the kernel bandwidth,
and (4) providing companion software. We illustrate the use of the method through simulations and empirical examples.
Matching methods improve the validity of causal inference by reducing model dependence and offering intuitive diagnostics. Although they have become a part of the standard tool kit across disciplines, matching methods are rarely used when analysing time‐series cross‐sectional data. We fill this methodological gap. In the proposed approach, we first match each treated observation with control observations from other units in the same time period that have an identical treatment history up to the prespecified number of lags. We use standard matching and weighting methods to further refine this matched set so that the treated and matched control observations have similar covariate values. Assessing the quality of matches is done by examining covariate balance. Finally, we estimate both short‐term and long‐term average treatment effects using the difference‐in‐differences estimator, accounting for a time trend. We illustrate the proposed methodology through simulation and empirical studies. An open‐source software package is available for implementing the proposed methods.
We evaluate whether a lack of guidance on how to choose the matching variables used in the Synthetic Control (SC) estimator creates specification‐searching opportunities. We provide theoretical results showing that specification‐searching opportunities are asymptotically irrelevant if we restrict to a subset of SC specifications. However, based on Monte Carlo simulations and simulations with real datasets, we show significant room for specification searching when the number of pre‐treatment periods is in line with common SC applications, and when alternative specifications commonly used in SC applications are also considered. This suggests that such lack of guidance generates a substantial level of discretion in the choice of the comparison units in SC applications, undermining one of the advantages of the method. We provide recommendations to limit the possibilities for specification searching in the SC method. Finally, we analyze the possibilities for specification searching and provide our recommendations in a series of empirical applications.
The synthetic control method is an increasingly popular tool for analysis of program efficacy. Here, it is applied to a neighborhood-specific crime intervention in Roanoke, VA, and several novel contributions are made to the synthetic control toolkit. We examine high-dimensional data at a granular level (the treated area has several cases, a large number of untreated comparison cases, and multiple outcome measures). Calibration is used to develop weights that exactly match the synthetic control to the treated region across several outcomes and time periods. Further, we illustrate the importance of adjusting the estimated effect of treatment for the design effect implicit within the weights. A permutation procedure is proposed wherein countless placebo areas can be constructed, enabling estimation of p-values under a robust set of assumptions. An omnibus statistic is introduced that is used to jointly test for the presence of an intervention effect across multiple outcomes and post-intervention time periods. Analyses indicate that the Roanoke crime intervention did decrease crime levels, but the estimated effect of the intervention is not as statistically significant as it would have been had less rigorous approaches been used. Supplementary materials for this article are available online.
In the absence of unobserved confounders, matching and weighting methods are widely used to estimate causal quantities including the Average Treatment Effect on the Treated (ATT). Unfortunately, these methods do not necessarily achieve their goal of making the multivariate distribution of covariates for the control group identical to that of the treated, leaving some (potentially multivariate) functions of the covariates with different means between the two groups. When these "imbalanced" functions influence the non-treatment potential outcome, the conditioning on observed covariates fails, and ATT estimates may be biased. Kernel balancing, introduced here, targets a weaker requirement for unbiased ATT estimation, specifically, that the expected non-treatment potential outcome for the treatment and control groups are equal. The conditional expectation of the non-treatment potential outcome is assumed to fall in the space of functions associated with a choice of kernel, implying a set of basis functions in which this regression surface is linear. Weights are then chosen on the control units such that the treated and control group have equal means on these basis functions. As a result, the expectation of the non-treatment potential outcome must also be equal for the treated and control groups after weighting, allowing unbiased ATT estimation by subsequent difference in means or an outcome model using these weights. Moreover, the weights produced are (1) precisely those that equalize a particular kernel-based approximation of the multivariate distribution of covariates for the treated and control, and (2) equivalent to a form of stabilized inverse propensity score weighting, though it does not require assuming any model of the treatment assignment mechanism. An R package, KBAL, is provided to implement this approach.
What are the returns to office in an authoritarian parliament? A new dataset shows that over 500 deputies to China's National People's Congress are CEOs of various companies. Entropy balancing is used to construct a weighted portfolio of Chinese companies that matches companies with NPC representation on relevant financial characteristics prior to the 11th Congress (2008-2012). The weighted fixed effect analysis suggests that a seat in the NPC is worth an additional 2.01 to 2.04 percentage points in returns and a 6.92 to 7.46 percentage point boost in operating profit margin in a given year. These micro-level findings buttress cross-national evidence and contribute to an emerging literature on the nature of representation in these institutions.