Reliable estimators for propensity score matching estimation

Abstract

We investigate the final sample properties of a large number of estimators that are suitable in the case when a selection-on-observables identification strategy is thought plausible, like inverse probability weighting, kernel and other variants of matching, as well as different parametric models. We also investigate the various tuning parameters related to many of those methods. To avoid arbitrary design dependence of the results, the Monte Carlo simulations are based on real data used for the evaluation of labour market programmes. We vary several dimensions of the design that are of practical importance, like sample size, outcome variable, and the selection process. We find that (i) trimming individual observations that have a 'too-large' weight is important for all estimators (even without common support problems); (ii) the choices of the various tuning parameters is important; (iii) simple matching estimators are inefficient and have considerable small sample bias; (iv) no estimator is superior in all designs; (v) particular bias adjusted radius matching estimators perform best on average, but may have fat tails for small samples; and finally, (vi) flexible, but simple parametric approaches do almost as well, because their gain in precision frequently overcompensates for their larger bias. One summary of this finding is that the choice of the broad class of estimators is not so important. Instead using an optimally tuned estimator within such a broad class of estimators is the relevant decision to make.