Martens EP, Pestman WR, Klungel OH.. Re: Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study

Institute for Clinical Evaluative Sciences, Toronto, Ont., Canada.
Statistics in Medicine (Impact Factor: 1.83). 02/2007; 26(4):754-68. DOI: 10.1002/sim.2618
Source: PubMed


Propensity score methods are increasingly being used to estimate causal treatment effects in the medical literature. Conditioning on the propensity score results in unbiased estimation of the expected difference in observed responses to two treatments. The degree to which conditioning on the propensity score introduces bias into the estimation of the conditional odds ratio or conditional hazard ratio, which are frequently used as measures of treatment effect in observational studies, has not been extensively studied. We conducted Monte Carlo simulations to determine the degree to which propensity score matching, stratification on the quintiles of the propensity score, and covariate adjustment using the propensity score result in biased estimation of conditional odds ratios, hazard ratios, and rate ratios. We found that conditioning on the propensity score resulted in biased estimation of the true conditional odds ratio and the true conditional hazard ratio. In all scenarios examined, treatment effects were biased towards the null treatment effect. However, conditioning on the propensity score did not result in biased estimation of the true conditional rate ratio. In contrast, conventional regression methods allowed unbiased estimation of the true conditional treatment effect when all variables associated with the outcome were included in the regression model. The observed bias in propensity score methods is due to the fact that regression models allow one to estimate conditional treatment effects, whereas propensity score methods allow one to estimate marginal treatment effects. In several settings with non-linear treatment effects, marginal and conditional treatment effects do not coincide.

  • Source
    • "On the other hand, under certain non randomized treatment assignment, omission of the identified confounding covariate leads to no bias but variance change for the estimated treatment effect (Robinson and Jewell, 1991; Neuhaus, 1998); this phenomenon suggests that the identified confounding covariate has dispersing characters. Similar phenomena have been observed in the propensity score approach to the causal inference (e.g., Senn et al., 2007; Austin et al., 2007 and in survival analysis estimating treatment effect measured as hazard ratio (Ford et al., 1995). See also Lee and Nelder (2004) for insight into these phenomena from perspectives of the conditional vs. marginal models. "
    [Show abstract] [Hide abstract]
    ABSTRACT: As is well known, omission of non confounding covariates identified by the treatment assignment may lead to considerable bias for estimated treatment effect even in a simple randomized trial. In this article we identify confounding vs. dispersing covariates by the confounding influence characterizing variance change and bias risk of estimated treatment effect due to constraint on effects of these covariates. Consequently, consistent constraint on effects of identified confounding covariates reduces variance of estimated treatment effect whereas inconsistent constraint on effects of identified dispersing covariates—such as omission of identified dispersing covariates—leads to little bias for estimated treatment effect.
    Full-text · Article · Dec 2013 · Communication in Statistics- Theory and Methods
  • Source
    • "We performed a propensity score analysis to adjust for imbalances in baseline characteristics between patients with-and without maintenance TMS (D'Agostino, 1998). For this purpose, we first developed a nonparsimonious logistic regression model to derive a propensity score for patients receiving TMS based on all the covariates (Austin et al., 2007). This logistic regression model yielded a c-statistic (Area Under Curve) of 0.73, indicating a satisfactory ability to discriminate between maintenance TMS or no additional TMS treatment. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The effectiveness of repetitive transcranial magnetic stimulation (TMS) is well established while studies of maintenance TMS are lacking. We aim here to determine whether maintenance is associated to a decrease in the relapse rate of depression, following successful acute treatment. We enrolled 59 consecutive patients with pharmacoresistant depression who have responded (>50% decrease in symptom severity) up to 6 weeks of acute TMS treatment. These patients received either 20 weeks of maintenance TMS (n=37) or no additional TMS treatment (n=22). We performed propensity adjusted-analysis to examine the association between the relapse rate over this 20-week period and maintenance TMS. Propensity analysis eliminated differences in baseline characteristics between patient with and without maintenance TMS and approximated the conditions of random site-of-treatment assignment. At 20 weeks, relapse rate was significantly different between the two groups (p=0.004, propensity analysis): 14 patients in the maintenance TMS group (37.8%) vs. 18 in the non-maintenance TMS group (81.8%), with an adjusted Hazard Ratio (HR)=0.288 (0.124-0.669). Maintenance TMS was associated with a significantly lower relapse rate in patients with pharmacoresistant depression in routine practice among responders.
    Full-text · Article · Jun 2013 · Journal of Affective Disorders
  • Source
    • "The choice of the odds ratio as a measure of treatment effect has been debated. Indeed, the choice of the appropriate PS based estimators for conditional and marginal non-linear treatment effects has been thoroughly discussed in the recent literature [21,36-38]. Actually, the problem with OR, usually described as non-collapsibility, refers to the fact that conditional and marginal effects might differ unless the true effect is null [21,39,40]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Propensity score (PS) methods are increasingly used, even when sample sizes are small or treatments are seldom used. However, the relative performance of the two mainly recommended PS methods, namely PS-matching or inverse probability of treatment weighting (IPTW), have not been studied in the context of small sample sizes. Methods We conducted a series of Monte Carlo simulations to evaluate the influence of sample size, prevalence of treatment exposure, and strength of the association between the variables and the outcome and/or the treatment exposure, on the performance of these two methods. Results Decreasing the sample size from 1,000 to 40 subjects did not substantially alter the Type I error rate, and led to relative biases below 10%. The IPTW method performed better than the PS-matching down to 60 subjects. When N was set at 40, the PS matching estimators were either similarly or even less biased than the IPTW estimators. Including variables unrelated to the exposure but related to the outcome in the PS model decreased the bias and the variance as compared to models omitting such variables. Excluding the true confounder from the PS model resulted, whatever the method used, in a significantly biased estimation of treatment effect. These results were illustrated in a real dataset. Conclusion Even in case of small study samples or low prevalence of treatment, PS-matching and IPTW can yield correct estimations of treatment effect unless the true confounders and the variables related only to the outcome are not included in the PS model.
    Full-text · Article · May 2012 · BMC Medical Research Methodology
Show more