Interpreting Treatment-Effect Estimates with Heterogeneity and Choice: Simulation Model Results

College of Pharmacy, University of Iowa, Iowa City, Iowa 52242, USA.
Clinical Therapeutics (Impact Factor: 2.73). 05/2009; 31(4):902-19. DOI: 10.1016/j.clinthera.2009.04.007
Source: PubMed


Background: Researchers using observational data in health-services research use various treatment-effect estimators to reduce the bias associated with unmeasured confounding variables and have focused on estimate differences to indicate the relative ability of these estimators to mitigate bias. However, available estimators may identify different treatment-effect concepts; if treatment effects are heterogeneous across patients and treatment choice reflects “sorting on the gain,” then treatment-effect estimates should differ regardless of confounding. Risk-adjustment approaches yield estimates of the average treatment effect on the treated (ATT), whereas instrumental variable approaches yield estimates of a local average treatment effect (LATE).

9 Reads
  • Source
    • "We modified a simple simulation model of treatment choice and outcome that was used in previous research (Brooks and Fang 2009). In this model, covariates affecting treatment choice are divided between those measured and unmeasured by a subsequent researcher. "
    [Show abstract] [Hide abstract]
    ABSTRACT: OBJECTIVE: To assess the covariate balancing properties of propensity score-based algorithms in which covariates affecting treatment choice are both measured and unmeasured. DATA SOURCES/STUDY SETTING: A simulation model of treatment choice and outcome. STUDY DESIGN: Simulation. DATA COLLECTION/EXTRACTION METHODS: Eight simulation scenarios varied with the values placed on measured and unmeasured covariates and the strength of the relationships between the measured and unmeasured covariates. The balance of both measured and unmeasured covariates was compared across patients either grouped or reweighted by propensity scores methods. PRINCIPAL FINDINGS: Propensity score algorithms require unmeasured covariate variation that is unrelated to measured covariates, and they exacerbate the imbalance in this variation between treated and untreated patients relative to the full unweighted sample. CONCLUSIONS: The balance of measured covariates between treated and untreated patients has opposite implications for unmeasured covariates in randomized and observational studies. Measured covariate balance between treated and untreated patients in randomized studies reinforces the notion that all covariates are balanced. In contrast, forced balance of measured covariates using propensity score methods in observational studies exacerbates the imbalance in the independent portion of the variation in the unmeasured covariates, which can be likened to squeezing a balloon. If the unmeasured covariates affecting treatment choice are confounders, propensity score methods can exacerbate the bias in treatment effect estimates.
    Health Services Research 12/2012; 48(4). DOI:10.1111/1475-6773.12020 · 2.78 Impact Factor
  • Source
    • "IV estimators are distinct from risk-adjustment estimators in both the assumptions required to yield unbiased estimates and in the interpretation of the eventual estimates [14–18]. Risk-adjustment estimators assume that unmeasured covariates affecting treatment choice are unrelated to outcomes and yield parameters that are properly interpreted as estimates of the average treatment effectfor the patients that received treatment [12–16, 19, 20]. In contrast, IV estimators yield estimates of a local average treatment effect, that is, the average effect for the subset of patients whose treatment choices were affected by measured factors called “instruments” [17, 21, 22]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Despite a 20-year-old guideline from the National Institutes of Health (NIH) Consensus Development Conference recommending breast conserving surgery with radiation (BCSR) over mastectomy for woman with early-stage breast cancer (ESBC) because it preserves the breast, recent evidence shows mastectomy rates increasing and higher-staged ESBC patients are more likely to receive mastectomy. These observations suggest that some patients and their providers believe that mastectomy has advantages over BCSR and these advantages increase with stage. These beliefs may persist because the randomized controlled trials (RCTs) that served as the basis for the NIH guideline were populated mainly with lower-staged patients. Our objective is to assess the survival implications associated with mastectomy choice by patient alignment with the RCT populations. We used instrumental variable methods to estimate the relationship between surgery choice and survival for ESBC patients based on variation in local area surgery styles. We find results consistent with the RCTs for patients closely aligned to the RCT populations. However, for patients unlike those in the RCTs, our results suggest that higher mastectomy rates are associated with reduced survival. We are careful to interpret our estimates in terms of limitations of our estimation approach.
    International Journal of Surgical Oncology 08/2012; 2012(2090-1402):127854. DOI:10.1155/2012/127854
  • Source
    • "Hence, in our situation, the effect of ICU admission on hospital mortality is not captured by the IV approach for the patients who, whatever the value of the physician's specialization, i.e., the chosen instrument, would have always been accepted or rejected from the ICU. Thus, it is important for researchers to state the treatment-effect concept that they are trying to identify before beginning estimation [53]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The assessment of the causal effect of Intensive Care Unit (ICU) admission generally involves usual observational designs and thus requires controlling for confounding variables. Instrumental variable analysis is an econometric technique that allows causal inferences of the effectiveness of some treatments during situations to be made when a randomized trial has not been or cannot be conducted. This technique relies on the existence of one variable or "instrument" that is supposed to achieve similar observations with a different treatment for "arbitrary" reasons, thus inducing substantial variation in the treatment decision with no direct effect on the outcome. The objective of the study was to assess the benefit in terms of hospital mortality of ICU admission in a cohort of patients proposed for ICU admission (ELDICUS cohort). Using this cohort of 8,201 patients triaged for ICU (including 6,752 (82.3%) patients admitted), the benefit of ICU admission was evaluated using 3 different approaches: instrumental variables, standard regression and propensity score matched analyses. We further evaluated the results obtained using different instrumental variable methods that have been proposed for dichotomous outcomes. The physician's main specialization was found to be the best instrument. All instrumental variable models adequately reduced baseline imbalances, but failed to show a significant effect of ICU admission on hospital mortality, with confidence intervals far higher than those obtained in standard or propensity-based analyses. Instrumental variable methods offer an appealing alternative to handle the selection bias related to nonrandomized designs, especially when the presence of significant unmeasured confounding is suspected. Applied to the ELDICUS database, this analysis failed to show any significant beneficial effect of ICU admission on hospital mortality. This result could be due to the lack of statistical power of these methods.
    BMC Medical Research Methodology 09/2011; 11(1):132. DOI:10.1186/1471-2288-11-132 · 2.27 Impact Factor
Show more