An Empirical Comparison of Tree-Based Methods for Propensity Score Estimation

Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill, University of North Carolina, Chapel Hill, NC.
Health Services Research (Impact Factor: 2.78). 05/2013; 48(5). DOI: 10.1111/1475-6773.12068
Source: PubMed

ABSTRACT OBJECTIVE: To illustrate the use of ensemble tree-based methods (random forest classification [RFC] and bagging) for propensity score estimation and to compare these methods with logistic regression, in the context of evaluating the effect of physical and occupational therapy on preschool motor ability among very low birth weight (VLBW) children. DATA SOURCE: We used secondary data from the Early Childhood Longitudinal Study Birth Cohort (ECLS-B) between 2001 and 2006. STUDY DESIGN: We estimated the predicted probability of treatment using tree-based methods and logistic regression (LR). We then modeled the exposure-outcome relation using weighted LR models while considering covariate balance and precision for each propensity score estimation method. PRINCIPAL FINDINGS: Among approximately 500 VLBW children, therapy receipt was associated with moderately improved preschool motor ability. Overall, ensemble methods produced the best covariate balance (Mean Squared Difference: 0.03-0.07) and the most precise effect estimates compared to LR (Mean Squared Difference: 0.11). The overall magnitude of the effect estimates was similar between RFC and LR estimation methods. CONCLUSION: Propensity score estimation using RFC and bagging produced better covariate balance with increased precision compared to LR. Ensemble methods are a useful alterative to logistic regression to control confounding in observational studies.

5 Reads
  • Source
    • "Cox proportional hazard modeling was repeated, entering IPTW values as a covariate in the regression model [24]. In order to avoid overestimation of effects from covariates included in the logistic regression model used to create propensity scores, these covariates were excluded from the IPTW-adjusted Cox regression model [22] [25]. Sensitivity analyses consisted of repeating the Cox proportional hazard modeling with IPTW for each propensity score quintile, confirming validity of findings in all quintiles [26]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: For node-positive vulvar cancer, adjuvant radiotherapy has an established benefit, whereas the impact of chemotherapy is unknown. A National Cancer Data Base (NCDB) analysis was conducted to determine patterns of care and evaluate the survival impact of adjuvant chemotherapy. The NCDB was queried for vulvar cancer patients diagnosed from 1998-2011 who underwent extirpative surgery with confirmed inguinal nodal involvement treated with adjuvant radiotherapy. Patients with inadequate follow-up or non-squamous histologies were excluded. Chi-square test, logistic regression analysis, log-rank test and multivariable Cox proportional regression modeling with adjustment using propensity score with inverse probability of treatment weights (IPTW) were conducted to establish factors associated with utilization and survival. A total of 1,797 patients were identified: 26.3% received adjuvant chemotherapy and 76.6% had 1-3 involved lymph nodes. Adoption of adjuvant chemotherapy significantly increased over time, from 10.8% in 1998 to 41.0% in 2006 (p<0.001). Lower utilization was seen in older patients, Northeast or Southern facilities, and patients with more extensive nodal dissection, whereas greater number of involved nodes, stage IVA disease and positive surgical margins led to a higher probability of receiving chemotherapy. Unadjusted median survival without and with adjuvant chemotherapy was 29.7 months and 44.0 months (p=0.001). On IPTW-adjusted Cox proportional regression modeling, delivery of adjuvant chemotherapy resulted in a 38% reduction in the risk of death (HR 0.62, 95% CI 0.48-0.79, p<0.001). In a large population-based analysis, adjuvant chemotherapy resulted in a significant reduction in mortality risk for node-positive vulvar cancer patients who received adjuvant radiotherapy. Copyright © 2015. Published by Elsevier Inc.
    Gynecologic Oncology 04/2015; 137(3). DOI:10.1016/j.ygyno.2015.03.056 · 3.77 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Children born very low birth weight (VLBW) are at an increased risk of delayed development of motor skills. Physical and occupational therapy services may reduce this risk. Among VLBW children, we evaluated whether receipt of physical or occupational therapy services between 9 months and 2 years of age is associated with improved preschool age motor ability. Using data from the Early Childhood Longitudinal Study Birth Cohort we estimated the association between receipt of therapy and the following preschool motor milestones: skipping eight consecutive steps, hopping five times, standing on one leg for 10 seconds, walking backwards six steps on a line, and jumping distance. We used propensity score methods to adjust for differences in baseline characteristics between children who did and did not receive physical or occupational therapy, since children receiving therapy may be at higher risk of impairment. We applied propensity score weights and modeled the estimated effect of therapy on the distance that the child jumped using linear regression. We modeled all other end points using logistic regression. Treated VLBW children were 1.70 times as likely to skip eight steps (RR 1.70, 95 % CI 0.84, 3.44) compared to the untreated group and 30 % more likely to walk six steps backwards (RR 1.30, 95 % CI 0.63, 2.71), although these differences were not statistically significant. We found little effect of therapy on other endpoints. Providing therapy to VLBW children during early childhood may improve select preschool motor skills involving complex motor planning.
    Maternal and Child Health Journal 07/2013; 18(4). DOI:10.1007/s10995-013-1306-x · 2.24 Impact Factor