An Empirical Comparison of Tree-Based Methods for Propensity Score Estimation

Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill, University of North Carolina, Chapel Hill, NC.
Health Services Research (Impact Factor: 2.78). 05/2013; 48(5). DOI: 10.1111/1475-6773.12068
Source: PubMed


To illustrate the use of ensemble tree-based methods (random forest classification [RFC] and bagging) for propensity score estimation and to compare these methods with logistic regression, in the context of evaluating the effect of physical and occupational therapy on preschool motor ability among very low birth weight (VLBW) children.
We used secondary data from the Early Childhood Longitudinal Study Birth Cohort (ECLS-B) between 2001 and 2006.
We estimated the predicted probability of treatment using tree-based methods and logistic regression (LR). We then modeled the exposure-outcome relation using weighted LR models while considering covariate balance and precision for each propensity score estimation method.
Among approximately 500 VLBW children, therapy receipt was associated with moderately improved preschool motor ability. Overall, ensemble methods produced the best covariate balance (Mean Squared Difference: 0.03–0.07) and the most precise effect estimates compared to LR (Mean Squared Difference: 0.11). The overall magnitude of the effect estimates was similar between RFC and LR estimation methods.
Propensity score estimation using RFC and bagging produced better covariate balance with increased precision compared to LR. Ensemble methods are a useful alterative to logistic regression to control confounding in observational studies.

5 Reads
  • Source
    • "Cox proportional hazard modeling was repeated, entering IPTW values as a covariate in the regression model [24]. In order to avoid overestimation of effects from covariates included in the logistic regression model used to create propensity scores, these covariates were excluded from the IPTW-adjusted Cox regression model [22] [25]. Sensitivity analyses consisted of repeating the Cox proportional hazard modeling with IPTW for each propensity score quintile, confirming validity of findings in all quintiles [26]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: For node-positive vulvar cancer, adjuvant radiotherapy has an established benefit, whereas the impact of chemotherapy is unknown. A National Cancer Data Base (NCDB) analysis was conducted to determine patterns of care and evaluate the survival impact of adjuvant chemotherapy. The NCDB was queried for vulvar cancer patients diagnosed from 1998-2011 who underwent extirpative surgery with confirmed inguinal nodal involvement treated with adjuvant radiotherapy. Patients with inadequate follow-up or non-squamous histologies were excluded. Chi-square test, logistic regression analysis, log-rank test and multivariable Cox proportional regression modeling with adjustment using propensity score with inverse probability of treatment weights (IPTW) were conducted to establish factors associated with utilization and survival. A total of 1,797 patients were identified: 26.3% received adjuvant chemotherapy and 76.6% had 1-3 involved lymph nodes. Adoption of adjuvant chemotherapy significantly increased over time, from 10.8% in 1998 to 41.0% in 2006 (p<0.001). Lower utilization was seen in older patients, Northeast or Southern facilities, and patients with more extensive nodal dissection, whereas greater number of involved nodes, stage IVA disease and positive surgical margins led to a higher probability of receiving chemotherapy. Unadjusted median survival without and with adjuvant chemotherapy was 29.7 months and 44.0 months (p=0.001). On IPTW-adjusted Cox proportional regression modeling, delivery of adjuvant chemotherapy resulted in a 38% reduction in the risk of death (HR 0.62, 95% CI 0.48-0.79, p<0.001). In a large population-based analysis, adjuvant chemotherapy resulted in a significant reduction in mortality risk for node-positive vulvar cancer patients who received adjuvant radiotherapy. Copyright © 2015. Published by Elsevier Inc.
    Gynecologic Oncology 04/2015; 137(3). DOI:10.1016/j.ygyno.2015.03.056 · 3.77 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Children born very low birth weight (VLBW) are at an increased risk of delayed development of motor skills. Physical and occupational therapy services may reduce this risk. Among VLBW children, we evaluated whether receipt of physical or occupational therapy services between 9 months and 2 years of age is associated with improved preschool age motor ability. Using data from the Early Childhood Longitudinal Study Birth Cohort we estimated the association between receipt of therapy and the following preschool motor milestones: skipping eight consecutive steps, hopping five times, standing on one leg for 10 seconds, walking backwards six steps on a line, and jumping distance. We used propensity score methods to adjust for differences in baseline characteristics between children who did and did not receive physical or occupational therapy, since children receiving therapy may be at higher risk of impairment. We applied propensity score weights and modeled the estimated effect of therapy on the distance that the child jumped using linear regression. We modeled all other end points using logistic regression. Treated VLBW children were 1.70 times as likely to skip eight steps (RR 1.70, 95 % CI 0.84, 3.44) compared to the untreated group and 30 % more likely to walk six steps backwards (RR 1.30, 95 % CI 0.63, 2.71), although these differences were not statistically significant. We found little effect of therapy on other endpoints. Providing therapy to VLBW children during early childhood may improve select preschool motor skills involving complex motor planning.
    Maternal and Child Health Journal 07/2013; 18(4). DOI:10.1007/s10995-013-1306-x · 2.24 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Purpose: Mantle cell lymphoma (MCL) is a rare, albeit aggressive subset of nonHodgkin lymphoma, resulting in varied treatment approaches. Given the paucity of data defining the optimal management for early-stage MCL, we conducted an analysis using the National Cancer Data Base (NCDB) to identify practice patterns and outcomes. Methods and Materials: The NCDB was queried for patients with stage I to II MCL diagnosed from 1998 to 2012 receiving chemotherapy (CT) or radiation therapy (RT), or both (CTþRT). Univariate and multivariable analyses for factors associated with treatment selection were completed using logistic regression. Propensity scores with inverse probability treatment weighting (IPTW) were calculated based on the conditional probability of receiving CTþRT. The log-rank test and Cox proportional hazards modeling with IPTW adjustment were conducted for the survival analyses. Results: In total, 2539 patients were identified. The key characteristics were as follows: 69% were male, 71% were aged 60 years, 28% had extranodal involvement, and 51% had stage I disease. Of the 2539 patients, 70% underwent CT, 11% underwent RT, and 19% underwent CTþRT. The use of CTþRT decreased from 23.1% to 14.1% in 1998 to 2002 and 2010 to 2012 (P<.001). CTþRT usage was lower for patients with the following characteristics: age 60 years, female sex, stage II disease, and the presence of B symptoms. With a median follow-up period of 42.8 months, the unadjusted 3-year overall survival estimates for patients receiving CT, RT, or CTþRT were 67.8%, 72.4%, and 79.8%, respectively (P<.001). After correcting for indication bias through IPTW-adjusted modeling, CTþRT reduced the risk of overall mortality compared with monotherapy (hazard ratio 0.65, PZ.029). Conclusions: Although uncommon, patients with stage I-II MCL can have favorable outcomes. Despite a continued decline in the usage of consolidative RT, combined modality therapy improves survival in this cohort compared with monotherapy.
    International journal of radiation oncology, biology, physics 07/2015; 93(3):668-676. DOI:10.1016/j.ijrobp.2015.07.2265 · 4.26 Impact Factor