The role of the c-statistic in variable selection for propensity score models

Department of Obstetrics and Gynecology and Duke Global Health Institute, Duke University, Durham, NC, USA.
Pharmacoepidemiology and Drug Safety (Impact Factor: 2.94). 03/2011; 20(3):317-20. DOI: 10.1002/pds.2074
Source: PubMed


The applied literature on propensity scores has often cited the c-statistic as a measure of the ability of the propensity score to control confounding. However, a high c-statistic in the propensity model is neither necessary nor sufficient for control of confounding. Moreover, use of the c-statistic as a guide in constructing propensity scores may result in less overlap in propensity scores between treated and untreated subjects; this may require the analyst to restrict populations for inference. Such restrictions may reduce precision of estimates and change the population to which the estimate applies. Variable selection based on prior subject matter knowledge, empirical observation, and sensitivity analysis is preferable and avoids many of these problems.

Download full-text


Available from: Daniel Westreich, Sep 21, 2014
  • Source
    • "It also corresponds to the area under the receiver operating characteristic (ROC) curve, which displays sensitivity as a function of 1- specificity for all the possible thresholds of the predictor[34]. If we consider an intervention allocation (intervention vs. control), the c-statistic is the probability that a subject receiving the intervention has a higher value for the predictor than a subject in the control group[35]. It can be estimated as follows: "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: Despite randomization, baseline imbalance and confounding bias may occur in cluster randomized trials (CRTs). Covariate imbalance may jeopardize the validity of statistical inferences if they occur on prognostic factors. Thus, the diagnosis of a such imbalance is essential to adjust statistical analysis if required. Methods: We developed a tool based on the c-statistic of the propensity score (PS) model to detect global baseline covariate imbalance in CRTs and assess the risk of confounding bias. We performed a simulation study to assess the performance of the proposed tool and applied this method to analyze the data from 2 published CRTs. Results: The proposed method had good performance for large sample sizes (n =500 per arm) and when the number of unbalanced covariates was not too small as compared with the total number of baseline covariates (≥40 % of unbalanced covariates). We also provide a strategy for pre selection of the covariates needed to be included in the PS model to enhance imbalance detection. Conclusion: The proposed tool could be useful in deciding whether covariate adjustment is required before performing statistical analyses of CRTs.
    Full-text · Article · Dec 2016 · BMC Medical Research Methodology
  • Source
    • " statistic , Hosmer – Lemeshow statistic , or any other measure of goodness - of - fit to select variables for inclusion in our models for the weights because doing so can lead to bias ( from unbalanced confounders or balanced nonconfounders including instrumental variables ) , reduced precision , nonpositivity , and / or restricted infer - ence ( Westreich et al . 2011 ) . To informally assess the bias – variance tradeoff ( Winer 1978 ) , we progressively truncated the overall stabilized weights by resetting weights less ( or greater ) than a certain percentile to the value of that percentile ( Cole and Hernán 2008 ) . Regarding the ORs derived from the untruncated weights as the " true " values , we "
    [Show abstract] [Hide abstract]
    ABSTRACT: BACKGROUND: Pesticide exposure may be positively associated with depression. Few previous studies have considered the episodic nature of depression or examined individual pesticides. OBJECTIVE: We evaluated associations between pesticide exposure and depression among male private pesticide applicators in the Agricultural Health Study. METHODS: We analyzed data for 10 pesticide classes and 50 specific pesticides used by 21,208 applicators enrolled in 1993-1997 who completed a follow-up telephone interview in 2005-2010. We divided applicators who reported a physician diagnosis of depression (n = 1,702; 8%) into those who reported a previous diagnosis of depression at enrollment but not follow-up (n = 474; 28%), at both enrollment and follow-up (n = 540; 32%), and at follow-up but not enrollment (n = 688; 40%) and used polytomous logistic regression to estimate odds ratios (ORs) and 95% CIs. We used inverse probability weighting to adjust for potential confounders and to account for the exclusion of 3,315 applicators with missing covariate data and 24,619 who did not complete the follow-up interview. RESULTS: After weighting for potential confounders, missing covariate data, and dropout, ever-use of two pesticide classes, fumigants and organochlorine insecticides, and seven individual pesticides- the fumigants aluminum phosphide and ethylene dibromide; the phenoxy herbicide (2,4,5-trichlorophenoxy) acetic acid (2,4,5-T); the organochlorine insecticide dieldrin; and the organophosphate insecticides diazinon, malathion, and parathion-were all positively associated with depression in each case group, with ORs between 1.1 and 1.9. CONCLUSIONS: Our study supports a positive association between pesticide exposure and depression, including associations with several specific pesticides.
    Full-text · Article · Jun 2014 · Environmental Health Perspectives
  • Source
    • "The model was developed with baseline characteristics reported at the time of the first open-heart surgery. Univariate modeling was performed to identify potential confounders and covariates with a significant association with the outcome of mortality [13,14]. The discriminatory power of the model was assessed using the area under the receiver operating characteristic curve (AUC), or C statistic; however, this model diagnostic was not used to guide variable selection into the propensity score model. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Given their frequency of occurrence in the United States, cancer and heart disease often coexist. For patients requiring open-heart surgery, this raises concern that the use of cardiopulmonary bypass (CPB) may cause a transient immunosuppression with the potential to promote the spread and growth of coexisting cancer cells. This study examined the association of cardiopulmonary bypass with cancer progression in a large population-based setting using linked data from several state-wide registries. A retrospective cohort study of cancer risk, stage, and mortality in 43,347 patients who underwent coronary artery bypass graft (CABG) surgery with and without CPB in New Jersey between 1998--2004 was conducted. A competing risk analogue of the Cox proportional hazards model with propensity score adjustment and regression on the cause-specific hazard was used to compute relative risk ratios (95% confidence intervals [CIs]) for patients undergoing CABG surgery with and without CPB. An increased risk for overall cancer incidence (17%) and cancer-specific mortality (16% overall, 12% case fatality) was observed; yet these results did not reach statistical significance. Of 11 tumor-specific analyses, an increased risk of skin melanoma (1.66 [95% CI, 1.08-2.55: p=0.02]) and lung cancer (1.36 [95% CI, 1.02-1.81: p=0.03]) was observed for patients with pump versus off-pump open-heart surgery. No association was found with cancer stage. These results suggest that there may be a relationship between CPB and cancer progression. However, if real, the effect is likely modest at most. Further research may still be warranted with particular focus on skin melanoma and lung cancer which had the strongest association with CPB.
    Full-text · Article · Nov 2013 · BMC Cancer
Show more