The role of the c-statistic in variable selection for propensity score models

Department of Obstetrics and Gynecology and Duke Global Health Institute, Duke University, Durham, NC, USA.
Pharmacoepidemiology and Drug Safety (Impact Factor: 2.94). 03/2011; 20(3):317-20. DOI: 10.1002/pds.2074
Source: PubMed


The applied literature on propensity scores has often cited the c-statistic as a measure of the ability of the propensity score to control confounding. However, a high c-statistic in the propensity model is neither necessary nor sufficient for control of confounding. Moreover, use of the c-statistic as a guide in constructing propensity scores may result in less overlap in propensity scores between treated and untreated subjects; this may require the analyst to restrict populations for inference. Such restrictions may reduce precision of estimates and change the population to which the estimate applies. Variable selection based on prior subject matter knowledge, empirical observation, and sensitivity analysis is preferable and avoids many of these problems.

Download full-text


Available from: Daniel Westreich, Sep 21, 2014

Click to see the full-text of:

Article: The role of the c-statistic in variable selection for propensity score models

36.65 KB

See full-text
  • Source
    • " statistic , Hosmer – Lemeshow statistic , or any other measure of goodness - of - fit to select variables for inclusion in our models for the weights because doing so can lead to bias ( from unbalanced confounders or balanced nonconfounders including instrumental variables ) , reduced precision , nonpositivity , and / or restricted infer - ence ( Westreich et al . 2011 ) . To informally assess the bias – variance tradeoff ( Winer 1978 ) , we progressively truncated the overall stabilized weights by resetting weights less ( or greater ) than a certain percentile to the value of that percentile ( Cole and Hernán 2008 ) . Regarding the ORs derived from the untruncated weights as the " true " values , we "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: Pesticide exposure may be positively associated with depression. Few previous studies have considered the episodic nature of depression or examined individual pesticides. Objective: We evaluated associations between pesticide exposure and depression among male private pesticide applicators in the Agricultural Health Study. Methods: We analyzed data for 10 pesticide classes and 50 specific pesticides used by 21,208 applicators enrolled in 1993-1997 who completed a follow-up telephone interview in 2005-2010. We divided applicators who reported a physician diagnosis of depression (n = 1,702; 8%) into those who reported a previous diagnosis of depression at enrollment but not follow-up (n = 474; 28%), at both enrollment and follow-up (n = 540; 32%), and at follow-up but not enrollment (n = 688; 40%) and used polytomous logistic regression to estimate odds ratios (ORs) and 95% CIs. We used inverse probability weighting to adjust for potential confounders and to account for the exclusion of 3,315 applicators with missing covariate data and 24,619 who did not complete the follow-up interview. Results: After weighting for potential confounders, missing covariate data, and dropout, ever-use of two pesticide classes, fumigants and organochlorine insecticides, and seven individual pesticides- the fumigants aluminum phosphide and ethylene dibromide; the phenoxy herbicide (2,4,5-trichlorophenoxy)acetic acid (2,4,5-T); the organochlorine insecticide dieldrin; and the organophosphate insecticides diazinon, malathion, and parathion-were all positively associated with depression in each case group, with ORs between 1.1 and 1.9. Conclusions: Our study supports a positive association between pesticide exposure and depression, including associations with several specific pesticides.
    Environmental Health Perspectives 06/2014; 122(9). DOI:10.1289/ehp.1307450 · 7.98 Impact Factor
  • Source
    • "The model was developed with baseline characteristics reported at the time of the first open-heart surgery. Univariate modeling was performed to identify potential confounders and covariates with a significant association with the outcome of mortality [13,14]. The discriminatory power of the model was assessed using the area under the receiver operating characteristic curve (AUC), or C statistic; however, this model diagnostic was not used to guide variable selection into the propensity score model. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Given their frequency of occurrence in the United States, cancer and heart disease often coexist. For patients requiring open-heart surgery, this raises concern that the use of cardiopulmonary bypass (CPB) may cause a transient immunosuppression with the potential to promote the spread and growth of coexisting cancer cells. This study examined the association of cardiopulmonary bypass with cancer progression in a large population-based setting using linked data from several state-wide registries. A retrospective cohort study of cancer risk, stage, and mortality in 43,347 patients who underwent coronary artery bypass graft (CABG) surgery with and without CPB in New Jersey between 1998--2004 was conducted. A competing risk analogue of the Cox proportional hazards model with propensity score adjustment and regression on the cause-specific hazard was used to compute relative risk ratios (95% confidence intervals [CIs]) for patients undergoing CABG surgery with and without CPB. An increased risk for overall cancer incidence (17%) and cancer-specific mortality (16% overall, 12% case fatality) was observed; yet these results did not reach statistical significance. Of 11 tumor-specific analyses, an increased risk of skin melanoma (1.66 [95% CI, 1.08-2.55: p=0.02]) and lung cancer (1.36 [95% CI, 1.02-1.81: p=0.03]) was observed for patients with pump versus off-pump open-heart surgery. No association was found with cancer stage. These results suggest that there may be a relationship between CPB and cancer progression. However, if real, the effect is likely modest at most. Further research may still be warranted with particular focus on skin melanoma and lung cancer which had the strongest association with CPB.
    BMC Cancer 11/2013; 13(1):519. DOI:10.1186/1471-2407-13-519 · 3.36 Impact Factor
  • Source
    • "First, in an observational study using a propensity score method, the purpose of estimation in the logistic regression model is not to precisely estimate probability of assignment to a particular treatment, but to get a variable used to balance on covariates. Therefore, statistical criteria such as Hosmer–Lemeshow test statistics or AUC (the Area Under the ROC Curve), are not informative [15, 16]. Secondly, all variables expected to show a relationship to the outcome should be included, regardless of whether they are significantly related to the assignment [17]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The purpose of this study is to evaluate the effectiveness of a single session intervention designed to reduce emotional distress in first-time mothers. We held a parenting class for first-time mothers who had given birth at a university hospital in Tokyo, Japan. The program of the class consists of lectures on infant care and group discussion, which is a common form of intervention in Japan. The effectiveness of intervention is assessed according to differences in emotional distress experienced by class participants and nonparticipants, and analyzed by the use of a propensity score method to avoid self-selection bias. In order to be more confident about our results, we employ several variations of this method. Results from statistical analysis show that although the effectiveness of the intervention was limited, it was able to alleviate subjects' loss of self-confidence as mothers. Because this outcome shows a good degree of consistency across methods, it can be considered robust. Moreover, it is roughly consistent with previous studies. Effectiveness can probably be increased by developing a program that improves upon the intervention.
    Maternal and Child Health Journal 08/2012; 17(6). DOI:10.1007/s10995-012-1088-6 · 2.24 Impact Factor
Show more