Ranking USRDS provider specific SMRs from 1998–2001

Department of Public Health, University of Massachusetts Amherst, Rm 411 Arnold House, 715 N. Pleasant Rd., Amherst, MA 01003, USA.
Health Services and Outcomes Research Methodology 01/2009; 9(1):22-38. DOI: 10.1007/s10742-008-0040-0

ABSTRACT Provider profiling (ranking/percentiling) is prevalent in health services research. Bayesian models coupled with optimizing
a loss function provide an effective framework for computing non-standard inferences such as ranks. Inferences depend on the
posterior distribution and should be guided by inferential goals. However, even optimal methods might not lead to definitive
results and ranks should be accompanied by valid uncertainty assessments. We outline the Bayesian approach and use estimated
Standardized Mortality Ratios (SMRs) in 1998–2001 from the United States Renal Data System (USRDS) as a platform to identify
issues and demonstrate approaches. Our analyses extend Liu et al. (2004) by computing estimates developed by Lin et al. (2006)
that minimize errors in classifying providers above or below a percentile cut-point, by combining evidence over multiple years
via a first-order, autoregressive model on log(SMR), and by use of a nonparametric prior. Results show that ranks/percentiles
based on maximum likelihood estimates of the SMRs and those based on testing whether an SMR = 1 substantially under-perform
the optimal estimates. Combining evidence over the four years using the autoregressive model reduces uncertainty, improving
performance over percentiles based on only one year. Furthermore, percentiles based on posterior probabilities of exceeding
a properly chosen SMR threshold are essentially identical to those produced by minimizing classification loss. Uncertainty
measures effectively calibrate performance, showing that considerable uncertainty remains even when using optimal methods.
Findings highlight the importance of using loss function guided percentiles and the necessity of accompanying estimates with
uncertainty assessments.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The development of high throughput screening (HTS) assays in the field of nanotoxicology provide new opportunities for the hazard assessment and ranking of engineered nanomaterials (ENMs). It is often necessary to rank lists of materials based on multiple risk assessment parameters, often aggregated across several measures of toxicity and possibly spanning an array of experimental platforms. Bayesian models coupled with the optimization of loss functions have been shown to provide an effective framework for conducting inference on ranks. In this article we present various loss-function-based ranking approaches for comparing ENM within experiments and toxicity parameters. Additionally, we propose a framework for the aggregation of ranks across different sources of evidence while allowing for differential weighting of this evidence based on its reliability and importance in risk ranking. We apply these methods to high throughput toxicity data on two human cell-lines, exposed to eight different nanomaterials, and measured in relation to four cytotoxicity outcomes. This article has supplementary material online.
    Journal of Agricultural Biological and Environmental Statistics 06/2013; 18(2):159. DOI:10.1007/s13253-013-0129-y · 0.78 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Within the context of California's public report of coronary artery bypass graft (CABG) surgery outcomes, we first thoroughly review popular statistical methods for profiling healthcare providers. Extensive simulation studies are then conducted to compare profiling schemes based on hierarchical logistic regression (LR) modeling under various conditions. Both Bayesian and frequentist's methods are evaluated in classifying hospitals into ‘better’, ‘normal’ or ‘worse’ service providers. The simulation results suggest that no single method would dominate others on all accounts. Traditional schemes based on LR tend to identify too many false outliers, while those based on hierarchical modeling are relatively conservative. The issue of over shrinkage in hierarchical modeling is also investigated using the 2005–2006 California CABG data set. The article provides theoretical and empirical evidence in choosing the right methodology for provider profiling.
    Journal of Applied Statistics 01/2014; 41(1). DOI:10.1080/02664763.2013.830086 · 0.45 Impact Factor
  • Source


1 Download
Available from