Shrinkage Estimators for a Composite Measure of Quality Conceptualized as a Formative Construct

School of Management, Boston University, Boston, MA.
Health Services Research (Impact Factor: 2.78). 06/2012; 48(1). DOI: 10.1111/j.1475-6773.2012.01437.x
Source: PubMed


To demonstrate the value of shrinkage estimators when calculating a composite quality measure as the weighted average of a set of individual quality indicators.
Rates of 28 quality indicators (QIs) calculated from the minimum dataset from residents of 112 Veterans Health Administration nursing homes in fiscal years 2005–2008.
We compared composite scores calculated from the 28 QIs using both observed rates and shrunken rates derived from a Bayesian multivariate normal-binomial model.
Shrunken-rate composite scores, because they take into account unreliability of estimates from small samples and the correlation among QIs, have more intuitive appeal than observed-rate composite scores. Facilities can be profiled based on more policy-relevant measures than point estimates of composite scores, and interval estimates can be calculated without assuming the QIs are independent. Usually, shrunken-rate composite scores in 1 year are better able to predict the observed total number of QI events or the observed-rate composite scores in the following year than the initial year observed-rate composite scores.
Shrinkage estimators can be useful when a composite measure is conceptualized as a formative construct.

3 Reads
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective: To examine variation in culture change to a person-centered care (PCC) model, and the association between culture change and a composite measure of quality in 107 Department of Veterans Affairs nursing homes. Methods: We examined the relationship between a composite quality measure calculated from 24 quality indicators (QIs) from the Minimum Data Set (that measure unfavorable events), and PCC summary scores calculated from the 6 domains of the Artifact of Culture Change Tool, using 3 different methods of calculating the summary scores. We also use a Bayesian hierarchical model to analyze the relationship between a latent construct measuring extent of culture change and the composite quality measure. Results: Using the original Artifacts scores, the highest performing facility has a 2.9 times higher score than the lowest. There is a statistically significant relationship between the composite quality measure and each of the 3 summary Artifacts scores. Depending on whether original scores, standardized scores, or optimal scores are used, a facility at the 10th percentile in terms of culture change compared with one at the 90th percentile has 8.0%, 8.9%, or 10.3% more QI events. When PCC implementation is considered as a latent construct, 18 low performance PCC facilities have, on an average, 16.3% more QI events than 13 high performance facilities. Conclusions: Our results indicate that culture change to a PCC model is associated with higher Minimum Data Set-based quality. Longitudinal data are needed to better assess whether there is a causal relationship between the extent of culture change and quality.
    Medical care 11/2012; 51(2). DOI:10.1097/MLR.0b013e3182763230 · 3.23 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background: Two approaches are commonly used for identifying high-performing facilities on a performance measure: one, that the facility is in a top quantile (eg, quintile or quartile); and two, that a confidence interval is below (or above) the average of the measure for all facilities. This type of yes/no designation often does not do well in distinguishing high-performing from average-performing facilities. Objective: To illustrate an alternative continuous-valued metric for profiling facilities--the probability a facility is in a top quantile--and show the implications of using this metric for profiling and pay-for-performance. Methods: We created a composite measure of quality from fiscal year 2007 data based on 28 quality indicators from 112 Veterans Health Administration nursing homes. A Bayesian hierarchical multivariate normal-binomial model was used to estimate shrunken rates of the 28 quality indicators, which were combined into a composite measure using opportunity-based weights. Rates were estimated using Markov Chain Monte Carlo methods as implemented in WinBUGS. The probability metric was calculated from the simulation replications. Results: Our probability metric allowed better discrimination of high performers than the point or interval estimate of the composite score. In a pay-for-performance program, a smaller top quantile (eg, a quintile) resulted in more resources being allocated to the highest performers, whereas a larger top quantile (eg, being above the median) distinguished less among high performers and allocated more resources to average performers. Conclusion: The probability metric has potential but needs to be evaluated by stakeholders in different types of delivery systems.
    Medical Care 10/2014; 52(12). DOI:10.1097/MLR.0000000000000242 · 3.23 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background:Composite measures are useful for distilling quality data into summary scores; yet, there has been limited use of composite measures for cancer care.Objective:Compare multiple approaches for generating cancer care composite measures and evaluate how well composite measures summarize dimensions of cancer care and predict survival.Study Design:We computed hospital-level rates for 13 colorectal, lung, and prostate cancer process measures in 59 Veterans Affairs hospitals. We computed 4 empirical-factor (based on an exploratory factor analysis), 3 cancer-specific (colorectal, lung, prostate care), and 3 care modality-specific (diagnosis/evaluation, surgical, nonsurgical treatments) composite measures. We assessed correlations among all composite measures and estimated all-cause survival for colon, rectal, non-small cell lung, and small cell lung cancers as a function of composite scores, adjusting for patient characteristics.Results:Four factors emerged from the factor analysis: nonsurgical treatment, surgical treatment, colorectal early diagnosis, and prostate treatment. We observed strong correlations (r) among composite measures comprised of similar process measures (r=0.58-1.00, P<0.0001), but not among composite measures reflecting different care dimensions. Composite measures were rarely associated with survival.Conclusions:The empirical-factor domains grouped measures variously by cancer type and care modality. The evidence did not support any single approach for generating cancer care composite measures. Weak associations across different care domains suggest that low-quality and high-quality cancer care delivery may coexist within Veterans Affairs hospitals.
    Medical Care 11/2014; 53(1). DOI:10.1097/MLR.0000000000000257 · 3.23 Impact Factor
Show more

Similar Publications