National Release of the Nursing Home Quality Report Cards: Implications of Statistical Methodology for Risk Adjustment

Department of Medicine, University of California, Irvine, CA 92697, USA.
Health Services Research (Impact Factor: 2.78). 03/2009; 44(1):79-102. DOI: 10.1111/j.1475-6773.2008.00910.x
Source: PubMed


To determine how alternative statistical risk-adjustment methods may affect the quality measures (QMs) in nursing home (NH) report cards. DATA SOURCES/STUDY SETTINGS: Secondary data from the national Minimum Data Set files of 2004 and 2005 that include 605,433 long-term residents in 9,336 facilities.
We estimated risk-adjusted QMs of decline in activities of daily living (ADL) functioning using classical, fixed-effects, and random-effects logistic models. Risk-adjusted QMs were compared with each other, and with the published QM (unadjusted) in identifying high- and low-quality facilities by either the rankings or 95 percent confidence intervals of QMs.
Risk-adjusted QMs showed better overall agreement (or convergent validity) with each other than did the unadjusted versus each adjusted QM; the disagreement rate between unadjusted and adjusted QM can be as high as 48 percent. The risk-adjusted QM derived from the random-effects shrinkage estimator deviated nonrandomly from other risk-adjusted estimates in identifying the best 10 percent facilities using rankings.
The extensively risk-adjusted QMs of ADL decline, even when estimated by alternative statistical methods, show higher convergent validity and provide more robust NH comparisons than the unadjusted QM. Outcome rankings based on ADL decline tend to show lower convergent validity when estimated by the shrinkage estimator rather than other statistical methods.

Download full-text


Available from: Laurent Glance,
  • Source
    • "Each random-effects model adjusted for resident characteristics described above and model estimates were used to predict risk scores for all residents associated with a measure. Predicted scores were then aggregated to the nursing home level to calculate average expected process/outcome rates for all nursing homes, which in turn were used to calculate the final riskadjusted process/outcome rates using the observed-to-expected comparison approach described in detail in a previous methodological study (Li et al., 2009). After facility-level risk-adjusted rates were calculated, we ran bivariate Pearson correlation analyses among risk-adjusted measures and all types of consumer ratings. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Several states are currently collecting and publicly reporting nursing home resident and/or family member ratings of experience with care in an attempt to improve person-centered care in nursing homes. Using the 2008 Maryland nursing home family survey reports and other data, this study performed both facility- and resident-level analyses, and estimated the relationships between family ratings of care and several long-term care quality measures (pressure ulcers, overall and potentially avoidable hospitalizations, and mortality) after adjustment for resident characteristics. We found that better family evaluations of overall and specific aspects of care may be associated with reduced rates of risk-adjusted measures at the facility level (range of correlation coefficients: -.01 to -.31). Associations of overall experience ratings tended to persist after further adjustment for common nursing home characteristics such as nurse staffing levels. We conclude that family ratings of nursing home care complement other types of performance measures such as risk-adjusted outcomes. © The Author(s) 2015.
    Medical Care Research and Review 07/2015; DOI:10.1177/1077558715596470 · 2.62 Impact Factor
  • Source
    • "Roy and Mor drew attention to measurement of quality and the importance of unbiased coding in data collection [27]. Li et al. showed the importance of adjusting for individual and facility-level characteristics when describing nursing home care quality [28]. Austin and Reeves, in a study assessing models of hospital care quality, concluded that the c-statistic was of little use to assess model fit although widely-accepted [29]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background This paper considers approaches to the question “Which long-term care facilities have residents with high use of acute hospitalisations?” It compares four methods of identifying long-term care facilities with high use of acute hospitalisations by demonstrating four selection methods, identifies key factors to be resolved when deciding which methods to employ, and discusses their appropriateness for different research questions. Methods OPAL was a census-type survey of aged care facilities and residents in Auckland, New Zealand, in 2008. It collected information about facility management and resident demographics, needs and care. Survey records (149 aged care facilities, 6271 residents) were linked to hospital and mortality records routinely assembled by health authorities. The main ranking endpoint was acute hospitalisations for diagnoses that were classified as potentially avoidable. Facilities were ranked using 1) simple event counts per person, 2) event rates per year of resident follow-up, 3) statistical model of rates using four predictors, and 4) change in ranks between methods 2) and 3). A generalized mixed model was used for Method 3 to handle the clustered nature of the data. Results 3048 potentially avoidable hospitalisations were observed during 22 months’ follow-up. The same “top ten” facilities were selected by Methods 1 and 2. The statistical model (Method 3), predicting rates from resident and facility characteristics, ranked facilities differently than these two simple methods. The change-in-ranks method identified a very different set of “top ten” facilities. All methods showed a continuum of use, with no clear distinction between facilities with higher use. Conclusion Choice of selection method should depend upon the purpose of selection. To monitor performance during a period of change, a recent simple rate, count per resident, or even count per bed, may suffice. To find high–use facilities regardless of resident needs, recent history of admissions is highly predictive. To target a few high-use facilities that have high rates after considering facility and resident characteristics, model residuals or a large increase in rank may be preferable.
    BMC Medical Research Methodology 07/2014; 14(1):93. DOI:10.1186/1471-2288-14-93 · 2.27 Impact Factor
  • Source
    • "Dapatan juga menunjukkan bahawa stail berfikir eksekutif dominan dalam kalangan pelajar kerana stail ini adalah stail yang diterima dan diberi ganjaran dalam sesebuah institusi pendidikan seperti yang telah diketengahkan oleh Teori Governan Mental Sternberg (1997). Pernyataan ini juga turut disokong oleh Hamzan (2006), Ruslin (2007), Ju An dan Sook Yoo (2008) serta Stenberg et al. (2008) yang menyatakan bahawa stail berfikir eksekutif dominan dalam dunia pendidikan kerana pelajar dapat mengadaptasi pemikiran mereka selaras dengan hasrat institusi. Maka pihak pengurusan Kolej Kejururawatan Murni perlu memastikan agar dasar pembangunan keilmuan itu dinamik, tersebar luas dan mempunyai sistem sokongan yang mantap. "

Show more