Measuring Quality Improvement in Public Health: The Development and Psychometric Testing of a QI Maturity Tool

Muskie School of Public Service, University of Southern Maine, Portland, ME 04101, USA.
Evaluation & the Health Professions (Impact Factor: 1.91). 02/2012; 35(2):119-47. DOI: 10.1177/0163278711433065
Source: PubMed


There is growing interest and investment in improving the quality of public health services and outcomes. Following the lead of other sectors, efforts are underway to introduce systematic quality improvement (QI) tools and approaches to state and local public health agencies. Little is known, however, about how to describe and reliably measure the level of QI maturity within a public health agency. The authors describe the development of a QI Maturity Tool using research from the fields of organizational design, psychology, health care, and complexity theory. The 37-item assessment tool is based on four quality domains derived from the literature: (a) organizational culture, (b) capacity and competency, (c) practice, and (d) alignment and spread. The tool was designed to identify features of an organization that may be enhancing or impeding QI; monitor the impact of efforts to create a more favorable environment for QI; and define potential cohorts of public health agencies for evaluation purposes. The article presents initial steps in testing and validating the QI Maturity Tool including: (a) developing a theoretical framework, (b) assuring face and content validity, (c) determining the tool's reliability based on estimates of internal consistency, (d) assessing the dimensionality, and (f) determining the construct validity of the instrument. The authors conclude that there is preliminary evidence that the QI Maturity Tool is a promising instrument. Further work is underway to explore whether self-reported survey results align with an agency's actions and the products of their QI efforts.

29 Reads
  • [Show abstract] [Hide abstract]
    ABSTRACT: This article focuses on local health departments (LHDs) that are advanced in accreditation and quality improvement (QI) efforts and the barriers and facilitators associated with sustaining improvements and building an organizational culture of QI. To understand the barriers and facilitators associated with building and sustaining progress toward a QI culture in LHDs. Quantitative data from a self-reporting survey and qualitative data from telephone interviews. Twenty-two LHDs across the United States responded to the survey. Ten of the 22 LHD respondents participated in telephone interviews. QI lead staff at LHDs that are advanced in accreditation preparation and QI. Self-reported LHD survey ratings against indicators for a QI culture, and the identified barriers and facilitators around sustaining QI initiatives. Of the 6 domains of a QI culture measured in the survey, the percentages of respondents that scored themselves highly to at least 1 indicator in each domain are as follows: leadership commitment (100%); employee empowerment (100%); teamwork and collaboration (100%); continuous process improvement (86%); customer focus (72%); and QI infrastructure (64%). Qualitative data from 10 telephone interviews revealed that key barriers to sustaining progress around QI included staff turnover, budget cuts, and major crises or events that arise as priority. Key facilitators included leadership commitment, accreditation, and dedication of resources and staff time to QI. When engaging in QI, LHDs should consider investing efforts in gaining leadership support and dedicating staff time early in the QI journey to ensure that QI efforts and initiatives are sustained. Local health departments interested in developing a QI culture should also consider pursuing accreditation, as it provides a structured framework for continuous improvement. They should also actively develop QI knowledge and skills among all staff members to minimize the negative impact of staff turnover.
    No preview · Article · Jan 2014 · Journal of public health management and practice: JPHMP
  • [Show abstract] [Hide abstract]
    ABSTRACT: CONTEXT:: There has been an extensive investment in building public health organizational capacity to improve performance and prepare for accreditation. An evolving perspective has focused not only on the practice of quality improvement (QI) within the health department but also upon the extent the culture of QI is embraced within the agency. OBJECTIVE:: No studies have examined the current national baseline of QI culture implementation, nor estimated the degree of QI sophistication local health departments (LHDs) have attained. We attempt to fill this void by aligning the findings from the QI module of the National Association of County & City Health Officials (NACCHO) 2010 Profile of LHDs against the constructs defined by the QI Maturity Tool and the NACCHO QI Roadmap (Roadmap to a Culture of Quality Improvement). DESIGN:: Specific questions regarding QI activities from the 2010 Profile Study QI module were used to assign responding LHDs to stages within the Roadmap. We also used data from the QI Maturity Tool administered to all LHDs in the 16 participating Multi-State Learning Collaborative states in 2010 and 2011. On the basis of this matched set, we applied the summative domain scores algorithm, classified agencies into 1 of 5 groups, compared our findings with those of the NACCHO survey, and aligned our categories to those of the Roadmap. RESULTS:: Nearly 80% of LHDs classified using the NACCHO Profile data were assigned to group 3 or 4 versus 48% using the QI Maturity Tool. Results from the cross-tabulations of the matched data set between the QI Maturity Tool classifications and the NACCHO Profile classifications revealed exact alignment 30% of the time. Forty-nine of 163 agencies were classified in the same grouping in both schemata. In addition, 84% of the agencies were classified within 1 neighboring category. CONCLUSIONS:: The results revealed that half, if not most, LHDs fall within the middle categories of QI maturity and sophistication, regardless of which classification system was deployed.
    No preview · Article · Apr 2013 · Journal of public health management and practice: JPHMP
  • [Show abstract] [Hide abstract]
    ABSTRACT: Use of a widely accepted quality assurance tool is an essential procedure of effective and result-oriented quality management in the rehabilitation sector, and generally in health care and social services, but is still lacking in Greece. OBJECTIVE This study aims to explore to what extent a Quality Assurance System in Rehabilitation (QASR) in the Greek setting could respond to the needs for quality evaluation of the facilities for people with a disability and to discuss possibilities of its use in rehabilitation organizations, sites and hospitals. METHODS The European Quality in Social Services (EQUASS) Assurance self-assessment questionnaire was officially translated and used as the basis for the new tool, which consisted of 110 questions in 11 sections on development and 6 questions on its evaluation. This tool was tested in 15 specialized centers. RESULTS The study received a high (93.75%) response rate. Overall score ranged from 11% to one perfect 100%; 53.3% of the facilities fell short of the preset qualification standards, while 4 (26.7%) were qualified for level-1 accreditation. Evaluation of the QASR questionnaire for the function of the rehabilitation facilities for the disabled was extremely positive. CONCLUSIONS The EQUASS assurance-based Greek QASR has received proper attention in its first implementation and it was shown promising to assess the needs of sites that would like to improve their services. The next steps are to establish its validity and reliability so that it can significantly emerge as the standard system for guiding policy in the rehabilitation sector in Greece.
    No preview · Article · Jul 2014 · Disability and Health Journal
Show more