Measuring and Comparing Safety Climate in Intensive Care Units
ABSTRACT Learning about the factors that influence safety climate and improving the methods for assessing relative performance among hospital or units would improve decision-making for clinical improvement.
To measure safety climate in intensive care units (ICU) owned by a large for-profit integrated health delivery systems; identify specific provider, ICU, and hospital factors that influence safety climate; and improve the reporting of safety climate data for comparison and benchmarking.
We administered the Safety Attitudes Questionnaire (SAQ) to clinicians, staff, and administrators in 110 ICUs from 61 hospitals.
A total of 1502 surveys (43% response) from physicians, nurses, respiratory therapists, pharmacists, mangers, and other ancillary providers.
The survey measured safety climate across 6 domains: teamwork climate; safety climate; perceptions of management; job satisfaction; working conditions; and stress recognition. Percentage of positive scores, mean scores, unadjusted random effects, and covariate-adjusted random effect were used to rank ICU performance.
The cohort was characterized by a positive safety climate. Respondents scored perceptions of management and working conditions significantly lower than the other domains of safety climate. Respondent job type was significantly associated with safety climate and domain scores. There was modest agreement between ranking methodologies using raw scores and random effects.
The relative proportion of job type must be considered before comparing safety climate results across organizational units. Ranking methodologies based on raw scores and random effects are viable for feedback reports. The use of covariate-adjusted random effects is recommended for hospital decision-making.
- SourceAvailable from: John Bryan Sexton
[Show abstract] [Hide abstract]
- "However, seven of the 34 items showed high missing value rates (9.7% to 53.6%, primarily due to “not applicable” responses). Previous studies had no items whose missing values exceeded 13% [5,13,29,33]. Most MVs were on items related to the perception of management factor with items focusing on hospital management having far more MVs than those focusing on unit management. "
ABSTRACT: Improving patient safety has become a major focus of clinical care and research over the past two decades. An institution's patient safety climate represents an essential component of ensuring a safe environment and thereby can be vital to the prevention of adverse events. Covering six patient safety related factors, the Safety Attitudes Questionnaire (SAQ) is a validated and widely used instrument to measure the patient safety climate in clinical areas. The objective of this study was to assess the psychometric properties of the German language version of the SAQ. A survey was carried out in two University Hospitals in Switzerland in autumn 2009 where the SAQ was distributed to a sample of 406 nurses and physicians in medical and surgical wards. Following the American Educational Research Association guidelines, we tested the questionnaire validity by levels of evidence: content validity, internal structure and relations to other variables. Confirmatory factor analysis was used to examine factor structure. Cronbach's alphas and inter-item correlations were calculated to examine internal consistency reliability. A total of 319 questionnaires were completed representing an overall response rate of 78.6%. For three items, the item content validity index was <0.75. Confirmatory factor analysis showed acceptable model fit (RMSEA = 0.045; CFI = 0.944) for the six-factor model. Additional exploratory factor analysis could not identify a better factor model. SAQ factor scores showed positive correlations with the Safety Organizing Scale (r = .56 - .72). The SAQ German version showed moderate to strong internal consistency reliability indices (Cronbach alpha = .65 - .83). The German language version of the SAQ demonstrated acceptable to good psychometric properties and therefore shows promise to be a sound instrument to measure patient safety climate in Swiss hospital wards. However, the low item content validity and large number of missing responses for several items suggest that improvements and adaptations in translation are required for select items, especially within the perception of management scale. Following these revisions, psychometric properties should reassessed in a randomly selected sample and hospitals and departments prior to use in Swiss hospital settings.BMC Health Services Research 09/2013; 13(1):347. DOI:10.1186/1472-6963-13-347 · 1.71 Impact Factor
[Show abstract] [Hide abstract]
- "Subscales within the SAQ include Teamwork, Safety Climate, Job Satisfaction, Perceptions of Management, Working Conditions, and Stress Recognition. The SAQ has been used to assess opportunities for quality improvement in obstetrical settings , intensive care units [5,6], within single institutions [7-9], in multicentre studies [10-12], children’s hospitals , the Veteran’s Administration  and increasingly in international settings [8,15]. The SAQ has been well-described [16-18]. "
ABSTRACT: Our previous analyses using the Stress Recognition subscale of the Safety Attitudes Questionnaire (SAQ) resulted in significant effect estimates with equally opposing explanations. We suspected construct validity issues and investigated such using our own data and correlation matrices of previous published studies. The correlation matrices for each of the SAQ subscales from two previous studies by Speroff and Taylor were replicated and compared. The SAS Proc Factor procedure and the PRIORS = SMC option were used to perform Common Factor Analysis. The correlation matrices of both studies were very similar. Teamwork, Safety Climate, Job Satisfaction, Perceptions of Management and Working Conditions were well-correlated. The correlations ranged from 0.53 to 0.76. For Stress Recognition correlations ranged from -0.15 to 0.03. Common Factor Analysis confirmed the isolation of Stress Recognition. CFA returned a strong one-factor model that explained virtually all of the communal variance. Stress Recognition loaded poorly on this factor in both instances, and the CFA indicated that 96.4-100.0% of the variance associated with Stress Recognition was unique to that subscale, and not shared with the other 5 subscales. We conclude that the Stress Recognition subscale does not fit into the overall safety climate construct the SAQ intended to reflect. We recommend that this domain be omitted from overall safety climate scale score calculations, and clearly identified as an important yet distinct organizational construct. We suggest that this subscale be investigated for its true meaning, characterized as such, and findings conveyed to SAQ end users. We make no argument against Stress Recognition as an important organizational metric, rather we suggest that as a stand-alone construct its current packaging within the SAQ may be misleading for those intent on intervention development and evaluation in healthcare settings if they interpret Stress Recognition results as emblematic of safety climate.BMC Research Notes 07/2013; 6(1):302. DOI:10.1186/1756-0500-6-302
[Show abstract] [Hide abstract]
- "The validity and reliability of the SAQ has been documented in many countries such as the United States (Modak et al., 2007; Sexton et al., 2006a), Norway (Deilkas and Hofoss, 2008), Turkey (Kaya et al., 2010), Sweden (Nordé n-Hä gg et al., 2010) and the Netherlands (Poley et al., 2011). Furthermore, the instrument has also been adapted for the use in different settings and specializations such as intensive care units (ICUs) (France et al., 2010; Pronovost et al., 2008; Poley et al., 2011; Sexton et al., 2011), operating rooms (Carney et al., 2010), general inpatient settings such as medical and surgical wards (Deilkas and Hofoss, 2008), ambulatory clinics (Holden et al., 2009; Modak et al., 2007), pharmacies (Nordé n-Hä gg et al., 2010), and labour and delivery units (Sexton et al., 2006b). Also the SAQ has been extensively used to explore the relationship between safety climate scores and patient outcomes, e.g. "
ABSTRACT: Patient safety is fundamental to healthcare quality. Attention has recently focused on the patient safety culture of an organisation and its impact on patient outcomes. A strong safety climate appears to be an essential condition for safe patient care in the hospital. A number of instruments are used to measure this patient safety climate or culture. The Safety Attitudes Questionnaire is a validated, widely used instrument to investigate multiple dimensions of safety climate at the clinical level in a variety of inpatient and outpatient settings. The purpose of this study is to explore the face- and content validity and the internal consistency of the Safety Attitudes Questionnaire in a large Belgian academic medical center. The translation into Dutch was done by three researchers. A panel of fifteen Dutch speaking experts evaluated the translation and its content validity. Content validity was quantified by the content validity index (CVI) and a modified kappa index. Face validity was evaluated by two nurses and two physicians who assessed the Dutch version of the SAQ. A cross-sectional design was used to test internal consistency of the SAQ items by calculating Cronbach's alpha and corrected item-total correlations. Twenty-three of the 33 SAQ items showed excellent and seven items showed good content validity. One item had a fair kappa value (item 20) and two items had a low content validity index (items 15 and 16). The average CVI of the total scale was 0.83 and ranged from 0.55 to 0.97 for the six subscales. The face-validity was good with no fundamental remarks given. The SAQ's overall Cronbach's alpha was 0.9 and changed minimally when removing items. The item-total correlations ranged from 0.10 to 0.63, no single items were strongly correlated with the sum of the other items. We conclude that in this study the Dutch version of the Safety Attitudes Questionnaire showed acceptable to good psychometric properties. In line with previous evidence, this instrument seems to be an acceptable to adequate tool to evaluate the safety climate.International journal of nursing studies 03/2012; 49(3):327-37. DOI:10.1016/j.ijnurstu.2011.10.002 · 2.90 Impact Factor