Assessing hospital disaster preparedness: A comparison of an on-site survey, directly observed drill performance, and video analysis of teamwork

Department of Emergency Medicine, Harbor-UCLA Medical Center, Los Angeles, CA, USA.
Annals of emergency medicine (Impact Factor: 4.33). 02/2008; 52(3):195-201, 201.e1-12. DOI: 10.1016/j.annemergmed.2007.10.026
Source: PubMed

ABSTRACT There is currently no validated method for assessing hospital disaster preparedness. We determine the degree of correlation between the results of 3 methods for assessing hospital disaster preparedness: administration of an on-site survey, drill observation using a structured evaluation tool, and video analysis of team performance in the hospital incident command center.
This was a prospective, observational study conducted during a regional disaster drill, comparing the results from an on-site survey, a structured disaster drill evaluation tool, and a video analysis of teamwork, performed at 6 911-receiving hospitals in Los Angeles County, CA. The on-site survey was conducted separately from the drill and assessed hospital disaster plan structure, vendor agreements, modes of communication, medical and surgical supplies, involvement of law enforcement, mutual aid agreements with other facilities, drills and training, surge capacity, decontamination capability, and pharmaceutical stockpiles. The drill evaluation tool, developed by Johns Hopkins University under contract from the Agency for Healthcare Research and Quality, was used to assess various aspects of drill performance, such as the availability of the hospital disaster plan, the geographic configuration of the incident command center, whether drill participants were identifiable, whether the noise level interfered with effective communication, and how often key information (eg, number of available staffed floor, intensive care, and isolation beds; number of arriving victims; expected triage level of victims; number of potential discharges) was received by the incident command center. Teamwork behaviors in the incident command center were quantitatively assessed, using the MedTeams analysis of the video recordings obtained during the disaster drill. Spearman rank correlations of the results between pair-wise groupings of the 3 assessment methods were calculated.
The 3 evaluation methods demonstrated qualitatively different results with respect to each hospital's level of disaster preparedness. The Spearman rank correlation coefficient between the results of the on-site survey and the video analysis of teamwork was -0.34; between the results of the on-site survey and the structured drill evaluation tool, 0.15; and between the results of the video analysis and the drill evaluation tool, 0.82.
The disparate results obtained from the 3 methods suggest that each measures distinct aspects of disaster preparedness, and perhaps no single method adequately characterizes overall hospital preparedness.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The State University of New York at Downstate (SUNY) conducted a web-based long-distance tabletop drill (LDTT) designed to identify vulnerabilities in safety, security, communications, supplies, incident management, and surge capacity for a number of hospitals preceding the 2010 FIFA World Cup. The tabletop drill simulated a stampede and crush-type disaster at the Green Point Stadium in Cape Town, South Africa in anticipation of 2010 FIFA World Cup. The LDTT, entitled "Western Cape-Abilities", was conducted between May and September 2009, and encompassed nine hospitals in the Western Cape of South Africa. The main purpose of this drill was to identify strengths and weaknesses in disaster preparedness among nine state and private hospitals in Cape Town, South Africa. These hospitals were tasked to respond to the ill and injured during the 2010 World Cup. This LDTT utilized e-mail to conduct a 10-week, scenario-based drill. Questions focused on areas of disaster preparedness previously identified as standards from the literature. After each scenario stimulus was sent, each hospital had three days to collect answers and submit responses to drill controllers via e-mail. Data collected from the nine participating hospitals met 72% (95%CI = 69%-75%) of the overall criteria examined. The highest scores were attained in areas such as equipment, with 78% (95%CI = 66%-86%) positive responses, and development of a major incident plan with 85% (95% CI = 77%-91%) of criteria met. The lowest scores appeared in the areas of public relations/risk communications; 64% positive responses (95% CI = 56%-72%), and safety, supplies, fire and security meeting also meeting 64% of the assessed criteria (95% CI = 57%-70%). Surge capacity and surge capacity revisited both met 76% (95% CI = 68%-83% and 68%-82%, respectively). This assessment of disaster preparedness indicated an overall good performance in categories such as hospital equipment and development of major incident plans, but improvement is needed in hospital security, public relations, and communications ahead of the 2010 FIFA World Cup.
    Prehospital and disaster medicine: the official journal of the National Association of EMS Physicians and the World Association for Emergency and Disaster Medicine in association with the Acute Care Foundation 06/2011; 26(3):192-5. DOI:10.1017/S1049023X11006443
  • [Show abstract] [Hide abstract]
    ABSTRACT: In 2001, a survey of Canadian emergency departments indicated significant deficiencies in disaster preparedness. Since then, there have been efforts on the part of Provincial governments to remedy this situation. This survey repeats the original study with minor modifications to determine if there has been improvement. The Hospital Emergency Readiness Overview study demonstrates that despite improvements, there remain gaps in Canadian healthcare facility readiness for disaster, specifically one involving contaminated patients. It also highlights the lack of any standardized assessment of healthcare facilities' chemical, biological, radiological, or nuclear readiness.
    Prehospital and disaster medicine: the official journal of the National Association of EMS Physicians and the World Association for Emergency and Disaster Medicine in association with the Acute Care Foundation 06/2011; 26(3):159-65. DOI:10.1017/S1049023X11006212
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: STUDY OBJECTIVE: Efficient management and allocation of scarce medical resources can improve outcomes for victims of mass casualty events. However, the effectiveness of specific strategies has never been systematically reviewed. We analyze published evidence on strategies to optimize the management and allocation of scarce resources across a wide range of mass casualty event contexts and study designs. METHODS: Our literature search included MEDLINE, Scopus, EMBASE, Cumulative Index to Nursing and Allied Health Literature, Global Health, Web of Science, and the Cochrane Database of Systematic Reviews, from 1990 through late 2011. We also searched the gray literature, using the New York Academy of Medicine's Grey Literature Report and key Web sites. We included both English- and foreign-language articles. We included studies that evaluated strategies used in actual mass casualty events or tested through drills, exercises, or computer simulations. We excluded studies that lacked a comparison group or did not report quantitative outcomes. Data extraction, quality assessment, and strength of evidence ratings were conducted by a single researcher and reviewed by a second; discrepancies were reconciled by the 2 reviewers. Because of heterogeneity in outcome measures, we qualitatively synthesized findings within categories of strategies. RESULTS: From 5,716 potentially relevant citations, 74 studies met inclusion criteria. Strategies included reducing demand for health care services (18 studies), optimizing use of existing resources (50), augmenting existing resources (5), implementing crisis standards of care (5), and multiple categories (4). The evidence was sufficient to form conclusions on 2 strategies, although the strength of evidence was rated as low. First, as a strategy to reduce demand for health care services, points of dispensing can be used to efficiently distribute biological countermeasures after a bioterrorism attack or influenza pandemic, and their organization influences speed of distribution. Second, as a strategy to optimize use of existing resources, commonly used field triage systems do not perform consistently during actual mass casualty events. The number of high-quality studies addressing other strategies was insufficient to support conclusions about their effectiveness because of differences in study context, comparison groups, and outcome measures. Our literature search may have missed key resource management and allocation strategies because of their extreme heterogeneity. Interrater reliability was not assessed for quality assessments or strength of evidence ratings. Publication bias is likely, given the large number of studies reporting positive findings. CONCLUSION: The current evidence base is inadequate to inform providers and policymakers about the most effective strategies for managing or allocating scarce resources during mass casualty events. Consensus on methodological standards that encompass a range of study designs is needed to guide future research and strengthen the evidence base. Evidentiary standards should be developed to promote consensus interpretations of the evidence supporting individual strategies.
    Annals of emergency medicine 03/2013; 61(6). DOI:10.1016/j.annemergmed.2013.02.005 · 4.33 Impact Factor