Conference PaperPDF Available

Fantasy planning: the gap between systems of safety and safety of systems

Authors:

Abstract

Numerous man-made disasters have revealed fantasy plans—safety artefacts which do not represent the reality of operational risk. In contrast to “drift”, where operations gradually become less safe, fantasy planning describe protections that have never been fully implemented, understood, or operated as intended. The theory of fantasy planning originally comes from sociologist Lee Clarke. Clarke described how the oil spill contingency plan for the Port of Valdez, Alaska, claimed that approximately 130 k barrels of oil could be recovered from the sea after a spill. This was tested in 1989, when the Exxon Valdez tanker ran aground, spilling 260 k barrels of oil. Despite the plan’s claims, no amount of oil even close to that amount had ever been recovered from open waters. In the city of San Bruno, California, 2010, a buried gas pipeline owned by Pacific Gas and Electric (PG&E) ruptured in a massive explosion. Hayes and Hopkins in their book 'Nightmare Pipeline Failures' noted that despite PG&E’s strong commitment to safety and use of bespoke risk modelling techniques, PG&E’s understanding of risk was not aligned to “actual” field risk. Both the oil spill contingency plan and the IMS had little grounding in reality. They were, in part, fantasies—but not deliberately by people. Most concerning is that sometimes risk protections may never be implemented or capable of being effective. This paper explores why, contrary to efforts towards safety, organisations create systems that describe a physical reality that may never exist. It also speculates on ways to close the gap between safety systems and operational reality. Ultimately, this paper seeks to sensitise risk practitioners to the problem of fantasy planning, and the vulnerability it creates.
A preview of the PDF is not available
... Consider also how symbolic artefacts can contribute to risk creep. As we argued previously (Hutchinson et al., 2018), symbolic artefacts can facilitate the justification of risks and decisions by giving assumptions or beliefs an observable form (Ross et al., 2016). In the case of responding to large oil spills in the Port of Valdez, the emergency plan provided the illusion of control over these problems (Clarke, 1999). ...
Article
Full-text available
Safety Management Systems are developed to help manage occupational risk, but they can also increase an organisation’s exposure to risk. This contradictory effect may happen when written artefacts (plans, risk assessments etc.) enable work to happen by encouraging a belief that the risks have been managed, when in reality they have not been. In this paper we introduce the term “enabling device” to cover the situation where a written artefact facilitates the commencement of work. We explore how enabling devices can become excessively symbolic, where they facilitate work to commence even when they may be decoupled from the issues they were designed to manage. We argue that highly symbolic artefacts acting in their enabling function: a) become more speculative than functional, b) make assumptions and beliefs “appear more real” by giving them an observable form, c) fill a need for people to solve issues without actually having to solve the issue, and d) increasingly become the unit of management instead of the issues and then take on a life of their own. This work suggests that practitioners should more critically evaluate the often invisible and potentially pervasive symbolism vested in safety artefacts to direct effective and sustainable risk interventions.
... Crisis management practice has been accused of producing "fantasy documents" (Bowen, 2008;Clarke, 1999;Hutchinson et al., 2018;McConnell and Drennan, 2006), and crisis and exercise evaluation reports are no exception (Birkland, 2009;Sinclair et al., 2012). The latter Expectations of crisis management evaluations criticism refers to the absence of an honest attempt to learn after an event, and claims that most documents are created for rhetorical purposes for certain audiences. ...
Article
Purpose: This paper seeks to overcome the mismatch between evaluation reports and the expectations of the target audience, by identifying crisis management professionals' expectations. Design/methodology/approach: An adapted stakeholder information analysis was used to survey the expectations of 84 crisis management professionals in the Netherlands. A general inductive analysis was applied to qualitative data, from which five main themes emerged: purpose; object or focus; reasoning and (meta) analysis; result or conclusion, and the overall design of the evaluation. Findings: Currently, evaluation reports are seen merely as a way to share experience and support thinking about how to avoid repeating mistakes. However, most respondents expected them to contribute to learning and support improvement. They should provide actionable feedback on what could be done differently or better, and indicate how this can be achieved. Respondents emphasised the need to focus on the human factor and not neglect the context. The wide variety of views underlined that it is difficult to create one evaluation product that meets all expectations. Research limitations/implications: Although some major themes clearly emerged from the data, it is unclear how they relate to each other, and their relative importance. In addition, no distinction is made between evaluations of real events and simulations. Practical implications: Users should be encouraged to provide input into the evaluation process by clarifying their needs and how they use evaluation reports. Originality/value: This research is the first attempt to identify user expectations regarding what constitutes an effective evaluation.
Article
Shortcomings of incident‐based metrics such as Total Recordable Incident Frequency Rate (TRIFR) are well‐documented, including the lack of standardization, construct validity, statistical power, and predictive power. A low TRIFR is also no assurance against legal liability. There is considerable overlap between the research literature on safety as the presence of capacities to make things go well, and jurisprudence in labor and workplace safety law. In this paper we suggest an index that merges the two, measuring the capacities to acquire and maintain safety knowledge; to understand the nature of operations; to resource for safety; to respond to risks; to demonstrate engagement and compliance; and the capacity for assurance.
Article
The Hyogo and Sendai frameworks stress that risk assessments should inform plans. However, it is unclear how, in theory, risk assessments and plans are conceptually related and how, in practice, they inform each other. This study aims to fill this knowledge gap. Conceptually, it proposes an explicit link between risk assessments and plans via the capability of the responding organisation and its effect on the severity of consequences. By including capability descriptions in risk assessments and plans, the strength of knowledge increases. This, in turn, makes it easier to use the output from one document as input to the other. Empirically, the study focuses on the current Swedish practice and on an analysis of risk and vulnerability assessments and plans prepared by 25 local municipalities, and interviews with representatives from nine municipalities. The results show that the examined documents do not describe capability in a way that makes it possible to relate risk and vulnerability assessments and plans to each other. Moreover, no other link between the documents is evident. Interviews confirm that the activities do not inform each other, partly due to a lack of resources and a poor understanding of the work. Instead, the efforts seem to focus on compliance; to simply submit the required documentation. Such a focus could hinder reducing losses from disasters. To strengthen current practice, this study recommends including capability descriptions in risk assessments and plans in order to strengthen their knowledge foundation and facilitate the integration of the activities.
Article
Full-text available
The safety management literature describes two distinct modes through which safety is achieved. These can be described as safety management through centralized control, or safety management through guided adaptability. Safety management through centralized control, labelled by Hollnagel as ‘Safety-I’, aims to align and control the organization and its people through the central determination of what is safe. Safety management through guided adaptability, or ‘Safety-II’, aims to enable the organization and its people to safely adapt to emergent situations and conditions. Safety-II has been presented as a paradigm shift in safety theory, but it has created practical difficulties for safety professional practice. In this paper, we define the two modes of safety management and explain the challenges in changing the role of a safety professional to support Safety-II. When should safety professionals re-enforce alignment, and when should they support frontline adaptations? We outline specific activities for safety professionals to adopt in their role to move towards a guided adaptability mode of safety management. This will move the safety professional further towards their fundamental responsibility – ‘to create foresight about the changing shape of risk, and facilitate action, before people are harmed.’
Article
Full-text available
We review the progress of naturalistic decision making (NDM) in the decade since the first conference on the subject in 1989. After setting out a brief history of NDM we identify its essential characteristics and consider five of its main contributions: recognition-primed decisions, coping with uncertainty, team decision making, decision errors, and methodology. NDM helped identify important areas of inquiry previously neglected (e.g. the use of expertise in sizing up situations and generating options), it introduced new models, conceptualizations, and methods, and recruited applied investigators into the field. Above all, NDM contributed a new perspective on how decisions (broadly defined as committing oneself to a certain course of action) are made. NDM still faces significant challenges, including improvement of the quantity and rigor of its empirical research, and confirming the validity of its prescriptive models. Copyright © 2001 John Wiley & Sons, Ltd.
Article
Full-text available
Safety analysis frequently relies on human estimates of the likelihood of specific events. For this purpose, the opinions of experts are given greater weight than the opinions of non-experts. Combinations of individual judgements are given greater weight than judgements made by a lone expert. Various authors advocate specific techniques for eliciting and combining these judgements. All of these factors – the use of experts, the use of multiple opinions, and the use of elicitation and combination techniques – serve to increase subjective confidence in the safety analysis. But is this confidence justified? Do the factors increase the actual validity of the analysis in proportion to the increase in subjective confidence?
Article
Full-text available
Safety activities may provide assurance of safety even where such assurance is unwarranted. This phenomenon – which we will call “probative blindness” – is evident both in hindsight analysis of accidents and in the daily practice of safety work. The purpose of this paper is to describe the phenomenon of probative blindness. We achieve this by distinguishing probative blindness from other phenomena, identifying historical instances of probative blindness, and discussing characteristics and causes associated with these instances. The end product is an explanation of the features of probative blindness suitable for investigating the probative value of current safety activities, and ultimately for reducing the occurrence of probative blindness.
Article
Full-text available
Quantitative risk analysis (QRA) is widely applied in several industries as a tool to improve safety, as part of design, licensing or operational processes. Nevertheless, there is much less academic research on the validity and validation of QRA, despite their importance both for the science of risk analysis and with respect to its practical implication for decision-making and improving system safety. In light of this, this paper presents a review focusing on the validity and validation of QRA in a safety context. Theoretical, methodological and empirical contributions in the scientific literature are reviewed, focusing on three questions. Which theoretical views on validity and validation of QRA can be found? Which features of QRA are useful to validate a particular QRA, and which frameworks are proposed to this effect? What kinds of claims are made about QRA, and what evidence is available for QRA being valid for the stated purposes? A discussion follows the review, focusing on the available evidence for the validity of QRA and the effectiveness of validation methods.
Article
Full-text available
Risk assessment and management was established as a scientific field some 30-40 years ago. Principles and methods were developed for how to conceptualise, assess and manage risk. These principles and methods still represent to a large extent the foundation of this field today, but many advances have been made, linked to both the theoretical platform and practical models and procedures. The purpose of the present invited paper is to perform a review of these advances, with a special focus on the fundamental ideas and thinking on which these are based. We have looked for trends in perspectives and approaches, and we also reflect on where further development of the risk field is needed and should be encouraged. The paper is written for readers with different types of background, not only for experts on risk.
Article
The use of the risk matrix as a hazard management tool is a significant issue for industry due to (i) documented pitfalls and (ii) an attention to negative outcomes. Question sets inspired by the Functional Resonance Analysis Method (FRAM) were evaluated to understand if an alternative approach offered a more effective means of risk assessment, and thus was of greater value to both the stakeholders and the organisation. Iterative evaluations were limited to four work systems within a manufacturing environment. A FRAM based approach meant that total systems were considered in addition to the hazards and controls within them, and in doing so safety and productivity were assessed as one activity. In contrast, comparative risk matrix assessments did not provide enough requisite variety to understand the complete picture, only specific hazards and their controls in isolation. For each of the systems investigated work-as-done had been adapted in response to introduced variability to maintain success. The FRAM perspective provided by the use of the question sets afforded the identification of higher order controls though collaboration with all stakeholders. It is concluded that developed narratives provide deeper learnings of system performance in the management of variability.
Article
Using interview data (n = 30) of residents in Calhoun County, Alabama, we examine the meaning respondents give to the objects (shelter-in-place kits) disseminated to residents in order to manage risk of accidents from Anniston Army Depot’s Chemical Weapons storage and incinerator activities. Our research builds on previous research examining how organizations engage in ‘fantasy planning,’ where they create plans that objectively do not increase safety but do create the perception of safety within the organization and especially the public.