Content uploaded by David D Woods
Author content
All content in this area was uploaded by David D Woods on Dec 21, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Behind Human Error: Cognitive Systems, Computers and Hindsight
Abstract
This report goes beyond a characterization of human error as a causal factor of accidents. It discusses the larger system within which practitioners operate and show how "blunt end" factors such as organizational processes and technology design impact the cognition and behavior of those at the "sharp end." Examples from various domains are used to illustrate deficiencies in computerized devices, which can lead to breakdowns in interaction. such as mode error. Reasons are presented for why these deficiencies as "latent failures" can exist without giving rise to accidents. Also discussed is the role of outcome knowledge in the attribution of error.
... That is, the rebound curve does not explain what was done during the stressor event and what could have been done differently to improve the outcome. In the safety science literature, this confusion between process and outcome is a wellknown barrier to understanding how complex systems actually fail or how to improve things [100]. Ultimately, this makes the rebound curve a poor basis for decision-making in many technological, organizational, and social domains. ...
... A deeper look reveals basic processes that enable resilience [44,81] and demonstrates basic patterns in how failure results when these processes break down [106,102]. Confusing the trigger event with the root cause is a common fallacy that makes it easy to scapegoat blame while avoiding the more challenging work of understanding and improving system operations [100]. ...
... Real systems often have multi-dimensional and interdependent functions that cannot be reduced to a single metric or measured on a single axis. This is especially true when functions provided by the same system compete with each other and operators face conflicting goals [100]. For example, dams store water for flood control and buffer against drought, and they also release water for electricity production, recreation, and water provision. ...
The rebound curve remains the most prevalent model for conceptualizing, measuring, and explaining resilience for engineering and community systems by tracking the functional robustness and recovery of systems over time. (It also goes by many names, including the resilience curve, the resilience triangle, and the system functionality curve, among others.) Despite longstanding recognition that resilience is more than rebound, the curve remains highly used, cited, and taught. In this article, we challenge the efficacy of this model for resilience and identify fundamental shortcomings in how it handles system function, time, dynamics, and decisions — the key elements that make up the curve. These oversimplifications reinforce misconceptions about resilience that are unhelpful for understanding complex systems and are potentially dangerous for guiding decisions. We argue that models of resilience should abandon the use of this curve and instead be reframed to open new lines of inquiry that center on improving adaptive capacity in complex systems rather than functional rebound. We provide a list of questions to help future researchers communicate these limitations and address any implications on recommendations derived from its use.
... occur when SA is lost or when mental breakdowns appear in complex tasks [11]. For instance, a 20-year study by the National Transportation Safety Board of the United States found that 80% of human errors were associated with SA problems [4]. ...
... The cost of human error is measured not only in economic terms but also in hundreds to thousands of lives lost, with far-reaching consequences. This issue applies to "multi-risk landscapes and multi-actor networks" across various industries [11,43,44]. ...
... Furthermore, some studies use the term "erroneous action" or the action of failing to produce the expected result or producing unwanted consequence [11,47]. These definitions bring us closer to the possibility that perhaps, behind the label of human error, there are deeper concepts to be explored, such as the cognitive factors of human performance. ...
Background: This study explores forklift operators’ situational awareness (SA) and human errors in logistic operations using a multiphase approach as an innovative methodology. Methods: Ethnography, eye tracking, error taxonomy, and retrospective think-aloud (RTA) were used to study the diverse cognitive, behavioral, and operational aspects affecting SA. After analyzing 566 events across 18 tasks, this research highlighted eye tracking’s potential by offering real-time insights into operator behavior and RTA’s potential as a method for cross-checking the causal factors underlying errors. Results: Critical tasks, like positioning forklifts and lowering pallets, significantly impact incident occurrence, while high-cognitive demand tasks, such as hoisting and identifying pedestrians/obstacles, reduce SA and increase errors. Driving tasks are particularly vulnerable to errors and are the most affected by operator risk generators (ORGs), representing 42% of incident risk events. This study identifies driving, hoisting, and lowering loads as the tasks most influenced by system factors. Limitations include the task difficulty levels, managing physical risk, and training. Future research is suggested in autonomous industrial vehicles and advanced driver assistance systems (ADASs). Conclusions: This study provides valuable insights into how we may improve safety in logistics operations by proposing a multiphase methodology to uncover the patterns of attention, perception, and cognitive errors and their impact on decision-making.
... Over time, scholars have increasingly recognized that traditional approaches to safety management, which strive to completely eliminate adverse outcomes and errors, fall short in dealing with the complexities of modern systems and enhancing safety (e.g., Hollnagel, 2014a). Driven by this growing understanding, a significant paradigm shift took place: moving away from the 'old view' (e.g., Reason, 1990) to the 'new view of errors' (e.g., Cook et al., 1998;Dekker, 2001Dekker, , 2002Woods et al., 1994; for an overview see Le Coze, 2022), where errors are considered as "symptoms [of underlying issues within the system], not [the root] causes [of adverse outcomes]" (Woods et al., 1994, p. 108). In the 'new view', safety is the "ability to succeed under [and to adapt to] varying conditions" (Hollnagel, 2014a, p. 137). ...
... The common elements of Resilience Engineering, the HRA, and EMT are that they stress the importance of understanding and addressing the issues rooted in the system instead of punishing and blaming individuals for their errors (Woods et al., 1994) as well as of rapidly containing the negative effects of errors that do occur. However, these approaches differ in their focus: Resilience Engineering and the HRA emphasize both learning from success (i.e., what worked and went well; e.g., Hollnagel, 2011bHollnagel, , 2014a and near misses (Dwyer et al., 2023). ...
Errors may be a safety hazard, yet all organizations and managers have to deal with errors. Error management and high reliability are strategies for dealing with errors. While these strategies originate from different research approaches and have been well studied independently, they have not been directly compared in empirical studies. Based on a theoretical analysis of similarities and differences between these approaches, we developed a training based on each of them. For our High Reliability Approach (HRA) training, we deduced training principles based on the facets of safety organizing. For the Error Management Training (EMT) and the training in the error-avoidant control condition (EAT), we oriented on existing training studies. We trained university students (N = 359) in a relevant skill. Our study revealed that both EMT and HRA training led to better performance than EAT. Exploratory analyses revealed emotion control towards errors to be related to performance only in the EMT group. Our article suggests that in spite of similar effectiveness of EMT and HRA training, there may be differential processes in these two approaches.
... Par exemple le pattern « dérive de fonctionnement » qui constate un écart à un fonctionnement qui tend à devenir habituel peut être interprété comme une faute de la part des individus ou bien comme une action adaptative face aux contraintes locales de production. Malheureusement, c'est souvent le résultat qui permet a posteriori de qualifier l'action (Woods, Johannesen, Cook, & Sarter, 1994). ...
Analyse systémique et organisationnelle des violences en entreprise à des fins de prévention : le modèle systémique d'analyse des situations de violence au travail (MSVT)
Recent accidents such as the B737-MAX8 crashes highlight the need to address and improve the current aircraft certification process. This includes understanding how design characteristics of automated systems influence flightcrew behavior and how to evaluate the design of these systems to support robust and resilient outcomes. We propose a process which could be used to evaluate the compliance of automated systems looking through the lens of the 3Rs: Reliability, Robustness, and Resilience. This process helps determine where additional evidence is needed in a certification package to support the flightcrew in interacting with automated systems. Two diagrams, the Design Characteristic Diagram (DCD) and the What’s Next diagram, are used to uncover scenarios which complicate flightcrew response. The DCD is used to look at the relationship between characteristics in design and potential vulnerabilities which commonly occur when design does not support the flightcrew. The What’s Next diagram looks at the ability of the design to support the flightcrew in anticipating what will happen next. In our process, claims surrounding the 3Rs that are present in a certification package are systematically evaluated using these two diagrams to uncover additional areas of support for the flightcrew. Questions about when these claims may break down which are identified using the DCD can be tested using scenarios developed on the What’s Next diagram. Further vignettes looking at different versions of a scenario can be assessed to increase the robustness in the design. The FAA has sponsored this research through the Center of Excellence for Technical Training and Human Performance. However, the FAA neither endorses nor rejects the findings of this research. The dissemination of this research is in the interest of invoking academic or technical community comments on the results and conclusions of the research.
Focusing on "enabling approaches" that treat humans as partners can add another layer of protection to our cybersecurity defenses.
Il miglioramento della sicurezza e della protezione dei pazienti dalle minacce sanitarie sono obiettivi fondamentali della politica sanitaria dello Stato e della Unione Europea. Il perseguire la sicurezza, intesa come "dimensione della qualità dell'assistenza sanitaria, che garantisce, attraverso l'identificazione, l'analisi e la gestione dei rischi e degli incidenti possibili per i pazienti, la progettazione e l'implementazione di sistemi operativi e processi che minimizzano la probabilità di errore, i rischi potenziali e i conseguenti possibili danni ai pazienti" (Ministero della Salute, 2007), impone lo sviluppo delle capacità e delle competenze necessarie per gestire e realizzare modifiche di comportamento e quindi modifiche del sistema. Tanto, anche in ragione degli approvandi disposti normativi in materia di risk management che, al netto della loro stesura finale, prevedono un forte intervento di tipo culturale per la formazione sulla prevenzione dei rischi, rafforzando il concetto di sicurezza dei pazienti e quindi di qualità dell'assistenza, in ossequio a quanto già raccomandato dall'Organizzazione Mondiale della Sanità (WHO, 2006) e dall'Unione europea (Consiglio UE, 2009). In particolare, è auspicato che le attività di gestione del rischio clinico siano incardinate nell'ambito dei sistemi di gestione della sicurezza e qualità, così come avviene in tutte le organizzazioni a elevata complessità nel mondo. Il coordinamento delle precipue funzioni, poi, dovrebbe essere distinto in due ambiti, il primo dei quali inerente le attività cliniche di analisi e anticipazione degli eventi avversi, da porre in capo a operatori sanitari esperti sia nel proprio ambito specialistico che nell'analisi sistemica (clinical risk manager); il secondo, di contro, relativo alle attività di monitoraggio e valutazione della sicurezza e dell'appropriatezza, attribuite a professionisti di area psico-sociale e politecnica (patient safety manager). Orbene, se nel primo contesto appare ben acclarato il profilo delle capacità e delle competenze richieste per svolgere la funzione operativa di clinical risk manager, la figura del patient safety manager quale responsabile di attività di sicuro taglio manageriale, è ben lungi dall'essere definita sia in termini di capacità, che di competenze necessarie, nonché di percorsi formativi specifici. In prima battuta, questo lavoro si propone di delineare, attingendo dalla letteratura scientifica, dai documenti di politica sanitaria, piuttosto che dall'analisi dell'implementazione di particolari modelli organizzativi posti già in essere in specifiche realtà aziendali di sanità pubblica, il framework entro cui sviluppare la definizione del patient safety manager, determinandone la sua fisionomia e il necessario percorso formativo, prendendo le mosse dai principi del Curriculum dell'OMS per la sicurezza delle cure (WHO, 2009). Successivamente, si opererà una revisione sistematica della letteratura scientifica, al fine di confrontare il profilo come determinato del patient safety manager, con gli attuali modelli di formazione in ambito sanitario, relativamente alla gestione della sicurezza del paziente, sì da determinarne la sua sostanziale applicazione, ovvero individuare le aree di criticità che ne impediscono una efficace implementazione.
This report describes research conducted during 1989 and 1990 on the cognitive characteristics of a corpus of anesthesia critical incidents. The incidents were collected by monitoring and transcribing the regular quality assurance conferences in a large, university anesthesiology department. The 57 reports of incidents were analyzed by constructing protocols which traced the flow of attention and the knowledge activation sequence of the participants. Characteristics of the resulting protocols were used to divide the collection into five categories: acute incidents, going sour incidents, inevitable outcome incidents, airway incidents, and non-incident incidents. Of these, the acute and going sour categories represent distinct forms of incident evolution. The implications of this distinction are discussed in the report. Nearly all of the incidents involve human cognitive performance features. Cognition clearly plays a role in avoiding incidents but also in aborting and recovering from incidents in progress. Moreover, it is clear that subtle variations in cognitive function may playa crucial role in anesthetic disasters, of which incidents are taken to be prototypes. Review of the corpus reveals the different cognitive functions involved in anesthesia and anesthesia incidents. These cover a wide range including classic aspects of cognition, for example the direction of attention, and complex and poorly understood aspects such as situation awareness. The cognitive features include dealing with competing goals, dealing with competing indicators, the limitations of imperfect models, knowledge activation failures, the role of learned procedures and assumptions in reducing cognitive workload, failure to integrate multiple themes, organizational factors, and planning. These presence of these different cognitive features and cognitive failures in a single discipline is significant because it enhances and supports separate findings from other domains (e.g. nuclear power plant operation, commercial aviation) and also because it provides strong support for the contention that operators acting in these semantically complex, time pressured, high consequence domains face common problems and adopt similar strategies for dealing with them. The report demonstrates the way in which cognitive analysis of incidents can be accomplished in anesthesia and in other domains and suggests a system for categorizing the results obtained. It also raises questions about the adequacy of evaluations of risk and safety that do not explicitly account for the cognitive aspects of incidents and their evolution. In order to make real progress on safety in domains that depend critically on human operators it is necessary to examine and assess human cognitive performance, a process which requires large amounts of data and careful reconstruction. Such cognitive analysis is difficult. It requires substantial experience, skill, and effort and depends on acquiring and sifting through large quantities of data. This should not be suprising, since the domain itself is one characterized by experience, skill, effort, and large quantities of data. The challenge for us and for other researchers is to perform more such analyses and extend and refine the techniques described here and to link the analyses to those from other domains.
shows how one can go beyond spartan laboratory paradigms and study complex problem-solving behaviors without abandoning all methodological rigor / describes how to carry out process tracing or protocol analysis methods as a "field experiment" (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Human-System interaction has been and will continue to be of interest to many researchers of various disciplines: engineers, computer scientists, psychologists, and social scientists. The research in Human-System Interaction (HSI) has progressed from the era of using anthropomorphic data to design workspace to the current period which utilizes human and artificial sensors to design sensory-based cooperative workspace. In either of these developments, HSI has been known to be complex. In 1994, we initiated a series of symposiums on Human Interaction with Complex Systems. It was then that various ideas surrounding HSI for today and tomorrow were discussed by many scientists in the related disciplines. As a follow-up, in 1995 the Second Symposium was organized. The objective of this symposium was to attempt to defme a framework, principles, and theories for HSI research. This book is the result of that symposium. The 1995 symposium brought together a number of experts in the area of HSI. The symposium was more focused on expert opinions and testimonies than traditional meetings for technical papers. There were three reasons for that approach.
Human Factors is unlike other traditional divisions of knowledge and is more than the mere haphazard interdisciplinary collaboration between psychology and engineering. As such, it requires a unique theoretical structure that reflects the opportunities and constraints intrinsic to emergent complex dynamical operational spaces derived from the interplay of human, machine, task, and environment.
One result of recent research on human error and disaster is that the design of the human-machine system, defined broadly, modulates the potential for erroneous action. Clumsy space use of technological powers can create additional mental burdens or other constraints on human performance that can increase the chances of erroneous actions by people especially in high workload, high tempo operations. This paper describes studies of a computer based automated device that combine critical incident analysis, bench test evaluations of the device, and field observation in order to understand physician-device interaction in the context of heart surgery. The results link, for the same device, user group and context, three findings. First, the device exhibits classic human-computer interaction flaws such as lack of feedback on device state and behavior. Second, these HCI flaws actually do increase the potential for erroneous actions and increase the potential for erroneous assessments of device state and behavior. The potential for erroneous state assessment is especially troublesome because it impairs the user’s ability to detect and recover from misassemblies, misoperations and device failures. Third, these data plus critical incident studies directly implicate the increased potential for erroneous setup and the decreased ability to detect errors as one kind of important contributor to actual incidents. The increased potential for error that emanates from poor human-computer interaction is one type of latent failure that can be activated and progress towards disaster given the presence of other potentiating factors in Reason’s model of the anatomy of disasters.