Emilie M. Roth

Emilie M. Roth

About

208
Publications
38,915
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
6,287
Citations

Publications

Publications (208)
Article
Discussion Panel Abstract: The recent Boeing 737MAX accidents crystalized for the public the complexity of anticipating system and operator performance and developing a system design that prevents catastrophic outcomes. The operational situations, progression of flightcrew actions, and system behaviors that led to the two accidents had not been ant...
Article
New systems are often based on optimistic assumptions of how they will improve human performance. In the cognitive engineering tradition, these assumed benefits are regarded as hypotheses that need to be tested. An important element of a system user evaluation is to determine whether the hypothesized benefits are realized. Evaluation may also uncov...
Article
Introduction Understanding and managing clinician workload is important for clinician (nurses, physicians and advanced practice providers) occupational health as well as patient safety. Efforts have been made on developing strategies for managing clinician workload by improving patient assignment. The goal of the current study is to use electronic...
Article
Nurses working in the hospital setting increasingly have become overburdened by managing alarms that, in many cases, provide low information value regarding patient health. The current trend, aided by disposable, wearable technologies, is to promote patient monitoring that does not require entering a patient’s room. The development of telemetry ala...
Article
To date, studies have found scattered and inconsistent use of existing opioid prescription support tools, such as Prescription Drug Monitoring Systems (PDMPs) and guidelines, and little research has been done to identify barriers for adopting them and better ways to support clinician decision making. This pilot study aims to better understand the f...
Article
Background The current state of scientific knowledge regarding communication between emergency medicine (EM) providers indicates that communication is critical to safe and effective patient care. Objectives In this study, we identified communication needs of EM nurses and physicians; in particular, what information should be conveyed, when, how, a...
Article
This research is a first stage in developing a method for modeling the clinician workload associated with an emergency medicine patient in order to display workload for purposes of managing clinician workload and emergency department (ED) flow. We proposed a multi-stage approach of predicting patient-related drivers of clinician’s workload in the e...
Chapter
This chapter describes the discipline of human factors engineering and how it can be specifically applied to the study and improvement of clinical workflow. Human factors engineering is a well-established scientific discipline that studies the functional capabilities and limitations of humans and methods to integrate these findings into the design...
Technical Report
Full-text available
This report documents the findings and recommendations from an investigation into stop signal violations at several passenger railroads. The study collected data on accidents related to stop signal overruns (SSO) from the National Transportation Safety Board (NTSB) reports and from the Federal Railroad Administration’s (FRA) accident database to id...
Conference Paper
Designing effective explainable artificial intelligence (XAI) systems represents a fundamental challenge for trusted and cooperative human machine collaboration. In this paper we describe the development and evaluation of an explanation interface that addrresses some of these new XAI design challenges. Our team developed an interface design approac...
Article
OCCUPATIONAL APPLICATION Understanding and managing clinician workload is important both for the occupational health of clinicians and for the safety of patients. This study describes the integration of a new display concept for visualizing emergency medicine physician and nurse workload into a live electronic health record along with an investigat...
Article
There is growing recognition of a need to go beyond levels of automation frameworks to provide more detailed guidance for design of effective human automation interaction (HAI). Here we present some design questions that are important for designers of HAI to address as they develop the requirements for the software architecture and user interfaces...
Article
Full-text available
We conducted a study to examine the factors that contribute to run through switches (RTS) in passenger rail yard operations. RTS occur when a train runs through a misaligned switch. It can result in damage to the switch that, if not detected and repaired, can increase the risk of derailment of a subsequent train coming in the opposite direction. Th...
Technical Report
Full-text available
This document provides guidance on how railroads can use a Human Systems Integration (HSI) approach to incorporate human-centered design into their acquisition processes for procuring new systems. This work began following a request for guidance from the Association of American Railroads (AAR) to create next generation locomotive cab design standar...
Article
Full-text available
We describe the patterns and content of nurse to physician verbal conversations in three emergency departments (EDs) with electronic health records. Emergency medicine physicians and nurses were observed for 2 h periods. Researchers used paper notes to document the characteristics (e.g., partners involved, location of communication, who initiated c...
Article
Across a wide variety of systems, from aviation and ground transportation to energy and financial systems, there continues to be a move toward introducing more powerful technological support. Research and practice have clearly demonstrated that, for these complex systems, human centered designs are critical, especially when anomalous scenarios aris...
Article
Cognitive engineering is an applied field with roots in both cognitive science and engineering that has been used to support design of information displays, decision support, human-automation interaction, and training in numerous high risk domains ranging from nuclear power plant control to transportation and defense systems. Cognitive engineering...
Article
We describe a prototype cognitive work aid for airlift mission allocation and scheduling. A key design challenge was how to generate and visualize multiple operationally distinct options to broaden the set of alternative plans that Mission Planners consider, increasing overall efficiency and effectiveness of the mission planning organization. We fi...
Article
The relatively rapid transition from a paper-based system to a digital system in healthcare has not always employed a sophisticated integration of usability concepts. Yet usability is critical to safety and to effectiveness of the electronic health record, and regulators and policy makers have been increasingly focused on this area. This panel will...
Article
Background: The importance of leadership is recognized in surgery, but the specific impact of leadership style on team behavior is not well understood. In other industries, leadership is a well-characterized construct. One dominant theory proposes that transactional (task-focused) leaders achieve minimum standards and transformational (team-orient...
Article
The National Academies Board on Human-Systems Integration (BOHSI) has organized this session. An initial presentation by the director of BOHSI will provide information on recent projects and updates. Then a panel will discuss the application of human-systems integration (HSI) to the issue of cyber security. Cyber security attempts to protect device...
Technical Report
Full-text available
This report documents the findings and recommendations from an investigation into stop signal violations at an American passenger railroad. The investigation was requested by the Federal Railroad Administration (FRA) and the passenger railroad, and sponsored by FRA’s Office of Research & Development (ORD). This investigation assisted the passenger...
Article
Unlabelled: An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies cover...
Article
Full-text available
Cognitive Task Analysis (CTA) has become part of the standard tool set of cognitive engineering. CTAs are routinely used to understand the cognitive and collaborative demands that contribute to performance problems, the basis of expertise, as well as the opportunities to improve performance through new forms of training, user interfaces, or decisio...
Article
Full-text available
Intelligence analysts—overloaded with complex and disparate data—must incorporate information into cohesive and convincing narratives and explanations. For analysts, data overload can result in developing premature conclusions and limit their ability to effectively conduct comprehensive analyses. Historically, automated decision aids designed to he...
Book
Full-text available
Cognitive engineering is an interdisciplinary approach to the analysis, modeling, and design of engineered systems or workplaces in which humans and technologies jointly operate to achieve system goals. As individuals, teams, and organizations become increasingly reliant on information technology and automation, it is more important than ever for s...
Conference Paper
Full-text available
A strength of the field of cognitive engineering and decision-making lies in its wide applica-bility across the complex socio-technical systems, which are ubiquitous in modern society. Meth-ods and theoretical advances in CEDM have been both developed through, and adapted across, domains as diverse as nuclear power, health systems, and aviation. Wh...
Article
To address issues in enterprise systems, operations research (OR) analysts need to be able to understand, codify, and communicate various aspects of the issues, such as the environment’s conditions and their relationships. Models are a natural way to capture that information, but they must be understandable to a variety of stakeholders involved in...
Article
Full-text available
A common characteristic of domains that require planning for allocation of scarce resources is the need to dynamically revise the plan as new requirements emerge and priorities change. We describe a prototype decision support system for planning and scheduling airlift that we developed for a military transport organization that enables agile plan a...
Article
Full-text available
The cyber security task is an intensely cognitive task that is embedded in a large multi-layered sociotechnical system of analysts, computers, and networks. Effective performance in this world is hampered by enormous size and complexity of the network data, the adaptive nature of intelligent adversaries, the lack of ground truth to assess performan...
Article
Automated systems, ranging from robots to intelligent planning aids, are increasingly part of the cognitive landscape. A pressing question facing the human factors community is how to design the automation to foster effective collaboration between the human and automated agents. This panel brings together leading researchers actively engaged in des...
Article
Previous research has identified many effective methods to visualize different types of meta-information, or information qualifiers; however, these methods are often incorporated without understanding how the graphical codes are perceived and how the encoded information is interpreted by display users. This results in display designers selecting gr...
Article
The effect of adaptive display on pilot performance was assessed using objective and subjective measures from a sequence of 2 experiments. Combat aircraft advances have led to dramatic increases in the operational tempo facing the pilot, increasing the likelihood for situation awareness (SA) failures, biases, and information overload. Prior researc...
Article
To understand the etiology and resolution of unanticipated events in the operating room (OR). The majority of surgical adverse events occur intraoperatively. The OR represents a complex, high-risk system. The influence of different human, team, and organizational/environmental factors on safety and performance is unknown. We video-recorded and tran...
Article
Communication failure is a common contributor to adverse events. We sought to characterize communication failures during complex operations. We video recorded and transcribed six complex operations, representing 22 h of patient care. For each communication event, we determined the participants and the content discussed. Failures were classified int...
Article
In this paper we describe a collaborative automated scheduler that utilizes innovative visualizations and mixed initiative techniques to enable users to more effectively uncover, communicate and dynamically revise priorities as circumstances change. The system includes visualizations to foster improved situation awareness of available resources vs....
Article
"War stories" are commonplace in surgical education, yet little is known about their purpose, construct, or use in the education of trainees. Ten complex operations were videotaped and audiotaped. Narrative stories were analyzed using grounded theory to identify emergent themes in both the types of stories being told and the teaching objectives the...
Article
Full-text available
This paper reports the results of a cognitive task analysis (CTA) that examined the cognitive and collaborative demands placed on conductors and the knowledge and skills that experienced conductors have developed that enable them to operate safely and efficiently. A secondary aim of the CTA was to understand the implications of the Rail Safety Impr...
Article
Objective: To understand the etiology and resolution of unanticipated events in the operating room (OR). Background: The majority of surgical adverse events occur intraoperatively. The OR represents a complex, high-risk system. The influence of different human, team, and organizational/environmental factors on safety and performance is unknown. Met...
Article
Supporting complex decision making requires conveying relevant information characteristics or qualifiers. The authors tested transparency and numeric annotation for displaying uncertainty about object identity. Participants performed a “missile defense” game in which they decided whether to destroy moving objects (which were either threatening miss...
Article
Full-text available
To planners at United States Transportation Command (USTRANSCOM), a course of action (COA) is a transportation plan - a description of what vehicles, routes, and ports should be used to move sets of cargo and passengers throughout the world. A planner may be tasked with very quickly producing multiple COA options at any time in response to either r...
Article
Naturalistic decision-making studies of intelligence analysis have generally focused on information search, collection, and synthesis processes, deemphasizing the initial “problem formulation” phase, in which analysts interpret and contextualize the information request to determine which information to collect. We present the results of two studies...
Conference Paper
Full-text available
This paper presents a number of views from a panel discussion on the relationship between human reliability analysis (HRA) and human factors. HRA emerged concurrently with the field of human factors and now features a nearly fifty-year shared history. While built on human factors, HRA distinguished itself early on from human factors due to its emph...
Article
Cyber Network degradation and exploitation can covertly turn an organization's technological strength into an operational weakness. It has become increasingly imperative, therefore, for an organization's personnel to have an awareness of the state of the Cyber Network that they use to carry out their mission. Recent high-level government initiative...
Article
Previous Human-Centered Computing department articles have reflected on the mismatch that can occur between the promise of intelligent technology and the results of technological interventions. Part 1 on the Practitioner's Cycles illustrated ways in which actual world problems-the forces and constraints of procurement-are at odds with the goals of...
Article
Summary form only given. The management of team work and task work in modern high-paced, collaborative environments is challenging due to unyielding time constraints, constant switching of cognitive tasks, and collating incoming and outgoing information requests to and from collaborators who vary in their expertise. To ensure the effectiveness of c...
Article
Cognitive artifacts are created and used to support task performance in many domains. These artifacts may be essential components designed into a process, they may have been created by users as work-arounds to system shortcomings, or they may be extensions to systems that add functionalities to meet evolving needs. Examination of cognitive artifact...
Article
Cognitive systems engineering projects have found a need for and a value in combining particular methods of cognitive task analysis (CTA). A principle reason that is given is that different CTA methods have different strengths in terms of how they inform the study, analysis or design of cognitive work systems. Focus questions include: What specific...
Article
Advances in aircraft operational capabilities have led to a dramatic increase in the operational tempo facing air combat aviators, which has in turn led to SA failures, particularly with respect to secondary information. For example, when engaging an air threat, aviators will often overlook key information such as geopolitical boundaries, resulting...
Article
We describe the most recent work-centered design for a military airlift organization. Earlier design cycles produced a set of coordinated visualizations to support synchronized air mission replanning. In this phase of the program automated planning support was incorporated to help C2 staff solve complex constraint problems across multiple missions...
Article
Bayesian networks (BNs) are probabilistic models frequently used to capture domain knowledge for use in computational systems that can reason about states, causes, and effects. While BNs have many advantages, their complexity can hamper the process of knowledge elicitation and encoding. First, domain experts may not have expertise in artificial rea...
Article
Bayesian belief networks (BNs) are well-suited to capturing vague and uncertain knowledge. However, the capture of this knowledge and associated reasoning from human domain experts often requires specialized knowledge engineers responsible for translating the expert's communications into BN-based models. Across application domains, we have analyzed...
Article
Full-text available
Data fusion systems are being increasingly used to support military planning, decision making, and command and control functions in general. Typically, these systems are designed around the current capabilities of particular data collectors (e.g., sensors) and available processing algorithms. These algorithms incorporate an "ontology" that reflects...
Article
Full-text available
As the Department of Defense (DoD) moves into a more agile C2 environment, added requirements will be laid on operators and systems -- integration with outside systems and organizations, operation in an environment where rules change daily, and severely limited resources. We foresee an increased reliance on automated support tools -- optimizers, sc...
Article
We conducted two studies that investigated display characteristics related to color (hue, saturation, brightness, and transparency) and contrast with a background for displaying information qualifiers (termed meta-information) such as uncertainty, age, and source quality. Level of detail (or granularity) of the meta-information and task demands wer...
Article
Information, as well as its qualifiers, or meta-information, forms the basis of human decision-making. Human behavior models (HBMs) therefore require the development of representations of both information and meta-information. However, while existing models and modeling approaches may include computational technologies that support meta-information...
Conference Paper
Full-text available
Over the course of our studies of how intelligence analysis professionals perform their work, we have observed that the perceived utility of technosocial models varies with the degree that analysts could create and/or adapt a model to a particular problem or situation. Analysts failed to trust models (a form of validity) that were created outside o...
Article
New system design often focuses on opportunities afforded by technology without careful analysis of the needs of the people and work involved. A strong consensus has been found across diverse design traditions regarding the need for in-depth analysis of the context of use to inform system design. Analysis of context of use encompasses identifying a...
Article
Data fusion systems are increasingly being used to support military planning and decision making. Typically these systems are designed around the current capabilities of particular data collectors (e.g., sensors) and processing algorithms. They incorporate an 'ontology' that reflects the designer's perception of the key features of the world (e.g.,...
Article
In this article, the author provides an overview of cognitive analysis methods and how they can be used to inform system analysis and design. Human factors has seen a shift toward modeling and support of cognitively intensive work (e.g., military command and control, medical planning and decision making, supervisory control of automated systems). C...
Article
Increasingly, the study of cognition and action in complex sociotechnical systems has revealed that humans reason about both information and the qualifications of that information. These qualifications, or meta-information (e.g., pedigree, recency, uncertainty), play a role in human performance across work domains (Pfautz et al., 2006). Meta-inform...
Article
Computerized clinical reminders (CRs) were designed to reduce clinicians' reliance on their memory and to present evidence-based guidelines at point of care. However, the literature indicates that CR adoption and effectiveness has been variable. We examined the impact of four design modifications to CR software on learnability, efficiency, usabilit...
Article
Cognitive task and work analyses are approaches to the analysis and support of cognitive work (rather than primarily physical or perceptual activities). Although a variety of methods exist for performing cognitive task and work analyses, they share a common goal of providing information about two mutually reinforcing perspectives. One perspective f...
Article
Full-text available
The field of cognitive engineering and decision making (CEDM) has grown rapidly in recent decades. At this writing, it is the largest technical interest group within the Human Factors and Ergonomics Society. Work that falls into this area of research and study is also widely practiced across Europe and in countries around the world. Along with this...