Display and Interface Design: Subtle Science, Exact Art
... Cognitive systems engineering (CSE ;Norman 1986;Rasmussen 1986;Rasmussen, Pejtersen, and Goodstein 1994;Vicente 1999) provides an overarching framework for the analysis, design, and evaluation of effective computerized decision support. Ecological Interface Design (EID; Bennett & Flach, 2011a;Hall et al., 2012;Rasmussen & Vicente, 1989, 1992) is a compatible framework focusing on the development of principles for effective display and interface design (Hall et al., 2012). Over the past 30 years, the CSE/EID approach has been applied to a wide variety of work domains. ...
... Instead, sensory perception directly results from the coupling between the information available in the environment (e.g., the perceived object, affordances) and the agent (Chemero, 2009). Conventionally, this knowledge (perception) has been conceived to involve higher-level processes to compute mental images by collecting and assimilating ambiguous environmental cues (Bennett & Flach, 2011a). The theory of direct perception emphasizes that an organism interacts directly with their environment with no need for processes that transform or supplement incoming data. ...
... As such, people can directly perceive meaningful properties of the environment, or affordances, by collecting, discovering, and associating environmental cues without having to resort to mediating inferences, experiences or representations. Therefore, agents guide behavior by taking advantage of invariant optical relationships to bypass unnecessary cognitive demands and develop direct associations (Bennett & Flach, 2011a). ...
Four interfaces were developed to factorially apply two principles of ecological interface design (EID; direct perception and direct manipulation) to a flexible manufacturing system (FMS). The theoretical foundation and concepts employed during their development, with findings related to more significant issues regarding interface design for complex socio-technical systems, are discussed. Key aspects of cognitive systems engineering (CSE) and EID are also discussed. An FMS synthetic task environment was developed, and an experiment was conducted to evaluate real-time decision support during supervisory operations. Participants used all four interfaces to supervise and maintain daily part production at systematically varied levels of difficulty across sessions. Significant results provide evidence that the incorporation of direct perception and direct manipulation in interface design produced an additive effect, allowing for greater support for the supervisory agents.
... In the implementation of CDS, efficiency and efficacy rank at the top of user priorities, ahead of aesthetic preferences, however, well-designed aesthetics can work hand-in-hand with principles that promote efficiency and efficacy such as visibility, consistency, appropriate visual emphasis, and visual balance. Bennett and Flach [10] espouse a design approach of subtle science and exact art [11] and expand the conventional dyad of user interface to user-interpreted meaning to include a third component of ecology which represents the context, work domain, or applied problem being addressed. ...
... Research on human perception, cognition, and subsequent performance in a wide variety of basic and domain-specific applied human-computer tasks have led to the development of many design principles and guidelines [10,24,[26][27][28]. In Table 14-1, we identify 21 principles that are relevant to the use of visualization and information integration in the design of CDS. ...
... Even a perfectly reliable decision aid will fail if it is not seen, attended to, or its meaning mis-interpreted. These principles are expected to be widely known and integrated into good design practices of technology developers [10,24,[26][27][28][29]. However, innovation often emerges from those who know the problem space well. ...
A well-designed visualization can be rapidly interpreted and understood. The advantages of visual representations of quantitative data and meaningful integration of related patient information comes at the cost of much effort in identifying the right information to group together, how to present it, to whom, and in what context. Even small gains from these efforts become great when multiplied by many users making critical health care decisions. In this chapter, we draw on theoretical models of human cognition and visual perception, understanding of workflow and information needs in the context of clinical applications, and user-centered design methods to promote the design of effective information integration and visualization. We provide a grounding in how humans use vision to think which has led to the development of principles to guide the design of information integration and visualization. We present design methods, including contextual inquiry and participatory design activities, relevant to understanding clinical information integration and visualization needs. Finally, in a world of increasing information overload, we review examples of research and design efforts to identify and artfully present the most relevant patient information to support a variety of clinical decisions such as prioritizing resources to patients with the most urgent needs, detection of missed actions, and making the right treatment choices.
... Ainsi, les interfaces doivent être « affordantes », c'est-à-dire qu'elles doivent rendre visibles les actions possibles à réaliser. Afin de donner à l'utilisateur la bonne affordance (Bennett & Flach, 2011), autrement dit celle sur laquelle le traitement perceptif pourra se reposer tout en étant efficace, l'interface doit représenter le système dans toute sa complexité (Vicente, 1999) et elle doit également rendre saillants les raccourcis utilisés par les experts (Vicente & Rasmussen, 1992). ...
... Selon le cadre théorique de l'EID, afin de prendre la complexité du système en considération, une vision globale du problème est nécessaire : la hiérarchie d'abstraction ou hiérarchie fin-moyen est l'outil utilisé afin de représenter la complexité du système en tant qu'unité. Cette structure relie les buts aux moyens nécessaires pour les atteindre et ce sur cinq niveaux hiérarchisés (Bennett & Flach, 2011 ;Vicente, 1999). ...
... Notre micromonde a été développé, pendant un an, à partir des résultats obtenus lors de l'analyse de l'activité afin de reproduire, tout en les contrôlant, l'atelier, les contraintes qui s'exercent sur le processus de production et les tâches à réaliser par l'opérateur. La hiérarchie d'abstraction est utile pour comprendre les contraintes s'exerçant sur l'atelier étudié (Bennett & Flach, 2011). Cette analyse nous servira également plus tard (cf. ...
L’évolution des technologies et la mutation du rôle attribué aux opérateurs de l’industrie suscitent un essor de la littérature concernant l’assistance aux procédés de fabrication. Cette thèse de psychologie s’inscrit dans cette littérature. Son objectif est de concevoir un assistant de contrôle aux procédés de fabrication, le compagnon virtuel, qui permette de maintenir à la fois les performances et les compétences des opérateurs. Le système étudié, une usine aéronautique, nécessite le recours à la simulation de processus physiques dynamiques. Deux champs de la littérature sont mobilisés : les cadres théoriques autour de la conception d’interfaces pour les environnements dynamiques et ceux sur le développement des compétences. Une méthodologie en trois parties est utilisée : (1) analyse de l’activité des opérateurs dans l’atelier réel, (2) validation d’un micromonde et (3) test d’un compagnon virtuel dans le micromonde. Les résultats montrent que les opérateurs de l’atelier réel mobilisent un schème trompeur lors de leur activité. Ce schème trompeur est également présent chez des participants tout-venants et il a un effet sur les performances des participants entrainés dans le micromonde. Enfin, le compagnon virtuel, conçu selon les principes de l’EID, ne permet pas de faire disparaitre le schème trompeur. Cependant, il permet aux participants qui n’utilisent pas le schème trompeur d’obtenir de bonnes performances. Les résultats obtenus étayent l’idée qu’il est pertinent de vérifier si des opérateurs ne mobilisent pas des schèmes trompeurs qui pourraient être un frein à l’utilisation des interfaces proposées pour les assister.
... On the one hand, UCD focuses on the capabilities and limitations of human operators, and seeks to amplify and extend their perceptual, cognitive, and performance capabilities ( [36]; [37]). On the other hand, EID focuses on the work domain, and seeks to design tools that support human operators by leveraging their perception, action, or cognitive capabilities ( [38]; [39]; [40]). The UCD attempts to identify information that is needed to complete only certain well-defined tasks. ...
... Here, the visual display stands between the cyber defender and work domain. The success of the visualization depends on the compatibility between the work domain and the visual display, and compatibility between the display and the cyber defender ( [40]; [17]). ...
... Aggregation hierarchy provides 'part-whole' models of the work domain by decomposing or aggregating items on each level of the abstraction hierarchy. A model of the structure of the work domain is obtained by using abstraction and aggregation hierarchies in work analysis [40]. ...
Cyber security visualization designers can benefit from human factors engineering concepts and principles to resolve key human factors challenges in visual interface design. We survey human factors concepts and principles that have been applied in the past decade of human factors research. We highlight these concepts and relate them to cybersecurity visualization design. We provide guidelines to help cybersecurity visualization designers address some human factors challenges in the context of interface design. We use ecological interface design approach to present human factors-based principles of interface design for visualization. Cyber security visualization designers will benefit from human factors engineering concepts and principles to resolve key human factors challenges in visual interface design.
... EID includes collecting data regarding the work domain and the decision processes employed by system users. Analysis techniques are employed to identify the salient characteristics of the work domain, including the cognitive and physical activities being performed along with the decisions and information needed to execute the activities to achieve the user's goals [16]. During design, provisions are made to present goal-relevant information, thereby providing the user with knowledge of affordances related to goal accomplishment [17]. ...
... The resulting design requirements take the form of RDRs organized into visualizations in a DTD, thus addressing RQs 4a and 4b. The RDR development approach incorporated the EID principles for intentdriven domains by developing requirements that reflect the work domain structure that governs goal outcomes [16]. The design requirements outlined in this step were then paired with a supporting concept for the challenged goal to form the envisioned solution. ...
This research addresses the data overload faced by intelligence searchers in government and defense agencies. The study leverages methods from the Cognitive Systems Engineering (CSE) literature to generate insights into the intelligence search work domain. These insights are applied to a supporting concept and requirements for designing and evaluating a human-AI agent team specifically for intelligence search tasks. Domain analysis reveals the dynamic nature of the ‘value structure’, a term that describes the evolving set of criteria governing the intelligence search process. Additionally, domain insight provides details for search aggregation and conceptual spaces from which the value structure could be efficiently applied for intelligence search. Support system designs that leverage these findings may enable an intelligence searcher to interact with and understand data at more abstract levels to improve task efficiency. Additionally, new system designs can support the searcher by facilitating an ‘Ambient Awareness’ of non-selected objects in a large data field through relevant system cues. Ambient Awareness achieved through the supporting concept and AI teaming has the potential to address the data overload problem while increasing the breadth and depth of search coverage.
... In addition to creating buffers to protect the system from risky situations, it is important to directly face the difficult challenge of making the actual risk boundaries visible (i.e., providing the feedback that operators need to anticipate and avoid dangerous situations). This insight is the motivation for the Ecological Interface Design approach for creating representations or visualizations for safety critical systems (e.g., [2,11]). ...
... In terms of interface design to support complex work, representations tend to be framed as either geometric analogues or metaphors that have multiple levels of structure to reflect the multiple levels of means-ends constraints associated with complex work (e.g., [2,5]). For both analogical and metaphorical representations, there are two related principles that are fundamental to the quality of the representation: semantic mapping [3] and systematicity [7]. ...
A central challenge in designing stable control systems is to identify the states that must be fed back to enable successful control. The quality of control (including safety) depends on our ability to visualize the state space underlying the functional dynamics of the work being managed. Building concrete visualizations is both a useful tool for knowledge elicitation with domain experts to discover the meaningful functional work constraints that determine this state space, and an essential part of interface design to support safe work in complex systems.
... Affordances. In the field of cognitive systems engineering, any attempted intervention is deemed useful or effective if it considers the affordances provided by all the interacting components of the system (Bennett & Flach, 2011). Affordances are "the acts or behaviors permitted by objects, places, and events" (Michaels & Carello, 1981, p. 42). ...
... Like custom interventions, custom assessmentswhich are essentially a specific type of intervention-are precise, sophisticated, and malleable enough to account for the specific combination of affordances in the interaction of content areas, assessment methods, and assessment criteria. This is one reason why custom interventions-and assessment instruments-are generally considered to be superior in predicting criteria of interest than OTS interventions or assessments (Bennett & Flach, 2011). ...
Programmatic curricular assessment is both necessary and desirable. Yet, the sheer scale of assessment projects can be daunting and there is currently no widely agreed upon model for assessment. Our assessment initiative’s novelty and scholarly value becomes apparent when viewed holistically as an evidence-based, multistep, comprehensive, in-house, sustainable assessment model. In this article, we detail the design, implementation, and results of our five-step curricular assessment initiative. First, the department faculty generated measurable departmental learning outcomes. Second, faculty mapped how individual courses in our curriculum met departmental goals. Third, faculty teams constructed common syllabi. Fourth, these teams generated multiple-choice questions (MCQs) and rubrics to measure knowledge acquisition and skill development. Fifth, to “close the assessment loop,” faculty members received assessment results to inform their teaching and further enhance students’ learning. Our iterative process also highlighted shortcomings in our procedures and instruments.
... In addition to the raw volume of data, the form in which data is presented to a user has a significant impact on cognitive workload and performance. Data should be integrated into meaningful information in a way that helps users understand its significance and anticipate how events and data are likely to evolve over time (Flach and Bennett 2011;Endsley and Jones 2024). A well-designed UI can present large quantities of potentially relevant information in a digestible format that highlights critical components of the work domain while simultaneously allowing details to be investigated, resulting in reduced cognitive workload, enhanced SA, and improved decision-making capabilities. ...
This report summarizes the cognitive workload challenges related to human-machine integrated formation (HMIF) drawing from scientific literature. It specifically addresses four key issues:
1) Individual cognitive load related to user interfaces and human–automation interactions.
2) Team-level impacts of cognitive load that extend beyond the individual operator.
3) Challenges to scaling cognitive demands with multiple autonomous systems.
4) The variability and evolution of cognitive workload and task demands over time in complex environments.
Areas for further consideration and specific recommendations are provided for HMIF development including additional language necessary for requirements documents, measures necessary in professional military education to prepare Soldiers for cognitive overload, and updates to policy (including DoDI 5000.95).
... One of the major challenges then, is for visual analytics designers to create designs that "... facilitate the discovery of meaningfulness of the situation ... not as a property of the mind, but rather as a property of the situation or functional problems that operators are trying to solve ... [by] developing representations that specify the meaningful properties of a work domain ... so that operators can discover these meaningful properties and can guide their actions appropriately" [BF11]. ...
Visual analytics systems combine machine learning or other analytic techniques with interactive data visualization to promote sensemaking and analytical reasoning. It is through such techniques that people can make sense of large, complex data. While progress has been made, the tactful combination of machine learning and data visualization is still under-explored. This state-of-the-art report presents a summary of the progress that has been made by highlighting and synthesizing select research advances. Further, it presents opportunities and challenges to enhance the synergy between machine learning and visual analytics for impactful future research directions.
... DSI may also shorten the time for the driver to understand spatial information by supporting the driver's understanding of the spatiotemporal structure of the scene. Furthermore, drivers may not have to rely on high-level knowledge, experience, and reasoning to take action, thus potentially preserving more attentional resources and cognitive capacity for other elements in the environment (Bennett and Flach, 2011;Merenda et al., 2018). DSI assists drivers' decision-making rather than replacing it, and the benefit is that it does not cause significant conflicts between interface information and drivers' mental models and SA when processing dangerous situations. ...
... We therefore believe that interface www.nature.com/scientificreports/ design should be grounded on a solid theoretical basis, while upholding the meaningfulness of perception as part of an agent/environment relationship 44 . Only in this way will designers be able to develop interface displays that convey this meaning to the operator. ...
When attempting to land on a ship deck tossed by the sea, helicopter pilots must make sure that the helicopter can develop sufficient lift to be able to safely touchdown. This reminder of affordance theory led us to model and study the affordance of deck‑landing‑ability, which defines whether it is possible to land safely on a ship deck depending on the helicopter’s available lift and the ship’s deck heave movements. Two groups of participants with no piloting experience using a laptop helicopter simulator attempted to land either a low‑lifter or a heavy‑lifter helicopter on a virtual ship deck by either triggering a pre‑programmed lift serving as the descent law if it was deemed possible to land, or aborting the deck‑landing maneuver. The deck‑landing‑ability was manipulated by varying the helicopter’s initial altitude and the ship’s heave phase between trials. We designed a visual augmentation making visible the deck‑landing‑ability, and thus enabling participants to maximize the safety of their deck‑landing attempts and reduce the number of unsafe deck‑landing. The visual augmentation presented here was perceived by participants as a means of facilitating this decision‑making process. The benefits were found to have originated from the clear‑cut distinction it helped them to make between safe and unsafe deck‑landing windows and the display of the optimal time for initiating the landing
... For example, the Ecological Interface Design approach borne from ecological psychology stresses engineering sophisticated domain understanding into tool design (Bennett & Flach, 2011;Vicente & Rasmussen, 1992). Other approaches stem from notions of situated cognition and focus on respecting the joint nature of engineered cognitive systems (Woods & Roth, 1988). ...
... The lack of a solid theory situates a field at the level of an art. In the case of UIs, this has been recognized by several authors, for example, Laurel [11] and Bennett and Flach [3]. In a paper published in 1981, T. P. Moran wrote about the UI: "it should be efficient to use and easy to learn; it should be consistent, logical, 'natural'; amen. ...
... Empirical research on EID is primarily based on a small-scale, representative thermohydraulic process simulator called Dual Reservoir System Simulation II (DURESS; Vicente, 1999) and college students (Vicente, 2002) to demonstrate the benefits of ecological interfaces relative to conventional ones. Subsequent research includes a number of highly representative studies employing high-fidelity simulators and professional operators (see e.g., Bennett & Flach, 2011;Jamieson, 2007). ...
Testing and evaluation of technology design for complex systems cannot readily attain conclusive results. This is because skilled professionals are often not available for testing while non-professionals may not be capable of operating the actual systems or high-fidelity simulators. Thus, practitioners and applied scientists can be challenged with decisions on selecting participant groups, which can severely constrain choices in the experimental tasks. This article presents the perspective of consequential validity, highlighting that general validity or rules to participant selection probably do not exist. Most importantly, the validity of a testing method or an empirical finding critically rests on the decisions of interest that must take into account nuances or idiosyncrasies of specific situations and desired outcomes. This perspective stands in contrast to how the literature predominantly portrays validity of testing methods or empirical findings as universal rather than focusing on outcomes within the confines of the study methods. The perspective of consequential validity calls for studies on how classical metrics of reliability and validity could manifest in consequence of specific decisions informed by empirical testing.
... Modeling or interacting with a socio-technical environment through a new display design is not a completely robust or scientific process (Bennett & Flach, 2011). A challenge with complex socio-technical environments is that they involve many variables that change over time, posing an infinite number of unique scenarios that cannot be studied or trained with users using a display design in advance. ...
Interface design and interaction testing is not a completely robust or scientific process. Rather, it is an artful, iterative, social process regulated by inter- or multi-disciplinary design teams. Lamentably, the design trajectory and iterations are often left out or partially communicated in the description of a published design. We offer display design considerations considering AI and propose a modular structure for the interface design and interaction testing process of teams. The key feature of the structure is that it considers display design and display interaction testing and general experimental design and enables teams to transparently communicate how they start, traverse, and end their design cycles. The system can be accessed via an established Industrial Engineering Society on the world wide web. Such a structure would enable studying where innovation and failure happen through iterations of steps taken by different design teams. Subsequently, the information can be used to enrich future research.
... In human factors literature, information displays and visualizations based on extensive analysis of a task domain have been studied extensively (Vicente, 1999;Vicente and Rasmussen, 1992;Burns and Hajdukiewicz, 2004;Bennett and Flach, 2019). Usually, these information displays are built to allow reasoning at multiple levels and allow users to access the information they need at the right time to make complex decisions or monitor the status of a system and deal with uncertainties. ...
Increasingly, artificial intelligence (AI) is being used to assist complex decision-making such as financial investing. However, there are concerns regarding the black-box nature of AI algorithms. The field of explainable AI (XAI) has emerged to address these concerns. XAI techniques can reveal how an AI decision is formed and can be used to understand and appropriately trust an AI system. However, XAI techniques still may not be human-centred and may not support human decision-making adequately. In this work, we explored how domain knowledge, identified by expert decision makers, can be used to achieve a more human-centred approach to AI. We measured the effect of domain knowledge on trust in AI, reliance on AI, and task performance in an AI-assisted complex decision-making environment. In a peer-to-peer lending simulator, non-expert participants made financial investments using an AI assistant. The presence or absence of domain knowledge was manipulated. The results showed that participants who had access to domain knowledge relied less on the AI assistant when the AI assistant was incorrect and indicated less trust in AI assistant. However, overall investing performance was not affected. These results suggest that providing domain knowledge can influence how non-expert users use AI and could be a powerful tool to help these users develop appropriate levels of trust and reliance.
... Ecological Interface Design (EID: Vicente and Rasmussen, 1992;Burns and Hajdukiewicz, 2004;Bennett and Flach, 2011) is a framework that helps designers develop displays that allow workers to cope with novelty and change. EID uses the Abstraction Hierarchy (AH; Rasmussen, 1985) as a modeling tool to identify the work domain constraints in terms of information content and structure. ...
... The feature acquisition was made using the Neurosky's Mindwave device for measuring the brain activity [23]. This simple and affordable device includes two electrodes, one for EEG dataset records (Fp1 channel), another for reference signals (the ear clip) and a power switch. ...
Human-Computer Interaction (HCI) is a multidisciplinary research area aiming the design of user-friendly systems. Even though systems are increasingly complex, recurring more and more to multimodal interactions, there are very basic aspects, such as colour and its correct perception, that continue to be crucial for effective and pleasurable communication. These aspects will be reflected in several areas and may be particularly useful in teaching-learning systems.
... The feature acquisition was made using the Neurosky's Mindwave device for measuring the brain activity [23]. This simple and affordable device includes two electrodes, one for EEG dataset records (Fp1 channel), another for reference signals (the ear clip) and a power switch. ...
Brain-Computer Interaction (BCI) technology can be used in several areas having recently gained increased interest with diverse applications in the area of Human Computer Interaction (HCI). In this area one of the central aspects relates to the ease of perceiving information. Typography is one of the central elements that, when properly used, can provide better readability and understanding of the information to be communicated. In this sense, this multidisciplinary work (typography and cognitive neuroscience) examines how the brain processes typographic information using EEG technology. In this context, the main goal of this work is to obtain information about the users when reading several words written in different typefaces and deduce theirs mental states (fatigue, stress, immersion) through user’s electroencephalogram signals (EEG). Additionally, several EEG features were extracted, namely the energy of Theta, Alpha and Beta waves, as well as, the variability of these bands’ energy. It is considered that this is a preliminary study in this area and may be extended to another type of design features.
... Each fish has two fins used to indicate trajectory. The length and angle between the fins is used to indicate the rate of travel using five discrete states to optimize scan-ability [12]: (1) long-close-below for rapidly unhealthy; ...
Cybersecurity operations are highly complex, requiring the coordination of specialized skills across multiple teams to successfully execute missions. Command and control within security operations centers is dominated by fragile mental models, demonstrating a need for systems that reinforce shared situational awareness across the organization. In this paper, we present the results of our research to: (1) define the needs associated with tactical cyber situational awareness; and (2) evaluate the usability and utility of a prototype tactical situational awareness dashboard. We found that incident tracking, tasking structure, execution timeline, and resource health constitute the essential aspects of tactical cyber situational awareness. Evaluations of prototypes suggest that three visualizations are well suited for conveying this information. We believe these results generalizable and will enable the development of tactical situational awareness capabilities in Security Operations Centers across public and private enterprises.
... Although the literature provides clues for designing tutorials (Bennett and Flach, 2011) and examples of tutorials more specifically for CMVs (e.g. Andrienko and Andrienko, 2001;Plaisant, 2005), there are no insights into how first-time users freely explore CMV geovisualization tools, nor how they learn to use them. ...
Maps are frequently combined with data displays in the form of coordinated and multiple views (CMV). Although CMV are valuable geovisualization tools, novice users may find them complex and thus require explanation. However, no tutorial guidelines have been developed that indicate what is helpful in understanding CMV geovisualization tools. We therefore conducted a study on the learnability of a CMV tool, informed with eye-tracking data, talk-aloud and interaction logs. We have investigated how untrained users work with a CMV geovisualization tool. The study revealed that: (1) despite their initial confusion, users found the tested tool pleasant to play with while getting to grips with how dynamic brushing works, (2) when examining the tool's interface, participants mainly looked freely at explanatory elements, such as labels and the legend, but they explored interactive techniques only to a limited degree. We conclude with tips about tutorial design and layout design for CMV tools.
The growing pervasiveness of robotic and embodied artificial intelligence systems in daily life and within cyber-physical environments highlights a complex web of challenges at the intersection of robotic safety, human-to-robot trust, and cybersecurity. This article explores these challenges by emphasising the crucial role of security in establishing and maintaining trust between humans and robots, which is integral to successfully adopting and operating these systems in human environments. Safety considerations include mitigating the risks of physical harm and environmental damage due to robotic malfunctions or cyberattacks, particularly in autonomous robots requiring high built-in safety measures. From a cybersecurity perspective, these systems face unique challenges due to their complex, interconnected software and hardware components that necessitate robust protection against data breaches to ensure secure data communication. Additionally, the dynamic interaction of these systems with the physical environment adds a layer of complexity, which makes the safety, security, and reliability of these interactions a vital component of the overall security strategy. This paper reviews these areas within the cyber-physical systems paradigm by focusing on engineering fail-safe mechanisms, the importance of trust and ethical responsibility in human-robot interactions, and the need for resilient cybersecurity measures. At this nexus, a table of crossover challenges illustrates the intricacy of integrating safety, trust, and security in robotic systems. This paper introduces “secure robotics” as a new paradigm to address these collective challenges with a novel model to provide a structured methodology for evaluating and enhancing robotic system performance that symbolises the convergence of theoretical constructs with empirical analysis. By defining secure robotics, the paper establishes a framework for advancing robotics in the cyber-physical era in alignment with current technological trends while anticipating future developments. This framework positions secure robotics as a key contributor to the evolution of cyber-physical systems.
How do we integrate artificial and human intelligence to use artificial intelligence to enhance our productivity, safety, and creativity? This book proposes that integrating AI into human teams will provide greater advantage than attempting to apply AI to replace human cognitive processing. Additionally, it provides methods for designing effective systems that include one or more humans and one or more AI entities to change the structure of human work.
Integrating Artificial and Human Intelligence through Agent Oriented Systems Design explores why teamwork is necessary today for complex work environments. The book explains the processes and methods humans employ to effectively team with one another and presents the elements of artificial agents that permit them to function as team members in joint human and artificial agent teams. It discusses design goals and illustrates how methods to model the complex interactions among human and artificial agents can be expanded to enable interaction design to attain shared goals. Model-Based Systems Engineering (MBSE) tools that provide logical designs of human–agent teams, the AI within these teams, training to be deployed for human and artificial agent team members, and the interfaces between human and artificial agent team members are all covered. MBSE files containing profiles and examples for building MBSE models used in the design approach are featured on the author’s website (https://lodesterresci.com/hat).
This book is intended for students, professors, engineers, and project managers associated with designing and developing AI systems or systems that seek to incorporate AI.
Checklists are a type of cognitive aid used to guide task performance; they have been adopted as an important safety intervention throughout many high-risk industries. They have become an ubiquitous tool in many medical settings due to being easily accessible and perceived as easy to design and implement. However, there is a lack of understanding for when to use checklists and how to design them, leading to substandard use and suboptimal effectiveness of this intervention in medical settings. The design of a checklist must consider many factors including what types of errors it is intended to address, the experience and technical competencies of the targeted users, and the specific tools or equipment that will be used. Although several taxonomies have been proposed for classifying checklist types, there is, however, little guidance on selecting the most appropriate checklist type, nor how differences in user expertise can influence the design of the checklist. Therefore, we developed an algorithm to provide guidance on checklist use and design. The algorithm, intended to support conception and content/design decisions, was created based on the synthesis of the literature on checklists and our experience developing and observing the use of checklists in clinical environments. We then refined the algorithm iteratively based on subject matter experts’ feedback provided at each iteration. The final algorithm included two parts: the first part provided guidance on the system safety issues for which a checklist is best suited, and the second part provided guidance on which type of checklist should be developed with considerations of the end users’ expertise.
Real-world events like the COVID-19 pandemic and wildfires in Australia, Europe, and America remind us that the demands of complex operational settings are met by multiple, distributed teams interwoven with a large array of artefacts and networked technologies, including automation. Yet, current models of human-automation interaction, including those intended for human-machine teaming or collaboration, tend to be dyadic in nature, assuming individual humans interacting with individual machines. Given the opportunities and challenges of emerging artificial intelligence (AI) technologies, and the growing interest of many organisations in utilising these technologies in complex operations, we suggest turning to contemporary perspectives of sociotechnical systems for a way forward. We show how ideas of distributed cognition, joint cognitive systems, and self-organisation lead to specific concepts for designing human-AI systems, and propose that design frameworks informed by contemporary views of complex work performance are needed. We discuss cognitive work analysis as an example.
Objective
As the world increasingly adopts renewable and sustainable energy systems, transitionary solutions include nuclear power, which currently provides 20% of the United States’ electricity and is the largest single source of carbon-free electricity generation. Advanced reactors are a critical component of a carbon-free mixed energy portfolio that require careful design of first-of-a-kind control rooms.
Background
The application of Human Factors Engineering (HFE) is essential for scientific and iterative testing of novel human–system interface (HSI) concepts to ensure effective, efficient, and safe plant operations. Microworlds are simulators that use simplified physics models and control systems to distill nuclear power operations into essential functions.
Method
HFE scientists used the Rancor Microworld Simulator to obtain preference and performance metrics for novel and traditional static HSI design styles. Participants comprised advanced reactor company employees and nuclear industry consultants. A mixture of quantitative and qualitative data was captured.
Results
There was a preference for the basic graphical style that included high contrast and traditional color scheme elements. No single HSI design outperformed the others, and the participants did not perform better using their preferred HSI style.
Conclusion
This experiment is the first in a series of HFE testing for HSIs in advanced reactor control room development. Clear user preferences emerged for elements within static displays. The cutting-edge neumorphic style was the least preferred. Future directions include tests of dynamic displays.
Application
HFE is used in evaluating and designing HSI devices that will improve the efficiency and safety of advanced nuclear power operations.
This concluding chapter on visualisation for, and of, safety, weaves together ideas put forward by the volume’s contributors. It analyses how visualisations and their role have changed over time, their co-evolution with key concepts in safety science and impact on cognitive representations built by practitioners, whether success should be assessed by level of adoption or by impacts on safety outcomes. A number of open questions for future research are outlined.KeywordsVisualisationUsabilityEvaluationSafety scienceCommunication
This chapter seeks to offer some explanation for the ubiquity of different types of visual representations in safety science. In particular, the chapter focuses on what these tell us about the thinking of safety researchers and practitioners, as well as how diagrams and other visual material influence their use of safety methods and tools.
Safety research and practice has struggled with how to describe, define and represent safety in order to improve understanding or to communicate its importance. Though visual representations are widely used, little research on visualisation and its impact has been undertaken. We provide a brief overview of existing work in this area, in areas including cognitive engineering and ethnography, and provide an introduction to the chapters that constitute this volume on the visualisation of safety.KeywordsVisualisationVisual artefactsMediaSafetyRisk
Safety visualisations and their influences on safety concepts are presented. Visualisations like safety posters show a clear message of fear and guilt. This changes after World War II, due to a more tolerant atmosphere. Latent, organisational factors as decisive elements of accident processes appear in visualisations. An example shows a method to follow accident scenarios in real time.
Maritime navigation is a demanding and complex domain that involves risks for people, the environment, and economic activity. The tasks associated with its execution require advanced training, expertise, experience, and a collaborative Navigation Team. Furthermore, naval operations demand higher readiness, accuracy, and resilience due to additional constraints. The response to these challenges has been integrating further automation and information systems. However, the effectiveness of innovative trends had been questioned by recent naval accidents like those involving the US and Norwegian naval ships.In bridge crews, collaboration is progressively more dependent on technological means since they are the information sources, and team members need to share and exchange different information formats besides audio. Furthermore, the increasing number of control functions and information systems required to strengthen the bridge situational awareness came with an additional cost to human operators. Therefore, navigation teams need further assistance in this challenging context to achieve a consistent and coherent situational awareness regarding the integrated systems in use, comprising technological and human agents' activities. The proposed solution under development is a Collaborative Decision Support System (C-DSS) fitted to the vessels' bridge systems requirements to reduce the cognitive workload, enhance collaboration between team members and information systems, and strengthen team situational awareness and sensemaking.Several studies addressed the need to provide enhanced interfaces with higher levels of abstraction representation, adjusted to the changed role of human operators, easily adaptable; improved collaboration between humans and automated agents, and superior information integration from internal and external environments. The most critical property of interfaces is to simplify the "discovery of the meaningfulness" of the problem space. World's representation should include the relevant and critical elements tailored to the task, augmenting the interaction experience, increasing the decision-making skill, and assisting the discovery of significant phenomena. The used methodology was an anthropocentric approach to innovation - design thinking. The process was performed with five phases: empathy, definition, idealization, prototyping and tests. Interface design prototypes were made with Mockups, covering the following several team roles. Usability tests, questionnaires and interviews were applied to validate and assess the C-DSS. Five focus group tests were made iteratively, with fifteen SMEs, twice with navigators, and once with SMEs from the other role, three in each iterative evaluation test, with a 1.5-hour duration. Following a snowball selection principle, participants were recruited from the Portuguese navy with the organization's guidance to ensure that all participants had an extensive seagoing experience.At the current stage of the C-DSS development, the results indicate significant potential for interface strategies. Results show that end-users would like to have the C-DSS, considering it innovative, friendly, easy to learn and with the information they need. The usability test allowed us to correct and improve numerous user interface design issues. The main difficulties maintained in terms of usability were related to recording data. The envisaged C-DSS is fitted to the vessels' bridge systems requirements embracing several prerequisites like being portable and customizable, enabling goals and priorities' management, logging performance and behavioural data, sharing different information formats, supporting information synchronization, providing situational awareness information about the system and operators.This study contributes to the understanding of the collaborative decision-making process in navigation teams through two objectives: first, systematising the main difficulties and challenges and, second, presenting a desirable solution, possible from a technological and financially viable point of view. The developed prototype has four distinct graphic interfaces, that complement each other and are oriented to the context of the user's role, based on the continuous contribution of target users, that is, elements belonging to navigation teams. The contributions allowed an improved understanding of the problem, idealise the solution, and improve the C-DSS, from design to insertion and adaptation of new functions.In the validation process of the prototype, it was found that the experts would like to use the C-DSS, stating that they would have greater autonomy and, even so, would be able to make an exceptional contribution to the team. Finally, the design thinking approach provided a basis for continuous feedback from end-users, becoming a twofold benefit by triggering new ideas of possible solutions to be deployed onboard.
The engineering disciplines supporting, and supported by, simulation are in the scope of this chapter of the SCS M&S Body of Knowledge. The chapter provides descriptions for systems engineering, virtual and augmented reality engineering, and visualization engineering.KeywordsModeling and simulationSystems engineeringM&S based systems engineeringSimulation systems engineeringLife cycle engineeringVirtual realityAugmented realityHuman-Computer Interaction Technology (HCI)
We applied the cognitive systems engineering (CSE)/ecological interface design (EID) approach to the work domain of flexible manufacturing systems (FMS). Work domain analyses (WDA) and activity analyses were conducted. Previous concerns regarding the suitability of CSE for FMS are addressed. Principles of EID (i.e., direct perception, direct manipulation, and visual momentum) were applied in designing an ecological interface for a flexible manufacturing system (EcoFlex). The critical links between the products of a WDA and the content of an ecological interface (a key innovation of EID) are made explicit. The evaluations of this interface were very positive across a wide variety of traditional and novel dependent variables (please see the companion manuscript). The overall success of this research program clearly demonstrates that the CSE/EID approach is capable of being applied to FMS. Contributions of this manuscript include both theoretical insights (defining properties of the FMS work domain, concrete examples of the principles of EID, critical links between CSE and EID, resolution of concerns about the overall framework) and practical applications (specific design solutions for FMS and similar work domains, and successful examples of the CSE/EID approach which can be studied and leveraged by students).
This chapter first provides a quick overview of user interface design process, and then discusses how Cognitive Systems Engineers apply work analysis to explore the larger context of use. Next, it provides an example to illustrate some of the thinking involved in designing a representation to support cognitive work. The chapter uses current work to develop a representation for managing a flexible manufacturing system to illustrate the process of translating a work analysis into a graphical representation. It focuses on the application of the abstraction/aggregation hierarchies during the preliminary stages of the flexible manufacturing systems (FMS) work domain analysis. The chapter illustrates how the products of the modeling effort are being used to inform the design of an ecological interface to support FMS operators in performing supervisory control activities. Finally, the chapter considers some of the broader emotive dimensions that are the concern of the user experience designer.
This book provides a great collection of work design testimonies with transferable lessons across many industry sectors and domains. It discusses physiological and cognitive parameters, teamwork, social aspects, organizational, and broader factors that influence work design initiatives.
It is important to learn from practitioner stories and real-world conditions that affect the theoretical applications of work design. Readers will benefit from understanding the struggles and successes of the authors. The chapters cover a wide spectrum of human factors and user needs, including decision making in (ab)normal and safety-critical situations, physical ergonomics, design-in-use modifications, and tailored training. The text examines holistic approaches that lead to improved work methods, worker engagement, and effective system-wide interventions.
Ergonomic Insights: Successes and Failures of Work Design is primarily written for professionals and graduate students in the fields of ergonomics, human factors, and occupational health and safety. Educators will also benefit from using these case studies in class lessons.
The cognitive systems engineering (CSE)/ecological interface design (EID) approach was applied in developing decision support for the flexible manufacturing system (FMS) work domain. Four interfaces were designed via the factorial application/non-application of direct perception (DP) and direct manipulation (DM). The capability of these interfaces to support performance in a simulated FMS was evaluated using a variety of traditional and novel dependent variables. The ecological interface (with DP, DM and an intact perception-action loop) provided clearly superior decision support (32 favorable significant results) relative to the other three interfaces (a combined total of 28 favorable significant results). The novel dependent variables were very sensitive. The results are interpreted from three different perspectives: traditional EID, the quality of constraint matching between triadic system components and closed-loop, dynamical control systems. The rationale for an expanded theoretical framework which complements, but does not replace, the original principles of CSE/EID is discussed. The potential for both specific interface features and novel dependent variables to generalize to real-world FMS applications is addressed. The expanded theoretical framework is universally relevant for the development of decision making and problem solving support in all computer-mediated work domains.
Ergonomics for environmental sustainability has been rapidly gaining attention in the scientific community. So far, a large part of the literature has focussed on specific dimensions of ergonomics for environmental sustainability, such as green designs, green buildings, environmental education, and sustainability frameworks. However, there is a necessity for an integrated study that presents the summary of published literature supported by detailed bibliometric characteristics. To address this gap, this study examined 418 articles on ergonomics for environmental sustainability and analysed them through bibliometric and network analysis. Major findings reveal the publication trends in ergonomics for environmental sustainability starting from 2011 to the present, the most productive and influential authors, and the most influential articles. This study also identifies the co-citation structure, bibliographical couplings and keyword co-occurrences among these articles. Findings from this study also provide a summary of the current research and present a robust roadmap for future directions in ergonomics for environmental sustainability.
Practitioner summary: This paper presents a bibliometric and network analysis of the academic literature in the domain of ergonomics for environmental sustainability. The study provides comprehensive insights into the relevant literature and identifies global research foci and future scopes. This study can guide practitioners in identifying the specific aspects of ergonomics for environmental sustainability to reduce global environmental impacts.
Effective on‐road safety requires proper maintenance of vehicles. In the trucking sector in India, there is a need for supporting predictive maintenance to decrease downtime and improve safety. Improving maintenance in this sector involves certain challenges. First, most trucks are owned by small‐scale fleet owners (trucks < 5). Second, maintenance is often handled by small‐scale mechanic workshops. The fault diagnosis is very often limited to recognition by the driver and later reassessed by the mechanic by relying on the feel or the sound of the vehicle. Third, a majority of stakeholders in this sector—drivers, mechanics, and owners—have low levels of education. Despite these challenges, with the increase in the rate of digitalization, in the future, it will be easier to monitor the health of the parts of a truck. In addition, there is a developing trend of mobile phone and internet penetration in India that has leapfrogged a majority of Indians into becoming “emergent users” of information technology. Therefore, this article shows that sociotechnical approaches such as ecological interface design can be used to develop mobile interfaces for supporting predictive maintenance through health and usage monitoring of trucks for small‐scale fleet owners in India. To develop the interface, a field study was conducted at several sites in the state of Tamil Nadu, India. The insights were used to develop scenarios and the abstraction hierarchy, which were later used creatively to develop the interface design for emergent users.
Interactive media art combines technical elements such as two-way communication, a hallmark of interactive media art, convergence expressed through complex media, and dynamic information transferability, which uses the network structure to propagate information dynamically. The technical elements of the winning works in the Prix Ars Electronica International Art Competitions “Interactive Art+” category were examined and analyzed, and it was identified that fusion with art is diversifying as technology advances. “Architecture of Radio,” by Richard Vijgen, reminds us that our digital lives are surrounded by physical signal systems that are not visible in the real world but in virtual space. Mary Flanagan’s “Help Me Know the Truth” claims that the surveillance system’s definition of a dangerous person reflects people’s unconscious bias. Lauren Lee McCarthy’s work “Someone” acknowledges the presence of innovative home services in a private space through the communication process between participants and visitors and the view that data and real-time filmed images are shared. Interactive media art fields that use these technological elements to converge art and technology are expected to accelerate as computer programming technology, multimedia technology, Internet and information communication technology, and digital content production.
This research focused on design concepts for the development of automation to support shared control of multiple unmanned aerial vehicles (UAVs) and the associated sensors used for surveillance in a distributed work system. Scenarios and storyboards were developed to study these design concepts in the context of troops in contact arising during convoy and search and rescue missions where multiple UAVs could be controlled by soldiers locally (at the site of an ambush) or remotely (from a Tactical Operations Center for a battalion or brigade), or by automation.
Storyboards were developed for three such scenarios and were used to conduct two cognitive walkthroughs with a total of 9 experienced soldiers. The results were used to complete cognitive task analyses defining roles and responsibilities and associated data and information requirements and to develop requirements for the design of such a distributed system. Generalizations regarding effective design for human-automation interaction in such a distributed work system were identified, emphasizing the need to provide benefit without being intrusive by supporting control and information display at different levels of abstraction through the use of pre-defined,mission-specific plays, by allowing fluid shifts in roles and responsibilities in order to redistribute the work, and by transitioning among different control paradigms (Management by Directive, Management by Permission, Management by Collaboration and Management by Exception).
Discussion Panel Abstract: The recent Boeing 737MAX accidents crystalized for the public the complexity of anticipating system and operator performance and developing a system design that prevents catastrophic outcomes. The operational situations, progression of flightcrew actions, and system behaviors that led to the two accidents had not been anticipated by the manufacturer or the regulator. These accidents were only the most recent examples of our failure to anticipate and manage operational complexities and operator performance. The art and science of human factors has yet to perfect risk assessment (or safety assessment) for complex systems.
In the not-so-distant past, system risk assessment made estimates of human error probabilities (HEPs) for specific operational tasks, which were combined with estimated equipment failure rates to produce an overall risk estimate. Indeed, these Human Reliability Analysis (HRA) techniques have evolved over decades and are still being developed (e.g., IDHEAS-ECA, Xing et al., 2020), partly because they satisfy the need for a simple quantitative threshold that can be used by industry and regulators: if risk probability is too high, change the design or some other aspect of operations.
Through the years, there have been critiques of the HRA approach (e.g., Hollnagel, 1998) that led to revisions, such as focusing on cognitive functions instead of operator tasks, but not to the basic quantitative risk-estimation approach. Other approaches to assessing risk/safety have wandered down other paths: attempting to capture system complexity from an operator’s perspective (Roth, Mumaw, Lewis, 1994; Nuclear Regulatory Commission, 2000), or better documenting the many ways in which system operators manage complexity daily to find ways to improve their capacity (Hollnagel, Woods, & Leveson, 2006). These approaches have used different measures than HEPs; e.g., measures of operator performance, measures of interface usability/design, measures of task complexity, and the analysis of system constraints.
In this panel, we offer different perspectives on risk/safety assessment as it relates to operator performance in complex systems. Foundational to assessment is deciding the nature of safety and the role of operator performance. Another important question is, as you move away from simple quantitative measures, how do you establish safety thresholds? That is, what guidance can we give to industry and regulators regarding how to measure safety and how to decide that action is required on the basis of safety.
This article aims to extend the traditional Ecological Interface Design (EID) process. With the rapid pace of digitalization in technological systems, there is a need for a design process that can handle the systemic design goals and the experiential basis of interaction. Currently, the traditional EID is well‐positioned to address these challenges. However, methodologically, it needs to be substantiated to make it accessible for designers, designing for novel arenas, such as Industry 4.0. Further, substantiating the design process will help designers, engineers, and human factors researchers with an accessible pathway that links the design brief to the final interface form. Therefore, this article delves into the “design” basis in the work of the Risø group and other prominent EID researchers and practitioners. In addition, it draws from a variety of themes in the discipline of design that addresses methodology. In particular, key insights are drawn from the design methods movement (1960s—); design, communication, and complexity from the Ulm school of design, 1953–1968; cognitive research conducted on designers and design activities; and finally, interaction design and communication design processes and models. All of these insights have been used syncretically to create the new integrated EID (iEID) process. This new design process consists of nine stages divided into three phases of divergence, transformation, and convergence, converting the initial design brief to the final interface. The steps of iEID are demonstrated using the example of interface design for a “digital twin” in the manufacturing sector.
Human operators do not necessarily perform better when receiving assistance from an automated aid than without the automated aid. The current work explored the impact of integrating the automated aid with the task information in low prevalence conditions. Specifically, we compared displays where the automated aid was integrated with task information in general or with another visual decision support aid. Subjects performed a speeded judgment task with the assistance of an automated aid, varying in display type, difficulty, and prevalence. Results indicated that participants performed less efficiently with the automated relative to without and that there was no added benefit of the visual decision support in terms of response times. Both decision supports improved participant’s sensitivity over no support, which may be beneficial for weakening the performance consequences of the low prevalence effect. Unexpectedly, we found that participants’ performance with each display was strongly dependent on when they experienced each display. It is possible that participants might be using strategies that complement one display over another, depending on the condition they see first. Automated aids could be used in real-world contexts to alleviate the effects of low target prevalence; however, the effectiveness may depend on experience with other interfaces.
New automated driving systems are constantly being developed and integrated into vehicles. At the current state of technology, these features still require drivers to monitor performance and resume control when required by the systems. To cue drivers to take control, a takeover request (TOR) is presented with auditory, visual, and haptic cues. To characterize current TOR practices, a literature review was conducted to review types of human-machine interfaces (HMI’s) and their associated message presentation. Twenty-six articles were identified after searching keywords across journal articles and conference proceedings. HMIs and message types were identified and classified. Results indicated that TORs are more commonly used as general alerts to gain driver attention to driving tasks, rather than to request drivers to engage in a specific action or explain context of the TOR. Literature suggests that future systems may focus more on not only alerting drivers but providing additional context to those alerts.
Operating two-wheeled vehicles in four-wheel-dominant environments presents unique challenges and hazards to riders, requiring additional rider attention and resulting in increased inherent risk. Emerging display and simulation solutions offer the unique ability to help mitigate rider risk–augmented, mixed, and virtual reality (collectively extended reality; XR) can be used to rapidly prototype and test concepts, immersive virtual and mixed reality environments can be used to test systems in otherwise hard to replicate environments, and augmented and mixed reality can fuse the real world with digital information overlays and depth-based sensing capabilities to enhance rider situational awareness. This paper discusses the use of multimodal applications of XR and integration with commercial off the shelf components to create safe riding technology suites. Specifically, the paper describes informal and formal research conducted regarding the use of haptic, audio, and visual hazard alerting systems to support hands-on, heads-up, eyes-out motorcycle riding, as well as the use of an immersive mixed reality connected bicycle simulator for rapidly and representatively evaluating rider safety-augmenting technologies in a risk-free environment.
Der vorgestellte Ausschnitt einer Forschungsarbeit zentriert die Frage: Wie
können schon in der Gestaltungsphase einer Benutzungsoberfläche, unabhängig
von der Expertise des Designers, die zusätzlichen informatorischen und arbeitsorganisatorischen
Anforderungen durch Mehrfachaufgaben Berücksichtigung
finden?
Um sich dieser Fragestellung zu nähern, wird eingangs ein Überblick über die
daran beteiligten Forschungsfelder gegeben (siehe Abbildung 1), z.B. Task
Switching und Ecological Interface Design. Parallel zur Erläuterung des theoretischen
Rahmens folgt die Beschreibung des Anwendungskontextes Leitzentralen
von Schleusen an Bundeswasserstraßen der Wasser- und Schifffahrtsverwaltung
des Bundes. Der Anwendungskontext wird weiterführend mittels Arbeitsgebietsanalyse
und Abstraktions-Dekompositions-Matrix beschrieben. Auf die Präsentation
dieser Ergebnisse folgen Zusammenfassung und Ausblick.
ResearchGate has not been able to resolve any references for this publication.