No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Economic Benefits of Human Performance Models in Systems Engineering
Abstract
Human performance modeling (HPM) is a method of quantifying human behavior, cognition, physical performance, and processes used for the development of systems designed for optimal user experience and interaction (Seibok and Wickens, 2013). Human Performance Models (HPMs) are an effective tool for Model Based Systems Engineering (MBSE) (Steinberg et al, 2017), with high economic benefit. Human Engineering practitioners use HPMs to define crew design requirements including the number of operators needed, the benefits of automation, and distribution of member task responsibilities required to operate a system. The application of HPMs throughout the development process helps to influence the system design by integrating with MBSE processes and informing many aspects of system definition and development including performance trades of algorithms, hardware, command and control functions, automation, and the need for situation awareness to monitor the system. In addition to these design benefits, HPMs save development costs and for a described case study reduced costs $4.3M demonstrating the economic benefits for systems engineering. This paper will outline the HPM process for a Command and Control (C2) system.
ResearchGate has not been able to resolve any citations for this publication.
The out-of-the-loop performance problem, a major potential consequence of automation, leaves operators of automated systems handicapped in their ability to take over manual operations in the event of automation failure. This is attributed to a possible loss of skills and of situation awareness (SA) arising from vigilance and complacency problems, a shift from active to passive information processing, and change in feedback provided to the operator. We studied the automation of a navigation task using an expert system and demonstrated that low SA corresponded with out-of-the-loop performance decrements in decision time following a failure of the expert system. Level of operator control in interacting with automation is a major factor in moderating this loss of SA. Results indicated that the shift from active to passive processing was most likely responsible for decreased SA under automated conditions.
The recent wave of interest in rhythms in human behavior is erroneously attributed to the recent developments in the study of biological rhythms. In a recent historical review of the field of behavioral rhythmic research lavie(1980) uncovered several independent roots of research unrelated to questions regarding the nature and functions of biological rhythms. One of those roots was the rather naive ambition of educational psychologists to schedule school hours according with the optimal times for cognitive functioning such as mathematics, reading, etc, on the one hand, and activities requiring psychomotor skills, on the other. This research which attracted quite a number of investigators around the turn of the century, died away around the mid 1920's. Recently it has been revived by the renewed interest in biological rhythms, sleep rhythms, and their interaction with behavior. The notion of an optimal schedule of human behavior is indeed an attractive one. In nature, optimal scheduling and synchronization of different behaviors with the geophysical environment is for many species a crucial survival issue. Displaying courting behavior at the wrong times of the year is dangerously maladaptive, while synchronization of courting and mating behavior with suitable environmental conditions ensures offspring survival.
An ultradian rhythm in waking performance on the order of 100 min. was hypothesized based on the period of the rapid eye-movement—non-rapid eye-movement intrasleep cycle. Errors on a visual continuous performance task (signal detection) over 6 hr. were analyzed for 19 Ss. Data analysis by fitting sinusoidal curves showed a peak at the expected frequency of 100 min.
The relationship between subjective workload and human behaviors
is considered. A model of the dynamics of this relationship is proposed.
Results of three simulation experiments are reported that show simple
linear identification algorithms are surprisingly robust in online
identification of noisy, nonlinear versions of this model. The
implications for developing adaptive aiding and decision support systems
are noted
Verification and validation (V&V) for human performance models (HPMs) can be likened to building a house with no bricks, since it is difficult to obtain metrics to validate a model when the system is still in development. HPMs are effective for performing trade-offs between the human system designs factors including number of operators needed, the role of automated tasks versus operator tasks, and member task responsibilities required to operate a system. On a recent government contract, our team used a human performance model to provide additional analysis beyond traditional trade studies. Our team verified the contractually mandated staff size for using the system. This task demanded that the model have sufficient fidelity to provide information for high confidence staffing decisions. It required a method for verifying and validating the model and its results to ensure that it accurately reflected the real world. The situation caused a dilemma because there was no actual system to gather real data to use to validate the model. It is a challenge to validate human performance models, since they support design decisions prior to system. For example, crew models are typically inform the design, staffing needs, and the requirements for each operator’s user interface prior to development. This paper discusses a successful case study for how our team met the V&V challenges with the US Air Force model accreditation authority and successfully accredited our human performance model with enough fidelity for requirements testing on an Air Force Command and Control program.
Objective:
We aimed to assess the impact of task demands and individual characteristics on threat detection in baggage screeners.
Background:
Airport security staff work under time constraints to ensure optimal threat detection. Understanding the impact of individual characteristics and task demands on performance is vital to ensure accurate threat detection.
Method:
We examined threat detection in baggage screeners as a function of event rate (i.e., number of bags per minute) and time on task across 4 months. We measured performance in terms of the accuracy of detection of Fictitious Threat Items (FTIs) randomly superimposed on X-ray images of real passenger bags.
Results:
Analyses of the percentage of correct FTI identifications (hits) show that longer shifts with high baggage throughput result in worse threat detection. Importantly, these significant performance decrements emerge within the first 10 min of these busy screening shifts only.
Conclusion:
Longer shift lengths, especially when combined with high baggage throughput, increase the likelihood that threats go undetected.
Application:
Shorter shift rotations, although perhaps difficult to implement during busy screening periods, would ensure more consistently high vigilance in baggage screeners and, therefore, optimal threat detection and passenger safety.
Human performance modeling provides a tool to facilitate the development of well-designed systems that support their users. However, models need to be based on and compared against empirical human-in-the-loop data to ensure that the human performance predictions are correct. This paper describes an approach to gathering already-existing empirical data to support the development of a model of astronaut performance in long-duration space missions.
Over the past 50 years, computers have undergone transformation from monolithic number crunchers, to centralized repositories of management information systems, to distributed, networked, cyberspace support systems. During the same period, uses of computers have moved from computational problems to life support, from machine language to GUIs, from abstractions of work to virtual reality on the World-Wide Web. These transformations have brought with them situations that have ethical implications.
Adverse drug events (ADEs) are a major cause of morbidity in hospitalised and ambulatory patients. Computerised provider order entry (CPOE) combined with clinical decision support systems (CDSS) are being widely implemented with the goal of preventing ADEs, but the effectiveness of these systems remains unclear.
We searched the specialised database Agency for Healthcare Research and Quality (AHRQ) Patient Safety Net to identify reviews of the effect of CPOE combined with CDSS on ADE rates in inpatient and outpatient settings. We included systematic and narrative reviews published since 2008 and controlled clinical trials published since 2012.
We included five systematic reviews, one narrative review and two controlled trials. The existing literature consists mostly of studies of homegrown systems conducted in the inpatient setting. CPOE+CDSS was consistently reported to reduce prescribing errors, but does not appear to prevent clinical ADEs in either the inpatient or outpatient setting. Implementation of CPOE+CDSS profoundly changes staff workflow, and often leads to unintended consequences and new safety issues (such as alert fatigue) which limit the system's safety effects.
CPOE+CDSS does not appear to reliably prevent clinical ADEs. Despite more widespread implementation over the past decade, it remains a work in progress.
Associations of computing specialists have adopted codes of ethics parallel to those of professional societies of engineers and scientists. These codes have attempted to address the special ethical dilemmas posed by computing. Difficulties in obtaining consensus about ethical imperatives make the development, revision, and maintenance of these codes problematic. The codes of professional ethics from 10 associations are compared with respect to content and focus. A recent revision of the Code of Ethics for the Association for Computing Machinery (ACM) is offered as a model for social science computing specialists. Keywords: ethics, codes of ethics, computer professionals, computers and society.
To measure dispensing accuracy rates in 50 pharmacies located in 6 cities across the United States and describe the nature and frequency of the errors detected.
Cross-sectional descriptive study.
Chain, independent, and health-system pharmacies (located in hospitals or managed care organizations).
Pharmacy staff at randomly selected pharmacies in each city who accepted an invitation to participate.
Observation by a pharmacist in each pharmacy for 1 day, with a goal of inspecting 100 prescriptions for dispensing errors (defined as any deviation from the prescriber's order).
Dispensing errors on new and refill prescriptions.
Data were collected between July 2000 and April 2001. The overall dispensing accuracy rate was 98.3% (77 errors among 4,481 prescriptions; range, 87.2%-100.0%; 95.0% confidence interval, +/- 0.4%). Accuracy rates did not differ significantly by pharmacy type or city. Of the 77 identified errors, 5 (6.5%) were judged to be clinically important.
Dispensing errors are a problem on a national level, at a rate of about 4 errors per day in a pharmacy filling 250 prescriptions daily. An estimated 51.5 million errors occur during the filling of 3 billion prescriptions each year.
Workload Thresholds for Human Performance Models Proceedings of the Human Factors and Ergonomics Society Annual Meeting
- D. Steinberg
- D. Donohoo
- L. Strater
- A. Diggs
Guidelines for Bias‐Free Writing
- M. Schwartz
Using meta‐analyses results and data gathering to support human performance model development Proceedings of the Human Factors and Ergonomics Society Annual Meeting
- A. Sebok
- C. Wickens
- R. Sargent