Article

Workload Thresholds for Human Performance Models

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Human performance modeling (HPM) can be an effective tool to use for determining crew designs. Crew design includes determining the number of operators needed, the role of automation, and member task responsibilities required to operate a system. Without effective measures of performance and thresholds for assessing success, design decisions from HPM will be erroneous. Operator tasks can be assigned and allocated to crew members in a simulation to estimate the workload for each operator during a period of performance. The methods for determining when an operator exceeds workload thresholds create challenges for those using HPM for crew design. Some types of analysis have more clearly defined thresholds. For example, if a military operator has too many tasks to complete to effectively initiate countermeasures between the times they receive a warning until the time the threat arrives, they are overloaded and cannot complete their mission. However, many missions do not have such a severe penalty for not completing the tasks within a given time. For example, pharmacists, satellite managers, traffic managers, food service workers do not have such stringent task timing completion thresholds. For example, the penalty for a food service provider to be overloaded is typically extended wait times rather than risk of a loss of life. For these types of operational situations, determining overload is much more challenging. This paper describes a new workload thresholds for operator workflow models. It incorporates the vigilance effort, the maximum time a crew member will be fully loaded, and determining the maximum time worked without a break.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
The out-of-the-loop performance problem, a major potential consequence of automation, leaves operators of automated systems handicapped in their ability to take over manual operations in the event of automation failure. This is attributed to a possible loss of skills and of situation awareness (SA) arising from vigilance and complacency problems, a shift from active to passive information processing, and change in feedback provided to the operator. We studied the automation of a navigation task using an expert system and demonstrated that low SA corresponded with out-of-the-loop performance decrements in decision time following a failure of the expert system. Level of operator control in interacting with automation is a major factor in moderating this loss of SA. Results indicated that the shift from active to passive processing was most likely responsible for decreased SA under automated conditions.
Article
Full-text available
The recent wave of interest in rhythms in human behavior is erroneously attributed to the recent developments in the study of biological rhythms. In a recent historical review of the field of behavioral rhythmic research lavie(1980) uncovered several independent roots of research unrelated to questions regarding the nature and functions of biological rhythms. One of those roots was the rather naive ambition of educational psychologists to schedule school hours according with the optimal times for cognitive functioning such as mathematics, reading, etc, on the one hand, and activities requiring psychomotor skills, on the other. This research which attracted quite a number of investigators around the turn of the century, died away around the mid 1920's. Recently it has been revived by the renewed interest in biological rhythms, sleep rhythms, and their interaction with behavior. The notion of an optimal schedule of human behavior is indeed an attractive one. In nature, optimal scheduling and synchronization of different behaviors with the geophysical environment is for many species a crucial survival issue. Displaying courting behavior at the wrong times of the year is dangerously maladaptive, while synchronization of courting and mating behavior with suitable environmental conditions ensures offspring survival.
Article
Full-text available
Observers' perceptual sensitivity to critical target events can deteriorate when they must remain alert for prolonged periods. The dominant characterization of this sensitivity decrement has been the R. Parasuraman and D. R. Davies (1977) taxonomy of vigilance, which attributes the decline in perceptual ability to the type of discrimination (successive-absolute vs simultaneous-comparative judgment) and the event rate (rate of stimulus presentation) required for task completion. According to the model, sensitivity decrements occur chiefly in tasks that couple successive discrimination with a high event rate. This meta-analysis of 42 vigilance studies attempted to refine the taxonomy by identifying other task characteristics related to the sensitivity decrement. The analysis confirmed that the magnitude of this sensitivity decrement is substantial and that it is a function of the type of discrimination and the event rate, but it also revealed that the current taxonomy should be revised to incorporate a new dimension: the sensory–cognitive distinction. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Following the logic of a prior experiment (Seligman et al., 1990) with varsity collegiate swimmers, sixty student volunteers performed a brief, but highly stressful vigilance task. Half were then given false feedback indicating poor performance; the others experienced a non-evaluative display at that point. All were then asked to repeat the vigil. Half the observers were assessed as high optimists and half as high pessimists. The pessimists showed a steeper vigilance decrement than the optimists, consistent with a model (Scheier and Carver, 1987) that proposes that pessimists are more emotion-focused under stress than optimists, and hence would be less attentive to the vigilance display, regardless of feedback condition. The false negative feedback actually turned out to be ineffective; hence, based on Seligman's account of the swimmer study, which requires effective negative feedback, there should have been no vigilance performance difference between optimists and pessimists, either pre- or post-feedback.
Article
Full-text available
An ultradian rhythm in waking performance on the order of 100 min. was hypothesized based on the period of the rapid eye-movement—non-rapid eye-movement intrasleep cycle. Errors on a visual continuous performance task (signal detection) over 6 hr. were analyzed for 19 Ss. Data analysis by fitting sinusoidal curves showed a peak at the expected frequency of 100 min.
Article
Full-text available
In the first part of the reported research, 12 instrument-rated pilots flew a high-fidelity simulation, in which air traffic control presentation of auditory (voice) information regarding traffic and flight parameters was compared with advanced display technology presentation of equivalent information regarding traffic (cockpit display of traffic information) and flight parameters (data link display). Redundant combinations were also examined while pilots flew the aircraft simulation, monitored for outside traffic, and read back communications messages. The data suggested a modest cost for visual presentation over auditory presentation, a cost mediated by head-down visual scanning, and no benefit for redundant presentation. The effects in Part 1 were modeled by multiple-resource and preemption models of divided attention. In the second part of the research, visual scanning in all conditions was fit by an expected value model of selective attention derived from a previous experiment. This model accounted for 94% of the variance in the scanning data and 90% of the variance in a second validation experiment. Actual or potential applications of this research include guidance on choosing the appropriate modality for presenting in-cockpit information and understanding task strategies induced by introducing new aviation technology.
Article
Full-text available
Robertson, Manly, Andrade, Baddeley, and Yiend (1997) proposed that the decline in performance efficiency over time in vigilance tasks (the vigilance decrement) is characterized by "mindlessness" or a withdrawal of attentional effort from the monitoring assignment. We assessed that proposal using measures of perceived mental workload (NASA-TLX) and stress (Dundee Stress State Questionnaire). Two types of vigilance task were employed: a traditional version, wherein observers made button-press responses to signify detection of rarely occurring critical signals, and a modified version, developed by Robertson et al. to promote mindlessness via routinization, wherein button-press responses acknowledged frequently occurring neutral stimulus events and response withholding signified critical signal detection. The vigilance decrement was observed in both tasks, and both tasks generated equally elevated levels of workload and stress, the latter including cognitions relating to performance adequacy. Vigilance performance seems better characterized by effortful attention (mindfulness) than by mindlessness. Actual or potential applications of this research include procedures to reduce the information-processing demand imposed by vigilance tasks and the stress associated with such tasks.
Article
Full-text available
The workload and stress associated with a 40-min vigilance task were examined under conditions wherein observers monitored an auditory or a visual display for changes in signal duration. Global workload scores fell in the midrange of the NASA Task Load Index, with scores on the Frustration subscale increasing linearly over time. These effects were unrelated to the sensory modality of signals. However, sensory modality was a significant moderator variable for stress. Observers became more stressed over time as indexed by responses to the Dundee Stress State Questionnaire, with evidence of recovery in the auditory but not the visual condition toward the end of the watch. This result and the finding that signal detection accuracy - although equated for difficulty under alerted conditions - favored the auditory mode, indicate that display modality and time on task should be considered carefully in the design of operations requiring sustained attention in order to enhance performance and reduce stress. Actual or potential applications of this research include domains in which monitoring is a crucial part, such as baggage screening, security operations, medical monitoring, and power plant operations.
Article
The theoretical framework presented in this article explains expert performance as the end result of individuals' prolonged efforts to improve performance while negotiating motivational and external constraints. In most domains of expertise, individuals begin in their childhood a regimen of effortful activities (deliberate practice) designed to optimize improvement. Individual differences, even among elite performers, are closely related to assessed amounts of deliberate practice. Many characteristics once believed to reflect innate talent are actually the result of intense practice extended for a minimum of 10 years. Analysis of expert performance provides unique evidence on the potential and limits of extreme environmental adaptation and learning.
Article
Objective: We aimed to assess the impact of task demands and individual characteristics on threat detection in baggage screeners. Background: Airport security staff work under time constraints to ensure optimal threat detection. Understanding the impact of individual characteristics and task demands on performance is vital to ensure accurate threat detection. Method: We examined threat detection in baggage screeners as a function of event rate (i.e., number of bags per minute) and time on task across 4 months. We measured performance in terms of the accuracy of detection of Fictitious Threat Items (FTIs) randomly superimposed on X-ray images of real passenger bags. Results: Analyses of the percentage of correct FTI identifications (hits) show that longer shifts with high baggage throughput result in worse threat detection. Importantly, these significant performance decrements emerge within the first 10 min of these busy screening shifts only. Conclusion: Longer shift lengths, especially when combined with high baggage throughput, increase the likelihood that threats go undetected. Application: Shorter shift rotations, although perhaps difficult to implement during busy screening periods, would ensure more consistently high vigilance in baggage screeners and, therefore, optimal threat detection and passenger safety.
Article
A function of North American Aerospace Defence (NORAD) in North Bay, Ontario, is to identify all aircraft entering Canadian airspace. The first step in performing this task is to detect visually the presence of aircraft from either radar or transponder information presented on display consoles. This challenging, real-world vigilance task was used to investigate factors affecting detection latencies. The experiment revealed that performance varied as a function of geographic area of coverage, the midnight shift was particularly sensitive to vigilance decrements, and a vigilance decrement effect can occur in a real-world task, but this effect is not as strong as those reported in laboratory studies.
Article
"Laboratory researches on prolonged visual search led to the development of an experimental situation in which the trend of ability at synthetic look-out duties was studied throughout long watchkeeping spells by the automatic production of occasional brief and barely visible signals. These stimuli were given at irregular intervals… ." Interpretation of results is made in terms of a conditioned stimulus formulation. 25 references. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
For half a century, Sleep and Wakefulness has been a valuable reference work. It discusses phases of the sleep cycle, experimental work on sleep and wakefulness, sleep disorders and their treatment, and such sleep-like states as hypnosis and hibernation.
Article
The percentage of detection of 37 studies of vigilance, using simple signals, was found to depend primarily on the initial or pretest detection level, the nature of the signal, i.e., whether it is a dynamic signal (requires movement or change of state of the eye) or static, and the duration of the watch. Using the data of these studies, a watchkeeping decremental function was developed. It was shown that the function can be used to estimate a tradeoff between time of watch and the initial percentage of detection. The loss of detection associated with static signals was found to be relatively small. It is suggested that it is this small loss which represents a vigilance process. The greater decrements associated with dynamic stimuli are assumed to be due to an additional ocular demand.
Article
Boredom and monotony are generally conceded to be negative factors that can have adverse effects on morale, performance, and quality of work. This article examines the evidence for yet another claimed effect of boredom and monotony, viz., that these factors are stressors, and that because they are stressors, they may produce effects even more detrimental than those mentioned above. Both laboratory and field studies are examined for evidence of increased neuroendocrine activity during exposure to conditions determined to be, or generally acknowledged to be, boring or monotonous. It is concluded that the available data offer no support for the belief that boredom or monotony per se produces the syndrome of stress. However, monotony coupled with a need to maintain high levels of alertness could represent a combination capable of eliciting considerable stress.
Article
To measure dispensing accuracy rates in 50 pharmacies located in 6 cities across the United States and describe the nature and frequency of the errors detected. Cross-sectional descriptive study. Chain, independent, and health-system pharmacies (located in hospitals or managed care organizations). Pharmacy staff at randomly selected pharmacies in each city who accepted an invitation to participate. Observation by a pharmacist in each pharmacy for 1 day, with a goal of inspecting 100 prescriptions for dispensing errors (defined as any deviation from the prescriber's order). Dispensing errors on new and refill prescriptions. Data were collected between July 2000 and April 2001. The overall dispensing accuracy rate was 98.3% (77 errors among 4,481 prescriptions; range, 87.2%-100.0%; 95.0% confidence interval, +/- 0.4%). Accuracy rates did not differ significantly by pharmacy type or city. Of the 77 identified errors, 5 (6.5%) were judged to be clinically important. Dispensing errors are a problem on a national level, at a rate of about 4 errors per day in a pharmacy filling 250 prescriptions daily. An estimated 51.5 million errors occur during the filling of 3 billion prescriptions each year.
The Way We’re Working Isn’t Working
  • T Schwartz
  • J Gomes
  • C Mccarthy
Argumenation and Debate: Critical Thinking for Reasoned Decision Making
  • A J Freely
  • D L Steinberg
Researches on the measurement of human performance Oxford, UK: His Majesty’s Stationery Office
  • N H Mackworth
Sustained Military Performance in Continuous Operations: Combatant Fatique, Rest and Sleep Needs
  • Gerald P Krueger
Argumenation and Debate: Critical Thinking for Reasoned Decision Making, 13 th Edition. Wadsworth, Cengage Learning
  • E Flynn
  • K Barker
  • B Carnahan
  • A J Freely
  • D L Steinberg
  • G G Globus
  • R L Drury
  • E C Phoebus
  • R Boyd
Flynn, E, Barker, K., and Carnahan, B., "National Observational Study of Prescription Dispensing Accuracy and Safety in 50 Pharmacies" J Am Pharm Assoc. 2003;43:191-200. Freely, A.J., Steinberg, D.L., Argumenation and Debate: Critical Thinking for Reasoned Decision Making, 13 th Edition. Wadsworth, Cengage Learning. 2014. Globus, G.G., Drury, R.L., Phoebus, E.C. and Boyd, R. Ultradian rhythms in human performance. Percept. Mot. Skills 33, 1171-1174, 1971.
Ultradian Rythms in Prolonged Human Performance" US Army Research Institute For the Behavioral and Social Sciences. 95-30
  • P Lavie
  • J Zomer
  • D Gopher
Lavie, P., Zomer, J, Gopher D, "Ultradian Rythms in Prolonged Human Performance" US Army Research Institute For the Behavioral and Social Sciences. 95-30. 1995. Mackworth, N. H. (1948). The breakdown of vigilance during prolonged visual search. Quarterly Journal of Experimental Psychology, 1, 6-21. doi: 10.1080/17470214808416738
When and Why Threats Go Undetected: Impacts of Event Rate and Shift Length on Threat Detection Accuracy During Airport Baggage Screening
  • N H Mackworth
  • R Meuter
  • I Lacherez
  • P F Pigeau
  • R A Angus
  • R G Neill
Mackworth, N. H. (1950). Researches on the measurement of human performance (Medical Research Council Special Report Ser. No. 268.). Oxford, UK: His Majesty's Stationery Office. Meuter, R., F., I., Lacherez, P.F., "When and Why Threats Go Undetected: Impacts of Event Rate and Shift Length on Threat Detection Accuracy During Airport Baggage Screening," Human Factors Journal, April 2016. Pigeau, R. A., Angus, R. G, O'Neill, P., "Vigilance Latencies to Aircraft Detection among NORAD Surveillance Operators.", Human Factors. Vol 37, Sept 1995. See, J. E., Howe, S. R., Warm, J. S., & Dember, W. N. (1995). Meta-analysis of the sensitivity decrement in vigilance. Psychological Bulletin, 117, 230-249. doi: 10.1037/0033 2909.117.2.230
Meta-analysis of the sensitivity decrement in vigilance
  • J E See
  • S R Howe
  • J S Warm
  • W N Dember
See, J. E., Howe, S. R., Warm, J. S., & Dember, W. N. (1995). Meta-analysis of the sensitivity decrement in vigilance. Psychological Bulletin, 117, 230-249. doi: 10.1037/0033 2909.117.2.230