Article

Applied Cognitive Task Analysis (ACTA) Instructional Software: A Practitioner's Window into Skilled Decision Making

Authors:
  • Applied Decision Science, LLC
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... CTA interviews. Using multiple CTA methods (Crandall et al., 2006), we captured the critical tasks and their relationships, beginning interview sessions with the task diagram method (Hutton & Militello, 1996). To understand the associated challenges of operational experience relevant to future unmanned helicopter resupply, we applied the critical decision method (CDM), a type of CTA method (Crandall et al., 2006;Klein, Calderwood, & Macgregor, 1989). ...
... Probes are then used to elicit aspects of expertise. To flesh out the envisioned-world aspects of the current problem, we used simulation interviews (Hutton & Militello, 1996) to walk through a range of anticipated AACUS mission scenarios and anomalies. ...
Article
Full-text available
Ensuring that unmanned aerial systems' (UAS) control stations include a tight coupling of systems engineering with human factors, cognitive analysis, and design is key to their success. We describe a combined cognitive task analysis (CTA) and design thinking effort to develop interfaces for an operator controlling an autonomous helicopter, a prototype system that the Office of Naval Research is developing. We first conducted CTA interviews with subject-matter experts having expertise in UAS flight operations, helicopter resupply, military ground forces, and marine airspace control. Data informed the development of analysis products, including human-system interface requirements, which drove the creation of design concepts through ideation sessions using design thinking methods. We validated and refined the design concepts with UAS pilots. We provide an overview of our process, illustrated by details of a timeline display development. Significant aspects of our work include close integration of CTA and design thinking efforts, designing for an "Üenvisioned world" of interaction with highly autonomous helicopter systems, and the importance of knowledge elicitation early in system design. This effort represents a successful demonstration of an innovative design process in developing UAS interfaces.
... We elicited information about the environments in which AACUS will be used, the challenges associated with operating manned and unmanned rotorcraft, the cues to which they attended during these missions, and other cognitive factors associated with the future AACUS mission and operating environments. We used CTA methods such as the task diagram (Hutton & Millitello, 1996) to understand procedural tasks involved in flying and resupply and the critical decision method (Crandall & Getchell-Reiter, 1993) to elicit context-rich lived incidents. We also used the simulation interview method (Hutton & Millitello, 1996) in which participants were asked to roleplay as a Marine logistics specialist at a COP, interacting with an autonomous helicopter providing resupply, as we walked them into a detailed scenario with contingencies and 'what-ifs.' ...
... We used CTA methods such as the task diagram (Hutton & Millitello, 1996) to understand procedural tasks involved in flying and resupply and the critical decision method (Crandall & Getchell-Reiter, 1993) to elicit context-rich lived incidents. We also used the simulation interview method (Hutton & Millitello, 1996) in which participants were asked to roleplay as a Marine logistics specialist at a COP, interacting with an autonomous helicopter providing resupply, as we walked them into a detailed scenario with contingencies and 'what-ifs.' ...
Article
Full-text available
This paper reports on a research project that combined cognitive task analysis (CTA) methods with innovative design processes to develop a handheld device application enabling a non-aviator to interact with a highly autonomous resupply helicopter. In recent military operations, unmanned helicopters have been used to resupply U.S. Marines at remote forward operating bases (FOBs) and combat outposts (COPs). This use of unmanned systems saves lives by eliminating the need to drive through high-risk areas for routine resupply. The U.S. Navy is investing in research to improve the autonomy of these systems and the design of interfaces to enable a non-aviator Marine to safely and successfully interact with an incoming resupply helicopter using a simple, intuitive handheld device application. In this research, we collected data from multiple stakeholders to develop requirements, use cases, and design storyboards that have been implemented and demonstrated during flight tests in early 2014.
... Due to the complexity of the task environment, each error may impact task performance in several different and sometimes subtle ways, making tractability difficult. Generally, a task analysis (TA) is a method used to examine the interactions between mental processes and behavioral responses (e.g., human-machine interaction, training effectiveness, etc.) that lead to effective complex task performance (1,2,3), so that the sources of likely errors can be addressed in task redesigning and training. ...
... Thus, it is not intended to be a finalized version at any level, but rather to provide a guide for more detailed conversations with soldiers and related subject-matter experts. In order to evaluate and refine the preliminary HTA, the researcher will interact with experienced soldiers in structured meetings within the laboratory to develop a comprehensive concept mapping of their knowledge (1,2). The knowledge elicitation process will also include requesting detailed descriptions of routine and non-routine tasks (e.g., Critical Decision Method; 8), highlighting where difficulties most often arise. ...
Article
Full-text available
Military personnel involved in convoy operations are often required to complete multiple tasks within tightly constrained timeframes, based on limited or time-sensitive information. Current simulations are often lacking in fidelity with regard to team interaction and automated agent behavior; particularly problematic areas include responses to obstacles, threats, and other changes in conditions. More flexible simulations are needed to support decision making and train military personnel to adapt to the dynamic environments in which convoys regularly operate. A hierarchical task analysis approach is currently being used to identify and describe the many tasks required for effective convoy operations. The task decomposition resulting from the task analysis provides greater opportunity for determining decision points and potential errors. The results of the task analysis will provide guidance for the development of more targeted simulations for training and model evaluation from the driver's perspective.
Article
Clinical decision support (CDS) is a process for enhancing health-related decisions and actions with pertinent, organized clinical knowledge and patient information that can significantly improve health outcomes and healthcare delivery. However, their impact on clinical outcomes has been inconsistent. Rigorous and continuous evaluation of CDS is necessary for improving CDS. A User and Task analysis was conducted to understand the stakeholder roles, their goals and tasks involved in the evaluation of CDS. This study describes a framework for evaluating CDS effectiveness for improving quality outcomes based on the analysis.
Article
Full-text available
Wildland firefighting often involves the creation of a fireline (a break in vegetative fuels), an operation commonly hindered by the break-down of gas-powered chainsaws. Some firefighters may not possess the knowledge and skills needed to address break-downs quickly, which threatens productivity and safety. The Applied Cognitive Task Analysis method was used to examine the troubleshooting process of an expert wildland fire sawyer. This included elicitations of key steps in this process, specific pieces of valuable knowledge, and sources of expertise. These results show that much of the expert understanding of complex faults was developed on-the-job rather than during a formal training program. This study highlights areas where training and job aids may be improved to support wildland firefighters in chainsaw troubleshooting and provides preliminary support of ACTA as a tool for training specialists in this domain.
Conference Paper
This paper summarizes on-going work on 3D sensing and imaging for unmanned aerial vehicles (UAV) carried laser sensors. We study sensor concepts, UAVs suitable for carrying the sensors, and signal processing for mapping and target detection applications. We also perform user studies together with the Swedish armed forces (SwAF), to evaluate usage in their mission cycle and interviews to clarify how to present data. Two ladar sensor concepts for mounting in UAV are studied. The discussion is based on known performance in commercial ladar systems today and predicted performance in future UAV applications. The small UAV is equipped with a short-range scanning ladar. The system is aimed for real-time situational analysis of small areas and for documentation of situations. The large UAV is equipped with a high-performing photon counting ladar with array detector. Its purpose is to support large-area surveillance, intelligence and mapping operations. Based on these sensors and their performance, signal and image processing support for data analysis is analyzed. Genera ted data amounts are estimated and demands on data storage capacity and data transfer is analyzed. The usage of 3D mapping have been tested together with the SwAF, where 3D maps was used in the planning phase and as last-minute intelligence update of the target area. Feedback from these tests are presented. Interviews with various military professions are performed, to get better understanding of how 3D data are used and interpreted. Approaches of how to present data from 3D imaging sensors for users are also discussed. Keywords: 3D data, UAV, target detection, mapping
Article
Full-text available
Recent years have witnessed strong progress in understanding how people make decisions in operational settings. The emerging field of Naturalistic Decision Making (NDM) is at a point to afford system developers (including design engineers, human factors engineers, ergonomics specialists) different tools and methods for designing interfaces/systems that will better support decision making in those settings. Decision requirements can be identified from the early conceptual design phase through redesign. The NDM framework attempts to describe the way in which people handle difficult conditions within the context of the overall setting or task. This SOAR describes various decision strategies used by individuals and teams to assess a situation, diagnose a problem, and select a course of action. The impact of stress upon these strategies is also considered. To help understand what people are thinking as they perform difficult tasks, the procedures for conducting Cognitive Task Analyses to examine design requirements are also examined.
Article
Full-text available
The representation of physics problems in relation to the organization of physics knowledge is investigated in experts and novices. Four experiments examine (a) the existence of problem categories as a basis for representation; (b) differences in the categories used by experts and novices; (c) differences in the knowledge associated with the categories; and (d) features in the problems that contribute to problem categorization and representation. Results from sorting tasks and protocols reveal that experts and novices begin their problem representations with specifiably different problem categories, and completion of the representations depends on the knowledge associated with the categories. For, the experts initially abstract physics principles to approach and solve a problem representation, whereas novices base their representation and approaches on the problem's literal features.
Article
Full-text available
To place the arguments advanced in this paper in alternative points of view with regard to mental models are reviewed. Use of the construct in areas such as neural information processing, manual control, decision making, problem solving, and cognitive science are discussed. Also reviewed are several taxonomies of mental models. The available empirical evidence for answering questions concerning the nature and usage of mental models is then discussed. A variety of studies are reviewed where the type and form of humans' knowledge have been manipulated. Also considered are numerous transfer of training studies whose results provide indirect evidence of the nature of mental models. The alternative perspectives considered and the spectrum of empirical evidence are combined to suggest a framework within which research on mental models can be viewed. By considering interactions of dimensions of this framework, the most salient unanswered questions can be identified.
Article
Full-text available
Cognitive simulations are runnable computer programs that represent models of human cognitive activities. We show how one cognitive simulation built as a model of some of the cognitive processes involved in dynamic fault management can be used in conjunction with small-scale empirical data on human performance to uncover the cognitive demands of a task, to identify where intention errors are likely to occur, and to point to improvements in the person-machine system. The simulation, called Cognitive Environment Simulation or CES, has been exercised on several nuclear power plant accident scenarios. Here we report one case to illustrate how a cognitive simulation tool such as CES can be used to clarify the cognitive demands of a problem-solving situation as part of a cognitive task analysis.
Article
This paper presents the findings of a study on how experienced naval officers make decisions in a complex, time-pressured command and control setting, the Combat Information Center of AEGIS cruisers. The decision processes invoked by the officers were consistent with the recognition-primed decision model. The majority of decisions concerned situation awareness and diagnosis in which the decision makers used feature-matching and story generation strategies to build situation awareness. Furthermore, awareness of the situation enabled the officers to recognize appropriate actions from published procedures or past experience. A recognitional strategy was used to identify 95% of the actions taken; decision makers compared multiple options in only 4% of the cases. These findings are discussed in terms of their implications for framing command-and-control problems, for emphasizing situation awareness, for a descriptive model of decision making, and for designing decision supports.
Article
Evidence is reviewed which suggests that there may be little or no direct introspective access to higher order cognitive processes. Subjects are sometimes (a) unaware of the existence of a stimulus that importantly influenced a response, (b) unaware of the existence of the response, and (c) unaware that the stimulus has affected the response. It is proposed that when people attempt to report on their cognitive processes, that is, on the processes mediating the effects of a stimulus on a response, they do not do so on the basis of any true introspection. Instead, their reports are based on a priori, implicit causal theories, or judgments about the extent to which a particular stimulus is a plausible cause of a given response. This suggests that though people may not be able to observe directly their cognitive processes, they will sometimes be able to report accurately about them. Accurate reports will occur when influential stimuli are salient and are plausible causes of the responses they produce, and will not occur when stimuli are not salient or are not plausible causes.
Article
Through the use of the critical incident technique one may collect specific and significant behavioral facts, providing "… a sound basis for making inferences as to requirements… " for measures of typical performance (criteria), measures of proficiency (standard samples), training, selection and classification, job design and purification, operating procedures, equipment design, motivation and leadership (attitudes), and counseling and psychotherapy. The development, fundamental principles, present status, and uses of the critical incident technique are discussed, along with a review of studies employing the technique and suggestions for further applications.
Technical Report
Developing effective instruction for complex problem-solving tasks requires analysis of the cognitive processes and structures that contribute to task performance. This report describes the data collection procedures associated with a cognitive task analysis technique known as the PARI (precursor, action, result, and interpretation) methodology. The methodology is being developed under the Basic Job Skills (BJS) program and constitutes one component of an integrated technology for developing and delivering training of cognitively complex tasks. The data collection procedures can be considered an extension of existing task analysis techniques and are based on studies of over 200 Air Force technicians in aircraft maintenance specialties whose primary task is troubleshooting. The procedures derived from these studies impose a structure on the knowledge acquisition task which captures the cognitive as well as the behavioral components of troubleshooting skill. The structured interview approach yields data that allow qualitative comparisons of problem-solving performances within and across technical skill levels. Such analyses have informed instruction developed under the BJS program by revealing the developmental course of skill acquisition and the components of expertise which are the training targets. More recent analyses have identified skill and knowledge commonalities across maintenance specialties and are informing training designed to facilitate knowledge transfer. A future goal of the BJS program is to examine the generality of the PARI methodology and the extent to which it can be applied to problem-solving tasks in nonmaintenance domains.
Article
A cognitive task analysis was performed to analyze knowledge structures, mental models, skills, and strategies of en route controllers to provide an understanding of the key cognitive components of the air traffic controller's job. This article presents the results of three procedures as they contributed to an understanding of controller expertise: paper problem solving, performance modeling, and structured problem solving. The procedures resulted in the identification of (a) 13 primary tasks, (b) a mental model representing expert controller's organization of domain knowledge, (c) three categories of controller strategies, and (d) a hierarchy of goals. These results are being used to specify the instructional content and sequencing for the new Federal Aviation Administration en route air traffic control curriculum.
Article
paper, I would have offered a paradoxical analysis, 'Accept as is - in spite of the author's sensitive appreciation of the strengths and liabilities of self-report measures'. The remainder of this comment will attempt to flesh-out the logic and the paradox behind my hypothetical assessment. Paradox rarely rears its ugly head in science, yet in spite of careful analyses like Spector's, and growing evidence of the validity of self-reports, it seems as if self-report-bashing might be an article of faith of some Scientific Apostle's Creed, 'I believe in good science; the empirical determination of theory choice, the control of extraneous variables, and the fallibility of self-report measures . . .' Spector (1994) acknowledges that cross-sectional self-report studies are sometimes inadequate. At times the problems are due to the studies' passive observational nature (and might be equally problematic had non-self-report measures been employed) and sometimes due to the self-report measurement strategy itself. The former cluster of problems should not be heard as a critique of self-reports per se, so I will instead direct my attention to the latter group of problems. There are known contaminations to self-report measures (e.g. social desirability, selective memory, etc.) that need to be considered. However, Donald Campbell's famous aphorism that good scientists are ontological realists but epistemological fallibalists - suggests that the fallibility of self-reports is not in itself a cogent critique. The immediate question arises: What measurement strategy do you propose to use instead of a self-report (e.g. behavioral, physiological, significant-other, expert judge, archival), and what are the grounds for believing that your alternative measurement strategy is less fallible than a self-report?
Article
Previous studies of expert decision makers have concluded that experts, because of cognitive limitations, are generally inaccurate, unreliable, biased, lack self-insight, and gain little with experience. Overall, previous psychological studies have painted a rather bleak picture of the decision-making abilities of experts. The research reviewed here provides a different view of experts in two respects. First, expert decision makers have been found to use strategies, such as reliance on group feedback, willingness to make adjustments, and a divide-and-conquer approach, which help them overcome the effects of cognitive limitations. Second, top decision makers in agriculture, personnel selection, health care, accounting/auditing, and management have been observed to share psychological characteristics such as perceptiveness, communication skills, self-confidence, and creativity under stress. These findings have implications for (1) image and expectations of experts, (2) classifying different types of experts, (3) training and/or selecting novices to become experts, and (4) design of expert systems.
Conference Paper
A formal, but pragmatic, method of recording and organizing human expertise into a knowledge-based system is presented. Practical considerations and methods which increase system validity while minimizing demands on human domain specialists are explored. The methodology concentrates on domain definition (background knowledge, references, situations, and procedures), on fundamental knowledge formulation (elementary rules, beliefs, and expectations), and on basal knowledge consolidation (review and correction cycles). Experience gained from testing and from expert feedback is described. I BACKGROUND Until recently. Artificial Intelligence researchers have developed knowledge-based systems (KBS) for problems in which the human ex­ pert was constantly available and involved. However, the very scarcity of the expertise to be modelled may be the motivation for system development. The expertise may be heuristic and dispersed among several individuals while a single expert may be difficult to locate, let alone interview.
Article
The first step in the development of an expert system is the extraction and characterization of the knowledge and skills of an expert. This step is widely regarded as the major bottleneck in the system development process. To assist knowledge engineers and others who might be interested in the development of an expert system, I offer (1) a working classification of methods for extracting an expert's knowledge, (2) some ideas about the types of data that the methods yield, and (3) a set of criteria by which the methods can be compared relative to the needs of the system developer. The discussion highlights certain issues, including the contrast between the empirical approach taken by experimental psychologists and the formalism-oriented approach that is generally taken by cognitive scientists.
Article
The Critical Decision Method (CDM) is an approach to cognitive task analysis. The method involves multiple-pass event retrospection guided by probe questions. The CDM has been used in the elicitation of expert knowledge in diverse domains and for applications including system development and instructional design. The CDM research illustrates the sorts of knowledge representation products that can arise from cognitive task analysis (e.g., Situation Assessment Records, time lines, decision requirements). The research also shows how one can approach methodological issues surrounding cognitive task analysis, including questions about data quality and method reliability, efficiency, and utility. As cognitive task analysis is used more widely in the elicitation, preservation, and dissemination of expert knowledge and is used more widely as the basis for the design of complex cognitive systems, and as projects move into even more field applications and real-world settings, the issues we discuss become increasingly critical.
Article
Information on knowledge elicitation methods is widely scattered across the fields of psychology, business management, education, counseling, cognitive science, linguistics, philosophy, knowledge engineering and anthropology. The purpose of this review is to (1) identify knowledge elicitation techniques and the associated bibliographic information, (2) organize the techniques into categories on the basis of methodological similarity, and (3) summarize for each category of techniques strengths, weaknesses, and recommended applications. The review is intended to provide a starting point for those interested in applying or developing knowledge elicitation techniques, as well as for those more generally interested in exploring the scope of the available methodology.
Article
Discusses the use of self-report studies to understand organizational phenomena and examines issues of construct validity and of appropriate inferences that can be made from cross-sectional self-report studies (CSRSs). It is argued that CSRSs have 2 weaknesses. First, the use of the job incumbent as the only source of data leaves many alternative explanations for observed correlations other than that the intended traits are related. Second, cross-sectional designs do not allow for confident causal conclusions. Despite these weaknesses, this design can be useful in providing a picture of how people feel about and view their jobs and can determine intercorrelations among various feelings and perceptions. CSRSs should not be automatically dismissed as being inferior. However, the methodology used should match the research question asked, and for many organizational behavior questions the CSRS will not provide adequate answers. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Article
Two studies supporting the use of the Critical Decision method (CDM) in eliciting knowledge from expert neonatal intensive care unit (NICU) nurses are presented. The first examines the utility of CDM in the nursing profession. In this study, significantly more information was elicited in CDM interviews than in non-CDM interviews. In the second study, cues, indicators, and exemplars were extracted from CDM incident accounts to form a guide to early sepsis assessment in the NICU that contains information not available in the current literature. All evaluators rated the guide as useful. Implications for future research, including generalizability to other areas of nursing, are discussed.
Article
The psychological study of expertise has a rich background and has recently gained impetus in part because of the advent of expert systems and related technologies for preserving knowledge. In the study of expertise, whether in the context of applications or the context of psychological research, knowledge elicitation is a crucial step. Research in a number of traditions - judgment and decision making, human factors, cognitive science, expert systems - has utilized a variety of knowledge elicitation methods. Given the diversity of disciplines, topics, paradigms, and goals, it is difficult to make the literature cohere around a methodological theme. For discussion purposes, we place knowledge elicitation techniques into three categories: (1) analysis of the tasks that experts usually perform, (2) various types of interviews, and (3) contrived tasks which reveal an expert′s reasoning processes without necessarily asking about these processes. We illustrate types and subtypes of techniques, culminating in a discussion of research that has empirically evaluated and compared techniques. The article includes some recommendations about "how to do" knowledge elicitation, some cautionary tales, and a discussion of the prospects.
A critical decision method is described for modeling tasks in naturalistic environments characterized by high time pressure, high information content, and changing conditions. The method is a variant of a J.C. Flanagan's (1954) critical incident technique extended to include probes that elicit aspects of expertise such as the basis for making perceptual discriminations, conceptual discriminations, typicality judgments, and critical cues. The method has been used to elicit domain knowledge from experienced personnel such as urban and wildland fireground commanders, tank platoon leaders, structural engineers, design engineers, paramedics, and computer programmers. A model of decision-making derived from these investigations is presented as the theoretical background to the methodology. Instruments and procedures for implementing the approach are described. Applications of the method include developing expert systems, evaluating expert systems' performance, identifying training requirements, and investigating basic decision research issues
Knowledge acquisition for real-time knowledge-based systems
  • B Clar Ke
CLAR KE, B. 1987, Knowledge acquisition for real-time knowledge-based systems, Proceedings of the First European Workshop on Knowledge Acquisition For Knowledge Based Systems (United Kingdom: Reading University), September.
Clinical assessment skills of experienced neonatal intensive care nurses
  • B Crandall
  • R Ood
CRANDALL, B. and CALDERW OOD, R. 1989, Clinical assessment skills of experienced neonatal intensive care nurses. Final report, Klein Associates Inc., OH. Prepared under contract 1 R43 NR0191101 for The National Center for Nursing, NIH.
The two worlds of human factors
  • D Meister
MEISTER, D. 1985, The two worlds of human factors, in R. E. Eberts and C. G. Eberts (eds), Trends in Ergonomics/Human Factors II, (New York: North-Holland), 3 ± 11.
Early assessment of NEC in premature infants
  • L G Militello
  • L Lim
MILITELLO, L. G. and LIM, L. 1995, Early assessment of NEC in premature infants, Journal of Perinatal and Neonatal Nursing, 9, 1 ± 11.
  • J Rasmussen
  • A M Pejterse N
  • L P Goodstein
RASMUSSEN, J., PEJTERSE N, A. M. and GOODSTEIN, L. P. 1994, Cognitive Systems Engineering (New York: Wiley).
  • T L Redding
  • Kaem Pf
SEAMSTER, T. L., REDDING, R. E. and KAEM PF, G. L. 1997, Applied Cognitive Task Analysis in Aviation (Aldershot: Avebury Aviation).
The reliability of the critical decision method, KATR-863(B)-87-07F. Prepared under contract MDA903-86-C-0170 for the US Army Research Institute Field Unit
  • J Taynor
  • B Crandall
  • S Wiggins
TAYNOR, J., CRANDALL, B. and WIGGINS, S. 1987, The reliability of the critical decision method, KATR-863(B)-87-07F. Prepared under contract MDA903-86-C-0170 for the US Army Research Institute Field Unit, Leavenworth, KS, by Klein Associates Inc., OH.
Cognitive task analysis of critical team decision making during multiship engagements Prepared under subcontract RI82264X for Williams Air Force Base
  • M L Militello
THORDSEN, M. L., MILITELLO, L. G. and KLEIN, G. A. 1992, Cognitive task analysis of critical team decision making during multiship engagements Prepared under subcontract RI82264X for Williams Air Force Base, AZ, by Klein Associates Inc., OH.
Guide to early sepsis assessment in the NICU. Instruction manual prepared for the Ohio Department of Development under the Ohio SBIR Bridge Grant program by
  • B Crandall
  • V Gamblian
CRANDALL, B. and GAMBLIAN, V. 1991, Guide to early sepsis assessment in the NICU. Instruction manual prepared for the Ohio Department of Development under the Ohio SBIR Bridge Grant program by Klein Associates Inc., OH.
Knowledge acquisition with question probes and conceptual graph structures
  • H W Gordon
  • R T Gill
GORDON, H. W. and GILL, R. T. 1992, Knowledge acquisition with question probes and conceptual graph structures, in T. Laurer, E. Peacock and A. Graesser (eds), Questions and Information Systems (Hillsdale, NJ: Lawrence Erlbaum), 29 ± 46.
Training and Development in Work Organizations: Frontiers of Industrial and Organizational Psychology
  • W C Howell
  • N J Cooke
HOWELL, W. C. and COOKE, N. J. 1989, Training the human information processor: A look at cognitive models, in I.L. Goldstein (ed.), Training and Development in Work Organizations: Frontiers of Industrial and Organizational Psychology (San Francisco: Jossey-Bass), 121 ± 182.
A cognitive systems engineering application for interface design
  • D Klinger
  • M Gomes
KLINGER, D. and GOMES, M. 1993, A cognitive systems engineering application for interface design, Proceedings of the 37 th Annual Meeting of the Human Factors Society (Santa Monica, CA: Human Factors Society).
Models of natural intelligence for fault diagnosis tasks: Implications for training and aiding of maintenance personnel
  • W Rouse
ROUSE, W. 1984, Models of natural intelligence for fault diagnosis tasks: Implications for training and aiding of maintenance personnel, in Arti®cial Intelligence in Maintenance: Proceedings of the Joint Services Workshop, report no. AFHRL-TR-84-25, Air Force Human Resources Laboratory, Colorado, 193 ± 212.
Cognitive task analysis of critical team decision making during multiship engagements Prepared under subcontract RI-82264X for Williams Air Force Base
  • M L Thordsen
  • L G Militello
  • G A Klein
THORDSEN, M. L., MILITELLO, L. G. and KLEIN, G. A. 1992, Cognitive task analysis of critical team decision making during multiship engagements Prepared under subcontract RI-82264X for Williams Air Force Base, AZ, by Klein Associates Inc., OH.