Content uploaded by Richard Edward Clark
Author content
All content in this area was uploaded by Richard Edward Clark on Jan 26, 2016
Content may be subject to copyright.
A preview of the PDF is not available
Cognitive Task Analysis for Expert-Based Instruction in Healthcare
Abstract and Figures
This chapter presents an overview or the rationale and ev idence for the use of cognitive task analysis (CTA) in health care including the following: It presents a brief history and definition of CTA, the reason it is being adopted for healthcare education, evidence for its learning benefits when used in evidence-based instructional design and medical simulators, an example of how one of the evidence-based CTA methods was implemented in healthcare, and suggestions for future research. The point is made that when evidence-based CTA methods are used, learning from CTA-based healthcare instruction increases an average of 45 % when compared with current task analysis methods.
Figures - uploaded by Richard Edward Clark
Author content
All figure content in this area was uploaded by Richard Edward Clark
Content may be subject to copyright.
... Realistic learning environment Complexity of tasks High-risk, safety-critical environment requires professionals who can act appropriately in unfamiliar situations Construction of knowledge that can be drawn upon when required Student needs Common theme in instructor interviews Learning structured by increasing difficulty Communication of expert knowledge Difficulty in expressing knowledge that has been automated (e.g., Clark 2014) Ability to capture all information required for effective learning ...
In this study, we explore the current practices of experienced instructors in developing scenarios for maritime simulator training and assessment. Scenario design is fundamental to effective simulator-based training but remains underexplored in extant literature. Through a series of semi-structured interviews (n = 16), we identify the core concerns of instructors designing scenarios; (1) realism, authenticity, and fidelity; (2) designing for students' requirements; (3) the importance of clear learning objectives. Based on the results, we propose that functional congruence is a more fitting term than realism, authenticity, and fidelity in this context, as effective simulator training requires a balance between a simulator realistic enough for immersive learning experiences, and a focus on clear learning objectives. In consideration of the need to train students to function as part of a team of competent experts, moreover, we propose the introduction of an established instructional design model into the scenario design process. The proposed model has proven successful in simulator-based training in the similarly high-risk and safety-critical field of healthcare and has the potential to both complement the current practices of experienced instructors and act as a valuable resource for those newer to the role in designing training transferrable to professional practice.
... Reviews of the PARI methodology and its applications from various historical periods can be found in [49,207,201], the literature showing that the most important body of research related to PARI methodology was concentrated in the 1990's and early 2000's. However, several important studies using PARI have been also reported in the recent years in instructional design in domains such as healthcare or engineering [160,45] or in virtual environments applications [47]. ...
This paper offers a multi-disciplinary review of knowledge acquisition methods in human activity systems. The review captures the degree of involvement of various types of agencies in the knowledge acquisition process, and proposes a classification with three categories of methods: the human agent, the human-inspired agent, and the autonomous machine agent methods. In the first two categories, the acquisition of knowledge is seen as a cognitive task analysis exercise, while in the third category knowledge acquisition is treated as an autonomous knowledge-discovery endeavour. The motivation for this classification stems from the continuous change over time of the structure, meaning and purpose of human activity systems, which are seen as the factor that fuelled researchers' and practitioners' efforts in knowledge acquisition for more than a century. We show through this review that the KA field is increasingly active due to the higher and higher pace of change in human activity, and conclude by discussing the emergence of a fourth category of knowledge acquisition methods, which are based on red-teaming and co-evolution.
... Reason Requirement Difficulty in expressing knowledge that has been automated (e.g., Clark, 2014) Ability to capture all information required for effective learning The findings thus far make it apparent that there is a need for a systematic approach to scenario design (Baldauf et al., 2012) confirmed by interviews with instructors, and that the approach must account for the complexity of MET (see Table 2), but that current literature does not include any use of a formal instructional design model for the purposes of scenario design (see Table 1). However, simulator-based training is also used successfully in other high-risk, safety-critical fields, where formal instructional design models have been applied effectively (cf. ...
Within maritime education and training, the value of simulator-based training has long been recognized. Simulation is a powerful tool in providing students with immersive and experiential opportunities in practicing complex professional tasks in a controlled environment. In an industry where mistakes can have catastrophic consequences, the importance of providing students with an education which focuses on good seamanship, rather than simply training individual skills, is evident, with strong emphasis on training competencies such as communication, collaboration, and decision-making. Much focus in existing literature is placed upon factors such as the fidelity of the simulator and the role of the instructor during the simulation, for example, but very little has been published around the design of the scenario exercises themselves. This gap in understanding what constitutes a well-designed scenario, and the process involved in effective scenario design is addressed by this study. Semi-structured interviews have been conducted with experienced instructors from a range of institutions across Sweden, Norway, Finland, and the Netherlands. The goal of the interviews was to uncover valuable insights into the process of designing and developing scenario exercises, focusing on the key considerations that contribute to the success of such training programs. While the initial aim of the study was to develop a systematic approach to scenario design, the findings also revealed some key concerns that provide more in-depth insight into the complexity of good scenario design. Instructors emphasize the need for scenario exercises that not only mimic real-world situations, but also align with specific educational objectives. This highlights the importance of designing scenarios that accommodate the varying levels of knowledge and experience found in a group of students. The scenarios must be both concrete enough to ensure the objectives are met, and flexible enough that the instructor can cater to these individual student needs. In addition to the specific skills being trained, the scenario must be designed in a way which further embeds the non-technical skills, such as communication, teamwork, and reflexivity, central to good seamanship. Through increasing the understanding of effective scenario design, this study aims to contribute to the existing body of knowledge on simulator-based maritime education and training. The use of a recognized instructional design model, developed for training complex tasks, is recommended as complementing the existing approach taken by simulator instructors, formalizing their current practice, and providing a much-needed resource in the form of a systematic approach that captures the nuances of the instructors’ experience.
... While KCMs offer numerous benefits for digital learning platforms, the creation of KCs for each assessment is a timeintensive process that demands domain expertise. This usually requires a domain expert, such as the course instructor, to determine the necessary KCs for solving each assessment [11]. This process necessitates the identification of at least one KC per assessment item and can involve identifying up to three or more KCs, depending on the complexity of the assessment, the subject area, and the educational level [8]. ...
Knowledge Components (KCs) linked to assessments enhance the measurement of student learning, enrich analytics, and facilitate adaptivity. However, generating and linking KCs to assessment items requires significant effort and domain-specific knowledge. To streamline this process for higher-education courses, we employed GPT-4 to generate KCs for multiple-choice questions (MCQs) in Chemistry and E-Learning. We analyzed discrepancies between the KCs generated by the Large Language Model (LLM) and those made by humans through evaluation from three domain experts in each subject area. This evaluation aimed to determine whether, in instances of non-matching KCs, evaluators showed a preference for the LLM-generated KCs over their human-created counterparts. We also developed an ontology induction algorithm to cluster questions that assess similar KCs based on their content. Our most effective LLM strategy accurately matched KCs for 56% of Chemistry and 35% of E-Learning MCQs, with even higher success when considering the top five KC suggestions. Human evaluators favored LLM-generated KCs, choosing them over human-assigned ones approximately two-thirds of the time, a preference that was statistically significant across both domains. Our clustering algorithm successfully grouped questions by their underlying KCs without needing explicit labels or contextual information. This research advances the automation of KC generation and classification for assessment items, alleviating the need for student data or predefined KC labels.
... Our design experience has led us to believe that knowing what knowledge and skills constitute the "doing" of a particular skill informs the design process rather than the "doing" itself. In fact, a common approach to creating authentic learning tasks with 4C/ID, cognitive task analysis (Clark, 2014;Clark & Estes, 1996), follows a similar path of moving from "subject matter knowledge" to the design of a learning intervention, and not vice versa. Namely, when conducting a cognitive task analysis, a designer makes the implicit knowledge of an expert explicit by observing their performance, and it is based on this explicated knowledge that learning tasks are then created. ...
The original impetus for the design of our professional development (PD) program came from the first author’s interest in PD in general and what makes it effective. We begin our presentation with a brief explanation of the core literature that affected our thinking and then introduce the context in which and for which the PD program was designed. This is followed by a presentation of the steps we took to design our PD program with 4C/ID. For ease of reading, we split the design process into 15 distinct steps, each focusing on a major milestone in the design process and introducing the challenges we faced during the process. We finish the presentation with our reflections on the design process and final remarks.
... The CTA is an engineering framework for cognitive systems that identifies, analyzes, and structures the nonconscious knowledge and skills experts apply when they perform complex tasks. 4 The conclusions of CTA are typically constructed from observations and interviews involving the implicit and explicit knowledge of experts. 5 The CTA has been used frequently in surgical education to improve design since it elucidates critical information that is frequently omitted in free recall. ...
Introduction
With the Army’s emerging doctrine of prolonged field care, and with burns being a common injury among soldiers, non-expert providers must be trained to perform escharotomy when indicated. However, the existing physical simulators and training protocols are not sufficient for training non-experts for performing effective escharotomy. Hence, to provide guidance in developing realistic escharotomy simulators and effective training protocols suitable for prolonged field care, a cognitive task analysis (CTA) is needed. This work aims to obtain educative information from expert burn surgeons regarding escharotomy procedures via the CTA.
Materials and methods
The CTA was done by interviewing five subject matter experts with experience in performing escharotomy ranging from 20 to over 100 procedures and analyzing their responses. Interview questions were developed to obtain educative information from expert burn surgeons regarding the escharotomy procedure. A “gold standard protocol” was developed based on the CTA of each of the subject matter experts.
Results
The CTA helped identify general themes, including objectives, conditions that mandate escharotomy, signs of successful escharotomy, precautions, challenges, decisions, and performance standards, and specific learning goals such as the use of equipment, vital signs, performing the procedure, and preoperative and postoperative care. A unique aspect of this CTA is that it identifies the background information and preparations that could be useful to the practitioners at various levels of expertise.
Conclusions
The CTA enabled us to compile a “gold standard protocol” for escharotomy that may serve as a guide for practitioners at various levels of expertise. This information will provide a framework for escharotomy training systems and simulators.
... As the required skills and knowledge for teachers to provide DI in secondary education were not yet known, a cognitive task analysis (CTA) was conducted in the current study, to identify, analyze, and structure both the skills and knowledge that experts (in our case, teachers who differentiate well) use while carrying out a complex task (Clark, 2014). The teachers observed in the study do not necessarily have to differentiate perfectly during the observations. ...
Providing differentiated instruction (DI) is a complex teacher task that many secondary school teachers do not master well. In the current study, a cognitive task analysis of this teacher task was conducted by analyzing how expert teachers do this and why, resulting in an inventory of the necessary teacher skills and knowledge for providing DI, and a description of the factors that influence the complexity of DI. The results of this analysis show what providing DI in secondary education entails, which is valuable for designing teacher professional development programs for DI at that level.
... van Geel et al., 2019) we know that providing differentiated instruction requires knowledge and skills that cannot be directly observed. In order to identify, analyze, and structure the skills and knowledge used by experts during the performance of a complex task a cognitive task analysis (CTA) can be performed (Clark, 2014). In this chapter, we therefore combine the outcomes of two CTA's that were performed to identify knowledge and skills required, one for the complex task of implementing AfL and one for the complex task of providing DI. ...
Teachers are increasingly expected to adapt their teaching to students’ needs. This can be done by implementing differentiated instruction (DI) or assessment for learning (AfL). These concepts are regarded as two distinct approaches to identifying students’ needs and adapting instruction accordingly. In the current study, we aim to identify empirical similarities and differences in teacher knowledge and skills required for differentiated instruction and assessment for learning respectively. Based on combined insights from two cognitive task analyses (CTA’s), it appears that – in line with many other aspects of effective teaching – four phases are closely related for the task (either DI or AfL) as a whole: preparing a lesson series, preparing a lesson, enacting a lesson and, after this enactment, evaluating a lesson. The teacher skills required for DI and/or AfL in each of these phases are similar, however, the emphasis given to each skill differs in practice and this can be noted throughout all four interrelated phases. For AfL, the emphasis is on eliciting evidence during the lesson, for DI, the emphasis is on pro-active alignment of instruction and activities, based on students’ needs. Since teachers need the same underlying skills to be able to perform either DI or AfL, we can hypothesize that teachers who are proficient at either DI or AfL, will also be able to develop and implement AfL or DI in practice.
Background
Team leadership during medical emergencies like cardiac arrest resuscitation is cognitively demanding, especially for trainees. These cognitive processes remain poorly characterized due to measurement challenges. Using virtual reality simulation, this study aimed to elucidate and compare communication and cognitive processes-such as decision-making, cognitive load, perceived pitfalls, and strategies-between expert and novice code team leaders to inform strategies for accelerating proficiency development.
Methods
A simulation-based mixed methods approach was utilized within a single large academic medical center, involving twelve standardized virtual reality cardiac arrest simulations. These 10- to 15-minutes simulation sessions were performed by seven experts and five novices. Following the simulations, a cognitive task analysis was conducted using a cued-recall protocol to identify the challenges, decision-making processes, and cognitive load experienced across the seven stages of each simulation.
Results
The analysis revealed 250 unique cognitive processes. In terms of reasoning patterns, experts used inductive reasoning, while novices tended to use deductive reasoning, considering treatments before assessments. Experts also demonstrated earlier consideration of potential reversible causes of cardiac arrest. Regarding team communication, experts reported more critical communications, with no shared subthemes between groups. Experts identified more teamwork pitfalls, and suggested more strategies compared to novices. For cognitive load, experts reported lower median cognitive load (53) compared to novices (80) across all stages, with the exception of the initial presentation phase.
Conclusions
The identified patterns of expert performance — superior teamwork skills, inductive clinical reasoning, and distributed cognitive strategiesn — can inform training programs aimed at accelerating expertise development.
This chapter presents an overview of the current state of Cognitive Task Analysis (CTA)in research and practice. CTA uses a variety of interview and observation strategies to capture a description of the explicit and implicit knowledge that experts use to perform complex tasks. The captured knowledge is most often transferred to training or thedevelopment of expert systems. The first section presents descriptions of a variety ofCTA techniques, their common characteristics, and the typical strategies used to elicit knowledge from experts and other sources. The second section describes research on the impact of CTA and synthesizes a number of studies and reviews pertinent to issues underlying knowledge elicitation. In the third section we discuss the integration of CTA with training design. In the fourth section, we present a
number of recommendations forfuture research and conclude with general comments.
(17) (PDF) Cognitive task analysis. Available from: https://www.researchgate.net/publication/294699964_Cognitive_task_analysis [accessed Oct 09 2023].
A novel contribution to the age-old debate about free will versus determinism.
Do we consciously cause our actions, or do they happen to us? Philosophers, psychologists, neuroscientists, theologians, and lawyers have long debated the existence of free will versus determinism. In this book Daniel Wegner offers a novel understanding of the issue. Like actions, he argues, the feeling of conscious will is created by the mind and brain. Yet if psychological and neural mechanisms are responsible for all human behavior, how could we have conscious will? The feeling of conscious will, Wegner shows, helps us to appreciate and remember our authorship of the things our minds and bodies do. Yes, we feel that we consciously will our actions, Wegner says, but at the same time, our actions happen to us. Although conscious will is an illusion, it serves as a guide to understanding ourselves and to developing a sense of responsibility and morality.
Approaching conscious will as a topic of psychological study, Wegner examines the issue from a variety of angles. He looks at illusions of the will—those cases where people feel that they are willing an act that they are not doing or, conversely, are not willing an act that they in fact are doing. He explores conscious will in hypnosis, Ouija board spelling, automatic writing, and facilitated communication, as well as in such phenomena as spirit possession, dissociative identity disorder, and trance channeling. The result is a book that sidesteps endless debates to focus, more fruitfully, on the impact on our lives of the illusion of conscious will.
This volume is the first comprehensive history of task analysis, charting its origins from the earliest applied psychology through to modern forms of task analysis that focus on the study of cognitive work. Through this detailed historical analysis, it is made apparent how task analysis has always been cognitive.
Developing effective instruction for complex problem-solving tasks requires analysis of the cognitive processes and structures that contribute to task performance. This report describes the data collection procedures associated with a cognitive task analysis technique known as the PARI (precursor, action, result, and interpretation) methodology. The methodology is being developed under the Basic Job Skills (BJS) program and constitutes one component of an integrated technology for developing and delivering training of cognitively complex tasks. The data collection procedures can be considered an extension of existing task analysis techniques and are based on studies of over 200 Air Force technicians in aircraft maintenance specialties whose primary task is troubleshooting. The procedures derived from these studies impose a structure on the knowledge acquisition task which captures the cognitive as well as the behavioral components of troubleshooting skill. The structured interview approach yields data that allow qualitative comparisons of problem-solving performances within and across technical skill levels. Such analyses have informed instruction developed under the BJS program by revealing the developmental course of skill acquisition and the components of expertise which are the training targets. More recent analyses have identified skill and knowledge commonalities across maintenance specialties and are informing training designed to facilitate knowledge transfer. A future goal of the BJS program is to examine the generality of the PARI methodology and the extent to which it can be applied to problem-solving tasks in nonmaintenance domains.