ChapterPDF Available

Abstract and Figures

This chapter presents an overview or the rationale and ev idence for the use of cognitive task analysis (CTA) in health care including the following: It presents a brief history and definition of CTA, the reason it is being adopted for healthcare education, evidence for its learning benefits when used in evidence-based instructional design and medical simulators, an example of how one of the evidence-based CTA methods was implemented in healthcare, and suggestions for future research. The point is made that when evidence-based CTA methods are used, learning from CTA-based healthcare instruction increases an average of 45 % when compared with current task analysis methods.
Content may be subject to copyright.
A preview of the PDF is not available
... Instructional design development typically starts with an analysis of the desired learning outcomes, which, in competency-based medical education, are also termed "whole-tasks," "complex tasks," or "entrustable professional activities" (EPAs) [3,11]. To describe such a task from an expert's perspective, educators have used cognitive task analysis (CTA) to analyze and structure the relevant knowledge, skills, and attitudes of complex tasks in a range of disciplines-including medicine [12]. Research in other educational fields has shown that instructional designs based on CTA have resulted in a 31-46% post-training performance gain [12]. ...
... To describe such a task from an expert's perspective, educators have used cognitive task analysis (CTA) to analyze and structure the relevant knowledge, skills, and attitudes of complex tasks in a range of disciplines-including medicine [12]. Research in other educational fields has shown that instructional designs based on CTA have resulted in a 31-46% post-training performance gain [12]. The next step in instructional design development is to select and orchestrate specific instructional strategies and sequences. ...
... The research question was: "What instructional design strategies for teaching the psychiatric interview and MSE have been used or studied in the education of health professionals?" For charting the data and collating and summarizing the results, both a cognitive task analysis (CTA) [12] and the fourcomponent instructional design (4C/ID) [11,16] approach were used to derive instructional design implications and identify research gaps. ...
Article
Full-text available
Objective The psychiatric mental status examination is a fundamental aspect of the psychiatric clinical interview. However, despite its importance, little emphasis has been given to evidence-based instructional design. Therefore, this review summarizes the literature from an instructional design perspective with the aim of uncovering design strategies that have been used for teaching the psychiatric interview and mental status examination to health professionals. Methods The authors conducted a scoping review. Multiple databases, reference lists, and the gray literature were searched for relevant publications across educational levels and professions. A cognitive task analysis and an instructional design framework was used to summarize and chart the findings. Results A total of 61 articles from 17 countries in six disciplines and three educational levels were identified for data extraction and analysis. Most studies were from the USA, presented as educational case reports, and carried out in undergraduate education in the field of psychiatry. Few articles described the instructional rationale for their curriculum. None of the studies compared the effectiveness of different instructional design components. Reported learning activities for each task domain (knowledge, skills, and attitudes) and for each step of an instructional design process were charted. Most articles reported the use of introductory seminars or lectures in combination with digital learning material (videos and virtual patients in more recent publications) and role-play exercises. Conclusions Educators in psychiatry should consider all task domains of the psychiatric interview and mental status examination. Currently, there is a lack of empirical research on expertise acquisition and use of instructional design frameworks in this context.
... More broadly, a large body of research has examined the relations between student behavior, cognitive processes, and learning in different contexts (i.e., the relations between student actions, speech, gestures; e.g., Chu andKita (2011), Church, Kelly, andHolcombe (2014), Clark (2014), Goldin-Meadow and Singer (2003), Kelly, Healey, Özyürek, and Holler (2015)). Previous research on the coupling between speech and physical movements in communication has shown a stronger relationship between speech and gesture than between speech and action in the context of language production and language comprehension (Church et al., 2014;Kelly et al., 2015). ...
... This project aims to advance research on student behavior, reasoning, and learning by using machine-learning methods to analyze the interplay between learners' actions, speech, and gestures while completing measurement estimation tasks. Since physical behavior during problem solving (Clark, 2014) and gestures (Goldin-Meadow & Singer, 2003) reveal implicit knowledge, we hypothesize that observing students' physical actions while problem solving will also reveal valuable implicit knowledge of measurement concepts. Our goal is to use machine-learning methods to discern how student behavior is indicative of student performance and, consequently, conceptual understanding in the context of measurement, in order to inform the design of future activities and instructional support for students to develop procedural and conceptual measurement skills. ...
Article
Although interdisciplinary collaborations are becoming increasingly common, researchers typically use data analysis methods specific to their field in order to uncover how students learn. We present affordances of integrating theories of embodied cognition and design with machine-learning methods to study student learning in mathematics and inform the design of embodied learning activities. By increasing such collaborative research efforts, learning scientists can incorporate regularization in computational models and ultimately draw reliable conclusions to further inform theory and practice through the design of technology-augmented learning activities. To illustrate this point, we explored students’ conceptual understanding of measurement since limited research has identified measurement estimation strategies that should be emphasized in classroom instruction. By uniquely applying machine-learning methods to a small, multimodal dataset from a study on student behavior in mathematics, we identified behavioral profiles, patterns in speech, and specific actions and gestures that are predictive of performance. These findings may be used to inform the design of embodied learning activities for measurement. We discuss the contribution of these findings to the field of embodied design, and the affordances and challenges of conducting collaborative research in the learning sciences.
... Meta-analyses after 2000 examining learning outcomes (Bernard et al., 2004;Shachar and Neumann, 2003) typically concluded that modality, per se, was not a significant factor in learning outcomes; findings Clark drew upon to re-iterate his original claims (Clark and Feldon, 2005;Clark, 2007). This position statement remains essentially unchanged (Becker, 2010;Clark, 2014), despite some recent meta-analyses showing that university learning outcomes are generally better with OL modes (Jayakumar et al., 2015;Jurewitsch, 2012;Nguyen, 2015). Critics of this status-quo argued that Clark's commentary reflected a lack of understanding regarding educational applications of "new media" and their educational applications (e.g. ...
... The primary focus of our review was to determine, based on the weight of evidence over the past two decades, whether F2F or online teaching modalities provide greater efficacy regarding university learning outcomes or whether the "no significant difference" position (Clark, 2007(Clark, , 2014 continues to reflect their relative status. Our combined analysis provided a clear answer to our research question. ...
Article
Full-text available
Purpose – Following the rapid shift to online learning due to COVID-19, this paper aims to compare the relative efficacy of face-to-face and online university teaching methods. Design/methodology/approach – A scoping review was conducted to examine the learning outcomes within and between online and face-to-face (F2F) university teaching programmes. Findings – Although previous research has supported a “no significant difference” position, the review of 91 comparative studies during 2000–2020 identified 37 (41%) which found online teaching was associated with better learning outcomes, 17 (18%) which favoured F2F and 37 (41%) reporting no significant difference. Purpose-developed online content which supports “student-led” enquiry and cognitive challenge were cited as factors supporting better learning outcomes. Research limitations/implications – This study adopts a pre-defined methodology in reviewing literature which ensures rigour in identifying relevant studies. The large sample of studies (n = 91) supported the comparison of discrete learning modes although high variability in key concepts and outcome variables made it difficult to directly compare some studies. A lack of methodological rigour was observed in some studies. Originality/value – As a result of COVID-19, online university teaching has become the “new normal” but also re-focussed questions regarding its efficacy. The weight of evidence from this review is that online learning is at least as effective and often better than, F2F modalities in supporting learning outcomes, albeit these differences are often modest. The findings raise questions about the presumed benefits of F2F learning and complicate the case for a return to physical classrooms during the pandemic and beyond.
... (P3, Foldit, Expert) In fact, this professional vision is so ingrained, so intuitive, that participants struggled to talk about their expertise and learning process, which is perhaps an obvious finding for researchers familiar with cognitive task analysis (CTA). As CTA researcher Clark writes, "experts don't know what [others] don't know," in other words, experts significantly distort or omit details of their own expertise without specialized knowledge probing [33]. P12 (Foldit, Intermediate) says that they "can't really put into words what makes a well-designed protein." ...
... A much simpler, potentially cheaper strategy to develop a skill chain would be directly asking the expert players to draw the skill chain as they understand it. This direct, unguided approach is typically avoided in CTA literature because it can lead to reduced or less structured results [13]. However, two critical factors differ in this context. ...
Article
Full-text available
For citizen science games (CSGs) to be successful in advancing scientific research, they must effectively train players. Designing tutorials for training can be aided through developing a skill chain of required skills and their dependencies, but skill chain development is an intensive process. In this work, we hypothesized that free recall may be a simpler yet effective method of directly eliciting skill chains. We elicited 23 skill chains from players and developers and augmented our reflexive thematic analysis with 11 semi-structured interviews in order to determine how players and developers conceptualize skill trees and whether free recall can be used as an alternative to more resource-intensive cognitive task analyses. We provide three main contributions: (1) a comparison of skill chain conceptualizations between players and developers and across prior literature; (2) insights to the process of free recall in eliciting CSG skill chains; and (3) a preliminary toolkit of CSG skill-based design recommendations based on our findings. We conclude CSG developers should: give the big picture up front; embrace social learning and paratext use; reinforce the intended structure of knowledge; situate learning within applicable, meaningful contexts; design for discovery and self-reflection; and encourage practice and learning beyond the tutorial. Free recall was ineffective for determining a traditional skill chain but was able to elicit the core gameplay loops, tutorial overviews, and some expert insights.
... A CTA is a technique for identifying, analyzing, and structuring the skills and knowledge used by experts to perform a complex task [16], and leads to an integrative, coherent description of everything required to perform professional tasks adequately [17]. In this article, skills and constituent skills are identified by means of lesson observations and a cued retrospective interview with various expert teachers, and summarized in a skills hierarchy. ...
Article
Although many schools in the Netherlands have purchased adaptive learning systems (ALSs) to reduce workload and improve differentiated instruction, the use of ALSs with teacher dashboards in the classroom does not in itself necessarily improve differentiated instruction. The question is, what skills and knowledge do teachers need to provide differentiated instruction with the help of ALSs with teacher dashboards and how does this differ from a context in which such technologies are not used By means of a cognitive task analysis (CTA), consisting of classroom observations, interviews and several expert meetings, teachers skills and knowledge necessary for providing differentiated instruction when using teacher dashboards and ALSs were disentangled. These findings were and compared with the skills and knowledge needed by teachers providing differentiated instruction without these tools. Results show that teachers dashboards can support teachers when providing differentiated instruction by providing up-to-date information about students progress. Nevertheless, coping with the new information provided on the dashboards also requires new skills and knowledge such as interpreting information from the dashboard and understanding how the information in the ALS is calculated. Although there are some differences, the core principles of differentiated instruction1) being goal-oriented, 2) continually monitoring, 3) adapting instruction and assignments, 4) being ambitious, 5) stimulating students self-regulationare critical for high-quality differentiated instruction regardless of the available tools in the classroom.
... Studies spanning a wide variety of domains, including but not limited to medical field, business sector, and software design, indicated that 30-45% of learning performance was effectively increased using CTA-informed instruction compared to traditional methods of observation and analysis (Clark, 2014;Lim, 2015). Therefore, CTA can help in capturing observable expert knowledge, processes, and goals, eventually leading to using that captured knowledge in instruction, training, performance assessment, and systems development (Mutie, 2015). ...
Thesis
This empirical research aims to explore physical education teachers’ readiness and develop an evaluation platform for wearable technology as a digital innovation in physical education. The research adopts a mixed-method approach with qualitative and quantitative methodologies for collecting data through systematic literature review, case study, survey questionnaire, and an evaluation platform. 115 peer-reviewed articles formed the basis of the systematic review. Responses of 38 expert physical education teachers in Kuwait, collected from semi-structured interviews and cognitive task analysis, formed the basis for qualitative analysis. The responses of 346 physical education teachers in Kuwait, collected from a questionnaire, formed the basis for quantitative analysis. In addition, usability testing and responses of 13 physical education teachers shaped the evaluation platform. The systematic review identifies different research foci for educational research and loci for technological innovation with insights that pose theory-based behavioral and technological conundrums. The case study findings identifies cognitive tasks, organizational and technological conditions for wearables. The survey develops a measurement scale for teacher readiness and tests the influence of readiness on perceived innovativeness. Using case study and survey insights, an evaluation platform was developed to support physical education teachers. The research offers theoretical, practical, and policy implications, shaping policy for strategies to introduce digital innovations in physical education.
... To conclude, the proof of the pudding is in the eating. Previous research claims high effectiveness of CTA-based training (Clark, 2014;Tofel-Grehl & Feldon, 2013) and such designs are published regularly in medical education (see: Cannon-Bowers et al., 2013;Clark et al., 2012;Tjiam et al., 2012;Velmahos et al., 2004). The current paper translates this approach to the domain of teacher PD. ...
Article
Full-text available
This paper describes how an interdisciplinary design team used the Four-Component Instructional Design (4C/ID) model and its accompanying Ten Steps design approach to systematically design a professional development program for teaching differentiation skills to primary school teachers. This description illustrates how insights from a cognitive task analysis into classroom differentiation skills were combined with literature-based instructional design principles to arrive at the training blueprint for workplace-based learning. It demonstrates the decision-making processes involved in the systematic design of each of the four components: learning tasks, supportive information, procedural information, and part-task practice. While the design process was time and resource-intensive, it resulted in a detailed blueprint of a five-month professional development program that strategically combines learning activities to stimulate learning processes that are essential for developing the complex skill providing differentiated instruction in a mathematics lesson .
Article
Ophthalmoscopy is a core component of a complete ophthalmic examination. Due to its complex technical aspects and patients becoming uncooperative if the procedure is prolonged, it can be a difficult skill for a novice to learn and develop proficiency in. Skills instruction is typically provided by subject matter experts (SMEs) through free recall without an agreed-upon instructional framework. This can lead to unintentional omission of essential steps and knowledge required to perform skills correctly. Cognitive task analysis (CTA) allows for construction of standardized instructional protocols that encompass the knowledge and skills experts apply when performing tasks. The objectives of this study were to (a) develop a CTA-based teaching protocol for canine indirect ophthalmoscopy and (b) compare the steps verbalized or demonstrated by SMEs during free recall instruction versus those ultimately identified by CTA. Four SMEs participated in free recall instructional sessions and interviews used for the development of a CTA-based teaching protocol for novice learners. The CTA-based protocol identified 66 steps and sub-steps considered essential for successfully performing canine indirect ophthalmoscopy. During instructional sessions, SMEs on average failed to verbalize 57.1% of clinical knowledge steps, did not verbalize or demonstrate 68.3% and 9.5% of action steps and did not verbalize or demonstrate 73.2% and 40.4% of decision steps, respectively. This study demonstrates that SMEs teaching indirect ophthalmoscopy by free recall may unintentionally omit important steps, suggesting that compared with free recall, CTA may generate more comprehensive and thus potentially more effective instructional materials for teaching technical skills in veterinary medicine.
Article
Indirect fundoscopy is challenging for novice learners, as patients are often intolerant of the procedure, impeding development of proficiency. To address this, we developed a canine ocular simulator that we hypothesized would improve student learning compared to live dogs. Six board-certified veterinary ophthalmologists and 19 secondyear veterinary students (novices) performed an indirect fundic examination on the model and live dog. Prior to assessment, novices were introduced to the skill with a standardized teaching protocol and practiced (without feedback) with either the model (n = 10) or live dog (n = 9) for 30 minutes. All participants evaluated realism and usefulness of the model using a Likert-type scale. Performance on the live dog and model was evaluated in all participants using time to completion of task, performance of fundic examination using a checklist and global score, identification of objects in the fundus of the model, and evaluation of time spent looking at the fundus of the model using eye tracking. Novices (trained on simulator or live dogs) were compared in fundic examination performance on the live dog and identification of shapes in the model. In general, experts performed the fundic examination faster (p ≤ 0.0003) and more proficiently than the novices, although there were no differences in eye tracking behavior between groups (p ≥ 0.06). No differences were detected between training on simulator versus live dog in development of fundoscopy skills in novices (p ≥ 0.20). These findings suggest that this canine model may be an effective tool to train students to perform fundoscopy.
Article
Full-text available
Cognitive task analysis (CTA) is enjoying growing popularity in both research and practice as a foundational element of instructional design. However, there exists relatively little research exploring its value as a foundation for training through controlled studies. Furthermore, highly individualized approaches to conducting CTA do not permit broadly generalizable conclusions to be drawn from the findings of individual studies. Thus, examining the magnitude of observed effects across studies from various domains and CTA practitioners is essential for assessing replicable effects. This study reports the findings from a meta-analysis that examines the overall effectiveness of CTA across practitioners and settings in relation to other means for identifying and representing instructional content. Overall, the effect of CTA-based instruction is large (Hedges's g = 0.871). However, effect sizes vary substantially by both CTA method used and training context. Though limited by a relatively small number of studies, the notable effect size indicates that the information elicited through CTA provides a strong basis for highly effective instruction.
Article
This volume is the first comprehensive history of task analysis, charting its origins from the earliest applied psychology through to modern forms of task analysis that focus on the study of cognitive work. Through this detailed historical analysis, it is made apparent how task analysis has always been cognitive.
Technical Report
Developing effective instruction for complex problem-solving tasks requires analysis of the cognitive processes and structures that contribute to task performance. This report describes the data collection procedures associated with a cognitive task analysis technique known as the PARI (precursor, action, result, and interpretation) methodology. The methodology is being developed under the Basic Job Skills (BJS) program and constitutes one component of an integrated technology for developing and delivering training of cognitively complex tasks. The data collection procedures can be considered an extension of existing task analysis techniques and are based on studies of over 200 Air Force technicians in aircraft maintenance specialties whose primary task is troubleshooting. The procedures derived from these studies impose a structure on the knowledge acquisition task which captures the cognitive as well as the behavioral components of troubleshooting skill. The structured interview approach yields data that allow qualitative comparisons of problem-solving performances within and across technical skill levels. Such analyses have informed instruction developed under the BJS program by revealing the developmental course of skill acquisition and the components of expertise which are the training targets. More recent analyses have identified skill and knowledge commonalities across maintenance specialties and are informing training designed to facilitate knowledge transfer. A future goal of the BJS program is to examine the generality of the PARI methodology and the extent to which it can be applied to problem-solving tasks in nonmaintenance domains.
Article
Malcolm Gladwell; род. 3 сентября 1963, Хэмпшир) — канадский журналист, поп-социолог. В 2005 году «Time» назвало Малкольма Гладуэлла одним из 100 самых влиятельных людей. Книги и статьи Малкольма часто касаются неожиданных последствий исследований в социальных науках и находят широкое применение в научной работе, в частности в областях социологии, психологии и социальной психологии. Некоторые из его книг занимали первые строки в списке бестселлеров «The New York Times». В 2007 году Малкольм получил первую премию Американской Социологической ассоциации за выдающиеся достижения по отчетам в социальных вопросах. В 2007 году он также получил почетную степень доктора филологии Университета Ватерлоо. Малькольм Гладуелл описывает эксперименты, которые показывают, что человеку с поврежденными эмоциональными центрами крайне трудно принимать решения. Он рассказывает про одного такого пациента, которому было предложено прийти на прием либо во вторник, либо в пятницу. И пациент два часа решал во вторник ему прийти или в пятницу — в столбик выписывал плюсы и минусы, их сравнивал, группировал по разному, всяко переставлял. И в жизни своих домашних он просто убивал вот этим. Если его спрашивали, ты что хочешь: омлет или салат? — это задача минут на сорок. Обычный человек очень просто поступает. Он видит омлет, что-то чувствует и говорит: Хочу! Все. Выбор сделан легко и быстро.