No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Tracks to a Medical Diagnosis: Expertise Differences in Visual Problem Solving
Applied Cognitive Psychology
Abstract
This study focuses on the visual problem-solving process of clinical pathologists. Its aim is to find expertise-related differences in the temporal arrangement of this process, with a special focus on the orientation phase. A theoretical model of the visual diagnostic process of medical specialists is extended with general problem-solving theory. Participants were 13 experts, 12 intermediates, and 13 novices, who all diagnosed seven microscopic images. Their microscope movements and thinking aloud were recorded. To study temporal arrangement of the process, we applied a time-grid to the data. The results reflected several aspects of general problem-solving theory. Experts and intermediates showed a more extensive orientation phase and more refined schemata than novices. Intermediates also showed a control phase at the end of the diagnostic process. Novices showed a uniform process. These phases were reflected in microscope navigation and thinking aloud, which justifies the extension of the theoretical model.Copyright
... Several studies have revealed differences in the knowledge structures and problem solving strategies used by experts and non-experts during these tasks (Jaarsma et al. 2015;Jarodzka et al. 2010;Van Meeuwen et al. 2014). Experts, for example, typically spend more time constructing a problem representation, for which they spend more time on low magnification in the first phase of the diagnostic process (Jaarsma et al. 2016). Additionally, experts tend to analyse cases in a comparative manner, in terms of typicality (Jaarsma et al. 2014). ...
... The coding scheme was designed both in an inductive and in a deductive manner, following methods known as 'descriptive coding' and 'hypothesis coding', respectively (Saldana 2009). The hypothesis coding was based on our previous expertise difference studies carried out among clinical pathologists (Jaarsma et al. 2014(Jaarsma et al. , 2015(Jaarsma et al. , 2016 and formed a considerable part of the codes: among others, components of clinical reasoning such as findings, heuristics, and diagnoses were predetermined codes. However, as the data for this study consist of dialogues instead of thinking aloud, some new codes had to be formed by induction from the data (i.e., the descriptive coding part). ...
... The 15 content categories are the measures for the two constructs, interaction and adaptation. Based on previous studies (Jaarsma et al. 2014(Jaarsma et al. , 2015(Jaarsma et al. , 2016, predictions could be made for some of these measures (see Table 2). However, these studies did not provide a basis for predictions on all 15 content categories. ...
Visual problem solving is essential to highly visual and knowledge-intensive professional domains such as clinical pathology, which trainees learn by participating in relevant tasks at the workplace (apprenticeship). Proper guidance of the visual problem solving of apprentices by the master is necessary. Interaction and adaptation to the expertise level of the learner are identified as key ingredients of this guidance. This study focuses on the effect of increased participation of the learner in the task on the interaction and adaptation of the guidance by masters. Thirteen unique dyads consisting of a clinical pathologist (master) and a resident (apprentice) discussed and diagnosed six microscope images. Their dialogues were analysed on their content. The dyads were divided in two groups according to the experience of the apprentice. For each dyad, master and apprentice both operated the microscope for half of the cases. Interaction was operationalised as the equal contribution of both master and apprentice to the dialogue. Adaptation was operationalised as the extent to which the content of the dialogues was adapted to the apprentice’s level. The main hypothesis stated that the interaction and adaptation increase when apprentices operate the microscope. Most results confirmed this hypothesis: apprentices contributed more content when participating more and the content of these dialogues better reflected expertise differences of apprentices. Based on these results, it is argued that, for learning visual problem solving in a visual and knowledge-intensive domain, it is not only important to externalise master performance, but also that of the apprentice.
... The outlined process of recognition and knowledge integration will now be illustrated by means of a study by Jaarsma et al. (2014), Jaarsma et al. (2016) on diagnosing histopathological slides. In an experiment, they followed the recognition and decision making process of three groups of examiners with different levels of expertise in histopathology: 13 novices (second year medical students with no real experience in histopathology), 12 intermediates (with an average of 3 years training) and 13 clinical pathologists (with 21 years of experience including 5 years training). ...
... Their diagnoses are good, but they still have to find the right words to motivate the decisions. This is further analyzed in the second paper by Jaarsma et al. (2016). Finally, the experts reach a level where there is an instant recognition after a first look at a slide. ...
The question is discussed from where the patterns arise that are recognized in the world. Are they elements of the outside world, or do they originate from the concepts that live in the mind of the observer? It is argued that they are created during observation, due to the knowledge on which the observation ability is based. For an experienced observer this may result in a direct recognition of an object or phenomenon without any reasoning. Afterwards and using conscious effort he may be able to supply features or arguments that he might have used for his recognition. The discussion is phrased in the philosophical debate between monism, in which the observer is an element of the observed world, and dualism, in which these two are fully separated. Direct recognition can be understood from a monistic point of view. After the definition of features and the formulation of a reasoning, dualism may arise. An artificial pattern recognition system based on these specifications thereby creates a clear dualistic situation. It fully separates the two worlds by physical sensors and mechanical reasoning. This dualistic position can be solved by a responsible integration of artificially intelligent systems in human controlled applications. A set of simple experiments based on the classification of histopathological slides is presented to illustrate the discussion.
... Another important aspect that Fox and Faulkner-Jones (2017) point out is the lack in research on 3D and dynamic medical images. We fully agree with that, but would like to point towards research not mentioned by these authors, i.e., by Bertram, Helle, Kaakinen, and Svedström (2013) on CT images and our own research on interactive digital pathology slides (Jaarsma et al., 2016;Jaarsma, Jarodzka, Nap, Van Merriënboer, & Boshuizen, 2015;Jaarsma et al., 2014) and on patient-video cases (Balslev et al., 2012). Moreover, the authors mention on several occasions the potential eye tracking has for medical education. ...
... These analyses could be triangulated with other data. In our own research, we have also investigated the interplay between hand movements as navigations within a pathological digital slide, eye movements on this slide as well as the verbalization about this examination (Jaarsma et al., 2016;Jaarsma et al., 2015;Jaarsma et al., 2014). This approach was also already investigated in a more natural setting with mobile eye tracking, although in the non-medical task of tea making (Tatler et al., 2013). ...
Visual expertise in medicine has been a subject of research since many decades. Interestingly, it has been investigated from two little related fields, namely the field that focused mainly on the visual search aspects whilst ignoring higher-level cognitive processes involved in medical expertise, and the field that mainly focused on these higher-level cognitive processes largely ignoring the relevant visual aspects. Consequently, both research lines have traditionally used different methodologies. Recently, this gap is being increasingly closed and this special issue presents methods to investigate visual expertise in medicine from both research lines, namely those investigating vision (eye tracking, pupillometry, flash preview moving window paradigm), verbalisations, brain activity, and performance measures (ROC analysis, gesture coding, expert performance approach). We discuss the benefits and drawbacks of each method and suggest directions for future research that could help to unbox the black box of visual expertise in medicine.
... In a following step, we moved towards an interactive task stimulus, namely digital pathology (Jaarsma et al., 2016;Jaarsma, Jarodzka, Nap, Van Merriёnboer, & Boshuizen, 2014. In the first study (Jaarsma et al., 2014) we compared how participants of three expertise levels diagnosed 10 pathological slides based on a two seconds inspection. ...
... Even though this study yielded interesting findings, the task we used was not really representative for this profession. Hence, in following studies (Jaarsma et al., 2015(Jaarsma et al., , 2016, we used a digital version of a tissue sample that could be operated as under a regular microscope: zooming in and out as well as panning around the slide. Hence, this was a highly representative task. ...
Eye tracking is increasingly being used in Educational Science and so has the interest of the eye tracking community grown in this topic. In this paper we briefly introduce the discipline of Educational Science and why it might be interesting to couple it with eye tracking research. We then introduce three major research areas in Educational Science that have already successfully used eye tracking: First, eye tracking has been used to im-prove the instructional design of computer-based learning and testing environments, often using hyper- or multimedia. Second, eye tracking has shed light on expertise and its devel-opment in visual domains, such as chess or medicine. Third, eye tracking has recently been also used to promote visual expertise by means of eye movement modeling examples. We outline the main educational theories for these research areas and indicate where fur-ther eye tracking research is needed to expand them.
... 21 For routine tasks, experts complete tasks anywhere from 1.3 to multiple times faster. [22][23][24] In one study, experts were 17 times faster than novices in tying laparoscopic knots while using fewer movements. 23 From an instructional design standpoint, instructors need to allow sufficient time for novices to solve problems. ...
Objective: To address the stages of expertise development, what differentiates a novice from an expert, and how the development and differences impact how we teach our classes or design the curriculum. This paper will also address the downside of expertise and discuss the importance of teaching expertise relative to domain expertise. Summary: Experts develop through years of experience and by progressing from novice, advance beginner, proficient, competent, and finally expert. These stages are contingent on progressive problem solving, which means individuals must engage in increasingly complex problems, strategically aligned with the learner’s stage of development. Thus, several characteristics differentiate experts from novices. Experts know more, their knowledge is better organized and integrated, they have better strategies for accessing knowledge and using it, and they are self-regulated and have different motivations. © 2017, American Association of Colleges of Pharmacy. All rights reserved.
... Gaze-contingent window paradigms have also revealed better performance when larger regions of the image are visible, relative to conditions that restricted visibility to a smaller portion of the image (Kundel et al., , 1991. Moreover, in the field of pathology, when observers were permitted to choose their level of magnification, the experts spent a greater proportion of time at low magnification (i.e., the level of magnification that provided a more "global" view of the image), compared to both intermediates and novices (Jaarsma et al., 2015; for related findings, see Jaarsma et al., 2016). Taken together, these findings support the holistic processing view by revealing that experts benefit from conditions that permit the processing of overall patterns. ...
In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.
... Second, expert medical practitioners are better able to flexibly reason, often in a conceptual manner, about the relationships between visual information and diagnostic alternatives [21]. In contrast, novices may rely on memorized biomedical rules that are relatively inflexible and not necessarily grounded in prior perceptual experiences [24,25] Thus, in pathology increasing experience interpreting biopsy images results in more finely tuned memory for exemplars, increasing the distinctiveness of exemplar categories, and the efficiency and flexibility with which critical image features are identified and mapped to candidate diagnoses [26,27]. It is important to realize, however, that developing visual expertise may be faster and ultimately more successful for some individuals than others, suggesting that not all trainees are capable of becoming experts at interpreting and diagnosing images [28,29]. ...
Digital whole slide imaging is an increasingly common medium in pathology, with application to education, telemedicine, and rendering second opinions. It has also made it possible to use eye tracking devices to explore the dynamic visual inspection and interpretation of histopathological features of tissue while pathologists review cases. Using whole slide images, the present study examined how a pathologist's diagnosis is influenced by fixed case-level factors, their prior clinical experience, and their patterns of visual inspection. Participating pathologists interpreted one of two test sets, each containing 12 digital whole slide images of breast biopsy specimens. Cases represented four diagnostic categories as determined via expert consensus: benign without atypia, atypia, ductal carcinoma in situ (DCIS), and invasive cancer. Each case included one or more regions of interest (ROIs) previously determined as of critical diagnostic importance. During pathologist interpretation we tracked eye movements, viewer tool behavior (zooming, panning), and interpretation time. Models were built using logistic and linear regression with generalized estimating equations, testing whether variables at the level of the pathologists, cases, and visual interpretive behavior would independently and/or interactively predict diagnostic accuracy and efficiency. Diagnostic accuracy varied as a function of case consensus diagnosis, replicating earlier research. As would be expected, benign cases tended to elicit false positives, and atypia, DCIS, and invasive cases tended to elicit false negatives. Pathologist experience levels, case consensus diagnosis, case difficulty, eye fixation durations, and the extent to which pathologists' eyes fixated within versus outside of diagnostic ROIs, all independently or interactively predicted diagnostic accuracy. Higher zooming behavior predicted a tendency to over-interpret benign and atypia cases, but not DCIS cases. Efficiency was not predicted by pathologist- or visual search-level variables. Results provide new insights into the medical interpretive process and demonstrate the complex interactions between pathologists and cases that guide diagnostic decision-making. Implications for training, clinical practice, and computer-aided decision aids are considered.
... Second, expert medical practitioners are better able to flexibly reason, often in a conceptual manner, about the relationships between visual information and diagnostic alternatives [21]. In contrast, novices may rely on memorized biomedical rules that are relatively inflexible and not necessarily grounded in prior perceptual experiences [24,25] Thus, in pathology increasing experience interpreting biopsy images results in more finely tuned memory for exemplars, increasing the distinctiveness of exemplar categories, and the efficiency and flexibility with which critical image features are identified and mapped to candidate diagnoses [26,27]. It is important to realize, however, that developing visual expertise may be faster and ultimately more successful for some individuals than others, suggesting that not all trainees are capable of becoming experts at interpreting and diagnosing images [28,29]. ...
Whole slide imaging technology enables pathologists to screen biopsy images and make a diagnosis in a digital form. This creates an opportunity to understand the screening patterns of expert pathologists and extract the patterns that lead to accurate and efficient diagnoses. For this purpose, we are taking the first step to interpret the recorded actions of world-class expert pathologists on a set of digitized breast biopsy images. We propose an algorithm to extract regions of interest from the logs of image screenings using zoom levels, time and the magnitude of panning motion. Using diagnostically relevant regions marked by experts, we use the visual bag-of-words model with texture and color features to describe these regions and train probabilistic classifiers to predict similar regions of interest in new whole slide images. The proposed algorithm gives promising results for detecting diagnostically relevant regions. We hope this attempt to predict the regions that attract pathologists' attention will provide the first step in a more comprehensive study to understand the diagnostic patterns in histopathology.
Context:
Visual expertise is the superior visual skill shown when executing domain-specific visual tasks. Understanding visual expertise is important in order to understand how the interpretation of medical images may be best learned and taught. In the context of this article, we focus on the visual skill of medical image diagnosis and, more specifically, on the methodological set-ups routinely used in visual expertise research.
Methods:
We offer a critique of commonly used methods and propose three challenges for future research to open up new avenues for studying characteristics of visual expertise in medical image diagnosis. The first challenge addresses theory development. Novel prospects in modelling visual expertise can emerge when we reflect on cognitive and socio-cultural epistemologies in visual expertise research, when we engage in statistical validations of existing theoretical assumptions and when we include social and socio-cultural processes in expertise development. The second challenge addresses the recording and analysis of longitudinal data. If we assume that the development of expertise is a long-term phenomenon, then it follows that future research can engage in advanced statistical modelling of longitudinal expertise data that extends the routine use of cross-sectional material through, for example, animations and dynamic visualisations of developmental data. The third challenge addresses the combination of methods. Alternatives to current practices can integrate qualitative and quantitative approaches in mixed-method designs, embrace relevant yet underused data sources and understand the need for multidisciplinary research teams.
Conclusion:
Embracing alternative epistemological and methodological approaches for studying visual expertise can lead to a more balanced and robust future for understanding superior visual skills in medical image diagnosis as well as other medical fields.
It has long been known that differences among individuals have an effect on learning. Dick Snow's research on aptitude-treatment interactions (ATIs) was designed to investigate and quantify these effects, and more recent research in this vein has clearly established that these effects can be quantified and predicted. Technology has now reached a point where we have the opportunity to capitalize on these effects to the benefit of learners. In this article, we review some of the demonstrated effects of ATIs, describe how ATI research naturally leads to adaptive e-learning, and describe one way in which an adaptive e-learning system might be implemented to take advantage of these effects.
The chapter highlights the theoretical and applied contributions of eye movement research to the study of human expertise. Using examples drawn from the domains of chess and medicine, the chapter demonstrates that eye movements are particularly well-suited for studying two hallmarks of expert performance: the superior perceptual encoding of domain related patterns, and experts’ tacit (or implicit) domain related knowledge. Specifically, eye movement findings indicate that expertise is associated with a greater ability to process domain related visual information in terms of larger patterns of features rather than isolated features. Furthermore, in support of the role of tacit knowledge in expertise, there is evidence that the eye movements of experts may contain information that is not consciously accessible.
for several years, we have been trying to understand what constitutes radiological expertise and how that expertise is acquired / our goal is to understand the learning of a complex skill and thereby to stretch the limits of existing knowledge about expertise and its acquisition / this chapter is a progress report on that work
expert problem-solving (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Students in complex visual domains must acquire visual problem solving strategies that allow them to make fast decisions and come up with good solutions to real-time problems. In this study, 31 air traffic controllers at different levels of expertise (novice, intermediate, expert) were confronted with 9 problem situations depicted on a radar screen. Participants were asked to provide the optimal order of arrival of all depicted aircrafts. Eye-movements, time-on-task, perceived mental effort, and task performance were recorded. Eye-tracking data revealed that novices use inefficient means-end visual problem solving strategies in which they primarily focus on the destination of aircraft. Higher levels of expertise yield visual problem solving strategies characterized by more efficient retrieval of relevant information and more efficient scan paths. Furthermore, experts' solutions were more similar than intermediates' solutions and intermediates' solutions were more similar than novices' solutions. Performance measures showed that experts and intermediates reached better solutions than novices, and that experts were faster and invested less mental effort than intermediates and novices. These findings may help creating eye-movement modeling examples for the teaching of visual problem solving strategies in complex visual domains.
This study explored the value of eye movement data for uncovering relatively small expertise-related differences in electrical circuit-troubleshooting performance, and describes that value in relation to concurrent verbal protocols. Results show that in the ‘problem orientation’ phase, higher expertise participants spent relatively more time, had a shorter mean fixation duration, and fixated more on a major fault-related component than lower expertise participants. In the ‘problem formulation’ part of the ‘problem formulation and action decision’ phase, the mean fixation duration of the higher expertise participants was longer. In the ‘action evaluation and next action decision’ phase, higher expertise participants spent relatively more time than the lower expertise participants. Over the different phases, only the mean fixation duration of the higher expertise participants differed significantly. The relation between the eye movement and concurrent verbal protocol data is qualitatively described. The results are discussed in perspective of the combined value of eye tracking and concurrent reports for expertise research and instructional design. Copyright © 2005 John Wiley & Sons, Ltd.
In two studies the role of biomedical knowledge in the diagnosis of clinical cases was explored. Experiment 1 demonstrated a decrease in the use of biomedical knowledge with increasing expertise. This result appeared to be at variance with some findings reported in the literature (e.g., Lesgold, 1984), but supported those of others (e.g., Patel, Evans, & Groen, 1989). In Experiment 2, three possible explanations for this phenomenon were investigated: (1) rudimentation of biomedical knowledge, (2) inertia, and (3) encapsulation of biomedical knowledge under higher order concepts. Using a combined think-aloud and post-hoc explanation methodology, it was shown that experts have more in-depth biomedical knowledge than novices and subjects at intermediate levels of expertise. The findings generally support a three-stage model of expertise development in medicine consisting of acquisition of biomedical knowledge, practical experience, and integration of theoretical and experientall knowledge resulting in knowledge encapsulation.
To identify key features contributing to trainees' development of expertise in microscopic pathology diagnosis, a complex visual task, and to provide new insights to help create computer-based training systems in pathology.
Standard methods of information-processing and cognitive science were used to study diagnostic processes (search, perception, reasoning) of 28 novices, intermediates, and experts. Participants examined cases in breast pathology; each case had a previously established gold standard diagnosis. Videotapes correlated the actual visual data examined by participants with their verbal "think-aloud" protocols.
Investigators measured accuracy, difficulty, certainty, protocol process frequencies, error frequencies, and times to key diagnostic events for each case and subject. Analyses of variance, chi-square tests and post-hoc comparisons were performed with subject as the unit of analysis.
Level of expertise corresponded with differences in search, perception, and reasoning components of the tasks. Several discrete steps occur on the path to competence, including development of adequate search strategies, rapid and accurate recognition of anatomic location, acquisition of visual data interpretation skills, and transitory reliance on explicit feature identification.
Results provide the basis for an empirical cognitive model of competence for the complex tasks of microscopic pathology diagnosis. Results will inform the development of computer-based pedagogy tools in this domain
The goal of mammography screening is to detect breast cancer at early stages, but because of the complexity of the breast parenchyma and the variability of signs of the disease, many cancers go unreported when initially visible on the mammogram. We compared the visual search strategy used by experienced mammographers in a case set where they examined both the mammogram in which a malignant mass was discovered at screening mammography and the most recent prior mammogram.
Four experienced mammographers participated in this experiment. They read a case set of 20 two-view mammograms, of which 15 contained a malignant mass and 5 were lesion-free, in two trials. For each of the cancer cases, two versions were shown to the observers: the one in which the cancer was reported in the clinical practice, called the "current" mammograms, and the most recent prior. Each trial had a balanced mix of current and prior mammograms. In addition, the same set of lesion-free cases was shown to the observers in both trials. The eye movements of the observers were tracked, and visual search parameters such as time to hit the location of the malignant mass, dwell, and mean pupil size in the location of the cancer were collected. Statistical analyses were used to determine whether there were differences between the current and prior mammograms.
A total of 66% of the malignant masses in the current mammograms and 57% in the priors attracted some amount of visual attention. From these, 71% yielded a report on the current mammograms, but only 40% on the priors. In the cases where the observer saw the malignant mass, they did so within 2 seconds of image display, regardless of whether the mammogram was current or prior.
Most unreported malignant masses attracted some amount of visual attention, but it was in the processing of the information extracted in the location of the lesion that most errors occurred. In our experiment, approximately 70% of the total time used by the observers for visual scan of the cases was spent gathering information to corroborate the hypothesis already formed by the radiologist.
Advances in the psychology of human intelligence
- M. T. H. Chi
- R. Glaser
- E. Rees
Medical image perception and techniques
- C. F. Nodine
- C. Mello-Thoms
Tracks to a medical diagnosis Copyright ©
Tracks to a medical diagnosis
Copyright © 2016 John Wiley & Sons, Ltd.
Appl. Cognit. Psychol. (2016)
Human abilities: An information processing approach
- M T H Chi
- R Glaser
Chi, M. T. H., & Glaser, R. (1985). Problem-solving ability. In R. J.
Sternberg (Ed.), Human abilities: An information processing approach.
New York: W. H. Freeman.
The role of expertise in radiologic image perception
- C F Nodine
- C Mello-Thoms
Nodine, C. F., & Mello-Thoms, C. (2010). The role of expertise in radiologic image perception. In E. Samei & E. A. Krupinski (Eds.), Medical
image perception and techniques (pp. 139-156). New York: Cambridge
University Press.