Figure 2 - uploaded by Jesse D Flint
Content may be subject to copyright.
Change in miss rate across sessions 

Change in miss rate across sessions 

Source publication
Conference Paper
Full-text available
Effective training is a vital foundation for transportation security officers required to learn strategies for identifying anomalies within X-ray images that may indicate a potential threat. Past research has shown that adaptive training is a powerful tool to increase detection performance, however, adaptive training strategies in this domain have...

Context in source publication

Context 1
... 2x3 mixed effects repeated measures ANOVA was used to test for statistical significance with type of training (between subjects) and test session (within subjects) as independent variables and pre-post differences in performance outcomes as dependent variables. A significant difference was evident in sensitivity change ( Δ d’) from pre-training to post-training for type of training, F = 4.717, p < .05, ή 2 = .555, where Traditional training showed significantly greater average delta across all training sessions (Figure 1). No significant difference was found across test sessions for change in sensitivity (d’) from pre-testing. A significant difference was evident in miss rate change ( ∆ M) from pre-training to post-training across test sessions, F = 9.398, p < .01, ή 2 = .973, where the change in miss rate was lower in Session 2 and 3 compared to Session 1 (Figure 2). No significant differences were found for type of training. A significant difference was also evident in false alarm rate change ( ∆ FA) from pre-training to post-training for the main effect of type of training, F = 9.118, p < .05, ή 2 = .830, with Traditional training showing significantly lower increases in false alarm rates from pre-training scores across all sessions (Figure 3). There was no significant difference across test session. A significant difference was found in reaction time change from pre-training to post-training across training sessions, F = 22.684, p < .01, ή 2 = .996., where greater negative deltas were evident in Session 2 and 3 compared to session 1 (Figure 4). No significant differences were found for type of training. A significant difference was found in criterion change ( ∆ c) from pre-training to post-training across training sessions, F = 17.360, p < .01, ή 2 = 1.0, where both session 2 and 3 showed a significant decrease in delta c compared to session 1 (Figure 5). A significant difference was also found across type of training, F = 5.117, p < .05, ή 2 = .589, where Adaptive Training had significantly higher decreases in delta criterion across training sessions compared to Traditional Training. Collecting novice and expert scan data allowed for an initial analysis of the utility of gaze data in further refining root cause error analysis for visual search within a real-time adaptive system as used in the current study. A number of variables, including time to first fixation on the threat, average fixation duration on threats, number of fixations on threats and response time to classification, were tested between the experts and novice groups. A significant difference was found in average fixation duration between experts and novices ( t =4.59, df = 105.282, p < 0.05). As shown in Figure 6, experts showed a higher average fixation duration on AoIs compared to novices. Time to first fixation on threat, time to classification, number of fixations on threat, and total dwell time on threat did not show any significant differences between novices and experts. Training in general resulted in a significant impact on accuracy and throughput demonstrated by a reduction in miss rates and response time. The training provided in the current study produced the same lasting change in c as observed in the Wolfe et al. (2007) study. The change ( ∆ ) in c for Adaptive Training was greater than that for Traditional Training, indicating Adaptive Training resulted in fewer misses than Traditional Training. However, results also showed that Traditional Training resulted in greater sensitivity (d ′ ) and lower false alarm rates than Adaptive Training for the three training sessions completed in this study. The trends (though not significant) for d ′ and false alarm rates reverse for Adaptive training between session 2 and 3. Thus, with additional training sessions, false alarm rates may continue to decrease and d ′ may increase due to training with more, varied threats in successive training sessions. Together, these effects could lead to an overall increase in sensitivity due to Adaptive Training. Based on findings here and those reported elsewhere (Wolfe, 2007), changes in sensitivity appear to be more difficult to induce than changes in response bias ( ∆ c) during initial training trials. The resulting change in c has the desired effect of decreasing miss rates, but only at the cost of also increasing false alarm rates. Further investigation using a longer training time (i.e., more sessions) may provide further insight into the long term impact of training on false alarm rate, as a low miss rates, at the expense of an increased false alarm rates, is not an ideal solution for aviation security. Behavioral differences between novices and experts have been measured by eye tracking search patterns, percentage of time looking at AoIs, and fixations (Kurland et al., 2005), where experts tend to visually process faster (i.e., shorter fixation duration) and move in shorter jumps from location to location. Results from this study showed experts have longer average fixation duration on threats, which is in line with previous findings from intelligent imagery analysis studies (Hale et al., 2008), yet contradicts that from Kurland et al. (2005). Additional data collection is planned to increase the number of expert data points to further investigate eye tracking metrics for real-time adaptive training of visual search tasks and identify additional significant differences in scan and search strategies that can be evaluated in real-time and used to tailor training. The result that the overall dwell time on the image is negatively related to the contour density for the expert group (Figure 7) may be partially explained by the findings of Lohrenz and Beck (2010) that suggests novices avoid searching in highly cluttered regions of displays. Additional analyses are ongoing to examine other search ...

Citations

... These metrics can be improved, however, with effective training. Hale et al. (2012) showed that discrimination training can significantly improve novice baggage screeners' accuracy and response time (RT). Critically, even professional baggage screeners have shown accuracy and RT improvements after training (Halbherr et al. 2013). ...
Article
Full-text available
Visual search is required in many professions where an undetected threat, such as a weapon, can put the well-being of others at risk. Given the importance of detecting these threats, researchers have used various experimental techniques to improve performance in visual search tasks, albeit with varying degrees of success. Here, we explore two promising techniques to improve visual search using ecologically valid synthetic aperture radar stimuli: object recognition training and search strategy training. Search strategy training is intended to make observers search more systematically through a display, whereas object recognition training is intended to improve observers’ ability to recognize critical targets. Search strategy training was implemented by instructing participants to scan through the display in a pre-specified pattern. Object recognition training was implemented by having participants discriminate between targets and non-targets. We also manipulated whether observers received anodal or sham transcranial direct current stimulation (tDCS) during training, which has been shown to improve visual search performance and target learning. To measure the effectiveness of the training and stimulation conditions, we tested object recognition accuracy and overall visual search performance before and after three sessions of increasingly difficult training. Results indicated that object recognition training significantly improved object recognition accuracy relative to the search strategy group, whereas search strategy training was effective in improving visual search accuracy in those who adhered to the training. However, tDCS did not interact with training type, and although both training types yielded significant improvements, training-related improvements were not significantly different between the different approaches. This evidence suggests that strategy-based training could be as effective as the more prototypical object recognition training. Implications for future training protocols are discussed.
... To empirically evaluate training effectiveness, lab-based and field-based studies were completed. Lab-based studies focused on examining how the addition of eye tracking impacted the adaptive training paradigm, and were used to help develop the training platform and content [20, 21] . After initial system development was complete , a training effectiveness evaluation was conducted in the field. ...
Conference Paper
Full-text available
Transportation Security Officers (TSOs) are at the forefront of our nation’s security, and are tasked with screening every bag boarding a commercial aircraft within the United States. TSOs undergo extensive classroom and simulation-based visual search training to learn how to identify threats within X-ray imagery. Integration of eye tracking technology into simulation-based training could further enhance training by providing in-process measures of traditionally “unobservable” visual search performance. This paper outlines the research and development approach taken to create an innovative training solution for X-ray image threat detection and resolution utilizing advances in eye tracking measurement and training science that provides individualized performance feedback to optimize training effectiveness and efficiency.
Conference Paper
Full-text available
Augmented reality (AR) is defined as “a live direct or an indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input, such as sound, graphics or GPS data.” It is not uncommon to come face-to-face with smart devices that are equipped with multiple embedded sensory inputs such as mega pixel camera, microphones, speakers, high definition (e.g. Retina) displays, 3D displays, holographic displays and pico-projection technologies. Such technology has enabled application designers and developers to package information succinctly and efficiently without loss of clarity. Recently, AR applications (e.g. iPhone World Lens, Google goggles) have drawn mainstream attention. The military also has programs that represent a leap forward (e.g. DARPA Sandblaster program). These advances in AR have been influenced by developments in variety of technologies including low cost of advanced processors, light weight displays, ubiquitous computing afforded by omnipresent devices such as smart phones, tablets, etc. However, there are currently no human factors standards to aid the development. These technologies have great potential to enhance our abilities, but there is also the risk that they represent an annoyance or a significant safety risk. Specifically, improper system lag, reliability, display design (e.g., clutter or resolution) could lead to errors. The goal of this session is to discuss what research is needed to define these standards. It is likely that there is no one set of standards, but developing a framework for these standards will go a long way towards bridging the research-application gap.
Article
Inspection is an important step in ensuring product quality especially in aircraft industry where safety is the highest priority. Since safety is involved, effective strategies need to be set to improve quality and reliability of aircraft inspection/maintenance and for reducing errors. Humans play a critical role in visual inspection of airframe structures. Major advancements have been made in aircraft inspection, but General Aviation (GA) lags behind. Strategies that lead to improvement in inspection processes with GA environment will ensure reliability of the overall air transportation system. Training is one such strategy where advanced technology can be used for inspection training and reducing errors. A hierarchical task analytic (HTA) approach was used to systematically record and analyze the aircraft inspection/maintenance systems in geographically dispersed GA facilities. Using the task analytic approach a computer based training system (GAITS: General Aviation Inspection Training System) was developed for aircraft inspection that is anticipated to standardize and systematize the inspection process in GA. This report documents the work involved in the development of General Aviation Inspection Training Systems in the GA environment.