Content uploaded by Adrian Schwaninger
Author content
All content in this area was uploaded by Adrian Schwaninger on Dec 20, 2013
Content may be subject to copyright.
The 4th International
Aviation Security Technology Symposium
November 27 – December 1, 2006
Airport security human factors: From the weakest to the
strongest link in airport security screening
Adrian Schwaninger
University of Zurich, Department of Psychology, Visual Cognition Research Group (VICOREG),
Switzerland
Abstract
Airport security screening is a challenging task. In fact, according to several experts, the human operator
is often the weakest link of the security system. In this article, the results of human factor studies
conducted over the last five years involving several international airports in four European countries are
summarized. It is shown how human operators can in fact become the strongest link if they are selected
and trained effectively. Recent developments and results of scientific studies in the following areas are
briefly presented and discussed: Pre-employment assessment and selection tests, adaptive computer-
based training, 3rd generation threat image projection, certification and competency assessment of x-ray
screeners.
Introduction
Several factors determine the performance of human operators (screeners) at an airport security
checkpoint (Figure 1). There are large differences between people with regard to aptitudes and abilities
needed in security screening. This is the reason why selection tests as part of the pre-employment
assessment procedure are valuable instruments to increase performance in airport security screening. X-
ray screeners need to know which items are prohibited and what they look like in passenger bags.
Computer based training is a very important tool to achieve and maintain x-ray image interpretation
competency because it allows exposing
screeners to objects that they usually do not
encounter in real life (e.g. improvised
explosive devices). Although TIP is not a very
effective training tool, it is a good technology
to increase and maintain motivation and
alertness of screeners. Humans often fail to
react appropriately if something happens that
they do not expect. Therefore, it is also very
important to conduct frequent practical tests
with real threat objects so that screeners
become used to reacting appropriately if they
encounter a real threat during the screening
operation, for example when a covert
(infiltration) test is conducted. In order to achieve a good performance at the security checkpoint, these
different factors need to be taken into account. To coordinate all performance improvement efforts a
properly developed and maintained system of supervision and quality control is necessary. This should
include initial and recurrent screener certification and competency assessment.
Pre-Employment Assessment and Selection Tests
It has become clear in recent years, that there are large differences between people with regard to
aptitudes and abilities needed in airport security screening (see for example Schwaninger, Hardmeier, &
Hofer, 2005; Hardmeier, Hofer, & Schwaninger, 2005, 2006a). Performance can be increased
CBT
(Visual Knowledge)
Selection Tests
(Aptitudes & Abilities)
TIP
(Motivaton & Attention)
Practical Tests
(Correct Action)
Screener
Performance
Supervision & Quality Control
Figure 1 Determinants of screener performance.
1
The 4th International
Aviation Security Technology Symposium
November 27 – December 1, 2006
substantially if reliable and valid selection tests are used as part of pre-employment assessment. For
example Hardmeier et al. (2006a) could show that screeners selected with the X-Ray Object Recognition
Test (X-Ray ORT) performed already after one year significantly better than other screeners who worked
since several years at the airport but were not selected using this test. Moreover, Hardmeier, Hofer, and
Schwaninger (2006b, this volume) found that visual aptitudes and abilities needed for coping with image-
based factors such as viewpoint, superposition by other objects and bag complexity did not increase very
much with on the job experience and training. This result was predicted by Schwaninger, Hardmeier, and
Hofer (2005) and stresses the importance of using reliable and valid tests such as the X-Ray ORT as part
of a pre-employment assessment system.
Adaptive Computer Based Training
Visual cognition and object recognition studies have shown that you can only recognize an object if it is
similar to something you have seen before (Graf, Schwaninger, Wallraven, & Bülthoff, 2002;
Schwaninger, 2004b, 2005a). The
consequences for x-ray screening are
illustrated in Figure 2. Each bag contains a
threat object and each of them looks quite
different in the x-ray image than in reality. This
is one reason why many threat items are
difficult to recognize without training. A second
reason is that several objects are not known
from everyday experience, which accounts at
least for the self defense gas spray depicted in
Figure 2c and the improvised explosive device
(IED) depicted in Figure 2d. In addition, some
threat objects look quite similar to harmless
objects. For example the switch-blade knife
depicted in Figure 2b resembles a pen or a
laser pointer. Another problem is image
difficult resulting from viewpoint changes (see
Koller & Schwaninger, 2006 for a recent
study). If an object is depicted from an unusual
viewpoint, it becomes difficult to recognize. This is illustrated in Figure 3. Each of the three objects is
known from everyday experience. However, most people have great difficulties in recognizing the images
at the top without training because they are depicted from an unusual viewpoint. These examples
illustrate how important it is to use a large threat image library in which objects are depicted from many
different viewpoints. Based on a close collaboration between vision scientists and aviation security
experts we have built a multiple views library
that currently contains more than 50’000
threat item images based on more than 700
different types of threat objects depicted from
many different viewpoints. Based on police
and intelligence information, this image library
is constantly being updated and loaded into a
computer based training system (CBT) to
keep screeners ready and prepared using
weekly recurrent training. This CBT (X-Ray
Tutor) is based on the results of scientific
studies on how the human brain processes
visual information to recognize objects (Schwaninger, 2004b, 2005a). A sophisticated algorithm allows
virtual placement of threat objects in X-ray images of passenger bags. This automatic algorithm of X-Ray
Tutor is very effective, scientifically approved and takes into account effects of viewpoint, superposition by
other objects and bag complexity (Bolfing, Michel, & Schwaninger, 2006; Schwaninger, Hardmeier &
Hofer, 2005; Schwaninger, Michel & Bolfing, 2005). Due to this specialized feature, X-Ray Tutor always
ab
cd
Figure 2 Different types of threat items in x-ray images of passenger
b
ags. a Electric shock device, b switch blade knife, c self defense gas
spray “Guardian Angel”, d improvised explosive device (IED).
abc
Figure 3 Objects are difficult to recognize when depicted from an
unusual viewpoint.
2
The 4th International
Aviation Security Technology Symposium
November 27 – December 1, 2006
presents screeners fresh images which constantly challenge interpretational skills. Each training session
is adapted to each individual in order to achieve optimal training effects. Most importantly, screeners
receive immediate feedback for each image containing a threat object. The effectiveness of such adaptive
CBT has been proven in several scientific studies, showing that screeners become able to detect threat
items reliably within a few seconds of image inspection. In addition to substantially increasing detection
performance, training with X-Ray Tutor also increases efficiency by reducing false alarm rates and
processing times (for details see Ghylin, Drury, & Schwaninger, 2006; Schwaninger, 2004b; Schwaninger
& Hofer, 2004; Hofer & Schwaninger, 2004).
Figure 4 summarizes the results of a study conducted with 72 participants. None of them had received
CBT before. For the period of six months, each week 1-2 training sessions of 20 minutes were conducted
using X-Ray Tutor. Four tests were conducted in which new IEDs were used that had not been shown
previously during
training (for details of
the study see
Schwaninger & Hofer,
2004). As you can see
in Figure 4a, there were
large increases of
detection performance
as a result of training.
This was the case for
both display durations
of 4 and 8 seconds. In
order to assess training
effectiveness we
calculated % increase
as compared to baseline measurement (first test results). As you can see in Figure 4b relative detection
performance was increased by about 71%. The analysis of response times revealed interesting findings
with regard to efficiency. Training with X-Ray Tutor resulted in a much faster
detection of IEDs. The response times for hits, i.e. correct decisions on x-ray
images containing a threat item, dropped from about 5 seconds before
training to about 3.5 after six months of training. For harmless bags average
response times remained constant at about 5 seconds, consistent with a
thorough search process.
In a recent study the effects of recurrent CBT on screener detection
performance was investigated with 334 airport security screeners using X-
Ray Tutor during a period of two years (Hardmeier, Hofer, & Schwaninger,
2006b, this volume). Most screeners trained at least twice a week for 20
minutes. Detection performance was measured using the Prohibited Items
Test (PIT). This is a reliable and valid instrument to measure how well
screeners can detect all kinds of threat items in x-ray images (Schwaninger,
Hardmeier, & Hofer, 2005; Hardmeier, Hofer, & Schwaninger, 2006a). As
can be seen in Figure 5 there was a large increase in detection performance
before and after two years of recurrent CBT. Consistent with several earlier studies this result confirms
that adaptive CBT can increase the ability to detect threat items in x-ray images substantially.
3rd Generation TIP: Combining the Benefits of TIP with Effective CBT
Threat image projection (TIP) is a technology that allows projecting fictional threat items (FTIs) on x-ray
images of real passenger bags while screening them at the airport security checkpoint. If a screener does
not detect a TIP image within a specified amount of time (a “miss”), a feedback message appears
indicating that a FTI was missed. Feedback messages are also shown when a FTI is detected (a “hit”) or
in the case of a non-TIP alarm, i.e. when the screener indicated that there was threat but no FTI has been
projected. TIP is now used in several countries and the following advantages have been associated with
it: Increased alertness of screeners, increased motivation and work satisfaction, screeners can be
0
20
40
60
80
100
Dec/Jan Feb/Mar Mar/April April/May
% Increase in Detection d
'
4 sec
8 sec
1
2
3
4
Dec/Jan Feb/Mar Mar/April April/May
Detection Performance d'
4 sec
8 sec
ab
Figure 4 Results of the study by Schwaninger and Hofer (2004) showing large increases o
f
absolute detection performance (a) and % increase of detection performance relative to baseline
measurement (b).
Figure 5 Training effect afte
r
two years of recurrent CBT.
3
The 4th International
Aviation Security Technology Symposium
November 27 – December 1, 2006
exposed to threat items that are normally not found in real bags, analyzing TIP data can be used to
measure detection performance. Usually the TIP to bag ratio is set to a value between 1:50 and 1:200,
i.e. a TIP image is shown to a screener every 50 - 200 bags on average. Because screeners are only
exposed to a few threat items per day when using TIP, this technology is not an effective training tool.
However, TIP has a training value if it is combined with adaptive CBT. Figure 6 shows the architecture of
a 3rd generation TIP system (3i-TIP) which combines the benefits of TIP and CBT for increasing screener
performance (Schwaninger, 2004a). A large TIP image library of 20’000 images is used in which
hundreds of different types of threat objects are depicted from many different viewpoints (the library is
currently being upgraded to 50’000 FTIs). The 3i-TIP system uses a dual-mode. In the adaptive mode,
each screener starts with FTIs shown in easy viewpoints. View difficulty is then increased based on
individual TIP performance. The goal of the adaptive mode is to keep TIP challenging every day in order
to increase and maintain screener motivation and alertness. In testing mode, every screener sees the
same FTIs. This provides the basis for reliable individual competency assessment if data is aggregated
over several months and a large TIP image library is used (Hofer & Schwaninger, 2005). TIP data can be
analyzed off-line with TIP DataVis to create reports and graphics for risk assessment, quality control and
individual screener competency assessment purposes. Missed FTIs are sent from the x-ray machines via
the network to the 3i-TIP server which uses image processing to analyze image difficulty resulting from
viewpoint difficulty, superposition and bag complexity (Bolfing, Michel, & Schwaninger, 2006;
Schwaninger, Michel, & Bolfing, 2005). The images are then sent to the CBT system (X-Ray Tutor) for
missed images review. This means that at the beginning of an X-Ray Tutor training session a screener is
first exposed to the TIP images that were missed while working at the checkpoint. This provides specific
remedial training based on individual TIP data. In addition, the individually adaptive algorithms of X-Ray
Tutor provide efficient training tailored to each individual screener. During a training session with X-Ray
Tutor a screener is exposed to about 400 x-ray images whereas about half of them contain threat items.
This results in very efficient and effective CBT (see scientific studies mentioned in the previous section).
Thus, 3rd generation TIP technology combines the benefits of TIP (motivation and alertness) and
adaptive CBT (effective and efficient training). It thereby provides a solid basis for increasing performance
of screeners. However, as mentioned above, it is also essential to conduct practical tests with real threat
objects frequently so that screeners learn to react appropriately when they are exposed to real threats at
the checkpoint for example during a covert (infiltration) test.
Figure 6 Architecture of a 3rd generation TIP system (3i-TIP) which is operational at several European
airports.
4
The 4th International
Aviation Security Technology Symposium
November 27 – December 1, 2006
Certification and Competency Assessment
The main aim of certification procedures is to ensure that adequate standards are consistently and
reliably achieved across different airports and countries. Certification can be regarded as providing quality
control over the screening process. It can provide important information on strengths and weaknesses in
aviation security procedures in general as well as on each individual screener. Certification and
competency assessment can also be a valuable basis for qualifying personnel, measuring training
effectiveness, improving training procedures and increasing motivation. In short, certification and
competency assessment can be very important instruments to improve aviation security.
The implementation of certification procedures has several challenges. First, what should be assessed
has to be identified. Then, it should be considered how procedures for certifying different competencies
can be implemented. Several countries, organizations and even companies are currently developing their
own certification or quality control systems so that international standardization therefore becomes
another important challenge.
The results of screener certification are very important for appropriate authorities, aviation security
institutions and companies. Moreover, failing a test can have serious consequences, depending on the
regulations of the appropriate authority. Therefore, it is essential, that a test is fair, reliable, valid and
standardized. In the last 50 years, scientific criteria have been developed that are widely used in
psychological testing and psychometrics. These criteria are essential for the development of tests for
measuring human performance.
The Competency Assessment Working Group (CAWG) by InterTAG has addressed these issues
regarding x-ray image interpretation in a White Paper (Schwaninger, Bridges, Drury, Durinckx, Durrant,
Hodge, Hofer, Jongejan, Maguire, McClumpha, Neiderman, Steinmann, Wüest, 2005). This document
has been included in ECAC Doc 30 and it has also been adopted by ICAO. Guidance material on other
components of certification (e.g., theoretical and practical exams) are currently being developed in the
ECAC Training Task Force and by the ICAO Training Group. Such internationally coordinated efforts are
essential to achieve and maintain common standards in aviation security worldwide.
Summary
In recent years, large investments into technology have been made in order to adapt to the new threat
situation. However, the most expensive technology is of limited value if the humans who operate it are not
selected and trained appropriately. Security and efficiency can be increased dramatically if scientifically
approved selection tests are used in a pre-employment assessment procedure. In addition, effective CBT
is needed for initial and recurrent training to increase threat detection performance of x-ray screeners. TIP
is a valuable tool for increasing motivation and alertness of screeners while working at the security
checkpoint. 3rd generation TIP technology combines the benefits of TIP with effective CBT. However, it is
also essential to conduct practical tests with real threat objects frequently so that screeners learn to react
appropriately when they are exposed to real threats at the checkpoint, for example when a covert
(infiltration) test is conducted. Last but not least an initial and recurrent certification system is needed to
ensure that adequate standards in aviation security are consistently and reliably achieved.
References
Bolfing, A., Michel, S., & Schwaninger, A. (2006). Assessing image difficulty in x-ray screening using
image processing algorithms. Proceedings of the 2nd International Conference on Research in Air
Transportation, ICRAT 2006, Belgrade, Serbia and Montenegro, June 24-28, 2006, 253-258.
Ghylin, K.M., Drury, C.G., & Schwaninger, A. (2006). Two-component model of security inspection:
application and findings. 16th World Congress of Ergonomics, IEA 2006, Maastricht, The Netherlands,
July, 10-14, 2006.
Graf, M., Schwaninger, A., Wallraven, C., & Bülthoff, H.H. (2002). Psychophysical results from
experiments on recognition & categorisation. Information Society Technologies (IST) programme,
Cognitive Vision Systems - CogVis (IST-2000-29375).
Hardmeier, D., Hofer, F., & Schwaninger, A. (2005). The object recognition test (ORT) – a reliable tool for
measuring visual abilities needed in x-ray screening. IEEE ICCST Proceedings, 39, 189-192.
5
The 4th International
Aviation Security Technology Symposium
November 27 – December 1, 2006
Hardmeier, D., Hofer, F., & Schwaninger, A. (2006a). Increased detection performance in airport security
screening using the x-ray ort as pre-employment assessment tool. Proceedings of the 2nd International
Conference on Research in Air Transportation, ICRAT 2006, Belgrade, Serbia and Montenegro, June
24-28, 2006, 393-397.
Hardmeier, D., Hofer, F., Schwaninger, A. (2006b, this volume). The role of recurrent cbt for increasing
aviation security screeners’ visual knowledge and abilities needed in x-ray screening.
Hofer F., & Schwaninger, A. (2005). Using threat image projection data for assessing individual screener
performance. WIT Transactions on the Built Environment, 82, 417-426.
Hofer, F. & Schwaninger, A. (2004). Reliable and valid measures of threat detection performance in X-ray
screening. IEEE ICCST Proceedings, 38, 303-308.
Koller, S., & Schwaninger, A. (2006). Accessing X-ray image interpretation competency using the X-Ray
Tutor Competency Assessment Test X-Ray CAT. Proceedings of the 2nd International Conference on
Research in Air Transportation, ICRAT 2006, Belgrade, Serbia and Montenegro, June 24-28, 2006,
399-402.
Schwaninger, A. & Hofer, F. (2004). Evaluation of CBT for increasing threat detection performance in X-
ray screening. In: K. Morgan and M. J. Spector, The Internet Society 2004, Advances in Learning,
Commerce and Security (pp. 147-156). Wessex: WIT Press.
Schwaninger, A. (2004a). Increasing efficiency in airport security screening. Proceedings of AVSEC
World 2004, November 3-5, Vancouver, B.C., Canada.
Schwaninger, A. (2004b). Computer based training: a powerful tool to the enhancement of human factors.
Aviation Security International, FEB/2004, 31-36.
Schwaninger, A. (2005a). Object recognition and signal detection. In: B. Kersten (Ed.), Praxisfelder der
Wahrnehmungspsychologie (pp. 108-132). Bern: Huber.
Schwaninger, A., Bridges, A.P., Drury, C., Durinckx, F., Durrant, P., Hodge, T., Hofer, F., Jongejan, R.,
Maguire, R.L., McClumpha, A., Neiderman, E., Steinmann, C., Wüest, W. (2005). Principles and
Requirements for Assessing X-Ray Image Interpretation Competency of Aviation Security Screeners.
White Paper, International Aviation Security Human Factors Technical Advisory Group (InterTAG),
Competency Assessment Working Group (CAWG).
Schwaninger, A., Hardmeier, D., & Hofer, F. (2005). Aviation security screeners visual abilities and visual
knowledge measurement. IEEE Aerospace and Electronic Systems, 20(6), 29-35.
Schwaninger, A., Michel, S., & Bolfing A. (2005). Towards a model for estimating image difficulty in x-ray
screening. IEEE ICCST Proceedings, 39, 185-188.
6
