Conference PaperPDF Available

A Mobile Application of Augmented Reality for Aerospace Maintenance Training


Abstract and Figures

Aircraft maintenance technicians (AMTs) must obtain new levels of job task skill and knowledge to effectively work with modem computer-based avionics and advanced composite materials. Traditional methods of training, such as on-the-job training (OJT), may not have potential to fulfill the training requirements to meet future trends in aviation maintenance. A new instruction delivery system could assist AMTs with job task training and job tasks. The purpose of this research is to analyze the use of an augmented reality (AR) system as a training medium for novice AMTs. An AR system has the potential to enable job task training and job task guidance for the novice technician in a real world environment. An AR system could reduce the cost for training and retraining of AMTs by complementing human information processing and assisting with performance of job tasks. An AR system could eliminate the need to leave the aircraft for the retrieval of information from maintenance manuals for inspection and repair procedures. AR has the potential to supply rapid and accurate feedback to an AMT with any information that he/she needs to successfully complete a job task. New technologies that promote smaller computer-based systems make the application of a mobile AR system possible in the near future.
Content may be subject to copyright.
Tom Haritos and Nickolas D. Macchiarella
Embry-Riddle Aeronautical University
Daytona Beach, Florida, 32124-3900
Aircraft maintenance technicians (AMTs) must obtain new levels of job task skill and knowledge to
effectively work with modern computer-based avionics and advanced composite materials. Traditional methods
of training, such as on-the-job training (OJT), may not have potential to fulfill the training requirements to meet
future trends in aviation maintenance. A new instruction delivery system could assist AMTs with job task
training and job tasks. The purpose of this research is to analyze the use of an augmented reality (AR) system as
a training medium for novice AMTs. An AR system has the potential to enable job task training and job task
guidance for the novice technician in a real world environment. An AR system could reduce the cost for training
and retraining of AMTs by complementing human information processing and assisting with performance of job
tasks. An AR system could eliminate the need to leave the aircraft for the retrieval of information from
maintenance manuals for inspection and repair procedures. AR has the potential to supply rapid and accurate
feedback to an AMT with any information that he/she needs to successfully complete a job task. New
technologies that promote smaller computer-based systems make the application of a mobile AR system
possible in the near future.
Aviation maintenance facilities for general,
commercial, and governmental organizations
operate, inspect, and maintain complex aircraft
structures and systems with many highly
interrelated components. Many complex
interrelated systems are necessary to maintain
aircraft. The complex interrelated aspects of
aircraft maintenance can be divided into four key
categories: software, hardware, environment, and
liveware. These four categories are often identified
by the term SHEL [1]. Liveware refers to the
human element of the aircraft maintenance system.
Aircraft maintenance technicians (AMTs) are the
liveware in the maintenance system.
The U.S. Government Accounting Office [2]
reports that in the coming years this issue becomes
much more critical as the majority of experienced
AMTs will be replaced with much younger AMTs
that lack experience and have minimal exposure to
various defects and aircraft types. As work
experience increases, AMTs develop a schema for
visual inspection. The development of a schema
assists with perceiving faults and helps technicians
reason whether the item being inspected is
airworthy [3]. Unfortunately, many years of
experience are necessary for a novice AMT to
develop a schema equivalent to that of an expert for
visual inspection, fault recognition, and repair.
Additionally, a large number of experienced AMTs
will retire in the very near future, leaving novice
AMTs in the hands of less knowledgeable mentors
One key job task AMTs perform is visual
inspection. This method of inspection allows room
for innumerable errors due to the lack of human
reliability and lack of training methods available to
teach the skills for visual inspection. Visual
inspection and the majority of aviation maintenance
job tasks require perception and human reasoning.
Unfortunately for AMTs, the search and decision
making skills which involve human perception and
reasoning are typically flawed [4]. The
identification of components (e.g., fuel pump, oil
pump, etc.) or the identification of faults (e.g.,
corrosion, cracks, distortion, etc.) requires the
ability to perceive certain characteristics that should
not exist for specific items. AMTs must take into
account many characteristics of components in
order to recognize faults when they are present.
AMTs must identify the components, recognize the
fault, (e.g., crack, corrosion, etc.) and reason
whether the fault is critical through its
characteristics (e.g., length, width, location) that are
present [4]. Due to the aircraft’s highly interrelated
components and complex systems, it may be
difficult for AMTs to identify the characteristics of
all types of faults and subsequently troubleshoot
various systems from one aircraft to another.
AMTs also conduct routine maintenance and
replacement of parts; repair surfaces for both sheet
metal and composite materials; and inspect for
corrosion, distortion and cracks in the fuselage,
wings, and tail [2]. Regardless of the task, aircraft
technicians must always refer to the maintenance
manual for the specific aircraft to obtain proper
procedures and specifications. A great amount of
time is spent searching for instructions, which
increases maintenance time, worker stress, and
decreases overall job performance. Neumann and
Majoros identify that 45% of an AMTs shift is
spent finding and reading instructional procedures
for job tasks [5].
In order to increase passenger safety, minimize
incidents, and accidents associated with aircraft
maintenance errors, new computer-based
technologies could be integrated into the training of
AMTs. An augmented reality (AR) training system
could possibly reduce errors associated with aircraft
maintenance in two ways. First, AR could allow
for an efficient method of retrieving information.
The information retrieved could be the equivalent to
an expert’s recall from long-term memory. An
expert in aviation maintenance can be defined as
someone who has accomplished various aircraft
maintenance tasks numerous times; often through a
process of overlearning that leads to automaticity
for regularly occurring job tasks. Errors are less
likely to occur when experts perform a job task [5,
6]. Novice technicians are prone to error, especially
under stressful conditions [5]. An application of an
AR system that delivers expert levels of
information in real time to novice workers in a real
world setting could reduce job task related errors.
Second, AR could complement human
information processing by facilitating a transition to
expert levels of knowledge in a shorter span of
time. The supply of efficient and effective feedback
may reduce the likelihood of maintenance errors
because AR could provide virtual objects that have
locations associated with a real world environment
[5]. When an AMT views a work piece (e.g.,
aircraft part or component) both real and virtual
objects (e.g., text, flags, arrows, avatars, etc.) could
be presented in one spatially integrated scene.
The aircraft maintenance infrastructure should
consider the implementation of alternative training
methods that could act as job performance aids. The
aircraft maintenance infrastructure could benefit
from new technology that allows information to
become embedded into a system that an AMT uses
while performing job tasks. Training has been
identified as a primary intervention to improve the
effectiveness, efficiency, and overall performance
of AMTs. Unfortunately, traditional training
methods, such as on-the-job training (OJT), may
not be capable of fulfilling the requirements for
future trends in aviation maintenance and training
for maintenance; systems are becoming too
complex to allow workers with a novice level of
knowledge to perform maintenance. Additionally,
expert AMTs are often too busy with maintenance
tasks that are operationally necessary to divide their
efforts between conducting OJT and their primary
work function (i.e., aircraft maintenance).
Traditional OJT may not be able to provide the
necessary training to novice technicians as it has
done in the past. The sophistication of the aircraft
alone demands new training interventions for
AMTs [2]. The OJT environment (i.e., real world
maintenance setting) may be an excellent choice for
training AMTs if the proper feedback and
appropriate information became readily available
without having to leave the aircraft.
One way to reach the goal of delivering OJT at
the aircraft is through the use of an AR system. AR
is a machine vision and computer graphics
technology that merges real and virtual objects into
unified, spatially integrated scenes [7]. Unlike
virtual reality, AR systems blend the real world and
virtual world (e.g., objects, text) in a real-time
environment [7]. AR may have the capability of
assisting with meeting training needs and could
eliminate training and recurrent training because
cognitive tasks can be carried out for the human by
the AR system. Therefore, it may be possible to
design training functions into the AR system that
are concurrent with the job tasks [5].
AMT Job Description and
Job Description
AMTs work in a number of technical
occupations, which include avionics, airframe,
powerplants, and non-destructive testing. The
majority of work occurs in hangars, on flight lines,
and at certified repair stations. Aircraft maintenance
requires a high level of physical activity and the
physical demands can sometimes become grueling,
strenuous, and may decrease work performance [8].
AMTs should be comfortable with heights and
operating heavy equipment, along with squeezing
into and working in tight confined spaces. Hangars
tend to be noisy and hazardous work environments.
These characteristics are attributable to moving
aircraft and various types of equipment necessary to
perform and accomplish job tasks.
Despite adverse weather, work conditions,
heights, and harmful chemicals, AMTs must
execute a standard of high workmanship and
craftsmanship in accordance to certification. The
majority of maintenance and minor repairs for
scheduled airlines occur on the flight line. These
factors are not often considered but do affect the
performance and productivity of AMTs. These are
only a few examples of the demands on an AMT.
Aviation maintenance requires special attention due
to the complexity of the aircraft; it is imperative
that novice AMTs receive guidance during their
training. An AR system could allow for a high
level of workmanship and high work productivity at
an early stage of a AMTs career (i.e., while working
at a novice level).
The Federal Aviation Administration (FAA)
provides the regulations for the certification of
AMTs through a prescribed curriculum [9]. The
FAA addresses experience requirements and states
that each applicant must present proper
documentation of certification from a school
certified under Federal Aviation Regulation (FAR)
Part 147 Aviation Maintenance Technician Schools,
or provide evidence of:
(a) At least 18 months of practical
experience with the procedures, practices,
materials, tools, machine tools, and
equipment generally used in constructing,
maintaining, or altering airframes, or
powerplants appropriate to the ratings
sought; or,
(b) At least 30 months of practical
experience concurrently performing the
duties appropriate to both the airframe and
powerplant ratings [9].
Subpart D 65.79 Skill Requirements of FAR
147, states that each applicant must successfully
pass an oral and practical exam for the appropriate
rating. The tests cover the applicant’s basic skill in
performing practical projects on the subjects
covered by the written test for that rating. As an
example of work required, an applicant for a
powerplant rating must show his ability to make
satisfactory minor repairs to, and minor alterations
of, propellers [9].
Any further training for AMTs will consist of
OJT, computer based training (CBT) and/or
video-based training. Seminars may also be
included into a training program. Generally,
seminars and workshops are provided for
employees of scheduled airlines. On occasion,
general aviation facilities send their AMTs to
workshops and seminars for new general aviation
aircraft that have advanced technologies (e.g., glass
cockpits and composite structures).
Traditional Methods of Training
Overview of Current Training
Typically following certification, AMTs
receive training from the following three methods:
OJT, computer-based training (CBT), and face-to-
face training. These methods have proven useful
and helpful throughout the years. Unfortunately,
these training methods have limitations. The
sophistication of aircraft technology and the
reduced number of expert AMTs available to teach
novice technicians is negatively affecting novice
AMT training [2].
OJT is the traditional method for teaching
maintenance inspection and repair. Unfortunately,
OJT may not be the best method for training
because the feedback to learners may be infrequent
and unmethodical [3]. OJT may be viewed as an
apprenticeship where a novice AMT is mentored by
an AMT who is an expert. There are many reasons
why the hangar environment may not be optimal for
traditional OJT. First, the work environment tends
to be stressful during aircraft inspection and
maintenance. Organizational conflict can arise
when training needs and operational needs compete
for resources. Secondly, hangars are noisy because
of operating equipment (e.g., ground power units,
hydraulic pumps, compressors, etc.) necessary to
perform job tasks. Thirdly, training various job
tasks requires an explanation of theory, principles,
and functions. The majority of senior AMTs
specialize in aircraft maintenance, not instruction
and theory, so training maybe suboptimal.
Fourthly, many maintenance job tasks are in areas
of confined space that allow access for only one
AMT at a time; confined spaces make it almost
impossible to teach certain job tasks with two
AMTs colocated. Finally, complex job tasks may
require repetitive training to ensure mastery of the
associated skills. Difficult tasks, such as engine
removal/replacement or engine overhaul, cannot be
learned by a novice technician with a single
explanation. These factors affect OJT by increasing
worker/trainee stress, constraining available time,
and decreasing employee work productivity.
At times, OJT may be an unstructured
approach for learning job tasks. Often each AMT
has an individualized way of accomplishing a task
and subsequently he/she teaches the task in a unique
way. These individual teaching methods and job
task steps may not be captured in maintenance
manual procedures. An AR system may permit
standardization for training by allowing novice
AMTs the ability to retrieve standardized training
and guidance information.
AMTs visually inspect and perform routine
maintenance (e.g., hydraulic filter changes,
lubrication) in a timely fashion. Any discrepancies
or non-routine repairs (NRR) found must be
recorded up and corrected before the aircraft is
returned to service. Usually during inspection, there
are NRR’s found, especially with aging aircraft.
These discrepancies create more work to be
accomplished in the same span of time, thus
increasing time constraints and stress.
Manufacturers prescribe work hours for
maintenance job tasks. Some maintenance facilities
offer incentives to AMTs that performed faster than
the manufacturer prescribed time. Added pressure
on AMTs can increase stress and reduce job
Unfortunately, the pressure to accomplish job
tasks and NRR’s found increases stress, time
constraints and reduces or eliminates the possibility
for a novice technician to learn a task in an optimal
learning environment [10]. Such circumstances are
often found throughout aviation maintenance
facilities. The implementation of OJT is less than
optimal the majority of the time.
AMTs are expected to become productive in a
short span of time after their arrival to a repair
facility. This is a very high expectation with regard
to the complex nature of the trade. AR may have
the ability to provide a structure or framework with
increased levels of standardization when compared
to traditional methods of OJT.
Face-to-Face Training
Face-to-face training is conducted through
seminars and workshops. Often seminars and
workshops provide group training sessions to
introduce new aircraft technologies. Face-to-face
training is an effective way to teach new tasks, but
time away from the workplace can be a negative
factor. Seminars are usually no longer than a
workweek long. New technologies (e.g., composite
material repair and digital avionics repair) often
require attendance to follow-up seminars in order
for AMTs to fully learn the techniques associated
with new technologies. Seminars and workshops
can be effective, but these approaches to training
cost companies lost work-hour time and reduced
short-term productivity. Finally, seminars are
typically more useful for experienced technicians
that have knowledge of structural composite repairs
and trouble shooting techniques; seminars reflect
the operational need for expert AMTs to maintain
complex aviation systems. They are often not
designed to meet the needs of novice AMTs [2].
Computer-based training is typically designed
as a tutorial or role-play scenario [11]. CBT
programs can be interactive unlike video-based
media. These training programs have been
implemented to train AMTs. These applications
have shown to be effective for training technicians
[11]; however, CBT devices and video are located
in a classroom environment, which may set
limitations for application. Many tasks may be
better learned in an actual work environment with
the proper and necessary equipment. Furthermore,
the majority of CBT cannot be tailored to fit the
individual needs of the student.
Computer-based training is a regularly used
instructional medium for providing an initial
understanding of systems on an aircraft (e.g.,
landing gear, struts, generators, A/C packs etc.).
Extensive job task training through the use of CBT
is limited due to complexity and costs associated
with developing and maintaining instructional
material [11]. The need to capture various types of
aircraft components and faults into video and/or
CBT becomes an almost impossible task.
AR as a Training Medium
What is AR?
AR is any scene or case in which the real
environment is supplemented using computer
generated graphics. AR displays consist of a real
environment with graphical enhancements known
as augmentations. Milgram and Kishino [12]
effectively defined AR and created a mixed reality
continuum to classify AR’s degree of virtual and
real world element integration (see Figure 1).
Figure 1. Milgram’s Reality-Virtuality
Continuum [7]
Azuma [7, 13] defines the necessary
properties that exist in an AR system. AR systems
combine real and virtual objects into a mixed reality
environment, running interactively, in real time, and
accurately align real and virtual objects with each
other. Azuma categorizes the components of any
AR system into three subsystems: scene generator,
display device, and tracking device.
AR may be identified either as an optical
see-through technology or as a video-based
technology. An optical see-through system
employs a helmet or head mounted display (HMD)
that allows the user to view the real world with a
virtual world projected onto combiner lenses
stationed in front of the eye. Video-based systems
display the mixed reality world on a computer
monitor. Both types of systems typically use an
optical recognition approach for registering real and
virtual elements into one spatially integrated scene
The mobile AR system at Embry-Riddle
Aeronautical University (ERAU) uses an optical
recognition device to identify markers based upon
binary numbers (see Figure 2). The markers are
placed on the aircraft and on aircraft components.
The AR system identifies the binary numbers. The
scene generating function of the system and sends
users virtual information in the form of text onto a
see-through combiner lens the user can position in
front of either eye (see Figure 3).
Figure 2. Markers for Optical-Based AR
AR has shown promise in a number of
technical fields. AR systems have been used
successfully in medicine to provide surgeons
computer assistance for surgical procedures [15].
The laser projected AR system superimposes data
onto the patient and assists surgeons by delivering
virtual information to assist with the procedure [16].
Researchers are experimenting with AR systems to
assist with architectural construction, inspection,
and renovation [17] . The purpose of this study was
to develop an AR system for improving the
methods for construction, inspection, and
renovation. The AR system was designed to guide
workers through the assembly process of a space
frame structure to make certain that each piece of
the structure was properly assembled and fastened.
This AR system directed the workers to the parts
and verbal instructions indicate which part should
be picked up for use. The worker scanned the item
via a barcode to enable the computer to verify that
the proper part was selected for use. Once the
correct piece has been picked up, a virtual image
was superimposed to visually guide installation.
Computer generated verbal instructions detailed
how to correctly install the component [17].
AR could assist in the same manner with
aircraft maintenance (see Figures 3 and 4). As with
the surgical procedures in medicine and the
guidance procedure with architectural design, AR
systems have the potential of superimposing the
necessary information for a job task on a work
piece in real time. AR could allow the user to
access any maintenance procedures and inspection
criterion in a real world work setting. The amount
of time spent retrieving information from the
maintenance manuals to assist with the sensation,
perception, and reasoning of faults for
troubleshooting, removal and repair could be
AR Paradigm for Training AMTs
The aircraft maintenance infrastructure
could benefit from the use of a new instruction
delivery system. The use of an AR system may save
time and reduce the cost of training and retraining
of certain psychomotor and cognitive tasks (e.g.,
troubleshooting system malfunctions). These types
of highly technical procedures must be reintroduced
in order to keep technicians current. One important
goal of AR is to provide scenes that are annotated
with information that is acquired through training
along with supporting novice AMTs with inspection
tasks that are rarely encountered.
Neumann and Majoros [5, 14] propose that
an AR system will complement human information
processing during the performance of an aerospace
maintenance task by controlling attention,
supporting short and long-term memory, and aiding
information integration [14]. Some of the positive
benefits of an AR system include improved recall,
control of attention during training, and can provide
concurrent training. Furthermore, AR could assist
with the removal and repair of items by graphically
depicting the necessary information over the real
world without having to leave the aircraft.
An AR system could assist with a number
of training methods for aircraft technicians. The AR
system will supply all the necessary information
including location, description, function, and the
necessary instruction for inspection,
troubleshooting, removal and repair. AR could
generate rapid feedback allowing for a faster
development of a schema than with traditional OJT.
AR could drastically reduce the time necessary to
perform inspections and repairs. All inspection lists,
fault attributes, and trouble shooting techniques
have the ability to exist over the real world.
Figure 3. Mobile Augmented Reality System
with Helmet Mounted Display at ERAU
Figure 4: Oil Pump Labeled with Markers for
Optical Recognition
AR could also facilitate a standardization of
training. Training is conducted differently by
various organizations. AR could not only
standardize training, but may also redefine OJT by
generating rapid and accurate feedback to allow for
self-instruction. This method could be ideal for
training new technicians entering the field of
aviation maintenance. The system has the capability
of allowing novice technician to retrieve
information equivalent to an expert in the field.
Additionally, in the case of AR-based training,
skills first acquired in a mixed reality setting would
serve for subsequent skill application in the real
world and this may redefine how workers are
trained [15]. Majoros and Neumann [14] propose
that an AR system will make some training methods
needless because of the computers ability to
complete cognitive tasks for the user.
Cognition, Recall, and Long-Term Memory
An AR system aids in the interface of multiple
senses, which is believed to complement human
information processing by aiding information
integration through multi-modal sensory elaboration
[18]. AR can create a framework associated with
the specific job task that facilitates learning and aids
recall. Researchers at ERAU [18, 19] propose that
the multi-modal elaboration occurs through the use
of the multiple senses that include visual, spatial,
proprioceptive, and audio. Furthermore, the
combination of the visual and spatial senses could
allow AR to force learning advantages by
amplifying subject material through multiple
channels of memory [20].
Neumann and Majoros [5] propose that the
virtual objects associated with the work piece (e.g.,
oil pump, fuel pump, etc.) are the basis for a link
into memory. The links are referred to as an array
of graphical descriptions in a work piece. The scene
created by AR through the real and virtual world
may create the framework for AMTs to remember
lists of items. AR may allow for an increased
probability of storing information into long- term
memory. Proper encoding of information could also
aid in the recall process. The virtual text labels or
virtual overlays in general, become associated with
the real world object and encoded into memory as
one visual image. In addition, Neumann and
Majoros [5, 14] propose that spatial cognition is a
fundamental element of AR and of the learning
process. AR could complement the spatial cognition
of aircraft technicians by combining text and the
real object without the need to view external
sources (e.g., CD-ROM or microfiche) to gain
instruction for the task. In return, a successful
interpretation and relationship could exist between
human and machine allowing for more effective
and efficient inspections and repairs.
The purpose of this research is to teach a
selected job task from a 100 hour inspection
checklist in accordance with Appendix D of the
Federal Aviation Regulations [9]. The research to
be conducted in the Spring of 2006 will be a follow-
up study to replicate the findings of other
researchers at ERAU [21]. The task will be to
inspect the propeller mounting bolts and safety wire
for signs of looseness on a Cessna 172S.
The experiment will consist of 36 ERAU
students. Participants for this study will be
randomly drawn from the population of students
undergoing AMT training and initial certification at
ERAU. The experiment will measure the effects of
an AR training paradigm as it compares with the
traditional learning paradigms of print-based
material, video-based material. The participants will
be randomly assigned to one of the three groups.
The experiment is a three-way between subjects, or
a 3 x 2, design. There are two independent
variables. The first independent variable is
represented by the mode of information
presentation and it has three levels, or factors. The
factors are: video-based presentation, AR-based
presentation, and print-based presentation. The
second independent variable is represented by time
and it has two factors. The factors are a three-
minute break in time prior to the post-instructional
recall test and a seven day break in time prior to the
long-term retention recall test. The dependent
variables for this study are immediate recall and
long-term recall. Alpha is set to .05.
The participants will be randomly assigned to a
group on the given day of experimentation. The
participants will be given a synopsis of the purpose
and procedures for the experiment. Consent forms
will then be required in order to participate. After
consent has been given, the participants will be
given a test to determine visual acuity. The 36
participants will be divided into groups of 12 and
each will be given instruction for their assigned
group. Each group will then be given an eight-
minute instructional session to learn about the
propeller, its mounting bolts, and associated safety
wire. The participants will then be given a recall
test to measure how much knowledge was gained
through the instructional delivery mode they used.
A second recall test will occur seven days later. The
test will be the same as the immediate recall test;
the purpose of the second test is to measure how
much information the participant is able to recall
from long-term memory.
The aircraft maintenance system relies on a
complex system of systems often analyzed with the
SHEL model [1]. The liveware element of the
SHEL model, the AMT, is central to a successful
maintenance effort. Standardization of training is
needed and consideration must be given to
alternative methods for teaching job tasks and
training AMTs. The traditional methods of training
have proven useful throughout the years. However,
these methods may not be suitable for future trends
in aviation. Advanced training methods become
important with development of highly sophisticated
aircraft and aircraft systems. In order to maintain a
high standard of safety that minimizes incidents and
accidents associated with aircraft maintenance,
more emphasis should be placed on training AMTs
[2]. Furthermore, a great deal of time is spent
retrieving instruction for job tasks. AR may
eliminate the time spent retrieving information by
supplying the important information associated with
aircraft maintenance while eliminating the need to
leave the aircraft.
An AR system has shown promise in a number
of technical fields, such as medicine and
architectural construction and renovation. Aircraft
maintenance could benefit from AR, as have the
surgical procedures in medicine, by superimposing
virtual data onto the aircraft and its components.
AR could also reduce the cost of training and
retraining by configuring the real world work
setting and real world work piece into a training
medium. The system can incorporate any required
training criteria to allow novice AMTs to train on
the actual aircraft. The AR system could allow
novice technicians to retrieve information
equivalent to an expert’s retrieval from long-term
AR may also complement human information
processing during the performance of aerospace
maintenance tasks by controlling attention and
aiding recall through multi-modal sensory
elaboration. The combination of the visual and
spatial senses could allow AR to facilitate learning
by amplifying subject material through multiple
channels of memory. AR may increase the
probability of encoding information into long-term
memory, thus aiding in information recall.
An AR system could provide the method of
training AMTs necessary to meet the advanced
needs of sophisticated aircraft systems. AR has the
ability to structure OJT. This can be accomplished
by incorporating various training needs that are
necessary for training AMTs. AR could give novice
technicians the ability to receive training on real
work items by delivering vital information needed
to accomplish job tasks.
[1] D. J. Garland, J. A. Wise, and V. D.
Hopkin, Handbook of Aviation Human
Factors. Mahwah, NJ: Lawrence Erlbaum
Associates, Inc., 1999.
[2] United States General Accounting Office,
"FAA Needs to Update the Curriculum and
Certification Requirements for Aviation
Mechanics," United States General
Accounting Office, Washington March
[3] K. A. Gramopadhye, B. Melloy, J.
Bingham, S. Chen, R. Master, A. Bhomick,
N. Quadros, and K. Madhani, Using
ASSIST for Aircraft Inspection Training:
Performance and Usability Analysis: FAA,
[4] G. C. Drury and K. A. Gramopadhye,
"Training for Visual Inspection: Controlled
Studies and Field Implications," presented
at Federal Aviation Meeting on Human
Factors Issues in Aircraft Maintenance and
Inspection, Washington, D.C., 2001.
[5] U. Neumann and A. Majoros, "Cognitive,
Performance, and System Issues for
Augmented Reality Applications in
Manufacturing and Maintenance,"
Proceedings of IEEE the Virtual Reality
Annual International Symposium (VRAIS),
pp. 4-11, 1998.
[6] J. E. Driskell, C. Copper, and R. P. Willis,
"Effect of Overlearning on Retention,"
Journal of Applied Psychology, vol. 77, pp.
615-622, 1992.
[7] R. T. Azuma, Y. Baillot, R. Behringer, S.
Feiner, S. Julier, and B. MacIntyre, "Recent
Advances in Augmented Reality," IEEE
Computer Graphics and Applications, vol.
21, pp. 34-47, 2001.
[8] Federal Aviation Administration,
"Advisory Circular No. 65-30A, Overview
of the Aviation Maintenance Profession,"
Department of Transportation, Ed.: FAA,
2001, pp. 70.
[9] ASA, "FARAMT," vol. Title 14 of the
Code of Federal Aviation, F. A.
Administration, Ed.: Aviation Supplies and
Academics, Inc, 1999, pp. 756.
[10] J. Ormrod, Human Learning, 3rd ed.
Upper-Saddle River, NJ: Prentice-Hall,
[11] S. M. Alessi and S. R. Trollip, Multimedia
for Learning: Methods and Development,
3rd ed. Boston: Allyn and Bacon, 2001.
[12] P. Milgram and F. Kishino, "A Taxonomy
of Mixed Reality Visual Displays," IEICE
Transactions Information Systems, vol.
E77-D, pp. 1321-1329, 1994.
[13] R. T. Azuma, "A Survey of Augmented
Realty," Presence, vol. 6, pp. 355-385,
[14] A. Majoros and U. Neumann, "Support of
Crew Problem-Solving and Performance
with Augmented Reality," Bioastronautics
Investigators' Workshop, Galveston, TX
January 17-19 2001.
[15] R. Kalawsky, A. W. Stedmon, K. Hill, and
C. Cook, "A Taxonomy of Technology:
Defining Augmented Reality," presented at
Human Factors and Ergonomics Society
Annual Meeting, Santa Monica, CA, 2000.
[16] D. N. Glossop and Z. Wang, "Laser
Projection Augmented Reality System for
Computer Assisted Surgery," International
Congress Series, pp. 65-71, 2003.
[17] A. Webster, S. Feiner, B. MacIntyre, and
W. T. Massie, "Augmented Reality in
Architectural Construction, Inspection, and
Renovation," Proceedings of the ASCE
Third Congress on Computing in Civil
Engineering. June 17-19, Anaheim, CA, pp.
913-919, 1996.
[18] N. D. Macchiarella, "Effectiveness of
Video-Based Augmented Reality as a
Learning Paradigm for Aerospace
Maintenance Training," Dissertation
Abstracts International, vol. 65(09), pp.
3347A, (UMI No. 3148420), 2004.
[19] B. Valimont, "The Effectiveness of an
Augmented Reality Learning Paradigm,"
Embry-Riddle Aeronautical University,
Daytona Beach, FL 2002.
[20] N. D. Macchiarella, "Augmenting Reality
as a Medium for Job Task Training,"
Journal of Instruction Delivery Systems,
vol. 19, pp. 21-24, 2005.
[21] D. A. Vincenzi, B. Valimont, N. D.
Macchiarella, C. Opalenik, S.
Gangadharan, and A. Majoros, "The
Effectiveness of Cognitive Elaboration
Using Augmented Reality as a Training and
Learning Paradigm," Proceedings of the
Human Factors and Ergonomics Society
47th Annual Meeting. Denver CO, pp.
2054-2058, 2003.
Email Addresses
Nickolas D. “Dan” Macchiarella, Ph.D.
Tom Haritos
Digital Avionics Systems Conference
October 30, 2005
... The application of Virtual Reality in Industry 4.0 has a different orientation due to the level of immersion it provides to the user (17,18) . ...
... Overall, the simulation model has the advantage of dealing with complex difficulties when the model's situational behaviour fluctuates over time, such as discontinuing product delivery during specified times [18]. From the literature review, it is evident that anyLogistix software has not been used in the FVSC and this is the first paper to discuss enhancing supply chain efficiency in FVSC. ...
... In addition, the electrolysis process uses DC electricity from renewable energy sources such as solar, wind, and biomass. However, due to cost constraints, only R 4% of hydrogen can be created by electrolysis of water at the moment [18,19]. This figure is likely to rise in the near future, owing to rising use of renewable energy (solar, wind, and nuclear), while the European Energy Directive has set a target of using renewable energy sources for 14% of total energy requirements by 2020 [17]. ...
Conference Paper
Full-text available
Explore the spread of LSS culture in India by systematic literature review of research article in current decade
... Lisans düzeyinde 96 öğrenci üzerinde yapılan eğitimler ve ölçme değerlendirme sonuçlarında, AG teknolojisi kullanmanın kalıcı öğrenmeye olumlu yönde etki ettiği görülmüştür. Haritos & Macchiarella (2005), Cessna 172S' de gevşeklik belirtileri için pervane montaj cıvatalarını ve güvenlik telini inceleme görevi için AG uygulamasının etkisi üzerine araştırma yapmışlardır [11]. AG teknolojisi kullanımın diğer eğitim yöntemlerine göre (video, resim vb.) kalıcı öğrenmeyi, uzun süreli hatırlamayı olumlu etki ettiği görülmüştür. ...
... Lisans düzeyinde 96 öğrenci üzerinde yapılan eğitimler ve ölçme değerlendirme sonuçlarında, AG teknolojisi kullanmanın kalıcı öğrenmeye olumlu yönde etki ettiği görülmüştür. Haritos & Macchiarella (2005), Cessna 172S' de gevşeklik belirtileri için pervane montaj cıvatalarını ve güvenlik telini inceleme görevi için AG uygulamasının etkisi üzerine araştırma yapmışlardır [11]. AG teknolojisi kullanımın diğer eğitim yöntemlerine göre (video, resim vb.) kalıcı öğrenmeyi, uzun süreli hatırlamayı olumlu etki ettiği görülmüştür. ...
Conference Paper
Full-text available
Bu çalışmada havacılık sektörüne yönelik olarak hava aracı bakımı ve teknisyen yetiştirme eğitimleri kapsamında yapılan Artırılmış Gerçeklik (AG) çalışmaları incelenerek, uçak turbo motoru eğitimi üzerine bir AG eğitim uygulamasının geliştirilmesi anlatılmıştır. Akıllı telefon, tablet vb. mobil cihazlarda çalıştırılmak üzere Mesleki ve Teknik Anadolu Liselerinde Uçak Bakımı Alanında eğitimi verilen Uçak Motoru ve Yapıları eğitimine yönelik olarak içerik geliştirilmiştir. Uygulamanın geliştirilmesi için öncelikle bir eğitim senaryosu hazırlanmıştır. Uçak Motoruna ait 3 boyutlu model ve parçalanma animasyonu hazırlanmıştır. AG uygulamasını geliştirmek için, hazırlanan modeller Unity3D oyun motoru yazılımına aktarılmış ve Vuforia eklentisi kullanılarak android işletim sistemi tabanlı akıllı telefon, tablet vb. mobil cihazlarda çalışabilecek şekilde işaretleyici tabanlı bir AG uygulaması geliştirilmiştir. Geliştirilen uygulamanın, Milli Eğitim Bakanlığına bağlı resmi ve özel kuruluşlarda Mesleki ve Teknik Eğitim veren kurumlarda, Uçak Bakım Alanı öğrencilerinin eğitiminde kullanılması ile öğrencilerin derse olan ilgilerinin ve motivasyonlarının ve derslerdeki başarı oranlarının artacağı düşünülmektedir. Geliştirilen uygulama aynı zamanda kullanıcıların eğitiminde kolaylık, üretimde kalitenin artmasını ve daha hızlı bakım ve montaj yapılmasını sağlayacaktır.
... Heal and Kunreuther (2005) put forward the concern that any operating companies in the commercial airline industry could pose a threat to its security dependence since boundaries between airlines and outside parties become narrow even fade in such an information-intended globalized society. Digital measures will be considered to be applied in the website, to ensure database integrity and protect passengers' privacy, and physical security-related components, such as Automation in the flight deck and Augmented Reality (AR) technology (Haritos & Macchiarella, 2005). ...
Full-text available
This study aims to analyze how digitalization and digital technology impact passengers’ air travel, specifically in the door-to-door air travel process. As a booming industry that involves advanced technology, the aviation industry attracted an increasing number of passengers and occupies a large position in the global market. However, it is also one of the industries which are most affected by the COVID-19 pandemic. Thus, looking insight into how advanced technologies can help the industry to reverse the negative situation and get better performance is urgent. A survey has been conducted to investigate the extent that passengers were satisfied with current digital tools in their door-to-door travel process. Actually, more than half of the participants used digital tools every time when they were traveling, and the most common digital tool they used was airline apps. Besides, a majority of participants showed a positive attitude toward the digital technologies applied in different stages of their air travel. However, some issues, like security issue and less efficiency, need to be further improved to satisfy passengers’ requirements. All in all, the analyses and results of this study can provide a useful guideline for the aviation industry to develop an effective digital strategy and to identify how to help passengers enjoy a more satisfying journey.KeywordsDigitalizationDoor-to-door air travelDigital technologyAviation efficiencyCustomer survey
... The use of the augmented reality application was easy to use 310 3.77 You need help or assistance using the application 13 4 The application's functionalities are correctly integrated 211 3.69 There is flexibility in using the application 1 2 10 0.31 You think most people would quickly learn to use the application 1 12 3.92 Using the application made it very difficult for him to use ...
This document presents an augmented reality application for mobile devices, as a contribution to education through a technological learning tool that allows the management of industrial robotic arms, implementing advanced control algorithms, which allows the simulation of several selected desired trajectories by the user; and the incorporation of animations that allow to know its operation and to verify the follow-up of the proposed trajectory, as well as the visualization of control errors in each trajectory taken. The application is oriented to the simulation of industrial robotic arms within an intuitive and friendly augmented reality environment, which allows users a great interaction with the robot's structure, providing simulation programs with new immersion technologies, in the educational field. Tests in the augmented reality application demonstrate ease of use and user intuition, providing a better understanding of the operation and structure of programmable manipulators.
... The system consists a computer vision algorithm, which is able to evaluate, at each step of a maintenance procedure, if the user has correctly completed his task, or not. The areas with a great number of applications of AR in maintenance processes are: aerospace [57][58][59], automotive [60][61][62] and industrial plants [4,63]. Due to the increasing complexity of maintenance processes, various assistants (e.g., tablets with smart software, smartphones, and virtual assistants) increasingly support employees of technical departments. ...
Full-text available
Few decades ago, maintenance was mainly seen as something that was difficult to manage. Nowadays, taking into account the impact of maintenance on other company processes (e.g. production, logistics, occupational health and safety management, environment), the maintenance process is recognized as an important business function. The economic impact of improper designed maintenance processes is related to costs, downtime, breakdowns, defects, additional inventory, etc., which affect product quality and the company’s performance. Maintenance technology is about finding and applying cost-effective ways to avoid or overcome degradation in machine performance and availability. The main goal of the paper is to present an overview of Industry 4.0 technologies in maintenance on the newest paradigm in the maintenance management. By the using of the advanced Industry 4.0, technologies, it is possible to monitor the state of the manufacturing equipment, diagnose faults, predict and solve problems before they occur. Moreover, the digitization of maintenance requires the mastery of both new technologies and organizational issues related to their planning, implementation and maintenance.KeywordsIndustry 4.0Maintenance 4.0I4.0 technologies
Purpose The purpose of this study is to analyze the effect of human factor training in aircraft maintenance accident mitigation and aircraft safety in post COVID-19 aviation scenarios. The cause of aircraft accidents and details of three decades of selective aircraft maintenance accidents are analyzed to arrive to the significant aviation safety factor. The effect of COVID-19 pandemic and related technological applications to maintain high standards of safety and their applications in aircraft maintenance with respect to the view of human factors are discussed in details. Design/methodology/approach This paper details the overview of the human errors, error mitigation and need of human factor applications in aircraft maintenance industry for safe air travel. The criticality of aircraft maintenance in keeping aircraft in airworthy condition to provide safe air transportation without delay and to support airline economy is discussed in this study. Findings The cause of aircraft accidents and details of three decades of selective aircraft maintenance accidents are analyzed to arrive to the significant aviation safety factor. The effect of COVID-19 pandemic and related technological applications to maintain high standards of safety and their applications in aircraft maintenance with respect to the view of human factors are discussed in details. The route of error mitigation and need of high standard technological training with human factor knowledge, to aircraft maintenance students are analyzed in detail with the opportunity of percentages of error reduction. Originality/value This study bridges, gained knowledge for aircraft maintenance error mitigation, current accident rates and future training needs for safest air travel through high standard quality maintenance in aircraft and its systems.
Full-text available
Mixed Reality (MR) visual displays, a particular subset of Virtual Reality (VR) related technologies, involve the merging of real and virtual worlds somewhere along the 'virtuality continuum' which connects completely real environments to completely virtual ones. Augmented Reality (AR), probably the best known of these, refers to all cases in which the display of an otherwise real environment is augmented by means of virtual (computer graphic) objects. The converse case on the virtuality continuum is therefore Augmented Virtuality (AV). Six classes of hybrid MR display environments are identified. However quite different groupings are possible and this demonstrates the need for an efficient taxonomy, or classification framework, according to which essential differences can be identified. An approximately three-dimensional taxonomy is proposed comprising the following dimensions: extent of world knowledge, reproduction fidelity, and extent of presence metaphor.
A complete examination of issues and concepts relating to human factors in simulation, this book covers theory and application in space, ships, submarines, naval aviation, and commercial aviation. The authors examine issues of simulation and their effect on the validity and functionality of simulators as a training device. The chapters contain in depth discussions of these particular characteristics and issues. They also incorporate theories pertaining to the motivational aspects of training, simulation of social events, and PC based simulation.
This paper surveys the field of augmented reality (AR), in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.
This paper outlines a functional decomposition of Augmented Reality (AR), from both a technological and human factors perspective, with a view to providing a coherent and structured framework for its development and evaluation in the future. Without a formal framework to underpin the description and development of AR technology, confusion may arise in comparing systems and applications. The proposed framework allows for the user's capabilities to be described in all their sensory detail, which, in turn, furnishes the taxonomy with a power to account for these in the development of advanced technologies. This top-level analysis can then be decomposed into discrete elements representing the enabling technologies that make up a given AR domain. This provides a much richer descriptive power within the framework and consistent criteria for comparing systems, applications and indeed conventional human-computer interfaces with AR. Furthermore, as it is a ‘user-centred’ approach, the framework does not assume that visual augmentation is the only means of providing an AR facility. Implicit in the framework is the notion that AR can be delivered through other sensory augmentation, even if the technical means is not available at the moment. In this way, the taxonomy offers a concise and precise means of categorising the technology, understanding the application, and, ultimately, supporting the user of an AR system.
Until now, little research attention in the area of Augmented Reality (AR) has been paid to the cognitive benefits engendered by this emerging technology. AR, the synthesis of computer images and text in the real world (Azuma, 1997), affords a supplement to normal information acquisition that has yet to be fully explored and exploited. AR achieves a more smooth and seamless interface by complementing human cognitive networks, and aiding information integration through multi-modal sensory elaboration, by utilizing visual, verbal, proprioceptive, and tactile memory while the user is performing real world tasks. AR also incorporates visuo-spatial ability, which involves the representations of spatial information in memory. The use of this type of information is an extremely powerful form of elaboration. This study examined four learning paradigms: print (printed material) mode, observe (video tape) mode, interact (text annotations activated by mouse interaction) mode, and select (AR) mode. The results of the experiment indicated that the select (AR) mode resulted in better learning and recall when compared to the other 3 conventional modes of learning. Implications for training and curriculum design are discussed.
1.1 INTRODUCTION This report is divided into seven major sections. The Background outlines the need for training in inspection. The next three sections detail the ASSIST development effort, introduce the reader to its evaluation effort, and outline the methodology used to evaluate this system, respectively. Sections on performance and usability analysis describe the results of the evaluation effort. Finally the Conclusion outlines the implications of this study in using computer-based inspection training for improving aircraft inspection performance. This research was conducted with various industry partners to ensure its relevance and applicability to the aviation maintenance community.
A general-purpose augmented reality apparatus was developed. The system uses rapidly scanned lasers to display information directly onto the patient using the patient as a “projection screen”. The device is also capable of measuring the location of projected laser spots with a 3D camera to ensure that the display is accurate, to apply corrections to the projection path and to assist in registration. The projection system has its own Application Programmer's Interface (API) and is a stand-alone add-on unit to any host computer system.An initial host application was developed to demonstrate the new unit. Fiducials representing vertices along a proposed craniotomy were embedded into a plastic skull and a projection path was calculated. A feedback-based optimization of the plan was performed by comparing the measurement taken by the camera of these coordinates. The optimized plan was projected onto the skull. On average, the projection deviated by approximately 1 mm from the plan. Possible applications include identification of critical anatomical structures, visualization of preplanned paths and targets, and telesurgery or teleconsultation applications.
The effectiveness of overlearning in enhancing performance has been acknowledged by researchers within the training community for years. In spite of this general consensus, the empirical basis for this claim is often not clear. This article presents a meta-analysis of the effects of overlearning on retention. Results indicate that overlearning produces a significant effect on retention of moderate overall magnitude and that the effect of overlearning on retention is moderated by the degree of overlearning, type of task, and length of retention period. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Presented as the author's thesis (Ph. D)- Nova Southeastern University. Open to researchers under library restrictions. Includes bibliographical references (leaves 127-134). Thesis (Ph. D.) Nova Southeastern University, 2004. Typescript.