ArticlePDF Available

Human Factors Issues In Synthetic Vision Displays: Government, Academic, Military, and Industry Perspectives

Authors:
Human Factors Issues In Synthetic Vision Displays: Government,
Academic, Military, and Industry Perspectives
Lawrence J. Prinzel III, Ph.D.
NASA Langley Research Center
Hampton, VA
Organizer and Chair
Panelists
J. Raymond Comstock, Jr., Ph.D.
NASA Langley Research Center
Hampton, VA
Co-Chair
Tim Etherington
Rockwell Collins
Cedar Rapids, IA
Guy A. French, Ph.D.
Air Force Research Laboratory
Wright-Patterson AFB, OH
Michael P. Snow, Ph.D.
The Boeing Company
Seattle, WA
Mica R. Endsley, Ph.D.
SA Technologies, Inc.
Marietta, GA
Christopher D. Wickens, Ph.D.
University of Illinois
Urbana-Champaign, IL
Kevin M. Corker, Ph.D.
San Jose State University
San Jose, CA
Background
Aviation has been witness to rapid advancement in technologies that have significantly improved aviation
safety. The development of attitude indicators, flight management systems, radio navigation aids, and
instrument landing systems (ILS) have extended aircraft operations into weather conditions with reduced
forward visibility. However, as Brooks (1997) has noted, “…while standard instrumentation has served
us well, enabling aviation as we see it today, literally thousands of dead souls, victims of aviation
catastrophe, offer mute and poignant testimony to its imperfections. The simple, elegant dream of soaring
aloft visually, intuitively – bird-like – remain elusive” (p. 17). Using conventional displays, pilots must
integrate information from many separate sources to achieve situation awareness. This integration
process can lead to errors, which in some cases can have deadly consequences.
In commercial aviation, over 30% of all fatal accidents worldwide are categorized as controlled-
flight-into-terrain (CFIT). In general aviation, the largest accident category is Continued Flight into
Instrument Meteorological Conditions (IMC), where pilots with little experience continue to fly into
deteriorating visibility conditions and either collide with terrain and/or lose control of their aircraft
because of a lack of familiar external cues. Finally, the single largest factor causing airport delays is
limited runway capacity and increased air traffic separation required when weather conditions fall below
visual flight rule operations. Today, synthetic vision technology may allow this visibility problem to be
solved with a visibility solution, making every flight the equivalent of a clear, daylight operation.
Synthetic vision is a display system that presents a view of the outside world to the flight crew by
melding computer-generated scenes from on-board databases and guidance displays, with information
derived from on-board sensors that augment the database imagery to provide object detection and
improved integrity of the display. Synthetic Vision Systems (SVS) are characterized by the ability to
represent visual information and cues that are intuitive and resemble visual conditions with unlimited
ceiling and visibility (Figure 1). In terms of safety benefits, synthetic vision may help to reduce many
accident precursors including:
Loss of vertical and lateral spatial awareness
Loss of terrain and traffic awareness on approach
Unclear escape or go-around path even after recognition of problem
Loss of altitude awareness
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 48th ANNUAL MEETING—2004 75
Loss of situation awareness relating to the runway environment and incursions
Unclear path guidance on the surface
Lately, there has been a significant amount of human factors research published on the subject of
synthetic vision. Although research has demonstrated the efficacy of synthetic vision to reduce CFIT
accidents, runway incursions, spatial disorientation, and enhance operational capabilities, there remain
issues to be investigated by researchers. Therefore, the purpose of the panel is to allow researchers from
government, industry, academia, and military to share their perspectives on the human factors of synthetic
vision.
NASA Synthetic Vision Perspective for Commercial and General Aviation Aircraft
Lawrence J. Prinzel III, Ph.D.
Meeting national aviation safety goals will require mitigating or eliminating the etiologies of accidents. A
significant factor involved in many commercial and general aviation accidents is limited visibility. NASA
initiated a new research project to develop technologies to help overcome safety problems associated with
limited visibility. The NASA Synthetic Vision System (SVS) project is based on the premise that better
pilot situation awareness during low visibility conditions can be achieved by reducing the steps required
to build a mental model from disparate pieces of data through the presentation of how the outside world
would look to the pilot if their visibility were not restricted. SVS display concepts employ computer-
generated terrain imagery, on-board databases, and precise position and navigational accuracy to create a
three dimensional perspective presentation of the outside world, with necessary and sufficient information
and realism, to enable operations equivalent to those of a bright, clear, sunny day regardless of the outside
weather condition.
Several panelists argue that synthetic vision may enhance situation awareness but have significant
concerns and question whether synthetic vision is the right solution. While agreeing for the need for more
research, the government perspective is that synthetic vision is much more than just the display of
synthetic terrain and that safeguards can be implemented to ensure that such concerns do not plague
synthetic vision systems. The NASA Synthetic Vision System is composed of several technologies that
include synthetic vision head-down, head-up, and navigation displays; pathway and guidance symbology;
runway incursion prevention technology; database integrity monitoring equipment; enhanced vision
sensors; taxi navigation and surface maps; and advanced communication, navigation, and surveillance.
Together, these technologies represent a comprehensive solution to problems of restricted visibility ---
i.e., solving a visibility problem with a visibility solution. The presentation will describe several
synthetic vision systems that are being developed for commercial, business, and general aviation aircraft.
An overview of research will also be provided that demonstrates the potential of SVS to mitigate the
precursors to low-visibility accidents and significantly enhance operational capabilities.
Industry Perspective for Business and Regional Aircraft
Tim Etherington
Perspective flight displays have been widely researched for over 50 years. Widespread availability of
accurate database information and new survey techniques coupled with advances in mass storage and
graphics processing have finally enabled the application of this research to practical avionics
architectures. Processing power is never unlimited and storage is still not free so the question of minimum
requirements for synthetic vision systems is explored. These requirements are discussed for minimum
resolution of terrain data, texturing of the terrain frame and shading applied to the texture. Field of view
requirements, guidance concepts, relationship to required navigation performance, precision navigation
and database validation issues are discussed. The industry perspective is explored for the business and
regional aircraft market with possible operational benefits detailed.
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 48th ANNUAL MEETING—2004 76
Synthetic Vision Technology for Air Force Applications
Guy A. French, Ph.D.
Synthetic vision technologies may include databases of terrain, obstacles, threats, traffic, and weather
information that are rendered on one or more displays to aid the pilot in creating and maintaining an
accurate mental model of the aircraft situation relative to the environment outside the cockpit. While the
current location for displaying information to aircrew is primarily on multifunction displays, increasingly
both commercial and military flying organizations are turning to head-up or head-mounted displays due to
their “see through” nature. As these displays become more common, information beyond that required for
basic flight will begin to find its way onto the HUD and HMD leading to potential conflict between the
desire to provide more information and the desire to see through to the outside world. Various symbology
and formatting strategies may be employed to reduce or resolve this conflict.
Another potential issue is that the underlying databases are likely to have widely varying update
cycles due to the nature of the specific information each contains. Within the civil airspace processes and
procedures will be developed that allow for regular maintenance of database accuracy. In areas with a
more austere nature that might be of interest to USAF operators on short notice (e.g. open fields with no
instrumentation) such database maintenance is likely to be less frequent and/or complete. There are
several strategies that may be employed to ameliorate stale data in these remote locations, including
ground support teams, UAVs, and enhanced vision systems.
Synthetic Vision Industry Perspective for Large Transport Aircraft
Michael P. Snow, Ph.D.
Synthetic vision, the concept of displaying a forward view to the pilot based on a database, has been
researched and developed for several decades. It usually includes egocentric, perspective view of
commanded flight path with terrain, obstacle, traffic, airspace and other data. Research findings indicate
that use of these displays leads to greater pilot situation awareness, reduced workload, and reduced flight
technical error. Proposed benefits include increased safety and new operational capabilities such as
reducing approach minima and aircraft separation. The application of this technology in large transports
is discussed, emphasizing pragmatic reasons for including it in modern commercial flight decks – or not.
Obstacles to application in this environment include cost, cost-effectiveness and maturity of competing
technologies, required infrastructure and technology development, training impacts, regulatory concerns,
and remaining human factors issues. These are detailed and areas in which human factors professionals
might make an impact are outlined.
Situation Awareness and Synthetic Vision
Mica R. Endsley, Ph.D. and William M. Jones
Synthetic Vision Systems (SVS) are built on database-derived information that is used to aid the pilot in
visualizing the aircraft situation relative to information outside the cockpit. It may incorporate terrain,
obstacles, cultural features, weather, and/or traffic information. As an additional and intuitive source of
information, the SVS concept may aid pilot situation awareness (SA) in many ways. Due to the
limitations of display technology, however, it may also lead to certain SA difficulties, particularly if it is
used in place of out-of-the-window viewing under no or low visibility conditions. The many potential
improvements in pilot SA that can be enabled through SVS include:
Overcoming CFIT-inducing problems,
Augmented information for basic flight control,
Improved awareness of other traffic, and
Aiding in flight operations tasks such as:
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 48th ANNUAL MEETING—2004 77
Evaluation of changes in runway and approach
Evaluation of new ATC vector/clearance
Evaluation of aircraft spacing
Evaluation of timing and fuel usage on path
Awareness of poor weather conditions on route.
Along with these benefits, however, there are hidden traps that may undermine SA unless the SVS
systems are carefully designed and tested. These pitfalls include:
Difficulty in correctly perceiving the vertical flight profile in a 3D display,
The compelling influence of graphical 3D displays may overcome digitally presented instrument data,
leaving the pilot open to spatial disorientation,
The potential to believe no traffic (or other obstacles) are present if not displayed (type II errors),
when in fact this “false world” could result from a database or sensor limitation.
Although the veridical nature of the SVS display is its strong point— integrated information
presented in a very natural manner — this also is its Achilles’ heel. It is a far more compelling display
than any previously and more likely to suck pilots into any false or ambiguous information it presents.
While it can be easily argued that the information provided by the SVS is better than the very limited
information available today under low visibility conditions, the desire to increase aircraft throughput (and
reduce safety margins) under these conditions by use of the SVS (creating significant efficiency gains),
demands that any potential SA problems be detected in the evaluation process and corrected for prior to
its implementation in flight operations.
Compellingness and Synthetic Vision Systems
Christopher D. Wickens, Ph.D.
Endsley and Jones (this symposium) have made a “compelling” case for the advantage of SVS displays to
support terrain and flight path awareness, but have also alluded to the “hidden traps” associated with the
very compellingness of the display itself, in terms of overtrust of the information offered there, and over-
allocation of attention to that display. This paper will undertake a careful examination of the components
that induce compellingness in an SVS display: its photo-realism, its 3D egocentric frame of reference, and
the pathway guidance, which it often hosts. We will consider the implications of excessive attentional
tunneling on the display for the detection of off-normal events, and will summarize the existing data on
SVS, and similar systems, regarding the allocation of attention, and the detection of unexpected events.
Human Factors Analysis Evaluation of SVS Commercial and Business Systems
Kevin M. Corker, Ph.D. & Eromi Guneratne
We have analyzed one of the Aviation Safety Program’s interventions for safety, the Synthetic Vision
System for commercial and business jet (SVS-CAB) applications from a human performance/human
factors perspective. In performing that analysis, we identified categories of human factors issues
associated with the use of the technology as follows:
Visual presentation of information (e.g., compellingness, visual momentum and cross-reference to
other instrumentation)
Integration of several types of information into the SVS Display and integration for the SVS with
either out-the-window or other flight directive and navigational data available in the cockpit (flight
directive, predictive, terrain, traffic.),
Procedures for use of the technology (crew resource management)
Off-nominal or marginal operations
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 48th ANNUAL MEETING—2004 78
An analysis of these issues as applied to the SVS design was undertaken to determine the current state
of the art associated with human system interaction in similar or analog systems. This examination
yielded a document of “issues” associated with the presentation of flight guidance and terrain avoidance
and awareness in low or marginal visibility conditions. The state of the art was then compared with the
state of research and development of the SVS-CAB to produce a matrix of features of the visual-aiding
system mapped to the current state of knowledge about the human perceptual, cognitive and procedural
performance with such systems. This matrix provided rough estimates of the state-of-knowledge and the
functional features of the system on a three level scale: indicating first that the state of knowledge was
adequate and those principles well integrated into the design, or indicating some gaps in the state of
knowledge and correspondingly possible gaps in the system design, and finally indicating either a lack of
knowledge or a lack of currently observable consideration in the design. The results of these analyses
will be provided in this panel discussion.
References
Brooks, P.E. (1997). Highway in the sky: Our legacy is safety on the road ahead. Flightline,
July/August, 16-22.
Figure 1. Examples of Synthetic Vision and Advanced Navigation Displays
Not subject to U.S. copyright restrictions.
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 48th ANNUAL MEETING—2004 79
... " This finding highlights the need for navigation displays that depict the airport layout, the location of the gates and runways, and the cleared taxi route. Such navigation displays have been developed (e.g., Hooey, Foyle & Andre, 2001; Theunissen, Rademaker, Jinkins & deHaag, 2002) and are under consideration for implementation by industry (Comstock et al., 2004). The wide-scale deployment of such displays should be considered a minimum requirement for the coordinated runway-crossing concept. ...
Thesis
Full-text available
The dissertation at hand identifies and analyzes how well glider pilots use low-cost collision alerting systems. While being generally recognized as a commendable tool for helping glider pilots see and avoid other traffic, these systems have been cited as possible contributing factors in several accidents. In literature, no in-depth research on how glider pilots may interpret or misinterpret their indications was found. At the beginning of this dissertation, a market study of human-machine interfaces for low-cost collision alerting systems is presented. During the study, different human-machine interfaces are taxonomized. The low-complexity and radar-style displays were found to be popular display formats. Also it was discovered that a perspective presentation of traffic has been evaluated for military applications, but not for a gliding context. Thus, a perspective presentation of traffic is proposed. The prototype of a perspective display format for gliding is developed by relying on a user-centered design process. Then, the design features of the low-complexity, radar-style and perspective displays are compared. This results in several hypotheses comparing the usability of the three display formats being postulated. In order to experimentally evaluate these hypotheses, 137 glider pilots partake in a laboratory experiment. They are presented with traffic information on one of the three display formats installed in a flight simulator. The participants then indicate where they suspect the traffic to be located in the outside world while being exposed to different flight conditions. Performance and subjective satisfaction measurements are recorded during the experiment. Inferential statistics are used to evaluate the experimental data. The perspective display format results in the most precise estimates of where traffic is located. Generally, errors in estimating traffic position increase as the participants’ ownship deviates from straight and level flight. Reaction time does not vary notably between display formats or different flight conditions. Subjective learnability and usability ratings favor the perspective display format over the two other formats analyzed. Overall, the perspective display format exhibits optimized usability in all dimensions when compared to the other two formats. During the usability analysis, circumstantial evidence arises which suggests that not all participants might interpret the data shown on their display similarly. A probable cause for this may be different knowledge deficits which are experienced between participants. These deficits may result in participants mentally modeling the collision alerting system incorrectly, thus leading to incorrect coordinate systems for interpreting the traffic information. A method for identifying these mental models is developed. The ensuing analysis reveals that most participants using low-complexity or radar-style display formats incorrectly interpret traffic information in an ownship-fixed fashion. Contrary, most participants working with the perspective display format perform at least some of the required rotations of their personal coordinate systems. The concept of different mental models based on different personal coordinate systems shows potential as an analysis tool for future display designs. From these findings multiple recommendations are deduced. They are directed at different stakeholders in the gliding community, including glider pilots, aircraft owners and operators, regulatory authorities, glider manufacturers, flight schools, competition rule makers and organizers, as well as designers of collision alerting systems and associated human-machine interfaces. Closing this dissertation, the potential for future human factors research in the gliding community is highlighted.
Highway in the sky: Our legacy is safety on the road ahead. Flightline
  • P E Brooks
Brooks, P.E. (1997). Highway in the sky: Our legacy is safety on the road ahead. Flightline, July/August, 16-22.