Conference PaperPDF Available

Monitoring of reliability in Bayesian identification

Authors:
  • Lübeck University of Applied Sciences

Abstract and Figures

Identification of tracked objects is a key capability of surveillance and information systems for air, surface (maritime), ground, and space environments. It improves situational awareness and offers decision support to operational users. Bayesian-style identification processes provide an identity as result. As input for taking further operational action, judgement of the remaining uncertainty is important. An operational user needs to know, how much he can rely upon the provided identification result. In this contribution, a typical Bayesian identification process is described and analyzed with respect to the support of an operational user in judging the reliability of an identification result. Measures proposed in literature do not completely fulfill the requirements of operational users. Existing approaches only deal with selected aspects of reliability. Therefore, relevant aspects of the overall reliability are considered in this paper. In addition, a set of appropriate measures is proposed, which corresponds to these aspects.
Content may be subject to copyright.
A preview of the PDF is not available
... The process of target identification (TI) that traffic operators carry out can be defined as the determination of the identity of an object. In the air domain, the work by Kruger and Kratzke [26] is an interesting example of the use of Bayesian Networks (BNs) for automatically performing the TI tasks. ...
... The identity of an object describes object features, such as allegiance and intent, which is crucial information for defense operators to be able to perform their tasks. The identity of an object can be assessed not only through decoding answers to identification-friend-foe (IFF) transmissions, but also through the extraction of object characteristics such as track behavior, identification by origin, platform performance, adherence to traffic regulations, the existence of electronic support measures, as well as positional and kinematical information [26,27]. Every identity is associated with a specific combination of object attributes. ...
... Every identity is associated with a specific combination of object attributes. For example, a fighter aircraft has a typical radar signature, may be able to fly at high speeds and can occasionally fulfill certain attack profiles [26]. Such object attributes induce emissions, which can be measured by different sensors. ...
Article
Full-text available
Monitoring dynamic objects in surveillance applications is normally a demanding activity for operators, not only because of the complexity and high dimensionality of the data but also because of other factors like time constraints and uncertainty. Timely detection of anomalous objects or situations that need further investigation may reduce operators’ cognitive load. Surveillance applications may include anomaly detection capabilities, but their use is not widespread, as they usually generate a high number of false alarms, they do not provide appropriate cognitive support for operators, and their outcomes can be difficult to comprehend and trust. Visual analytics can bridge the gap between computational and human approaches to detecting anomalous behavior in traffic data, making this process more transparent. As a step toward this goal of transparency, this article presents an evaluation that assesses whether visualizations of normal behavioral models of vessel traffic support two of the main analytical tasks specified during our field work in maritime control centers. The evaluation combines quantitative and qualitative usability assessments. The quantitative evaluation, which was carried out with a proof-of-concept prototype, reveals that participants who used the visualization of normal behavioral models outperformed the group that did not do so. The qualitative assessment shows that domain experts have a positive attitude toward the provision of automatic support and the visualization of normal behavioral models, as these aids may reduce reaction time and increase trust in and comprehensibility of the system.
... The process of target identification (TI) that traffic operators carry out can be defined as the determination of the identity of an object. In the air domain, the work by Kruger and Kratzke [26] is an interesting example of the use of Bayesian Networks (BNs) for automatically performing the TI tasks. ...
... The identity of an object describes object features, such as allegiance and intent, which is crucial information for defense operators to be able to perform their tasks. The identity of an object can be assessed not only through decoding answers to identification-friend-foe (IFF) transmissions, but also through the extraction of object characteristics such as track behavior, identification by origin, platform performance, adherence to traffic regulations, the existence of electronic support measures, as well as positional and kinematical information [26,27]. Every identity is associated with a specific combination of object attributes. ...
... Every identity is associated with a specific combination of object attributes. For example, a fighter aircraft has a typical radar signature, may be able to fly at high speeds and can occasionally fulfill certain attack profiles [26]. Such object attributes induce emissions, which can be measured by different sensors. ...
... The class of a target reveals target features such as allegiance, intent and possible capabilities. The class can be assessed through analyzing sensor data (such as analyzing identification-friend-foe (IFF) replies according to pre-defined database setups), kinematical data (such as g-force, speed and altitude which reveal target behavior and platform performance characteristics) and through investigating team-based information (such as if another team-member has classified the target) (see [5,9] for more information). ...
... Different target classes can fulfill certain combinations of attributes of technical and behavioral characteristics and capabilities. For example, a fighter aircraft has a typical radar signature, may be able to fly at high speeds and can fulfill certain attack profiles [9]. These attributes can be measured by different sensors, and through matching these attributes with a database collection of known attribute setups, a probable target class can be generated. ...
Conference Paper
Full-text available
This paper presents an empirical study where the effects of three levels of system transparency of an automated target classification aid on fighter pilots’ performance and initial trust in the system were evaluated. The three levels of transparency consisted of 1) only presenting text–based information regarding the specific object (without any automated support), 2) accompanying the text-based information with an automatically generated object class suggestion and 3) adding the incorporated sensor values with associated (uncertain) historic values in graphical form. The results show that the pilots needed more time to make a classification decision when being provided with display condition 2 and 3 than display condition 1. However, the number of correct classifications and the operators’ trust ratings were the highest when using display condition 3. No difference in the pilots’ decision confidence was found, yet slightly higher workload was reported when using display condition 3. The questionnaire results report on the pilots’ general opinion that an automatic classification aid would help them make better and more confident decisions faster, having trained with the system for a longer period.
... The prototype consists of a simulation engine, a database and a threat identification engine, based on a rule matching process and the use of a Bayesian Network (Kruger & Kratzke, 2009). To resemble the capabilities of their current systems, a simplified version of the prototype presented in has been used during the empirical investigations, i.e., the participants themselves carry out the identification and prioritization tasks, analyzing and interacting with the data available through the prototype. ...
Article
This paper investigates the impact that domain expertise has on risk assessment when analyzing uncertain geographical and sensor data. The differences between novice and expert air traffic operators were examined taking into account the performance of identifying and classifying threatening targets, the time needed to carry out such classifications and the confidence reported for each decision. The results show that confidence was significantly higher for the expert group. This was supported by the after-test questionnaire since none of the novice participants reported being more confident with the visualizations of uncertainty provided. No significant differences regarding time and performance were found between the groups, even if experts needed on average, more time to make a decision. Based on the collected logs, the experienced participants more often accessed the detailed information for each object presented by the tool tip. Both the time taken and the data accessed might indicate that experts had better situation awareness. Finally, the experts reported higher workload values related to performance.
Chapter
The Identification and classification of targets is one of the key capabilities of Ground based C4ISR systems, military Command and Control Systems and Combat Management Systems. It is a precondition for situational awareness and supports operational users in decision making. A correct identification is an important prerequisite to prevent fratricide and civilian collateral damages and to complete the Situational Awareness. Modern Combat Management and Surveillance systems deal with thousands of tracked objects and such an operator is unable to handle the huge amount of targets and data in an operationally acceptable timeline. Therefore an automated identification and classification process is integrated in such military systems. Typical sensors used for this task are radars, IFF and ESM sensors complemented by sources like Tactical Data Links, civil and military Airspace Control Means and flight plans.
Article
Identification in an air surveillance system is applied to a large number of tracks and can be implemented by means of Bayesian classification. Performed in an automated system, there is a strong need to monitor the identification process and contributing sensors by the operator. This paper describes two sensor failure-tests based on frequencies' evaluation of conflicting evidence by statistical means. In a simulated air surveillance scenario, a Conflict Binomial and a Conflict Ratio Failure-Test are evaluated and compared, with particular respect to operationally relevant performance measures.
Conference Paper
Full-text available
Threat evaluation (TE) is concerned with determining the intent, capability and opportunity of detected targets. To their aid, military operators use support systems that analyse incoming data and make inferences based on the active evaluation framework. Several interface and interaction guidelines have been proposed for the implementation of TE systems; however there is a lack of research regarding how to make these systems transparent to their operators. This paper presents the results from interviews conducted with TE operators focusing on the need for and possibilities of improving the transparency of TE systems through the visualization of uncertainty and the presentation of the system rationale.
Book
Ecological Interface Design delivers the techniques and examples that provide you with a foundation to succeed in designing advanced display graphics. The opening chapters introduce the “art” of interface design by exposing the analytical methods behind designs, the most common graphical forms, and how these methods and forms are pulled together to create a complete design. The book then incorporates case studies that further emphasize techniques and results. Each example exemplifies a solution to a certain part of the EID puzzle. Some of the examples demonstrate the analysis phase, while others apply more scrutiny to graphical design. Each is unique, allowing allowing you to use them in the development of your own designs. The volume concludes with an analysis that connects ecological interface design with other common interface design methods, enabling you to better understand how to combine approaches in the creation of design solutions.
Article
For a decade, NATO nations, both on their own and within NATO groups have performed studies in the field of Twodimensional Inverse Synthetic Aperture Radar (2D-ISAR) and High Resolution Radar (HRR) Range Profiles. During that work high-quality radar imagery data of aircraft has become available through measurement campaigns. It was shown that the acquisition of a sufficiently large and representative reference database is a difficult and laborious process. Other database properties, such as its contents, size, upgradability, retrieval speed, operational use, etcetera have not received sufficient attention within these studies. In the future, NATO may be playing a major role in maintaining a NATO central database of radar signatures. This paper provides an initial understanding of the operational implications for performing such a task. It argues that, instead of providing a central database to NATO nations, NATO should stimulate the use of decentralized classifiers and databases and ensure that coalition partners can access them via a data network in operational situations.
Article
The process and the prototype presented here are dedicated to improve the overall identification capability. This is aimed to be achieved by making available all identification related information; i.e. local and remote data; fusing and interpreting them and supporting the decision process by offering a recommendation together with all explanation that might be desired. The paper presents a solution that uses the Identification Data Combining Process (IDCP) according to draft STANAG 4162 as baseline. The prototype assisting in identifying airborne objects is the result of an experimental system using simulated as well as live data in a German Control and Reporting Centre (CRC).
Conference Paper
Multisensor data fusion is a key enabling technology in which information from a number of sources is integrated to form a unified picture [1]. This concept has been applied to numerous fields and new applications are being explored constantly. Even though most multisensor data fusion applications have been developed relatively recently, the notion of data fusion has always been around. In fact, all of us employ multisensor data fusion principles in our daily lives. The human brain is an excellent example of an operational fusion system that performs extremely well. It integrates sensory information, namely sight, sound, smell, taste and touch data and makes inferences regarding the problem at hand. It has been a natural desire of researchers in different disciplines of science and engineering to emulate this information fusion ability of the human brain. The idea is that fusion of complementary information available from different sensors will yield more accurate results for information processing problems. Significant advances in this important field have been made but perfect emulation of the human brain remains an elusive goal.