No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Cost-Benefit Quantification of ISHM in Aerospace Systems
Abstract
Integrated Systems Health Management (ISHM) is an evolving technology used to detect, assess, and isolate faults in complex aerospace systems to improve safety. At the conceptual design level, system-level engineers must make decisions regarding the inclusion of ISHM and the extent and type of the sensing technologies used in various subsystems. In this paper, we propose a Cost-Benefit Analysis approach to initiate the ISHM design process. The key to this analysis is the formulation of an objective function that explicitly quantifies the cost-benefit factors involved with using ISHM technology in various subsystems. Ultimately, to determine the best ISHM system configuration, an objective is formulated, referred to as Profit, which is expressed as the product of system Availability (A) and Revenue per unit Availability (R), minus the sum of Cost of Detection (CD ) and Cost of Risk (CR ). Cost of Detection includes the cost of periodic inspection/maintenance and the cost of ISHM; Cost of Risk quantifies risk in financial terms as a function of the consequential cost of a fault and the probabilities of occurrence and detection. Increasing the ISHM footprint will generally lower Cost of Risk while raising Cost of Detection, while Availability will increase or decrease based upon the balance of the reliability and detectability of the sensors added, versus their ability to reduce total maintenance time. The analysis is conducted at the system functional level, with ISHM allocated to functional blocks in the optimization analysis. The proposed method is demonstrated using a simplified aerospace system design problem resulting in a configuration of sensors which optimizes the cost-benefit of the ISHM system for the given input parameters. In this problem, profit was increased by 11%, inspection interval increased by a factor of 1.5, and cost of risk reduced by a factor of 2.4 over a system with no ISHM.
... Collectively, these articles have covered a range of aspects of IVHM, with approximately 35 per cent describing the potential impacts or cost-benefit analyses (e.g. references [10], [15], and [29] to [31]), 15 per cent discussing design approaches (e.g. references [26] and [32] to [34]), and ∼25 per cent focusing on examples of either fielded or under development IVHM systems (e.g. ...
... These introductory discussions are typical to emphasize the benefits of IVHM, although a more in-depth assessment of the potential advantages is available from those authors who substantially focused on cost-benefit analysis (e.g. Byer et al. [3], Banks et al. [27], and Hoyle et al. [31]). For mission operations, adoption of IVHM can provide with adaptive control and improved survivability. ...
... The majority of authors (e.g. Williams [1], Banks et al. [27], and Hoyle et al. [31]) see the main barrier to the adoption of IVHM as the cost of the hardware and software that is needed to perform the IVHM tasks. Reichard et al. [7] indicate that this cost includes the development, qualification, and implementation of the sensors and data processing software, and also the penalty costs associated with additional weight, power, computing, and communication resources. ...
Integrated vehicle health management (IVHM) is a collection of data relevant to the present and future performance of a vehicle system and its transformation into information can be used to support operational decisions. This design and operation concept embraces an integration of sensors, communication technologies, and artificial intelligence to provide vehicle-wide abilities to diagnose problems and recommend solutions. This article aims to report the state-of-the-art of IVHM research by presenting a systematic review of the literature. The literature from different sources is collated and analysed, and the major emerging themes are presented. On this basis, the article describes the IVHM concept and its evolution, discusses configurations and existing applications along with main drivers, potential benefits and barriers to adoption, summarizes design guidelines and available methods, and identifies future research challenges.
... [9], [10]. It also provides excellent narratives of why projects chose not to use MBD after considering it [11]. ...
... The literature does provide existing cost models for related endeavors such as integrated vehicle health management [8], [9], [10]. It also provides excellent narratives of why projects chose not to use MBD after considering it [11]. ...
... A number of analytic approaches exist for examining the cost/benefit tradeoffs of Integrated Vehicle Health Management (IVHM) or Integrated System Health Management (ISHM) technologies [8], [9], [10]. IVHM is broader and somewhat orthogonal to model-based diagnosis and recovery as we have considered it, in that IVHM is typically concerned with all aspects of supporting operations of one or a fleet of systems. ...
Over the past 20 years, there has been much work in the area of model-based diagnosis (MBD). By this we mean diagnosis systems arising from computer science or artificial intelligence approaches where a generic software engine is developed to address a large class of diagnosis problems. Later, models are created to apply the engine to a specific problem. These techniques are very attractive, suggesting a vision of machines that repair themselves, reduced costs for all kinds of endeavors, spacecraft that continue their missions even when failing, and so on. This promise inspired a broad range of activity, including our involvement over several years in flying the Livingstone and Livingstone 2 on-board model-based diagnosis and recovery systems as experiments on two spacecraft. While a great deal was learned through a variety of applications to simulators, testbeds and flight experiments, no project adopted the technology in operations and the expected benefits have not yet come to fruition. This led us to ask what are the costs of using MBD for the operational scenarios we encountered, what are the benefits, and how do we approach the question of whether the benefits outweigh the costs? How are missions today approaching fault diagnosis and recovery during operations? If we characterize the cost and benefits of using MBD, how would it compare with traditional ways of making a system more robust? How did expectations for MBD compare to benefits seen in the field and why? The literature does provide existing cost models for related endeavors such as integrated vehicle health management. It also provides excellent narratives of why projects chose not to use MBD after considering it. However, we believe that this paper is the first to unpack and discuss the cost, benefit and risk factors that impact the net value of model-based diagnosis and recovery. We use experience with systems such as Livingstone as an example, so our focus is on-board model-based diagnosis and reco-
very, but we believe many of the insights and remaining questions on the costs and benefits are applicable to other diagnosis applications. Quantitative model of when on-board model-based diagnosis would be an effective choice, it lays out the cost/benefit proposition and identifies several disconnects that we believe prevent adoption as an operational tool. While we do not suggest metrics for every cost, benefit and risk factor we identify, we do discuss where each factor arises in development or operations and how model-based diagnosis and recovery tends to leverage or exacerbate each. As such we believe the analysis is of use to those developing MBD or related techniques and those who may employ them. It also serves as one example of how honest expectations based on technical capability can come to differ from the net impact on customer problems. In this paper we present a cost/benefit analysis for MBD, using expectations and experiences with Livingstone as an example. We provide an overview of common techniques for making spacecraft robust, citing fault protection schemes from recent missions. We lay out the cost, benefit and risk advantages associated with on-board MBD, and use the examples to probe each expected advantage in turn. We conclude our analysis with a summary of our method for analyzing the costs and benefits in a particular domain, and encourage others to come forward with analyses of costs and benefits for fielded systems. Finally, we discuss related work both in terms of similar analyses and fielded systems.
... Other methodologies exist for rationally allocating finite resources to subsystems [58,59,60,61]. Often these methods formulated the design trade-off problem as a multi-objective optimization problem. ...
... The main objective functions were revenue expressed as the product of revenue per unit availability and system availability, cost of risk, and cost of failure detection. These methods attempted to optimize the allocation of resources for health management and diagnostics in aerospace systems [59]. ...
Complex systems engineering projects are increasingly prevalent in our world. Technical requirements for complex systems usually break out individual subsystems parameters. For instance, each subsys- tem on a spacecraft can be assigned target mass, volume, power consumption, and other technical requirements. These tangible variables are often traded between subsystems engineers to maximize subsystem design utility. This in turn helps to maximize overall system utility. Design margins are also often assigned to these design variables during early stages of the design process. Risk, reliability, robustness, and uncertainty have until this point not been part of subsystems param- eter trading and design margins. This research aims to formalize a method of trading and margining these design variables among subsystems with the end eect of maximizing system utility and system integrity. Further, this research will investigate how dierent stakeholders in the complex system de- sign process value and perceive risk, reliability, robustness, and uncertainty. This research will also be extended to examine the role culture plays in the valuation of these variables. This paper presents a literature synthesis on the topics of the methods and tools of design Trade Studies, and the research and practice of risk and uncertainty in collaborative design and model based engineering. Initial thoughts are presented on how to incorporate risk and uncertainty into Trade Studies in collaborative design environments. A summary of future areas of research is included. Expected contributions of the overall research and a rough plan are outlined.
... • maximized usage of the component life while ensuring mission safety One example of cost-benefit quantification of ISHM in aerospace systems appears in (Hoyle et al., 2007). Their methodology analyzes the trade-offs between system availability, cost of detection, and cost of risk. ...
One of the most prominent technical challenges to effective deployment of health management systems is the vast difference in user objectives with respect to engineering development. In this paper, a detailed survey on the objectives of different users of health management systems is presented. These user objectives are then mapped to the metrics typically encountered in the development and testing of two main systems health management functions: diagnosis and prognosis. Using this mapping, the gaps between user goals and the metrics associated with diagnostics and prognostics are identified and presented with a collection of lessons learned from previous studies that include both industrial and military aerospace applications.
... These issues are fundamental for efficient maintenance management, in particular, in a maintenance (CBM) scenario. A clear and efficient methodology to evaluate the harshness of an event (that is able to give answers to the three problems introduced above on the base of a real flight envelope) may generate advantages for the final user in terms of maintenance accuracy, safety, and idle time [4]. Furthermore, also in the design phase (e.g., [5]) great advantages may occur because of the increased knowledge of the helicopter behavior under contingent loads. ...
A landing that exceeds design conditions is usually referred to as a hard or harsh landing. This condition is delimited between the usual operational conditions and the crash event. Helicopter structural damage due to harsh landings is generally not as severe as during a crash, but it may lead to unscheduled maintenance events, involving costs and idle time. The aim of the current paper is to define a methodology, which estimates (by means of hybrid, multi-environment analyses) the damage state of a part of a helicopter fuselage after a harsh landing event. In particular, a mixed multibody finite element modeling strategy is used to achieve computationally efficient results, which are obtained for various landing conditions, in terms of drop heights and the helicopter attitude at landing. The obtained model may be useful both in a design scenario and in exploring structural health monitoring applications. With the latter in mind, output data were deeply investigated with a dedicated focus on the correlation between the damage position and landing conditions. Stringers in a particular zone of the fuselage were identified as the most affected zone (by localized plasticization) during harsh landings. This result provides a possible basis for using focused structural health monitoring strategies. With this in mind, a zone selection criterion, as a tradeoff between sensorization extension and performance, was proposed.
... Shortly this case confirmed the CBM effect on increase in cost effectiveness, availability and safety practically. Hoyle et al. (2007) analyze cost benefit of Integrated Systems Health Management (ISHM) in Aerospace Systems. As Condition Based Maintenance Policy, ISHM detects, assesses and isolate faults and so improves safety and reliability. ...
... (1) Optimize planning, scheduling and decision making for maintenance 9-13 -For maintenance scheduling by operators of the PHM enabled system -For a contract based service provider that relies on PHM to guarantee uptime (2) Generate a set of alternative solutions given user's flexibility in relaxing various constraints 10,[14][15][16] -Sensitivity analysis to figure out the most critical components -Break even curves for various input parameter ranges -Define scope for service contracts by assessing which components are most profitable for PHM (3) PHM Designfor integrating into a legacy system or incorporating into the new system design [16][17][18][19][20][21][22] -Sensor selection and placement -Determine detection thresholds (e.g. on a RoC curve) for cost effective PHM -Down select and prioritize list of faults/subsystems/components for PHM capability (4) Assess effectiveness of PHM to reduce costs and improve reliability 14,16,17,[22][23][24][25][26][27][28][29][30][31][32] -Evaluate the economic promise of PHM compared to the cost (value) of the system itself -Assess safety and reliability benefits of PHM 33 -Assess savings in the overall Life Cycle Costs for an asset 10, 16, 30, 34, 35 (5) Compare various PHM approaches 24-26, 32, 34, 36 -Compare based on ROI in a given period of performance -Compare payback periods for various alternatives A variety of CBAs conducted in the above situations also differ in their technical approach. One of the most popular approaches has been to optimize a cost function while honoring the constraints on requirements and resources to arrive at a beneficial maintenance policy 9, 11-13, 21, 22 . ...
... Other researchers have also proposed similar cost-benefit formulations for diagnostic systems (Williams, 2006;Kurien & Moreno, 2008;Hoyle, Mehr, Tumer, & Chen, 2007). These approaches, however, are primarily concerned with higher-level trade-offs in integrating diagnostic solutions to provide health management functionality and focus on performance indices such as operational cost, and maintainability. ...
A variety of rule-based, model-based and data-driven techniques have been proposed for detec-tion and isolation of faults in physical systems. However, there have been few efforts to compara-tively analyze the performance of these approaches on the same system under identical conditions. One reason for this was the lack of a standard framework to perform this comparison. In this pa-per we introduce a framework, called DXF, that provides a common language to represent the sys-tem description, sensor data and the fault diag-nosis results; a run-time architecture to execute the diagnosis algorithms under identical condi-tions and collect the diagnosis results; and an eval-uation component that can compute performance metrics from the diagnosis results to compare the algorithms. We have used DXF to perform an em-pirical evaluation of 13 diagnostic algorithms on a hardware testbed (ADAPT) at NASA Ames Re-search Center and on a set of synthetic circuits typically used as benchmarks in the model-based diagnosis community. Based on these empirical data we analyze the performance of each algorithm and suggest directions for future development. 1 INTRODUCTION Fault Diagnosis in physical systems involves the detection of anomalous system behavior and the identification of its cause. Some key steps in diag-nostic inference are fault detection (is the output of the system incorrect?), fault isolation (what is
... (1) Optimize planning, scheduling and decision making for maintenance 9-13 -For maintenance scheduling by operators of the PHM enabled system -For a contract based service provider that relies on PHM to guarantee uptime (2) Generate a set of alternative solutions given user's flexibility in relaxing various constraints 10,[14][15][16] -Sensitivity analysis to figure out the most critical components -Break even curves for various input parameter ranges -Define scope for service contracts by assessing which components are most profitable for PHM (3) PHM Designfor integrating into a legacy system or incorporating into the new system design [16][17][18][19][20][21][22] -Sensor selection and placement -Determine detection thresholds (e.g. on a RoC curve) for cost effective PHM -Down select and prioritize list of faults/subsystems/components for PHM capability (4) Assess effectiveness of PHM to reduce costs and improve reliability 14,16,17,[22][23][24][25][26][27][28][29][30][31][32] -Evaluate the economic promise of PHM compared to the cost (value) of the system itself -Assess safety and reliability benefits of PHM 33 -Assess savings in the overall Life Cycle Costs for an asset 10, 16, 30, 34, 35 (5) Compare various PHM approaches 24-26, 32, 34, 36 -Compare based on ROI in a given period of performance -Compare payback periods for various alternatives A variety of CBAs conducted in the above situations also differ in their technical approach. One of the most popular approaches has been to optimize a cost function while honoring the constraints on requirements and resources to arrive at a beneficial maintenance policy 9, 11-13, 21, 22 . ...
With recent advancements in prognostics methodologies there has been a significant interest in maturing Prognostics and Health Management (PHM) to increase its technology readiness level for onboard deployments. Active research is underway both in industry and academia to address shortcomings in availability of run-to-failure data, accelerated aging environments, real-time prognostics algorithms, uncertainty representation and management (URM) techniques, prognostics performance evaluation, etc., to name a few. At this juncture it is highly desirable to close the loop by connecting the high level customer requirements for mission planning and execution to performance specifications for prognostics methodologies at the lower technical level. This calls for integrating the pragmatics of safety, reliability, cost, and real-time viability into the prognostics methodologies to establish a connection between top-down and bottom-up approaches currently pursued in the PHM community. In this paper we identify key areas that must be addressed to bridge these gaps and provide an overview of how these areas have been addressed in part at various levels. We also discuss on how these issues can be further developed into a comprehensive and more coherent portfolio of technologies that will ultimately lead to specifying guidelines for prognostics performance. Copyright © 2010 by the American Institute of Aeronautics and Astronautics, Inc.
... Prior work has addressed the need for integrating early risk assessment and management tools and methodologies into the vehicle and system design [34][35][36][37]. Most notably, the Function Failure Design Method (FFDM), promotes early identification of potential failures by linking them to product functions [38,39]. ...
Ensuring the reliability of complex software intensive systems is becoming a critical requirement for all military and commercial aerospace applications, and becomes especially more challenging when implemented for autonomous and evolving deployments required of such applications. To ensure reliability, this research asserts that knowledge, data, and models of such complex systems must be integrated with their intended systems starting from the early design stages, hence enabling designers and engineers to plan for contingencies, redundancies, and potential changes early, before costly design decisions have been made. In this paper, a general system-level design methodology is introduced to perform simulation-based failure identification and propagation analysis of software- hardware systems. In particular, the Functional Failure Identification and Propagation (FFIP) analysis framework is introduced as a novel approach for designing reliable software-intensive systems. A combination of function, structure, and behaviour modelling is proposed to simulate failure propagation paths and the resulting functional failures to determine mitigation options, integrating hierarchical system models with behavioural simulation and qualitative reasoning. The overall goal of this research is to develop a formal framework and simulation-based design tool for design and system engineering teams to evaluate and assess the potential of functional failures of software intensive systems throughout the lifecycle.
... Chase et al. propose a utility-based method based on science return for evaluating inclusion of new technologies into Mars rover missions [32]. Similar analyses have been applied to Integrated Vehicle Health Management (IVHM) technologies [28], [29], [30]. IVHM is broader and somewhat orthogonal to model-based diagnosis and recovery as we have framed it, in that IVHM is typically concerned with all aspects of supporting operations of one or a fleet of systems. ...
Experience developing and deploying model-based diagnosis (MBD) and recovery and other model-based technologies on a variety of testbeds and flight experiments led us to explore why our expectations about the impact of MBD on spacecraft operations have not been matched by effective benefits in the field. By MBD, we mean the problem of observing a mechanical, software, or other system and determining what failures its internal components have suffered using a generic inference algorithm and a model of the system's components and interconnections. These techniques are very attractive, suggesting a vision of machines that repair themselves, reduced costs for all kinds of endeavors, spacecraft that continue their missions even when failing, and so on. This promise inspired a broad range of activities, including our involvement over several years in flying the Livingstone and L2 onboard MBD and recovery systems as experiments on Deep Space 1 and Earth Observer 1 spacecraft. Yet, in the end, no spacecraft project adopted the technology in operations nor flew additional flight experiments. To our knowledge, no spacecraft project has adopted any other MBD technology in operations. In this paper, we present a cost/benefit analysis for MBD using expectations and experiences with Livingstone as an example. We provide an overview of common techniques for making spacecraft robust, citing fault protection schemes from recent missions. We lay out the cost, benefit, and risk advantages associated with onboard MBD and use the examples to probe each expected advantage in turn. We suggest a method for evaluating a mission that has already been flown and providing a rough estimate of the maximum value that a perfect onboard diagnosis and recovery system would have provided. By unpacking the events that must occur in order to provide value, we also identify the factors needed to compute the expected value that would be provided by a real diagnosis and recovery system. We then di-
-
scuss the expected value we would estimate that such a system would have had for the Mars Exploration Rover mission. This has allowed us to identify the specific assumptions that made our expectations for MBD in this domain incorrect.
Maintenance Planning plays a vital role in optimizing the benefits of Integrated Vehicle Health Management (IVHM). The challenge is to identify the right combinations of different types (Preventive, CBM and Run-to-Fail) of maintenance tasks for different subsystems or components of complex systems like an aircraft to achieve the most optimized solution in terms of availability, cost and safety. Maintenance Strategy plans most cost effective maintenance type for each fault of a sub-system in such a way that availability and safety are optimized. Also, the strategy should satisfy the important goals viz. technical feasibility and certifiability of the solution. This study presents a RCM based maintenance strategy framework with some modifications over the existing guidelines. The framework has been implemented and is demonstrated with a case study of EPGDS (Electrical Power Generation and Distribution System). The results with arbitrary costing for each task are outlined with the objective of demonstrating the effectiveness of the framework.
Prognostics and Health Management (PHM) principles have considerable promise to change the game of lifecycle cost of engineering systems at high safety levels by providing a reliable estimate of future system states. This estimate is a key for planning and decision making in an operational setting. While technology solutions have made considerable advances, the tie-in into the systems engineering process is lagging behind, which delays fielding of PHM-enabled systems. The derivation of specifications from high level requirements for algorithm performance to ensure quality predictions is not well developed. From an engineering perspective some key parameters driving the requirements for prognostics performance include: (1) maximum allowable Probability of Failure (PoF) of the prognostic system to bound the risk of losing an asset, (2) tolerable limits on proactive maintenance to minimize missed opportunity of asset usage, (3) lead time to specify the amount of advanced warning needed for actionable decisions, and (4) required confidence to specify when prognosis is sufficiently good to be used. This paper takes a systems engineering view towards the requirements specification process and presents a method for the flowdown process. A case study based on an electric Unmanned Aerial Vehicle (e-UAV) scenario demonstrates how top level requirements for performance, cost, and safety flow down to the health management level and specify quantitative requirements for prognostic algorithm performance.
Risk is becoming an important factor in facilitating the resource allocation in engineering design because of its essential role in evaluating functional reliability and mitigating system failures. In this work, we aim at expanding existing quantitative risk modeling methods to collaborative system designs regarding resource allocation in a distributed environment, where an overlapped risk item can affect multiple stakeholders, and correspondingly be examined by multiple evaluators simultaneously. Because of different perspectives and limited local information, various evaluators (responsible for same or different components of a system), though adopting the same risk definition and mathematical calculation, can still yield unsatisfying global results, such as inconsistent probability and/or confusing consequence evaluations, which can then cause potential barriers in achieving agreement or acceptable discrepancies among different evaluators involved in the collaborative system design. Built upon our existing work, a Risk-based Distributed Resource Allocation Methodology (R-DRAM) is developed to help system manager allocate limited resource to stakeholders, and further to components of the targeted system for the maximum global risk reduction. Besides probability and consequence, two additional risk properties, tolerance and hierarchy, are considered for comprehensive systematic risk design. Tolerance is introduced to indicate the effective risk reduction, and hierarchy is utilized to model the comprehensive risk hierarchy. Finally a theoretical framework based on cost-benefit measure is developed for resource allocation. A case study is demonstrated to show the implementation process. The preliminary investigation shows promise of the R-DRAM in facilitating risk-based resource allocation for collaborative system design using a systematic and quantifiable approach in distributed environment.
In this paper, the functional-failure identification and propagation (FFIP) framework is introduced as a novel approach for evaluating and assessing functional-failure risk of physical systems during conceptual design. The task of FFIP is to estimate potential faults and their propagation paths under critical event scenarios. The framework is based on combining hierarchical system models of functionality and configuration, with behavioral simulation and qualitative reasoning. The main advantage of the method is that it allows the analysis of functional failures and fault propagation at a highly abstract system concept level before any potentially high-cost design commitments are made. As a result, it provides the designers and system engineers with a means of designing out functional failures where possible and designing in the capability to detect and mitigate failures early on in the design process. Application of the presented method to a fluidic system example demonstrates these capabilities.
ResearchGate has not been able to resolve any references for this publication.