ArticlePDF Available

Abstract and Figures

Complex conceptual system design trade studies traditionally consider risk after a conceptual design has been created. Further, one person is often tasked with collecting risk information and managing it from each subsystem. This paper proposes a method to explicitly consider and trade risk on the same level as other important system-level variables during the creation of conceptual designs in trade studies. The proposed risk trading method advocates putting each subsystem engineer in control of risk for each subsystem. A risk vector is proposed that organizes many different risk metrics for communication between subsystems. A method of coupling risk models to dynamic subsystem models is presented. Several risk visualization techniques are discussed. A trade study example is presented based upon a simplified spacecraft model. Results from introducing the risk trading methodology into a simulated Collaborative Design Center are presented. The risk trading method offers an approach to more thoroughly consider risk during the creation of conceptual designs in trade studies.
Content may be subject to copyright.
A preview of the PDF is not available
... Of particular interest to this research is the early phase of systems engineering that is encompassed by the system architecting process [2,4,5]. System architecting includes developing customer needs statements, design reference missions, system requirements, functional system models, tradeoff studies, and a variety of other work products [2,5,6]. This information is generally stored in a database that can be interpreted by a system architecture framework [7] such as the Department of Defense Architecture Framework (DoDAF) [8] which was specifically modified in the V2.0 release to heavily encourage the use of a database [9]. ...
... Both the systems engineering process and the mechanical design process advocate for developing system or product requirements [5,106] at the start of their respective development cycles. Thus, this 6 Copyright © 2019 ASME method assumes that adequate requirements have already been developed for the system. Table 1 shows a generic set of requirements for a UAV. ...
... There are two potential approaches that practitioners may use at this step including: (1) identifying the highest probability of occurrence failure flow system emissions and rank ordering them according to their probabilities, and (2) examining the other systems within the SoS that may be impacted by the failure flow system emissions. The first approach is the standard approach often taken in FMEA and other analyses [6]. We advocate that the second approach be taken and provide further details below. ...
Conference Paper
Full-text available
Increasingly tight coupling and heavy connectedness in systems of systems (SoS) presents new problems for systems designers and engineers. While the failure of one system within a SoS may produce little collateral damage beyond a loss in SoS capability, a highly interconnected SoS can experience significant damage when one member system fails in an unanticipated way. It is therefore important to develop systems that are “good neighbors” with the other systems in a SoS by failing in ways that do not further degrade a SoS’s ability to complete its mission. This paper presents a method to (1) analyze a system for potential spurious emissions and (2) choose mitigation strategies that provide the best return on investment for the SoS. The method is suited for use during the system architecture phase of the system design process. A functional and flow approach to analyzing spurious emissions and developing mitigation strategies is used in the method. Use of the method may result in a system that causes less SoS damage during a failure event.
... While not studied in the context of prototyping, recent research has emerged to assess risk in engineering domain specific tasks and to investigate the relationship between risk and creativity during concept selection in engineering design. Specifically, to explore risk taking in an engineering contest the Engineering Domain Specific Risk Taking (E-DOSPERT) scale was developed by Van Bossuyt et al. [40] and found that risk taking attitudes are domain specific and can be trained. Barclift et al. [41] used this scale to investigate risk taking in an additive manufacturing design task and found that the risk aversion sub-scale could predict the novelty of ideas but not the quality of the ideas. ...
... These results do not support our hypothesis that prototype fidelity would impact the likelihood to move forward with a concept. While previous work has shown that higher fidelity prototypes are more likely to garner support and buy-in from external stakeholders [40], our results indicate that prototype fidelity does not a play a significant role in the decision-making behaviors of internal design team members. ...
... Through this future work, different risk taking scales should be investigated for their potential to identify risk taking attitudes in the creative concept selection stages of the design process. While financial risk taking [55] and the engineering domain specific risk taking [40] scales exist, they have not been thoroughly investigated for their ability to explain variations in the fuzzy front end of the design process and should be investigated, along with the preferences for creativity scale (PCS) [56], for their influence on this portion of the design process. ...
Article
Building prototypes is an important part of the concept selection phase of the design process, where fuzzy ideas get represented to support communication and decision making. However, the previous studies have shown that prototypes generate different levels of user feedback based on their fidelity and esthetics. Furthermore, prior research on concept selection has shown that individual risk attitude effects how individuals select ideas, as creative ideas are perceived to be riskier in comparison to less creative ideas. While the role of risk has been investigated in concept selection, there is lack of research on how risk is related to the selection of prototypes at various levels of fidelity. Thus, the purpose of this study was to investigate the impact of prototype fidelity, concept creativity, and risk aversion on perceived riskiness and concept selection through a between-subjects study with 72 engineering students. The results revealed that there was a "goldilocks" effect in which students choose concepts with "just the right amount" of novelty, not too much and not too little, as long as quality was adequate. In addition, the prototype fidelity of a concept had an interaction with uniqueness, indicating that unique concepts are more likely to be perceived as less risky if presented at higher levels of fidelity.
... While not studied in the context of prototyping, recent research has emerged to assess risk in engineering domain specific tasks and to investigate the relationship between risk and creativity during concept selection in engineering design. Specifically, to explore risk taking in an engineering contest, the engineering domain specific risk taking (E-DOSPERT) scale was developed by Van Bossuyt et al. [40] and found that risk taking attitudes are domain specific and can be trained. Barclift et al. [41] used this scale to investigate risk taking in an additive manufacturing design task and found that the risk aversion sub-scale could predict the novelty of ideas but not the quality of the ideas. ...
... These results do not support our hypothesis that prototype fidelity would impact the likelihood to move forward with a concept. While the previous work has shown that higher fidelity prototypes are more likely to garner support and buy-in from external stakeholders [40], our results indicate that prototype fidelity does not a play a significant role in the decision-making behaviors of internal design team members. ...
... Through this future work, different risk taking scales should be investigated for their potential to identify risk taking attitudes in the creative concept selection stages of the design process. While financial risk taking [55] and the engineering domain specific risk taking [40] scales exist, they have not been thoroughly investigated for their ability to explain variations in the fuzzy front end of the design process and should be investigated, along with the PCS [56], for their influence on this portion of the design process. ...
Conference Paper
Building prototypes is an important part of the concept selection phase of the design process, where fuzzy ideas get represented to support communication and decision making. However, previous studies have shown that prototypes generate different levels of user feedback based on their fidelity and aesthetics. Furthermore, prior research on concept selection has shown that individual risk attitude effects how individuals select ideas, as creative ideas are perceived to be riskier in comparison to less creative ideas. While the role of risk has been investigated in concept selection, there is lack of research on how risk is related to the selection of prototypes at various levels of fidelity. Thus, the purpose of this study was to investigate the impact of prototype fidelity, concept creativity, and risk aversion, on perceived riskiness and concept selection through a between-subjects study with 72 engineering students. The results revealed that there was a “goldilocks” effect in which students choose concepts with “just the right amount” of novelty, not too much and not too little, as long as quality was adequate. In addition, the prototype fidelity of a concept had an interaction with uniqueness, indicating that unique concepts are more likely to be perceived as less risky if presented at higher levels of fidelity.
... Risk attitude data from psychometric survey techniques has been found to be aspirational in nature while choice lotteries are generally predictive [32,36]. In this research, we take the perspective of aspirational risk attitude measures (i.e., psychometric risk surveys) in line with existing research on applying risk attitudes to engineering analyses and trade-off studies [32,37]. ...
Article
Full-text available
Systems engineering practices in the maritime industry and the Navy consider operational availability as a system attribute determined by system components and a maintenance concept. A better understanding of the risk attitudes of system operators and maintainers may be useful in understanding potential impacts the system operators and maintainers have on operational availability. This article contributes to the literature a method that synthesizes the concepts of system reliability, and operator and maintainer risk attitudes to provide insight into the effect that risk attitudes of systems operators and maintainers have on system operational availability. The method consists of four steps providing the engineer with a risk-attitude-adjusted insight into the system's potential operational availability. Systems engineers may use the method to iterate a system's design or maintenance concept to improve expected operational availability. If it is deemed necessary to redesign a system, systems engineers will likely choose new system components and/or alter their configuration; however, redesign is not limited to physical alteration of the system. Several other options may be more practical depending the system's stage in the life cycle to address low risk-adjusted operational availability such as changes to maintenance programs and system supportability rather than on component and system reliability. A simple representative example implementation is provided to demonstrate the method and discussion of the potential implications for Navy ship availability are discussed. Potential future work is also discussed.
... al. [43], and Van Bossuyt et. al. [44] provide several related methods for weighting significant outcomes using utility theory and other techniques. This is especially important in situations where the entire cost of a successful cyber attack may not be fully calculable. ...
Conference Paper
Full-text available
As complex systems such as nuclear power plants, naval ships, critical infrastructure, and other systems become more connected to the internet and digital control interfaces, the chance of a cyber attack causing physical damage to a system and failure of the system increases. In many systems, recovery actions can prevent an incipient failure from causing a system-wide failure. This paper presents a method of determining if a human operator or an automated system is more appropriate to complete a recovery action during a cyber attack. The method is useful during the conceptual phase of system design where architecture changes have minimal impact on the cost and schedule of the system design effort. Practitioners can use the method to make cost and probability-informed decisions. A case study of a spent fuel cooling pool in a nuclear power plant is presented to illustrate the method.
... With increasing system complexity, design methods used for relatively simple product design are replaced by design methodologies specifically suited for complex systems [1,2]. Functional modeling is often used in the early conceptual phase of system design (generally referred to as system architecture although this definition is not universally accepted) [3]. ...
Article
Full-text available
This paper presents a method of assessing cable routing for systems with significant cabling to help system engineers make risk-informed decisions on cable routing and cable bundle management. We present the Cable Routing Failure Analysis (CRFA) method of cable routing planning that integrates with system architecture tools such as functional modeling and function failure analysis. CRFA is intended to be used during the early conceptual stage of system design although it may also be useful for retrofits or overhauls of existing systems. While cable raceway fires, cable bundle severing events, and other common cause cable failures (e.g., rodent damage, chemical damage, fraying and wear-related damage, etc.) are known to be a serious issue in many systems, the protection of critical cabling infrastructure and separation of redundant cables is often not taken into account until late in the systems engineering process. Cable routing and management often happens after significant system architectural decisions have been made. If a problem is uncovered with cable routing, it can be cost-prohibitive to change the system architecture or configuration to fix the issue and a system owner may have to accept the heightened risk of common cause cable failure. Given the nature of cables where energy and signal functions are shared between major subsystems, the potential for failure propagation is significant.
Thesis
Development projects, with their processes from design to manufacturing, aim to deliver a product with desired characteristics. Global risk management in the product development should, then, consider all the mentioned aspects: from process to final product. Unfortunately, this is often not the case. The thesis, hence, aims at representing project and process risks and their interactions. The proposed model includes interactions within development process and its product individually, but also those related to FBS framework. The solution focuses on risk assessment and presents identification, analysis and evaluation of risks. Simulator has been developed for risk evaluation. This tool is based on multi-agent technology, fuzzy cognitive maps, epidemiological simulation and design theory. The solution provides comprehensive process-product view on risk management. It enables bottom-up and, both, qualitative and quantitative risk representation. Monte Carlo, multi-view and “what if” analysis provide global risk evaluation.
Article
Full-text available
Through the application of statistical models, the active mission success estimation (AMSE) introduced in this paper can be performed during a rapidly developing unanticipated failure scenario to support decision making. AMSE allows for system operators to make informed management and control decisions by performing analyses on a nested system of functional models that requires low time and computational cost. Existing methods for analyses of mission success such as probabilistic risk assessment or worst case analysis have been applied in the analysis and planning of space missions since the mid-twentieth century. While these methods are effective in analyzing anticipated failure scenarios, they are built on computational models, logical structures, and statistical models that often are difficult and time-intensive to modify, and are computationally inefficient leading to very long calculation times and making their ability to respond to unanticipated or rapidly developing scenarios limited. To demonstrate AMSE, we present a case study of a generalized crewed Martian surface station mission. A crew of four astronauts must perform activities to achieve scientific objectives while surviving for 1070 Martian sols before returning to Earth. A second crew arrives at the same site to add to the settlement midway through the mission. AMSE uses functional models to represent all of the major environments, infrastructure, equipment, consumables, and critical systems of interest (astronauts in the case study presented) in a nested super system framework that is capable of providing rapidly reconfigurable and calculable analysis. This allows for AMSE to be used to make informed mission control decisions when facing rapidly developing or unanticipated scenarios. Additionally, AMSE provides a framework for the inclusion of humans into functional analysis through a systems approach. Application of AMSE is expected to produce informed decision making benefits in a variety of situations where humans and machines work together toward mission goals in uncertain and unpredictable conditions.
Article
Methodologies are required to model, evaluate, and analyze the resilience of system architectures at the conceptual design level to support needs evaluations, policy development, or study of social decisions. The primary objective of this paper was to develop a generalized resilience analysis model for the characterization of multistate engineered systems subject to disruptions to support system design selection during trade studies. Each design variant within the tradespace is modeled using continuous time Markov chains that simulate the transient response of each design variant subject to a user-defined disruption. The operating condition of the multistate system is characterized by a finite number of machine states. Each design variant's utility response is generated as a function of the system's machine state response to the disruption. Resilience capacities and capabilities are generated to populate trade studies. A case study consisting of an unmanned surface vessel provides a system theoretic approach to illustrate the concepts presented within this paper and demonstrates the ability of this methodology to aid decision maker's selection of resilient system architectures. Use of an analytical framework serves as a unifying foundation for incorporating resilience behaviors of multistate engineered systems into early stage design considerations.
Conference Paper
Full-text available
In the early engineering design phases of complex systems and facilities with significant cable routing requirements, the protection of critical cabling infrastructure and separation of redundant cables is often not taken into account. Cable routing and management happens later in the design process after significant system architectural decisions have been made. Given the nature of cables where energy and signal functions are shared between major subsystems, the potential for failure propagation is significant. We propose the Cable Routing Function Failure Analysis (CRFFA) method of cable routing planning that integrates with functional modeling and function failure analysis of complex systems to be used during the early conceptual stages of design. Through a more complete understanding of power and data cabling requirements during the early phases of design, a system design can be developed that minimizes the potential for critical cable infrastructure to be collocated. Reductions in collocated critical cabling reduce potential failure propagation pathways. The method in this paper relies on functional failure propagation probability calculation methods to identify and avoid potentially high risk cable routing choices. The implementation of this method will help engineering practitioners to design complex systems and facilities that protect against cabling failure propagation events (cable raceway fires, cable bundle severing events, etc.) from the earliest phases of design. Thus, system reliability will increase while system failure probabilities, cost of system design, and design lifecycles will decrease.
Conference Paper
Full-text available
It is accustomed to call robust design a design that is resilient to noise. A product could be designed to be robust by methods such as Taguchi's. The idea is to manipulate the design parameters that could be controlled by designers to minimize the effect of the noise on the planned behavior in the designated environment. We are concerned with a broader perspective of robustness, one that arises from many environmental uncertainties including those related to technical knowledge, customers, and market conditions. In such interpretations, the product behavior includes physical behavior as well as customer satisfaction, cost, as well as any parameter that is related to the technical and market success of the product. In this context, we define a product as robust if a large variety of potential environmental uncertainties have little impact on its behavior. While the broad perspective of robustness applies to all design stages, we are in particular interested in the conceptual design stage that is considered to be the most critical step in product development. In this stage, an abstract description of the product is created that serves as the basis for subsequent design stages and decisions. To a large extent, the quality of the product concept determines the fate of the product. In (Ziv Av and Reich, 2005) we presented a method – SOS (subjective objective system) – for generating optimal concepts in diverse disciplines. In this work, we extend SOS to generate robust product concepts. In the context of SOS, robustness is defined as the stability of the optimal concept or configuration generated by SOS with respect to (1) variations in designers’ subjective judgment, (2) variations in available technology, (3) variations in organization context, and (4) variation in customers’ preferences. All these could have an impact on the results obtained by SOS. In order to assess the robustness, we run different tests with simulated changes and analyzed the results. For example, robustness with respect to designers’ judgment was tested by varying such judgment and checking the stability of the solution to such variations. Robustness with respect to customer preferences was calculated by sampling different preferences and finding their related optimal concepts. This data was subsequently analyzed to find robust concepts as well as risky concepts. For each preference we also analyzed the local robustness of the solution; that is, how much can we change the customer preferences from the available estimation and maintain the same solution. If these variations are large, our confidence in the solution increases. The robustness with respect to other variations is analyzed similarly. In the context of SOS, we define two types of robust concepts. In general, robust concept is a product concept that remains stable as different evaluations in SOS varies due to different circumstances. The first type of robustness – global robust concept – is defined as the concept that is most prevalent if we let SOS input values vary randomly in their allowable range. This is an operational definition because it specifies the method to find that concept. The second type of robustness – local robust concept – is defined as a concept designed for a particular set of inputs and that is remain intact even if these input values change significantly from their present values. This is also an operational definition. Since SOS automatically generates the concept from its inputs, we can run simulations with different input values and obtain the results that allow assessing the global and local robustness of a concept. A last perspective on concept robustness arises from the concept of a product family. In doing so, we depart from the traditional work on product family and platform that mainly deals with complete or detailed designs. In contrast to others, we deal with product family and platform at the concept generation stage. In the context of SOS, instead of finding a design concept that is prevalent across the space of SOS input values as we did in the global robustness analysis, we define robust platform concept to be a product concept family that addresses several markets and whose common platform is almost completely specified. Consequently, implementing the platform concept in multiple markets involves minimal customization.
Article
Full-text available
Risk assessments performed at the conceptual design phase of a product may offer the greatest opportunity to increase product safety and reliability at the least cost. This is an especially difficult proposition, however, as often the product has not assumed a physical form at this early design stage. This paper introduces the Risk in Early Design (RED) method, a method for performing risk assessments based on functions, rather than physical components, to address this challenge. In particular, this paper focuses on the function based mathematical mappings of the RED method for a preliminary risk assessment based on catalogued historical failure information. An example is presented on how the RED preliminary risk assessment method is used in the design process on a spacecraft thermal control subsystem. Also, heuristics for applying the particular types of risk assessments are discussed. The preliminary risk assessment method discussed offers a potentially paradigm-shifting approach to identifying potential areas of concern in a product during the early stages of design when risk mitigation is least expensive.
Article
The concept of function offers significant potential for transforming thinking and reasoning about engineering design as well as providing a common thread for relating together product risk information. This paper focuses specifically on risk data by examining how this information is addressed for a design team conducting early stage design for space missions. A fundamental set of risk elements is proposed based on a linguistic analysis of the risk information needs of the design team. Sample risk statements are then decomposed into a set of key attributes that are used to scrutinize the risk information using three approaches from the pragmatics sub-field of linguistics: (1) Gricean, (2) Relevance Theory, and (3) Functional Analysis. Based on the deficiencies identified in this analysis, a format for the communication of risk data by explicitly accounting for five risk attributes developed in this work is formulated.
Conference Paper
A method for propagating and mitigating uncertainty in conceptual-level space mission design is described. The method presented forms a normative method for decision makers seeking to balance cost, risk, and performance in preliminary mission design, with an emphasis on launch vehicle selection. Models representing each of these parameters are created. The variables of the models are then classified. Probabilistic methods, Bayesian techniques, and numerical simulation are used to characterize the resulting parameters. Lastly, results of this analysis are combined with the risk tolerance of the decision maker to guide in balancing these parameters. As an example application, the investigated method is applied to two mission scenarios. The first mission scenario investigates the uncertainty in mission success for the Mars Exploration Rover (MER) mission where the launch vehicle has been selected a priori. The second mission scenario investigates balancing the set of parameters for a generic sun-synchronous low-earth orbit satellite when two possible launch vehicles are available. Use of this approach with the first example yields a surprisingly high level of risk to the mission. Use of the approach for the second example provides quantitative results in estimating the launch cost and risk of the two different options.
Article
Uncertainty plays a critical role in the analysis for a wide and diverse set of fields from economics to engineering. The term 'uncertainty' has come to encompass a multiplicity of concepts. This paper begins with a literature survey of uncertainty definitions and classifications from various fields. A classification of uncertainty for the design and development of complex systems follows. The various classifications are more practical than theoretical: to make distinct the techniques used to address each type of uncertainty and to demonstrate the effects of each type of uncertainty in each field. The classification for the design and development of complex systems delineates ambiguity, epistemic, aleatory, and interaction uncertainty. Epistemic uncertainty is further subdivided into model-form, phenomenological, and behavioral uncertainty, each of which is described in detail. The uncertainty taxonomy presented is an integral part of ongoing research into propagating and mitigating the effect of all types of uncertainty in the design and development of complex multidisciplinary engineering systems.
Article
A method for determining margins in conceptual-level design via probabilistic methods is described. The goal of this research is to develop a rigorous foundation for determining design margins in complex multidisciplinary systems. As an example application, the investigated method is applied to conceptual-level design of the Mars Exploration Rover (MER) cruise stage thermal control system. The method begins with identifying a set of tradable system-level parameters. Models that determine each of these tradable parameters are then created. The variables of the design are classified and assigned appropriate probability density functions. To characterize the resulting system, a Monte Carlo simulation is used. Probabilistic methods can then be used to represent uncertainties in the relevant models. Lastly, results of this simulation are combined with the risk tolerance of thermal engineers to guide in the determination of margin levels. The method is repeated until the thermal engineers are satisfied with the balance of system-level parameter values. For the thermal control example presented, margins for maximum component temperatures, dry mass, power required, schedule, and cost form the set of tradable system-level parameters. Use of this approach for the example presented yielded significant differences between the calculated design margins and the values assumed in the conceptual design of the MER cruise stage thermal control system.
Conference Paper
Collaborative design centers often employ software tools to conduct trade studies. Commonly, this takes the form of a software program to aggregate and pass data between multiple computer workstations. This allows multiple people to concurrently create a conceptual design. Trade study software continues to evolve to meet the demands of modern collaborative design centers. However, the risks associated with moving from one trade study software tool to another are not well understood. Additionally, little is known about the software preferences of Collaborative Design Center (CDC) staff. This paper determines software preferences of two user groups consisting of graduate and undergraduate mechanical engineering students. This paper then explores the risks in deploying new trade study software in a collaborative design center. A method for estimating and mitigating risks with changing trade study software is presented. Recommendations for a smooth transition between software packages are given. The risk model developed in this paper offers a quick way of estimating and mitigating conversion risk for collaborative design centers.