No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Measuring system of systems performance
Abstract
Single systems and systems of systems, alike, demand management approaches focused on performance; but system of systems management can only responsibly address performance if it accounts for characteristics such as the presence of distinct subsystems pursuing possibly disparate purposes, a characteristic by definition of no concern to single system management. By extension then, with to manage in many ways to measure, systems of systems demand performance measurement schemes that accommodate the traits that set them apart from individual systems. Enterprise AID represents a means for measuring and hence managing the current or future performance of systems of systems. Enterprise AID - or simply AID, for assessment, improvement, and design - is a methodology for the design and use of performance measurement systems able to uniformly address problems encountered with extant or envisioned, single system or system of systems type enterprises. This paper describes advantages held by the AID methodology for system of systems performance measurement, and it does so within a context set by appropriate definitions, selected methodology elements, and an application example focused on the selected elements. This paper concludes with a recognition of performance related issues attendant to system of systems improvement or design.
... What emerges from this is self-referentiality on the SoS level, allowing it to reflect on its behavior and adjust. This self-reflection requires also that there are observable characteristics on the SoS level, such as indicators for its performance [57], and it is a part of governance to identify those indicators. Allowing these emergent properties to be observed sometimes requires the use of additional mediators [48]. ...
... When demand changes, the SoS needs to adjust its capacity by recruiting new members. A process of homeostasis can be employed to find stability so that capacity matches demand based on different SoS performance metrics [57]. However, since membership is voluntary, a shortage of members may require an adjustment of incentives, making it more attractive to join by giving a higher expected value to the agent. ...
... From the SoS perspective, on the other hand, there can be reasons for wanting a certain agent to be involved. Capabilities are not binary but come with performance parameters, such as cost or time to completion [57]. ...
Systems-of-systems are often characterized as systems where the constituent parts have some independence from the whole. Recent research has aimed at clarifying in more detail what this independence means. It has shown that independence requires the constituent systems to be agents that observe the system-of-systems from within and construct internal models of it as a basis for decisions. This view on observers as parts of the system-of-systems parallels development in the field of second-order cybernetics several decades ago, yet the connection between that field and systems-of-systems has not been explored previously. This paper, therefore, summarizes key concepts from second-order cybernetics, including the subtopic autopoiesis. It then discusses what the implications are on systems-of-systems engineering through the identification of 17 concerns. These concerns relate to the physical topology, behavior, and control of the system-of-systems. This paper shows how these concerns directly relate to the theoretical concepts of second-order cybernetics and autopoiesis, and thereby, opens the door to further exploitation of this theoretical foundation.
... The notion of reality being socially constructed by professionals working in their local environments drove the selection of a constructivist approach in this study. Meyers and Hester (2012), writing from the perspective of SOS engineering, observe that: ...
While a large body of literature exists documenting the technological vulnerabilities of the U.S. electrical grid’s technical components, less is known in the research community about the cybersecurity interorganizational dependencies and relationship networks that are present between the nation’s more than 3,000 electrical utilities. This exploratory qualitative case study investigated those interorganizational dependencies at the industry level of analysis, using constructs such as Galbraith’s (1977) organization information processing theory and systems of systems to facilitate the analysis. Thirteen cybersecurity professionals were interviewed as industry informants.
... AID = assessment, improvement, and design; PE = problem essential; PM = problem measure; SM = solution measure. multiple jurisdictions in system of systems-type environments (Meyers & Hester, 2013). Moreover, Sage and Cuppan's (2001) Table 1 descriptions strongly suggest that system of systems-type problems should be addressed only with methodological approaches that, like AID, can truly promote such problems' resolution. ...
Public works decision and policy makers daily face myriad management problems. The problem of stormwater runoff management, for one, has for several decades received a great deal of attention, with much of that focused on reconciling the needs of natural watershed systems with those of the man-made physical and nonphysicalsocietal systems that watersheds overlap. In other words, stormwater management has effectively been cast as a system of systems problem requiring a delicate balance among multiple natural and man-made systems. The authors propose a methodology with which stakeholders can first set and then realize expectations for stormwater management problems by using watershed- and system of systems- oriented perspectives. The methodology, Enterprise AID (assessment, improvement, and design) is particularly well suited to such a pairing of perspectives, and this article, therefore, shows how stakeholders in multiple and commonly disparate interests might best balance the stormwater management needs of watersheds and related societal constructs.
The assessment of performance and quality is essential to the engineering and evolution of Systems of Systems (SoS); however, there has been limited discussion of SoS measurement and evaluation in the literature. This paper reviews the measurement and assessment literature across multiple disciplines to address this gap and applies constructive research to design principles and concepts for SoS evaluation. Relevant challenges and best practice elements are identified within each discipline for comparison against the nature and characteristics of Systems of Systems. These elements are then synthesized into three key areas necessary for SoS evaluation: measure development, determination of good metrics, and assessment. While further work has been identified and initiated on methods for the generation of such metrics, this study provides practitioners with the core practices and principles necessary to enable tailored and enhanced SoS assessment.
This article describes an exploratory qualitative case study of information sharing among cybersecurity professionals in the critical U.S. electrical power industry. Drawing on organizational information processing theory, this study examined how information related to cybersecurity related threats and response alternatives was accessed and shared in the participants’ organizations and within their broader industry network. In-depth interviews of 13 participants from 10 organizations were conducted to identify the nature of information sharing, the networks used to access the information, and changes that could improve information sharing between cybersecurity professionals and their organizations.
The study found that information sharing networks exist at interpersonal, company-to-company, and company-to-multicompany levels of analysis. The role of trust in forming these networks was notable, as was the role of certain communication media, particularly threat briefings, testing exercises, and cybersecurity mutual assistance planning. The study found that certain types of network actors, such as law enforcement agencies, had strong relationship ties with the utility companies, while other actor types had weak or nonexistent ties. The implications for critical infrastructure protection are discussed.
The first perspective in the system of systems engineering (SoSE) methodology is to ensure that the engineering analysis is supported by an explicit understanding and framing of the problem under study. By explicitly framing the problem and its associated context, the SoSE methodology minimises the chance of a Type III error (i.e., correctly rejects the null hypothesis for the wrong reason) committed at study's outset. This paper addresses the methodology's Perspective 1, framing the system under study, and the nine component execution elements serving to operationalise it. The paper will illustrate those elements as tools with which SoS problems may be framed in terms of their most prominent contextual and environmental influences.
This paper describes an effort to determine the rationale and content for a research agenda in complex systems. This effort included a workshop conducted with 50 thought leaders in complex engineered, organizational, and natural systems. The results of this workshop were subsequently presented to seven groups in academia and industry across the United States. In this way, additional comments, suggestions, and insights were gained from roughly 200 participants in these presentations. The objectives of these eight events were to understand the underlying issues that cause us to perceive a system to be complex, and formulate a set of fundamental research questions whose pursuit would advance abilities to address these issues. © 2007 Wiley Periodicals, Inc. Syst Eng: 260–271, 2007
Many have observed that measurement, knowledge generation and advancement are companion activities. The evidence for this comes from numerous sources over the long history of mankind. Better understanding of these connections appears able to yield major advances in knowledge generation and use of measurement. Current formal understanding of the meaningful nature of measurements lags behind other aspects of the practice of measurement science. It is a major area needing attention through application of the differing kinds of thinking found in the hard and soft sciences. The state of understanding and trends of these relationships are reviewed from a measurement practitioner’s viewpoint.
Although “Measures of Effectiveness” (MOEs) are referred to in systems engineering documents such as EIA/IS-632, there is no universally accepted meaning for the term. This paper provides an overview of this important topic by suggesting a definition of MOEs. The role of MOEs as the standards for identifying successful solutions is examined. They are shown to be “mission” or “purpose” oriented and not concerned with the internal details of the solutions per se. A clear distinction is drawn between the MOEs as established by the stakeholder and the quantified MOE, or figure of merit, established as a result of the analysis and evaluation of particular solutions. MOEs are seen to be the precursor to, hence the “engines” of, test and evaluation and of criteria. A study of the roles of other forms of “measures” enables a clearer understanding of the role and nature of MOEs to be established. © 2000 John Wiley & Sons, Inc. Syst Eng 3: 50–58, 2000
This paper presents sufficient conditions for a multiattribute utility function to be either multiplicative or additive. The number of requisite assumptions to imply the main result is equal to the number of attributes. Because the assumptions involve only trade-offs between two attributes at a time or lotteries over one attribute, it is reasonable to expect that decision makers can ascertain whether these assumptions are appropriate for their specific problems. Procedures are given for verifying the assumptions and assessing the resulting utility functions. The paper concludes with a sketch of a recent application of the results to a six-attribute problem relating to the development of Mexico City's airport facilities.
Engineering activities in future organization development, including various information-based systems, vary, but all result in evolutions of an organization, its capabilities and systems. These evolutions occur in a context of Systems-of-Systems (SoS) where the organization must maintain a sustained, sustainable, and controlled SoS evolution as a whole. This paper presents an understanding of SoS challenges to the application of Systems Engineering (SE) in organizational evolutionary development and discusses the difference between “developing a SoS” and “developing systems in a SoS context” from an SE management perspective. A new approach to SE process organization and management is presented in order to help an organization cope with the high complexity of SoS evolutions and improve its architecture practice. Philosophically different from many SoS SE studies that consider mainly how to develop a SoS, the new approach is to add a dimension or components of SE practice at the organization level that is aimed at creating a better engineering environment to enable effective applications of traditional SE practice in implementing SoS evolutions. © 2003 Wiley Periodicals, Inc. Syst Eng 6: 170–183, 2003
Although Measures of Effectiveness (MOEs) are an important element of the Systems Engineering process, they have yet to gain acceptance from practitioners. This is caused not only by the confusion surrounding their definition but also the perceived difficulty in formulating appropriate MOEs. This paper builds on previously published papers that address the issue of the definition of MOEs and the distinction to be drawn with Measures of Performance. Following a brief review of this existing work, a process for formulating MOEs is suggested and explained. This is supplemented by several examples from everyday life and from historical references. The examples build on the issues raised in the discussion of the process and provide practical examples to reinforce what has been previously discussed. © 2002 Commonwealth of Australia. Exclusive worldwide publication rights in the article have been transferred to Wiley Periodicals, Inc. Syst Eng 5, 253–263, 2002
The nature of enterprises as systems is considered. This exploration begins with discussion of the work of enterprises, with emphasis on challenges rather than routine operations. This reflects a desire to support enterprises as they address essential challenges. Situations where enterprise transformation is needed to successfully deal with challenges are then discussed. The nature of enterprise transformation is discussed in terms of ends, means, and scope, as well as perspectives, approaches, and solutions. This leads to elaboration of a portfolio of important research issues that suggests a wealth of potential means for supporting people in enterprises to accomplish the work of these enterprises. © 2005 Wiley Periodicals, Inc. Syst Eng 8: 138–150, 2005
Today, many systems within government and industry are typically engineered, not as stand alone systems, but as part of an integrated system of systems, or a federation of systems, or systems family. A significant level of effort has been devoted in the past several years to the development, refinement, and ultimately, acceptance of processes for engineering systems or systems engineering processes. Today, we have four ldquostandardrdquo processes within present and past standards: EIA-632, IEEE 1220, ISO 15288, and MIL-STD-499C. We continue to use systems engineering processes espoused in our current set of standards, and are left to our devices to tailor these processes to one appropriate for a systems family context. This paper will examine the systems engineering processes in existence today and conclude with development of a process designed specifically for systems family applications.