Content uploaded by Peter Gorm Larsen
Author content
All content in this area was uploaded by Peter Gorm Larsen on Feb 03, 2016
Content may be subject to copyright.
XXXX
Systems of Systems Engineering: Basic Concepts, Model-based
Techniques, and Research Directions
CLAUS BALLEGAARD NIELSEN, Cranfield University
PETER GORM LARSEN, Aarhus University
JOHN FITZGERALD, Newcastle University
JIM WOODCOCK, University of York
JAN PELESKA, University of Bremen
The term ‘System of Systems’ (SoS) has been used since the 1950s to describe systems that are composed of
independent constituent systems, which act jointly towards a common goal through the synergism between
them. Examples of SoS arise in areas such as power grid technology, transport, production and military
enterprises. SoS engineering is challenged by the independence, heterogeneity, evolution and emergence
properties found in SoS. This paper focuses on the role of model-based techniques within the SoS engineering
field. A review of existing attempts to define and classify SoS is used to identify several dimensions that
characterise SoS applications. The SoS field is exemplified by a series of representative systems selected
from the literature on SoS applications. Within the area of model-based techniques the survey specifically
reviews the state of the art for SoS modelling, architectural description, simulation, verification and testing.
Finally, the identified dimensions of SoS characteristics are used to identify research challenges and future
research areas of model-based SoS Engineering.
Categories and Subject Descriptors: H.1.1 [Systems and Information Theory]: General systems theory;
I.6.0 [Simulation and Modelling]: General; F.4.0 [Mathematical Logic and Formal Languages]: Gen-
eral
General Terms: Design, Languages
Additional Key Words and Phrases: System of Systems, Systems Engineering, Model-based Engineering
ACM Reference Format:
Claus Ballegaard Nielsen, Peter Gorm Larsen, John Fitzgerald, Jim Woodcock and Jan Peleska, 2015. Sys-
tems of Systems Engineering: Basic Concepts, Model-based Techniques, and Research Directions. ACM Com-
put. Surv. V, N, Article XXXX ( 2015), 41 pages.
DOI: http://dx.doi.org/10.1145/2794381
1. INTRODUCTION
In many important domains, including infrastructure, healthcare, transportation,
emergency response and defence, reliance is placed on the delivery of a service by a
system composed of largely independent, typically pre-existing, systems. For example,
the successful treatment of a patient in an emergency results from the interaction
of several separately owned and managed systems including telephony, ambulance
assignment, information sharing, communications and hospital management. These
constituent systems may have existed before requirements (for example, to meet max-
This work was supported in part by the EC Framework 7 Project COMPASS (Grant Agreement 287829).
Corresponding Author’s address: Claus Ballegaard Nielsen, Centre for Systems Engineering, Cranfield
University, Defence Academy of the United Kingdom, Shrivenham, SN6 8LA, United Kingdom; email:
c.nielsen@cranfield.ac.uk. During the leading work Claus Nielsen was affiliated with Aarhus University,
but has since changed affiliation to Cranfield University.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights for components of this work owned
by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or repub-
lish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request
permissions from permissions@acm.org.
c
2015 ACM. 0360-0300/2015/-ARTXXXX $15.00
DOI: http://dx.doi.org/10.1145/2794381
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:2 C. B. Nielsen et al.
imum response times or to guarantee the confidentiality of patient data) were imposed
on their collective behaviour. Advances in network and communications technology
have made it possible to conceive of deliberately engineering and maintaining such
‘Systems of Systems’ (SoSs).
The SoS engineer faces several important challenges, not least the need to identify
the boundaries of the overall SoS and of the independent constituent systems within it.
These boundaries relate to both technical aspects such as interfaces, integration and
testing, and management aspects such as governance and stakeholder involvement.
Further challenges relate to the gaining of confidence in system operation, in terms
of behavioural correctness, performance qualities and their validation. Many of these
challenges are already the foci of work in the field of Systems Engineering [Hitchins
2005]. SoS Engineering is not a completely new or opposing discipline, but is a sub-
field of Systems Engineering that focuses on the boundaries and interactions between
independent, distributed and evolving constituent systems and their stakeholders.
The distributed constituent systems in an SoS are owned and operated by indepen-
dent stakeholders, and consequently there are limitations on the exchange of informa-
tion about them. On the other hand, SoS behaviour is dependent on emergent phenom-
ena observed at the system boundaries. Consequently, there is a need for engineering
techniques that address such characteristics. The challenges that SoS Engineering en-
counters to a higher degree than traditional Systems Engineering are described by
Dahmann et al. and can be summarized as: stakeholders with competing interests
and priorities, no centralised authority over all the systems, added complexity due
to multiple system life-cycles, as well as balancing testing, behaviour characteristics
and performance needs between the constituent systems and the SoS [Dahmann et al.
2008].
SoSs arise in increasingly broad domains. A widely used example is in emergency
response, in which agencies (such as fire, police, hospital) with independently owned
and managed systems nevertheless collaborate to deliver a service on which reliance
is placed. Other instances of SoSs arise in infrastructure systems, which typically de-
liver services through the collaborative operation of multiple providers. In road trans-
port, for example, local, regional and national agencies, often operating to different
priorities, manage flow in interlinked traffic networks, but must offer services that re-
main safe across their boundaries. Recent applications of SoS engineering have been
in less conventional domains, such as transport of radioactive source materials [mauss
et al. 2015], and the design of audio/video networks that stream content from mul-
tiple providers to heterogeneous sets of devices [Bryans et al. 2014]. After surveying
the state of the art in model-based techniques for SoS engineering, we discuss some
significant examples of SoSs in Section 4.
Although SoSs arise in very diverse domains and may differ in architecture and
application, the engineering problems that they present have some commonalities.
These include the need to gain assurance of key properties of the SoS as a whole
in spite of the operational and managerial independence of the constituent systems,
their distributed, concurrent character, and their heterogeneity. In addition, the range
of stakeholders involved, including the owners and operators of constituent systems,
their integrators, and ultimately those who experience the system behaviour of the
SoS, implies the need to employ methods and tools that support collaborative working
from the elicitation of requirements to testing and maintenance.
We regard a system as “a combination of interacting elements organized to achieve
one or more stated purposes” [INCOSE 2015; International Organization for Standard-
ization 2015]. An SoS is a system, some of whose elements are themselves designated
as systems. A discipline of SoS Engineering has begun to emerge, aiming to extend
Systems Engineering with the ability to develop, maintain and adapt SoSs effectively.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:3
In common with other areas of Systems Engineering, there has also been considerable
interest in the use of model-based techniques [Cantot and Luzeaux 2011]. We will use
the term ‘model’ to refer to an abstract description of a system of interest. The partic-
ular abstraction decisions made in a given model are determined by the model’s pur-
pose [Kramer 2007]. For example, a model of an emergency response SoS constructed
in order to analyse maximum response times is unlikely to include an explicit repre-
sentation of all the fields in a patient record.
Models may be used to describe real-world objects or phenomena to a certain level
of fidelity. Equally, models may be used during design to describe potential systems
that are yet to be realised. In SoS Engineering, as in Systems Engineering more gen-
erally, both kinds of model arise in the development and maintenance of a system or
SoS. In particular, descriptive models of the already existing elements of an SoS may
be combined with design models of elements that are to be constructed. Models can
cover such diverse aspects of an SoS as its structure, functionality, communications
and behaviour.
To gain confidence that an SoS architecture will respect key properties, it is
paramount to have a precise model of the constituents and the connectors between
them, the properties of the constituents and the SoSs environment. Such a model sup-
ports the “trade-off” of alternative designs at early development stages and the pre-
cise determination of the contract that exists between each constituent system and
the SoS. Model-based approaches are already well accepted in industry for their abil-
ity to manage and control the overall complexity of a system, reveal and document its
key structure and behaviour, and communicate these to stakeholders [Woodcock et al.
2009]. As the use of models evolves and matures, there is a clear need for verification
and validation technology that allows the value of models to be exploited.
Model-based SoS engineering is an active area, both of practice and research. There
is considerable evidence of activity in forums such as the IEEE Systems Engineering
Conference1, the IEEE Conference on System of Systems Engineering2, and INCOSE3,
which runs an industry-led working group on SoS engineering, as well as publications
of record in media such as the IEEE Systems Engineering Journal and the Journal
of System of Systems Engineering. Notable in the area of model-based methods is the
joint Model-Based Systems Engineering initiative of INCOSE and the Object Manage-
ment Group4. In 2010, the European Commission funded a group of projects specifi-
cally addressing SoS engineering5, and has held a series of expert workshops aiming to
help determine the potential of research in the field up to 2020 [European Commission
2012]. There is a considerable and growing volume of work in the area, but the subject
is young, and general lessons and patterns that cut across applications remain to be
learned. Therefore the exact extent to which model-based engineering can be applied
SoS engineering is still an open research question.
The purpose of this paper is to provide a structured view of the state of the art in
model-based techniques in SoS engineering, and to identify challenges for research in
this field. We focus on the use and potential of models that have formal semantic foun-
dations. Our 2009 review of industrial applications of formal methods suggested that
the benefits of formal modelling were being increasingly realised in industrial practice,
1http://www.ieeesyscon.org/
2For example, http://www.sosengineering.org/2015/
3http://www.incose.org
4http://www.omgwiki.org/MBSE
5These include two projects focussing on research in model-oriented methods for the design of SoSs: COM-
PASS (www.compass-research.eu) and DANSE (www.danse-ip.eu), T-AREA-SOS which promotes transat-
lantic cooperation (www.tareasos.eu/) and ROAD2SOS (www.road2sos-project.eu).
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:4 C. B. Nielsen et al.
particularly in hardware, and to a greater extent in software design [Woodcock et al.
2009]. The field of SoS engineering is nascent, exciting, and at a very early stage of
development with as yet no formal foundation able to scale from hardware and soft-
ware to the level of the SoS itself. The scope of the paper is confined to the modelling of
technical systems, since this covers the major part of work to date. However, progress
on socio-technical aspects of the interaction between humans and SoSs, and between
humans mediated through the SoS, is essential to success in providing a truly compre-
hensive approach to SoS engineering [Lock and Sommerville 2010]. Business aspects,
including requirements elicitation, tendering and procurement, although also outside
the scope of the paper, are essential to a comprehensive understanding of SoS [Holt
2012].
This paper concentrates on two aspects of model-based SoS engineering: firstly the
characteristics of SoSs that make them particularly challenging, and secondly the en-
gineering activities that form the core of model-based approaches, specifically mod-
elling, architectural description, simulation, testing and verification. A review of the
many attempts to define and classify SoSs (Section 2) suggests that it may be bene-
ficial to view SoSs in terms of the dimensions that categorise them (Section 3), and
that pose particular modelling challenges. A review of several exemplars of SoS engi-
neering from the literature helps to ground the subsequent discussion (Section 4). In
Section 5, current and promising technologies for formal model-based SoS engineer-
ing are explored. In Section 6, research challenges in realising the potential of each of
these technologies are identified and the steps necessary to provide a firmer foundation
for model-based SoS engineering is examined.
2. DEFINITION AND CHARACTERISTICS
As might be expected in an emerging field, there is yet no precise and widely accepted
definition of SoS to which the bulk of the literature conforms, making it difficult to
bound the field precisely. The literature is diverse, and there are many attempts to
define and characterise SoS. Several reviews have sought to achieve some conver-
gence [Keating 2005; Jamshidi 2005; Sharawi et al. 2006; Lane and Valerdi 2007;
Gorod et al. 2008; Jamshidi 2008]6. In this section, we review attempts to define, char-
acterise or describe SoS, starting with a historical overview (Section 2.1), followed by
a focus on literature that describe SoS via characteristics (Section 2.2) or Taxonomies
(Section 2.3. Views on SoS in Industry (Section 2.4), Academia (Section 2.5) and Engi-
neering Handbooks (Section 2.6) are presented, with the section ending with a focus
on literature that extend existing SoS definitions (Section 2.7).
2.1. Historical Overview of SoS Definition
The early ideas of SoS come from a variety of sources. Boulding’s paper on General
Systems Theory uses the term “system of systems” to describe the organisation of the-
oretical constructs [Boulding 1956]. The SoS concept is described as “the arrangement
of theoretical systems and constructs in a hierarchy of complexity”. Boulding’s classi-
fication of SoS consists of a static structure for system’s anatomy as well as a dynamic
dimension that enables the system to adapt over time, as the SoS is an “open sys-
tem” that can be affected by external events. There is a division of labour between
differentiated but mutually dependent parts, and the SoS has a “transcendent” and
“unknowable” element. Several features of Boulding’s description bear a clear resem-
6Note that some surveys include [Jackson and Keys 1984; M¨
uller-Merbach 1994] that use SoS to describe
a system for the arrangement and ordering of various “systems methodologies” concepts. These are not
included in this survey, as we concentrate on the relationships between and behaviour of operational con-
stituent systems, and not on creating structures for analysis methodologies.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:5
blance to aspects of the more recent engineering notions of SoS considered below. Later,
both urban city planning, systems science structures and biological systems came to
be characterised as SoS [Berry 1964; Ackoff 1971; Jacob 1974].
The United States’ Strategic Defense Initiative (SDI) from the late 1980s became a
key factor in establishing SoS as an engineering concept focused on joining indepen-
dent systems together [United States, Congress, Senate, Committee on Armed Ser-
vices 1988]. Subsequently, SoS research intensified in both academia and industry,
and attracted increased awareness. Nevertheless, it took another 15 years for SoS En-
gineering (SoSE) to develop as a recognised discipline, and by the early 21st Century
was still regarded as being in its infancy [Keating et al. 2003].
2.2. Defining SoS via Characteristics
In a response to a widening recognition of SoSs combined with the lack of a shared
agreement on an SoS definition, Maier [1996]7characterises SoS in terms of five prin-
cipal features sometimes referred to by the acronym “OMGEE”:
Operational Independence. Any system that is part of an SoS is independent and is
able to operate serviceably if the SoS is disassembled.
Managerial Independence. Despite collaborating with the other members of the
SoS, the individual systems are self-governing and individually managed so that
they “not only can operate independently, they do operate independently.”
Geographic Distribution. The parties collaborating in an SoS are distributed over
a large geographic extent. Although the geographic extent is defined vaguely, it
is stressed that the collaborating systems can only exchange information and not
considerable quantities of mass or energy.
Evolutionary Development. An SoS’s existence and development are evolutionary
in the sense that objectives and functionality can be under constant change, as they
can be added, modified or removed with experience. Thus an SoS never appears
completely formed.
Emergent Behaviour. Through the collaboration between the systems in an SoS a
synergism is reached in which the system behaviour fulfils a purpose that cannot
be achieved by, or attributed to, any of the individual systems.
Boardman and Sauser [2006] seek to identify characteristics that distinguish SoSs
from conventional systems and pay particular attention to the merging of new con-
stituents with existing systems to form the SoS. They identify five characteristics for
SoS (acronym “ABCDE”):
Autonomy. Each system is free and independent with its own purpose of operation.
Belonging. Systems function collaboratively to meet a common higher purpose.
Connectivity. Synergism is enabled by the highly dynamic distributed network.
Diversity. The constituents are heterogeneous self-sufficient systems that are open
for enhancement by evolution and adaptation.
Emerging. The cumulative actions and interactions between the constituents of an
SoS give rise to the behaviours that can be attributed to the SoS as a whole.
Abbott argues that the SoS term should be reserved for systems that are qualita-
tively and structurally different from traditional systems [Abbott 2006]. An SoS should
not be seen as a hierarchy of components, but as an environment were systems reside
7In the SoS literature, Maier [1998a] is widely cited. However, the characteristics originate from Maier
[1996] which also exists in a whitepaper version published online [Maier 1998b]. All these versions share
the title “Architecting Principles for Systems-of-Systems”.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:6 C. B. Nielsen et al.
and systems can join, operate and interact within it. Abbott defines three character-
istics of such environments: 1)Open at the top, meaning that an SoS is continually
open for addition of new applications and systems, without any top-level system defin-
ing the SoS. 2)Open at the bottom, meaning that the lowest level of the SoS, such as
a specific communication stack, may be changed at any time. 3)Continually evolving,
but slowly: an SoS is never complete as it evolves with changes in the surrounding
environment. At least three forms of system evolution exist: standards and interfaces
adjustment, technological changes, and feature modification.
2.3. Defining SoS via Taxonomies
Shenhar [1994] proposes a two-dimensional taxonomy for systems; a technological un-
certainty dimension describing the maturity of the technologies and the Scope level
dimension classifying systems from a single-purpose assembly to an array of geograph-
ically dispersed systems interacting to achieve a common purpose. Shenhar and Bonen
[1997] later identify SoS with the array type.
DeLaurentis and Crossley [2005] propose a taxonomy for SoS analysis that described
how an SoS materialises when needs are met by a combination of independent systems
that rely on the interrelationships between one another. The taxonomy emphasises
three dimensions: 1) “Connectivity”: analysis of interdependencies and the dynamic
topology changes over time, 2) “Control/autonomy”: balance between authority con-
trol versus autonomous behaviour, and 3) “System type”: the balance between hard-
ware/software and human enterprise.
Finally, Karcanias and Hessami [2010] considers SoS as an evolution of Composite
Systems, focused on the integration challenges of autonomous and independent sys-
tems in large-scale projects. SoSs are described as complex multi-systems that define
a global goal and aggregate interdependent constituent systems.
2.4. Industrial Views on SoS
In industry there has been a focus on the SoS challenges that occur in the communi-
cation and exchange of data between systems.
Noam describes the change in the interconnection between carriers and “telecom-
munications integrators” in new telecommunication infrastructures as a move from
“network of networks” to SoS, because networks start separating into dynamic sys-
tems, which will allow integrators to deliver services to the consumers, without them
owning the telecommunication network [Noam 1994].
Focusing on data transmission and hierarchical structures, Kotov [1997] defines an
SoS as “large-scale concurrent and distributed systems the components of which are
complex systems themselves”. These complex systems are described as a result of hard-
ware, software and network being merged into larger integrated system architectures
with constant growing complexity.
The SoS term is used to describe the type of systems emerging from rapidly improv-
ing military capabilities on intelligence gathering and sharing [Owens 1995], which
expands to a focus on the integration and interoperability between C4I (Command,
Control, Communication, Computers and Intelligence) and ISR (Intelligence, Surveil-
lance, Reconnaissance) systems [Manthorpe 1996] as being key in future battlefield
scenarios [Pei 2000].
Within Enterprise Systems, SoSs are challenging because the continuous emerging
behaviour makes it difficult to capture their business value and the heterogeneity of
constituents means that each system has its own capabilities, users and interfaces,
making integration a challenge [Carlock and Fenton 2001]. In the same way Cocks
[2006] describes SoS as the result of a system engineering process involving systems
for which integration and life-cycle development are not under centralised control.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:7
Boehm [2006] refers to Software-Intensive SoSs (SISoS) as a key factor in the com-
petitiveness of organisations that supply software services. Future demands will re-
quire dynamically evolving systems consisting of numerous independently-developed
systems, that will have emergent requirements and socio-technical issues as key chal-
lenges.
2.5. Academic Views on SoS
Within academia there has been a focus on engineering discipline and engineering
education for SoS. Eisner et al. [1991] identify the need of focusing on the challenges
that arise from the scale and complexity of SoSs (there denoted “S2”), and builds this
around seven characteristics including notions of interdependence and – unusually –
a requirement for overall control of the constituents.
In the same way as Eisner et al., Roe assume an overarching authority that has re-
sponsibility for overall SoS requirements, but focus on the use of formal specifications
to evaluate functional decomposition and interoperability between legacy systems. In
one of the first books on SoS Simulation and Modelling Cantot et al. describes an SoS
as an assembly of systems which are independently acquired and then operated in
order to maximise the performance of the global operation of the grouped systems at
certain periods [Cantot and Luzeaux 2009; 2011].
With a focus on education Lukasik [1998] describes an SoS as a self-organizing sys-
tem assembled from multiple distributed individual systems, and assembly that has
not been directly designed; instead it follows from the evolution of the integration of
constituents. Chen and Clothier [2003] focuses on systems engineering practice, but
attempts to improve and adapt traditional methods by focusing on the design of the
environment in which the SoS constituents reside.
Keating et al. [2003] provide a perspective in which an SoS is described as a meta-
system of interrelated complex subsystems, constructed out of systems that integrate
in order to reach a high goal, despite the individual parts being unlike in technology,
geography and operation. In relation to this Crossley [2004] suggests that a change in
industry and defence acquisition, moving from specification of single systems towards
less implementation-specific specifications of capabilities, has allowed for more exist-
ing and future systems interacting to fulfil a common mission.
2.6. Views in Professional Handbooks
The US Department of Defense Systems Engineering Guide for Systems of Systems
defines an SoS as “a set or arrangement of systems that results when independent
and useful systems are integrated into a larger system that delivers unique capa-
bilities” [OUSD(AT&L), DoD 2008] (Originally from the Defense Acquisition Guide-
book [Department of Defense 2004]). SoS development involves the creation of sys-
tems, which are collections of legacy, evolving and new systems, that must have a high
degree flexibility and adaptability. Two of the main considerations are: (1) the lack
of authority over the constituent systems because of their independent management,
funding and objectives that may not align with those of the SoS as a whole. (2) The
emergent behaviour adding a large degree of unpredictability, as overall system be-
haviour cannot be predicted by having individual knowledge of each constituents.
In its most recent edition, the INCOSE Systems Engineering Handbook [INCOSE
2015] (an update over the previous 2011 edition [INCOSE 2011]) treats an SoS as a
system whose elements are managerially and/or operationally independent systems,
and which together usually produce results that cannot be achieved by the individual
systems alone. It cites Maier’s characterisation of SoSs, and further cites Dahmann’s
“pain points” [Dahmann 2014] as significant challenges facing SoS engineering: ab-
sence of common authorities; leadership in the SoS organisational environment; va-
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:8 C. B. Nielsen et al.
riety of constituent systems’ perspectives; management of diverse requirements and
the overall SoS capability; autonomy, interdependencies and emergence; and the chal-
lenges of validation, testing and learning. Security is further identified as a pressing
concern. The handbook observes that SoS engineering demands a balance between lin-
ear procedural methods for systematic activity and holistic nonlinear methods in the
face of complexity arising from SoS emergence.
2.7. Extensions of Existing Definitions
Krygiel [1999] define SoS as “a set of different systems so connected or related as to
produce results unachievable by the individual systems alone.”, which is an adaption
of Maier and Rechtin [Maier and Rechtin 1997]. Sage and Cuppan [2001] build on
Maier’s SoS characteristics to apply a strategy for organisational structuring, while
Bar-Yam et al. [2004] use military, biological and sociological case studies to supple-
ment Maier’s characteristics with a notion of interdependency arising from the in-
teroperability of constituents. Fisher [2006] builds on Maier’s characteristics to de-
scribe a class of large-scale software-intensive systems that exhibit levels of complex-
ity for which traditional engineering methods are likely to prove fruitless. Baldwin and
Sauser [2009] use Maier’s characteristics to define an SoS as “a type of system com-
posed of traditional systems and distinguished by the dynamic properties of autonomy,
belonging, connectivity, diversity and emergence”.
Cook et al. [1999] attempts to establish a methodology for developing military SoS
via a structure of different levels of complexity (essentially based on [Boulding 1956]).
SoSs are regarded as self-maintaining, evolving, have a large human element and,
while they are loosely organised, they have a strong interaction through which they
work towards a common goal.
Based on a survey of SoS definitions, Sharawi et al. [2006] propose an SoS charac-
teristics that are considered either essential or desirable with respect to the modelling
and simulation of SoS, for which essential concepts are independence, interoperability
and a global goal, while a characteristic such as distribution is considered desirable.
2.8. Other System Types related to SoS
Different kinds of systems and notions exist in the literature which bears a resem-
blance to SoS by their name, structures or characteristics but are not identified as
such.
Sage and Cuppan [2001] describes a Federation of Systems (FoS) as having many of
the same challenges as SoS, but they are much more heterogeneous with respect to the
“trans-cultural and trans-national socio-political” dimensions. Krygiel [1999] describes
a FoS as a type of SoS, but with a very high degree of heterogeneity, autonomy and
greater geographic distribution.
Ultra-large-scale systems (ULS) are described as having most of the characteristics
of an SoS, but in a much larger scale. Instead of being seen as an SoS they are regarded
as socio-technical ecosystems on a large scale, as they involve multifaceted interactions
between people and technology that reside inside an environment of massively com-
plex technical and managerial challenges, in which an “erosion of the people/system
boundary” will occur [Northrop et al. 2006].
A comparable system type is described in the UK-based Large-Scale Complex IT
Systems initiative that focused on the challenges arising from the integration between
large, complex IT system [Sommerville et al. 2012]. These systems consist of a col-
lection of new and existing independently owned and managed software systems that
work together, which raise high demands for the interactions between systems, organ-
isations and people. A Large-Scale Complex IT System consist of systems, potentially
systems of systems in their own right, that work together out of mutual interest but
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:9
are owned by individual organisations that may be competing. This means that they
would be categorized as a virtual SoS in the categorization presented in Section 3.10.
Sommerville et al. directly address virtual SoS that however is found unintuitive, as
“virtual” is found to be a too ambiguous a term and the “system of” in SoS is seen
to imply something that has been intentionally designed to perform a purpose in an
organization, using Checkland’s definition of a system [Checkland 1999]. Instead the
term a “coalition of systems” is used to describe systems that work together out of mu-
tual interest without being originally design for this purpose and often having hostile
relationships between stakeholders.
3. DIMENSIONS OF SOS
As Section 2 suggests, there is no single precise definition of SoS, but the literature of-
fers a rich set of descriptions of SoS properties that allow fine distinctions to be drawn
between SoS instances. Equally, these many approaches share common concepts. What
do these mean for the use of model-based techniques to develop and maintain SoSs and
constituent systems? We argue that the “space” of SoSs might usefully be described in
terms of several dimensions based on shared concepts that have a bearing on mod-
elling and analysis. The intention is that positioning an SoS engineering problem in
the space defined by these dimensions might suggest modelling patterns and analy-
sis approaches. In this section we propose eight such dimensions derived from terms
used in the literature, taking account of the contexts in which the terms have been
used. For example the term “independence” may denote independence of operations
for self-governing constituents [Maier 1996; Crossley 2004], the independence of ca-
pabilities in the terms of the variances in resources of constituents [Karcanias and
Hessami 2010] , or it can refer to independent optimisation of the constituents [Sage
and Cuppan 2001].
3.1. Autonomy of Constituents
Autonomy is the extent to which a constituent system’s behaviour is governed by its
own rules rather than by others external to the constituent. This is especially seen
as a result of individual ownership of the systems. The property of managerial inde-
pendence identified by Maier (Section 2.2) entails that constituents perform their own
functions in accordance with their own rules, while also participating in the SoS. Au-
tonomy of constituents is central to SoS Engineering as described by Keating et al.
[2003]. For Boardman and Sauser [2006], autonomy refers to the capacity of a con-
stituent system to pursue a specific purpose: constituents that were conceived as parts
that exhibit no autonomy are really enabling elements of the SoS, rather than (con-
stituent) systems in their own right. Cook [2001] acknowledges the need for indepen-
dence in Maier’s sense, and also identifies a requirement for constituents to be ‘pur-
poseful’ and set their own goals.
Given the heterogeneity of an SoS, there is likely to be considerable variation in the
autonomy exhibited by constituents. Modelling and analysis techniques need to permit
the expression of a range of actions that an autonomous constituent may perform, but
which may not be precisely predicted at the SoS level. This suggests that there is a
need for looseness or underspecification of constituent system behaviour.
3.2. Independence
Independence is the capacity of constituent systems to operate when detached from the
rest of the SoS. This is Maier’s ‘operational independence’ characteristic, also identified
by Krygiel [1999] as the capability for independent action and by Jamshidi [2008] as
the extent to which systems are ‘independently operable’. Independence of both design
and operation is key to the SoS definition offered by Sharawi et al. [2006].
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:10 C. B. Nielsen et al.
Independence implies that a given constituent system may offer a range of be-
haviours, some related to its role in an SoS, and others independent of it. The rela-
tionship between these classes of behaviour, and specifically the dependencies between
them, might be hidden from the SoS engineer. Model-based techniques therefore need
to be able to support information hiding.
3.3. Distribution
Distribution refers to the extent to which constituent systems are dispersed so that
some form of connectivity enables communication or information sharing. Distribution
may denote a geographical distance such as “arrays” of systems dispersed over wide
geographical areas as described by Shenhar [1994] and Maier’s geographical distri-
bution characteristic [Maier 1996]. In Kotov’s characterisation of SoS, it is clear that
distribution may refer to a network distribution of concurrent processes as well as
physical separation [Kotov 1997]. Manthorpe [1996], considering joint military opera-
tions, identifies the need to accurately spread data to thousands of locations enabled
by mobile platforms and sensors.
Modelling frameworks that support distribution require the ability to assign con-
stituent system processes to computational infrastructure, linked by a communication
medium. Descriptions of concurrency, communication, and particularly failures of com-
munication media, are necessary.
3.4. Evolution
Many SoSs are long-lasting and subject to change, whether in the functionality de-
livered, the quality of that functionality, or in the structure and composition of con-
stituent systems. Maier [1996] identifies evolutionary development as a key charac-
teristic, and Carlock and Fenton [2001] identify the lack of a permanent state of the
SoS. Carney et al. [2005] view evolution as taking place through a series of largely
deliberate preservative or adaptive interventions caused by, for example, upgrades to
constituent systems or a need to respond to an evolving environment, as identified by
Crossley [2004] and Despotou et al. [2003]. Abbott [2006] emphasises that an SoS is
“continually evolving, but slowly”. Bloomfield and Gashi [2008] observe that evolution
is “incessant”.
Model-based approaches to SoS engineering require support for gaining assurance
of the preservation of specified properties under evolution steps. We may characterise
this as a need for verification of conformance of a constituent system’s interfaces to
those of the other constituents with which it must interact. Evolution may be mani-
fested as updates to constituent systems, requiring re-verification of conformance.
3.5. Dynamic Reconfiguration
Dynamic Reconfiguration is the capacity of an SoS to undertake changes to its struc-
ture and composition, typically without planned intervention. Several authors identify
the ability to undertake this kind of real-time change as an important characteristic,
especially in ensuring resilience of an SoS to faults and other threats. Boardman and
Sauser [2006] identify the need for “dynamic determination of connectivity”, requiring
the autonomy of the constituent systems to deliver the functions required to disconnect
and reconnect constituent systems, and to modify interfaces. Crossley [2004] regards
SoS as dynamic entities, while Schneider and Trapp [2009] discuss approaches to the
use of runtime safety models to enable dynamic reconfiguration of open SoSs.
In contrast with evolution, which refers to the capacity to support planned changes
on a slow scale through intervention, this dimension refers to the technical abilities
an SoS has to change its composition during operation, such as on-the-fly swap-in and
a pluggable architecture. To support the Dynamic Reconfiguration, SoS models must
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:11
have abstractions for the dynamic modification of architectures and interfaces, and the
capacity to reason about such changing structures.
3.6. Emergence of Behaviour
Within the SoS literature, emergence refers to the behaviours that arise as a result of
the synergistic collaboration of constituents. Reliance is typically placed on the deliv-
ery of some emergent behaviour in order to deliver a higher functionality than deliv-
ered by the constituents separately. Several significant papers, including Maier [1996]
and Boardman and Sauser [2006], refer directly to the need for emergence, and Abbott
[2006] develops the characteristic of SoS being “open at the top” in this sense.
The reliance placed on emergence establishes important demands on modelling and
analysis methods. It is vital that global properties can be described at the SoS level;
it will often be the case that an emergent property on which SoS stakeholders depend
can only be sensibly articulated at the SoS level, and not at the level of constituent
systems. Modelling and analysis tools should permit the statement and verification
of emergence, and permit identification of emergent behaviour that one would like to
avoid, such as feature interactions [Zave 1993].
3.7. Interdependence
Interdependence refers to the mutual dependency that arises from the constituent sys-
tems having to rely on each other in order to fulfil the common goal of the SoS. If the
objective of a constituent system depends on the SoS, then the constituent system it-
self may have to sacrifice some of its individual behaviour in order to meet the require-
ments of joining the SoS. Examples of terms that have been joined under “Interde-
pendent” are; interrelationships [Krygiel 1999], interdependencies [Sage and Cuppan
2001], interdependency [Bar-Yam et al. 2004] and belonging [Boardman and Sauser
2006].
Including both “Independence” and “Interdependence” may appear contradictory.
However, some authors take the view that an SoS requires trade-offs between the
degree of independence in the constituent systems and the interdependence required
to reach the common goal [Sage and Cuppan 2001; Bar-Yam et al. 2004]. So while
the individual constituent systems are independent, the relations and interoperability
between requires some degree of interdependence.
Modelling and analysis techniques should allow the explicit identification of inter-
dependence, the tracing of mutual dependencies, and the ability to use these links to
assess the impact of constituent system changes.
3.8. Interoperability
Interoperability refers to the ability of the SoS to incorporate a range of heterogeneous
constituent systems. This involves the integration and adaptation of interfaces, pro-
tocols and standards to enable bridging between legacy and newly designed systems.
The interoperability concept appears in the literature in the notions of simultaneous
functioning [Shenhar 1994], integration of capabilities [Manthorpe 1996], interoper-
ability and integration [Krygiel 1999], heterogeneity [Carlock and Fenton 2001; Cook
2001], “open at the bottom” [Abbott 2006] and diversity [Boardman and Sauser 2006].
The need for interoperability places several requirements on modelling and analysis
methods; it reinforces the need for techniques supporting the verification of confor-
mance of constituent system interfaces. Models of SoS exhibiting a need for interoper-
ability are likely to incorporate heterogeneous models of the constituents. Mechanisms
for ensuring semantic consistency of diverse models, and rigorous analysis of those
very distinct model types, are required in order to meet the needs for verification of
both emergence and conformance.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:12 C. B. Nielsen et al.
3.9. Mapping Concepts to SoS Dimensions
Table I shows how the eight dimensions are mapped to the descriptions, definitions
and characteristics the original authors have used to perceive and explore the SoS do-
main. The purpose of the mapping is to visualise the distribution of the dimensions in
Table I. Mapping Concepts to SoS Dimensions
Author(s)
Described in Section
Autonomy
Independence
Distribution
Evolution
Dynamic Reconfiguration
Emergence of Behaviour
Interdependence
Interoperability
Boulding 1956 2.1 • • • •
Ackoff 1971 2.1 • • •
Eisner et al. 1991 2.5 • • • •
Noam 1994 2.4 • • • • •
Shenhar et al. 1994 2.3 • • •
Manthorpe 1996 2.4 •••
Maier 1996 2.2 •••• •
Kotov 1997 2.4 • •
Lukasik 1998 2.5 • • • •
Krygiel 1999 2.7 • • • •
Roe 1999 2.5 • • • •
Cook et al. 1999 2.7 • • • • • •
Pei 2000 2.4 • • •
Carlock et al. 2001 2.4 • • • • •
Sage et al. 2001 2.7 •••• •
Chen et al. 2003 2.5 •• •••••
Keating et al. 2003 2.5 •• •••
Bar-Yam et al. 2004 2.7 • • • • •
Crossley 2004 2.5 • • • •
DeLaurentis et al. 2005 2.3 •••••••
Abbott 2006 2.2 • •
Boardman et al. 2006 2.2 • • • • • •
Cocks 2006 2.4 • • • •
Boehm 2006 2.4 • •
Fisher 2006 2.7 •••• ••
Sharawi et al. 2006 2.7 •••••• •
DoD SE Guide for SoS 2008 2.6 ••••••••
Karcanias 2010 2.3 •••• ••
INCOSE 2015 2.6 ••••••••
Count – 16 17 19 21 10 22 12 20
relation to both author and year. The mapping shows that some dimensions are used
more than others, with “Evolution” and “Emergence of Behaviour” as the most fre-
quent. This result is not unexpected as both are matters that are difficult to grasp and
for which there is still limited knowledge in a systems development context. There-
fore, they get more attention in defining publications, while areas such as “Dynamic
Reconfiguration” and “Distributed” systems are more researched. This however does
not mean that the other dimensions are not important in an SoS context, instead it
should be seen as an indicator of how they are regarded. For instance “Evolution” and
“Emergence of Behaviour” are properties sought for or managed in SoS engineering,
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:13
while “Independence” and “Autonomy” can be seen as properties of the constituent
systems, indicated by their ownership.
The mapping also shows diversity, as only a few authors have the same marks for
the eight terms. This might suggest vagueness in SoS as a concept. Looking at the
literature there is some truth to this, but it is also worth mentioning that it seems to
be the tendency that the newer publications include more of the eight dimensions than
earlier publications, and that the engineering handbooks contain the vast majority of
the eight. A reason for this diversity can be that the SoS literature derives from multi-
disciplinary collection of researchers, developers and managers, each approaching the
SoS field from their specific background. Some take an operational point of view, others
a focus on management, while this survey has a modelling perspective.
This survey has studied a large part of the literature in the search for SoS charac-
teristics, as seen from a modelling perspective. This does not entail that the results in
Table I cannot be used by other perspectives or as an identification of SoS in general.
It is merely important to bear in mind in what perspective the table was created. The
eight dimensions in Table I are not intended to define a boundary separating SoSs from
non-SoSs, but allow an individual candidate SoS to be characterised in a way that may
be useful in determining the model-based engineering techniques that are applicable.
In the remainder of this paper, these dimensions are used to structure the discus-
sions on current practice, research challenges and the anticipated steps needed for a
strengthened SoS engineering discipline. This is done within the four areas of mod-
elling and architectures, simulation, testing and verification. The eight dimensions
will be used as reference points to ensure that they are considered with relation to
each of these four areas.
3.10. Categorisation
The US Department of Defense makes use of four categories of SoS [OUSD(AT&L),
DoD 2008]:
Directed. The SoS is built to fulfil specific purposes. Constituent systems have the
ability to operate independently, but are managed to satisfy a concrete purpose.
Collaborative. The constituent systems are not compelled to follow a central man-
agement, but voluntarily participate in a collaboration to fulfil the goal.
Acknowledged. The SoS recognises a common purpose and goal, while the con-
stituent systems retain independent control and objectives. Evolution of the com-
mon purpose is based on collaboration between the SoS and the constituent sys-
tems.
Virtual. The SoS is without either managerial control or a common purpose. This
makes the behaviour and the fulfilled goals highly emergent, but also entails that
the exact means and structures producing the system functionality are difficult to
discern and distinguish.
Three of these categories were originally defined by Maier [1996], while the “Acknowl-
edged” type was later proposed by Dahmann and Baldwin [2008]. Maier argues that
SoSs should be considered equivalent merely because they have a similar complexity
and scope. A categorisation is required in order to guide the selection of architect-
ing principles. The categories are based on the degree of managerial control because
this determines how adaptable and cooperative each constituent system will be with
respect to requirements, interfaces, data formats and technologies. In turn, this influ-
ences the challenges faced when constructing the SoS.
In practice, the “Directed” SoS embodies a form of planned emergence because the
constituent systems are centrally managed. The other types of SoS have little or no
centralised managerial control. The “Collaborative” type has the notion of a centralised
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:14 C. B. Nielsen et al.
management but with very limited or no powers to enforce decisions, while the “Vir-
tual” type is without any degree of management. Dahmann and Baldwin [2008] adds
the “Acknowledged” type to describe the scenarios found in many military systems.
This type is focused on establishing collaborative management at the SoS level, while
keeping the managerial and technical independence at the constituent level. The goal
is that autonomy and ownership are maintained, while at the same time ensuring that
changes can be collaborative and decided upon on the basis of some common objectives.
In practice, one would not expect the SoS type to be uniform within an SoS, but local
subsystems of differing types might arise. Indeed the heterogeneity of constituent sys-
tem ownership could be expected to lead to a wide range of (possibly inconsistent)
stakeolders’ views of the levels of control actually offered within an SoS.
We hypothesise that differing degrees of control might be reflected in relatively
stronger or weaker models of constituent system interfaces. Directed and collaborative
SoSs would require a relatively strong specification of a central decision-making au-
thority. Whilst work such as that of Ingram et al. [2014] draws some correspondences
between modelling patterns in SysML and SoS types, a theory of SoS types embodied
in rigorous models remains to be developed.
4. ILLUSTRATIVE EXAMPLES
The literature on Systems of Systems Engineering (SoSE) covers a wide range of appli-
cation domains. In this section, we review a few applications described in the literature
in order to illustrate the SoS domain.
4.1. Transportation
Transport networks are often composed of independently owned and operated systems
that are geographically distributed. DeLaurentis [2005] illustrates how transportation
can be seen as an SoS covering all the eight dimensions from Section 3, emphasising
the evolutionary nature and the emergent properties as the main reasons why the
transportation sector needs SoS modelling and analysis techniques.
Air traffic management systems are often cited as examples of de facto SoSs. For
example Ball Sr [1997] and Geddes et al. [1998] consider the movement to a “Free
Flight” model of air traffic operations in the US as a transition from a directed to a
more distributed and collaborative SoS structure. Air traffic management routinely
has to balance competing concerns that dominate to different extents among stake-
holders. It is argued that these changes necessitate explicit mechanisms for the fol-
lowing: promoting initiative, sharing of purpose, situation and planning; and efficient
communications. It is argued that the current barriers to collaboration suggest that ex-
plicit support for the transition is required, and that a system of “associates” following
explicit sharing protocols can help to resolve them.
Mansouri et al. [2009] present a systematic discussion of the derivation of a frame-
work for the management of maritime SoS for the US Maritime Transport Sys-
tem (MTS). The MTSoS is represented in terms of five ’ABCDE’ characteristics derived
from Boardman and Sauser [2006]. Opposing forces are considered on each dimension.
For example, the ‘Autonomy’ dimension represents a tension between conformance and
independence; ‘Belonging’ balances centralisation and decentralisation; ‘Connectivity’
ranges from platform-centric to network-centric; ‘Diversity’ from homogeneity to het-
erogeneity; and ‘Emergence’ ranges from the foreseen to the indeterminable.
4.2. Smart Energy Grids
Smart grids integrate data on energy supply and consumption in order to deliver elec-
tricity in an cost-effective manner. There is a need to provide assurance of safety, secu-
rity of supply, and in doing so there is a significant engineering challenge in balancing
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:15
dynamically changing availability and price from independent and autonomous sup-
pliers against varying demand [European Commission 2012]. There have been some
implementations of smart grid technology, such as the Smart Grid/Distribution Net-
work Management System developed as part of the Customer-led Network Revolution
project in the UK8, which integrates computation and network technology on a power
generation and distribution network with multiple independent energy suppliers and
consumers.
From the perspective of model-based approaches, the heterogeneity of smart grid
systems is striking. Smart grids are cited as examples of cyber-physical systems re-
quiring novel combined models, for example in Dillon et al. [2011] and Ili´
c et al. [2010].
Miller et al. [2012] use simulation to conduct a sensitivity analysis based on a model
that captures the physics of demand response and the behaviours of humans both as
individuals and in networks. Agusdinata and DeLaurentis [2008] discuss the role of
models in policy-making for the energy sector, illustrating the trade-off between the
levels of resolution afforded by models and the ease with which high-level views of SoS
behaviour can be obtained. However, few if any model-based studies in SoS link all
three types of element: cyber, physical and human.
4.3. Emergency Management and Response
The agility required of an emergency response SoS, and the often unique circumstances
of each substantial emergency, lead to astronomically large models. In order to man-
age this, Liu [2011] proposes an hierarchical SoS for emergency response in China.
Even with such an hierarchical approach, modelling of such a complex structure is
only at an early stage. In more constrained environments, simulation can be used to
analyse SoS successfully. For example, Mahulkar et al. [2009] report the construction
of a substantial MATLAB-based model of an SoS on board a naval vessel in order
to explore the consequences of changes in the on-board technology using simulation
based on emergency response scenarios. Emergency response technologies, based on
networked sensors and actuators, provide a significant application area. Daniel et al.
[2009] present an architecture for swarms of micro-UAVs sensing and mapping toxic
atmospheric emissions following an incident such as an explosion or fire.
4.4. E-commerce
An example of SoS that many people encounter on near daily basis is E-commerce, the
buying and selling of goods and services via online shopping. E-commerce involves a
number of independent different systems working together in order to reach the overall
goal of sale and delivery. These systems perform functions that provides the virtual
market place, the handling of payment transactions, management of inventory and
supply chains, as well as handling shipping and delivery. [Ricker and Kalakota 1999].
Making all of these play well together requires an infrastructure that contains the
dimensions of a SoS. One of the premier E-commerce companies, Amazon.com requires
a highly scalable platform that can provide the performance, reliability and efficiency
needed to support millions of customers [DeCandia et al. 2007]. Amazon’s consists of
highly decentralized architecture consists of hundreds of services and systems that
need to integrated as seamlessly as possible.
Fig. 1 illustrates the multitude of systems involved in an E-commerce. The right
side of the dotted line contains the systems that are under the e-commerce company’s
ownership, while the left shows systems that are owned by other stakeholders. Large
8See http://www.networkrevolution.co.uk/. The network management system operates areas of network from
deployed on networks serving about 20,000 customers, using 17 smart enabled network interventions, as
well as central and distributed controllers.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:16 C. B. Nielsen et al.
Fig. 1. Illustration of constituent systems in an E-commerce business
E-commerce companies will have a number of fulfilment centres to physically handle
inventory, picking and packaging, as well as multiple data centres to deliver the data
storage and processing power required.
In order to run everything from sales, pricing to warehousing as well as handling a
large number of suppliers, the E-commerce companies have a wide range of systems
that are under their ownership. All of these systems, which may either be developed in-
house or by a third party, all work towards the same goal and the integration between
them are key to the efficiency of the company. They do not, however, express all of
the dimensions of an SoS. Having ownership of their systems enable the companies to
have better control of the development strategies, a greater degree of flexibility and
faster decision-making than they have with systems owned by third parties. There
are still vital systems that will be independently owned by other businesses, such as
payment transactions as well as the systems of suppliers and shipping companies.
This means that e-commerce companies are dependent on distributed development
strategies planned for the individual systems and on the quality of relationships and
communication between systems owners. This is where a combination of a multiple
SoS dimensions makes the development much more challenging.
4.5. Observations
The SoS engineering literature covers a very wide range of application areas be-
yond those illustrative examples mentioned above, including communication sys-
tems [White and Jean 2011] and healthcare [Wickramasinghe et al. 2007]. It is worth
noting that SoSs are not always explicitly identified as such. Many of the examples in
the literature illustrate “SoS Thinking” for the derivation of management frameworks,
rather than the technological activity aimed at developing ICT artefacts like networks,
machines or software. We find comparatively few accounts of applying model-based
SoS engineering techniques ‘in the wild’; the vast majority of application reports re-
main at the proof of concept or pilot study stages.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:17
5. STATE OF THE ART
Having reviewed a range of systems that form the subject of the SoS literature, we
focus on the state of the art in model-based techniques. In this section, we examine
the state of the art in modelling and architectural description (Section 5.1), simula-
tion (Section 5.2), testing (Section 5.3) and verification (Section 5.4).
5.1. Modelling and Architecture
As indicated in Section 1, modelling of systems, or specific aspects of systems, has been
carried out for many years in many different disciplines, and this has naturally spread
to SoS engineering [Cantot and Luzeaux 2011]. Models can be developed at a range of
abstraction levels, depending on their purpose and the forms of analysis that are to be
performed on them [Kramer 2007]. In order to gain the maximum engineering value
from model-based methods, there need to be well-defined relations between models and
their realisations. We will say that an implementation Irefines a model M, written
MvIif the properties shown to hold of Malso hold of I.
Models can be expressed in many forms ranging from graphical sketches or text to
mathematical formalisms. In this paper, we will pay particular attention to models
that are expressible in a form that can be given an unambiguous semantics to permit
machine-assisted confirmation or refutation of properties of interest. The state of the
art of model-based SoS engineering is generally such that models are developed using
existing notations and tools tailored to the application at hand. There have been few
attempts to define general–purpose languages for model-based SoS engineering [Lud-
wig et al. 2011], although Woodcock et al. [2012] have proposed a formal language for
expression of refinable models of SoSs. The difference between a formal language for
expressing refinable models of Sos, as distinct from a language for expressing refinable
models of systems, is that the former must directly address the different dimensions
of SoS described in Table 3.9.
5.1.1. Enterprise Architecture. In the context of this paper, the term ‘architecture’ covers
not only structural properties such a system hierarchies and interfaces, but also func-
tional properties that may be affected by architectural decisions and changes. In the
existing literature the direct usage of the terms “architectural models” and “architec-
tural modelling” in relation to SoS is rather sparse. Only a few authors have a direct
focus on SoS architecture, and mostly in relation to Enterprise Architectures [Lucke
et al. 2010].
Different Enterprise Architecture frameworks have been developed over the years,
including DoDAF and MODAF, originally to support the system engineering discipline.
They are used to create representations of large enterprises and to create methodolo-
gies for capturing their structure and dynamics. Looking at the current state of Enter-
prise Architectures a number of authors aim at generalising such enterprise architec-
tures to make them applicable to the engineering of SoSs [Carlock and Fenton 2001;
Harmon 2005]. However, in general these papers do not take into account the real
challenges of engineering SoSs, indicated by the SoS dimensions listed in Section 3.
5.1.2. Architectural Frameworks. Looking outside the scope of Enterprise Architectures,
Kilicay-Ergin and Dagli [2008] see a limitation in using static frameworks and method-
ologies for architecture development of SoS. These do not provide a way to analyse
dynamic evolution of system state or behaviour, and consequently they point to a shift
towards executable models. Here they take our Autonomy and Evolution dimensions
into account. In conclusion Kilicay-Ergin et al. determines that simulation tools are
needed that combine both structural and executable models, in order to include all of
the behavioural views needed to comprehend an SoS.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:18 C. B. Nielsen et al.
Selberg and Austin [2008] describe the evolution from systems to SoSs and the as-
sociated unmanageable increase of complexity. Here the SoS characteristics Evolution
and Emergence of Behaviour are considered explicitly. This paper demonstrates how
using more standardised interfaces will prepare better for the integration of new con-
stituent systems in the future. In addition, there is a recommendation for the use of
general purpose formal models supporting abstraction and analysis capabilities. How-
ever, there is still a lack of formal methods supporting proper engineering of SoS.
5.2. Simulation
One of the most frequently used forms of model analysis is execution in some form.
Typically, this is called model simulation, indicating that it is an approximation of how
the real system would behave in the same scenario. In the existing literature on model
simulations of SoSs, the focus is primarily on using the models for training purposes
for different kinds of personnel. Because of this, the majority of the publications on
SoS simulation are also focused on the distribution of simulators connected together
and abilities to make use of agent-based systems (typically used to simulate different
kinds of people operating the systems).
5.2.1. Agent-based Simulation. Agent-based technology is typically included when it is
desired to include the human in the simulation loop [Axelrod 1997]. Thus, such intel-
ligent agents are primarily used to explore socio-technical aspects in an SoS setting.
This has also been used to explore different design choices for internal interoperability
in a ship environment [Mahulkar et al. 2009]. Gutierrez-Garcia et al. [2009] enhance
the agent-setting with formally expressed constraints with an emergency management
case study. These agent-based technologies support the Autonomy dimension very well.
5.2.2. Focused Simulation. Distribution aspects in a simulation setting typically focus
on interoperability between simulations of different constituent systems. Here, the
majority make use of High-Level Architecture (HLA) simulation interoperability [Lees
et al. 2007], which has been adopted as an IEEE standard [IEEE1516 2010]. HLA has
primarily been used for high-fidelity, defence-related simulations coupling different
existing simulators together to examine, for example, different possible war scenar-
ios. However, HLA has also been used in an SoS setting for exploring design choices
around the weight of an aircraft [Sharawi et al. 2006]. HLA primarily improves the
Interoperability dimension of SoS.
Within the area of modelling and simulation, a large part of the existing literature
is military focused. For example, the US Future Combat System (FCS) has been mod-
elled and simulated using a specially developed SoS Application Toolkit (SoSAT) for
military missions [Campbell et al. 2005; Eddy et al. 2007]. This simulation uses State-
mate [Harel 1987] and fault tree analysis to conduct probabilistic simulations focusing
on potential faults appearing in constituent systems.
In other works, professional simulation engines, such as ExtendSim9have been
used [Jian et al. 2010]. These works are primarily focussed on limiting the SoS from
undesirable Emergence of Behaviour. Jian et al. uses the simulation engine for the
evaluation of SoS architectures using a two-level process. The higher level is focused
on capability requirements and on computing the SoS overall measure of performance.
At the lower level the quality of the capability requirements is obtained by simulation
of the SoS. The model simulates the interactions between the component systems in
order to assess the components’ ability to meet the specified capability requirements.
9www.extendsim.com
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:19
5.2.3. General Simulation. A few examples can be found that attempt to create a gen-
eral modelling and simulation approach aimed directly for SoS. One such example can
be found in [Kotov 1997], where a C++-based library is presented for modelling and
simulation in an SoS setting. Here, constituent systems are hosted at multiple com-
puting nodes and the communication between these is incorporated. A similar generic
approach extending the Unity language to an SoS setting can be found in [Gamble
and Gamble 2008]. All of these works can be seen as enhancing the Interoperability
between different models of constituent systems.
Sahin et al. [2007] presents a framework for the architectural representation and
simulation of an SoS based on Discrete Event System Specification (DEVS) and the ex-
change between constituent systems defined in XML. DEVS is also used by Berenji and
Jamshidi [2011] in a fuzzy-control setting for independent vehicles acting in a swarm
context. DEVS is a formalism for describing and simulating hierarchical systems via
components and their interconnection including the run-time addition/removal of cou-
plings and components. Support for HLA is also available. The use of DEVS in an SoS
setting is explained in [Mittal et al. 2009]. This is an example of a practice aiming at
increasing the Interoperability between models of constituent systems in an SoS, as
well as investigating the Dynamic Reconfiguration of the system.
SoS simulation capabilities using formal models of SoSs at a higher level of abstrac-
tion is found in the CML tool, Symphony [Coleman et al. 2012]. Ideas are also present
to cope with simulations of evolution of SoSs dynamically [Nielsen and Larsen 2012].
5.3. Testing
5.3.1. Compositionality Versus SoS System-Level Testing. Considering the time and effort
required for SoS system-level testing, it is a scientifically valid question whether it
could be made redundant by means of V&V activities performed on the constituent
systems that are part of the SoS: the concept of compositionality has been elaborated
in the field of formal methods and specifies conditions allowing to deduce emergent
properties of the complete system from the “local” properties of its constituents [Hoare
1978; Roscoe 2010]. Indeed, the distribution and local autonomy of constituent sys-
tems facilitate the application of compositional arguments, because prerequisites like
absence of shared resources (e.g., variables, processors) needed to apply such an argu-
ment are generally fulfilled.
These considerations, however, obviously contradict practical experience with SoS
testing on system level, where it quite frequently turns out that the composed sys-
tem does not fulfil its expected emergent properties. This experience is not caused by
the preconditions for the compositionality argument being violated. Instead, there are
quite different reasons for SoS to fail on the system level:
(1) Crucial non-functional emergent properties – in particular, safety and security
[Leveson 1995] – are non-compositional. As a consequence, the combination of safety
or security mechanisms does not necessarily lead to a safe or secure system.
(2) Constituent systems frequently show a lack of quality at the point in time when
they are first delivered for the purpose of integration testing, so system level tests
simply fail because some constituent systems do not meet their specifications.
(3) Constituent systems may show erroneous behaviour when operating in an SoS
configuration, due to undocumented assumptions made by the sub-system suppliers,
which are not fulfilled in the SoS configuration.
(4) System level tests fail because emergent properties have been insufficiently cap-
tured during the requirements elaboration phase.
It is the purpose of SoS system-level testing to reveal these deviations, as will be
described in the paragraphs below (see also [Peleska 2013]).
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:20 C. B. Nielsen et al.
5.3.2. Coordination of Testing Activities. Using the terms of the military domain, testing
activities of constituent systems may be structured into developmental test and eval-
uation (DT&E) which is a verification activity, and operational test and evaluation
(OT&E) which is a validation activity [OUSD(AT&L), DoD 2008, p. 43]. As empha-
sised in [OUSD(AT&L), DoD 2008, p. 11] and [Colombi et al. 2008], the managerial
independence between constituent systems does generally not allow to synchronise
life cycle activities between them. As a consequence, system integration testing on SoS
level cannot rely on all constituent systems being ready for this task at a given point
in time. The problem becomes even more severe if some constituent systems are mem-
bers of more than one SoS: at the point in time a system test is to be performed for
SoS 1, a constituent system may be occupied with changes targeted at its operation in
SoS 2 configurations. As a consequence, a synchronisation point where all constituent
systems of a SoS are in a baselined state that is ready for system integration testing
might never be reached.
This problem can be mitigated by interoperability test campaigns. Interoperability
testing verifies two or more constituent systems with the following objectives:
— Ensure their basic capabilities to exchange data over the intended interfaces.
— Verify that the synergetic properties expected from this cooperation are realised in
conformance with the SoS requirements.
The latter V&V activity is called end-to-end testing, since SoS functionality is inves-
tigated along the complete processing chain, from the initiating constituent system to
the systems supporting and finally to those utilising the established results.
In a systematically structured SoS test campaign it is first ensured that constituent-
level tests have been successfully completed to ensure systems comply with their
specifications. In particular, it should be ensured that conformance tests have been
performed in order to verify that each constituent system conforms to the applicable
(communication and/or functional) standards. Acceptance testing should ensure that
system-specific functional, structural and non-functional properties are fulfilled. Then
interoperability testing investigates whether the constituents cooperate adequately to
ensure the emergent SoS properties required.
5.3.3. Coping with complexity. Liang and Rubin [2009] address the combinatorial ex-
plosion problem caused by the size of SoS state and input vectors by adopting the
concepts of pairwise testing and orthogonal arrays for SoS system-level testing. These
concepts have been widely applied in software testing. Pairwise testing with orthogo-
nal arrays advocates test data generation according to the strategy by Tatsumi [1987],
which goes back to Taguchi’s original ideas on robust design [Taguchi 1987; Phadke
1989]: (1) Identify the input and state parameters influencing the System Under Test
(SUT) behaviour (these parameters are called factors in the Taguchi Method). (2) Parti-
tion each single input or state vector component into equivalence classes (called levels
in the Taguchi Method). (3) Select level combinations, that is, vectors of values, each
taken from an equivalence class of the associated factor.
The selection of level combinations is typically performed using the orthogonal array
approach. This is a method for selecting level combinations which are balanced in the
sense that all combinations of a given dimension (n-tuples of levels associated with
nfactors) occur an equal number of times. Typically, the orthogonal array method
is applied to pairs of different factors, so that for each pair the associated levels are
exhaustively combined, and each pair is exercised the same number of times (n= 2).
As may be expected (also in the light of several critical evaluations of the method,
such as [Bach and Schroeder 2004]), pairwise testing does not solve all complexity
issues in SoS testing, as will be discussed in more detail in Section 6.4.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:21
In the light of SoS characteristics (see Table I), Autonomy as well as Independence of
constituent systems is reflected by independent developmental and operational tests.
Conformance and Interoperability testing addresses the distributed nature of SoS. Evo-
lution of constituent system functions are addressed by coordinated constituent-level
and SoS-level testing campaigns, as long as the constituent system functionality af-
fects the SoS under consideration. Emergence of Behaviour is checked by means of
end-to-end tests.
5.4. Verification
The literature on verification of SoSs is rather thin, reflecting the fact that research
in this area is still in its infancy. However, there is an interest in introducing formal
verification at different levels of software architectures in an SoS setting [Michael et al.
2009]. In this section we discuss why this is the case, given that systems development
is relatively well–developed in comparison, as we review the key developments in SoS
verification.
5.4.1. Run-time Execution Monitoring. There are two general strategies that can be used
to verify a system: the first is to check in advance that the system has desirable prop-
erties; the second is to wait until run-time and check that an actual execution of the
system performs satisfactorily. Run-time Execution Monitoring (REM) is in the latter
category and is representative of methods for tracking the behaviour of an underlying
application. It ranges from the analysis of simple audit logs through to sophisticated
checking of states and transitions against formally specified assertions. Example uses
include the verification of the NASA Deep Impact fault-protection engine [Drusinsky
2003] and the verification of the US Ballistic Missile Defence System battle man-
ager [Caffall and Michael 2004]. REM was chosen in the latter case because of its
ability to scale and its support for real-time assertions. Much of the research on REMs
involves temporal logic for specifications [Drusinsky 2003; Havelund and Rosu 2001]
and there are both continuous [Maler and Nickovic 2004] and discrete time applica-
tions [Drusinksy and Shing 2005].
5.4.2. Behavioural and Stochastic Verification. A key difficulty in SoS development and de-
ployment is to verify required overall properties while individual component systems
are heterogeneous and autonomous. Individual component systems tend to be extraor-
dinarily large and diverse and an SoS relies for its successful operation on Emergence
of Behaviour and feature interactions. It must be possible to predict the objectives of
an SoS in a dependable way, in spite of different and potentially conflicting local goals.
[Calinescu and Kwiatkowska 2010] suggest that formal analysis and verification, in-
cluding model checking, quantitative model checking, and quantitative analysis and
verification, are key technologies for the verification of SoS policies, contributing to
SoS dependability management and assurance. They envisage a reconfigurable policy
engine that uses online formal analysis and verification techniques for the implemen-
tation of a wide class of autonomic computing policies [Calinescu and Kwiatkowska
2009].
5.4.3. Contract-based Specification. Payne and Fitzgerald [2010] argue that explicit con-
tracts should be recorded at the boundaries between constituent systems in order to
verify SoS-level properties. They survey different approaches to contract-based speci-
fication, both for behavioural and non-functional purposes, although they concentrate
on the latter. Raising those results to an SoS-level has also started [Payne et al. 2012].
In the design-by-contract paradigm, the emphasis is on specifying the interfaces be-
tween components, usually involving preconditions, postconditions, and state invari-
ants [Meyer 1992] to document assumptions and commitments. The contract is this: if
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:22 C. B. Nielsen et al.
the interface user guarantees to meet the precondition, then the interface implementer
guarantees to meet the postcondition. The implementer’s guarantee might be partial
correctness (if the interface operation terminates, then it is guaranteed correct), or it
may be total (the operation will terminate and be correct). Invariants are used to docu-
ment an abstract model satisfying these constraints. More sophisticated contracts deal
with concurrency and shared resources. For example, rely and guarantee conditions
may be used to document progress through intermediate states as well as reaching the
final state [Jones 1983b; 1983a]. Reactive systems may be specified using reactive pre-
and postconditions, where special auxiliary variables are used to record the history of
past interactions and the readiness of current events [Woodcock and Cavalcanti 2001;
2002].
Beugnard et al. [1999] describe contracts at architectural levels, suitable for describ-
ing SoSs. In their work, contracts are structured into four layers: a) The syntactic layer
describes operation signatures. b) The behavioural layer describes the effect of oper-
ations using preconditions and postconditions. c) The synchronisation layer describes
real-time scheduling of component interactions and message passing. d) The quality of
service layer describes non-functional aspects of operations.
5.4.4. Verifying Emergent Properties. The interaction of multiple system components
sometimes leads to the emergence of unexpected properties as the aggregation of dif-
ferent features react in unpredictable ways. Such as in a basic telephone system where
a telephone call involves precisely two parties, the initiator and the recipient. An ex-
tension to conference calling would allow many parties to enter and leave a call. An
undesirable emergent property is that there may be no one responsible for paying for
the call if the initiator is allowed to leave. This is known as weak emergence [Bedau
1997]; other examples occur in natural complex systems, including ant-foraging and
bird-flocking; it is a significant technical challenge to model such systems so as to re-
veal them. A classic example of weak emergence occurs in Conway’s Game of Life,
where global behaviours arise as the result of purely local rules. The game is played by
a cellular automaton with four rules for simulating a population of entities distributed
across the cells, whose life and death depends on the occupancy of adjacent cells. Po-
lack and Stepney [2005] argue that the weakly emergent behaviour of the well-known
glider pattern, made up from several entities, cannot be shown to be refined by the
local rules governing the life of entities in cells. Sanders and Smith [2012] show that it
can. Sanders & Smith consider a different perspective on this problem: instead of try-
ing to discover emergent properties of existing systems, they consider the development
of an implementation with a required emergent property. Engineering emergent prop-
erties is seen as the software engineering activity of refinement: taking an abstract
property and realising it in terms of local behaviours and component interactions.
5.4.5. Verifying SoS Dimensions. As noted at the beginning of this section, formal verifi-
cation of SoSs is still in its infancy. This means that there is no best practice available
for most of the dimensions mentioned in Table I; however, formal methods do already
have something to say about these dimensions. As we have seen, autonomous sys-
tems can be verified using model-checking and emergent behaviour using refinement.
Research at the systems level suggests that the best way to tackle independence, in-
terdependence, and interoperability is through compositional verification techniques,
whilst distribution, dynamic behaviour, and evolution can be modelled using process
algebra, Petri nets, and temporal logic and verified using theorem proving and model
checking. Theorem proving and model checking has been developed for the SoS specific
modelling language CML [Coleman et al. 2012; Couto and Payne 2013].
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:23
6. CHALLENGES IN SYSTEM OF SYSTEMS ENGINEERING AND STRENGTHENING THE
DISCIPLINE
Having identified the state of the practice in model-based techniques for SoS, the fo-
cus is shifted towards the research challenges in each of these areas and on how the
challenges can be faced by examining the steps necessary to provide a strengthened
foundation for model-based SoS engineering.
In this section a short review of the literature on SoS challenges is presented (Sec-
tion 6.1), followed by an examination of the challenges in modelling and architectural
description (Section 6.2), simulation (Section 6.3), testing (Section 6.4) and verifica-
tion (Section 6.5).
6.1. Research Challenges for SoS in the Literature
A research agenda for SoS architecting is presented by Valerdi et al. [2008], with ten
major research challenges linked to academic and industrial problems. The work fo-
cuses on the entire life–cycle of an SoS and includes considerations from e.g. buyers,
developers, and maintainers. Some of the challenges include: (a) Evolution, research is
needed to develop methodologies that can deal with evolving and emergent behaviours
in SoS that dynamically adapt and absorb deviations in the system structure and (b)
“System vs. SoS attributes” that denotes a range of trade-offs between attributes such
as adaptability and modularity that can be simulated in SoS architecture models.
A different set of research challenges are presented by Maier [2005], that are fore-
most focused on the representation and analysis of SoS. Methods and tools that can
be used for describing and analysing the “upper layers” of SoS are wanted. The upper-
layers refer to interactions among network elements where division of functionality
and the interaction between systems are particular challenges.
The wish for better methods is also included by Ring and Madni [2005], who opt for
better theory, methods and tools in order to anticipate unintended behaviour in SoS
operation.
These research challenges match well with those being proposed for system types
that resemble SoS, such as for Ultra-large-scale systems and Large-Scale Complex IT
Systems. Northrop et al. [2006] see challenges in topics such as orchestrating activities
between diverse stakeholders, in measuring and evaluating the effectiveness of system
design, and in creating adaptive system infrastructures. This calls for more research
in how to establish stronger support for system design at many levels of abstraction
and in increased computational engineering through automated tool-support for as-
sessing the behaviour of evolving system compositions. Sommerville et al. [2012] com-
plements this list by proposing research challenges related to: the knowledge sharing
between dispersed stakeholders, the creation of adaptable models based on real-time
system monitoring data, handling resilience and recovery from failure, and on per-
forming verification of systems with no firm specification. These research topics are
all directly applicable to SoS; however Northrop et al. and Sommerville et al. have a
stronger emphasis on research related to socio-technical systems, human interaction
and the handling of acquisition and regulation policies, than what is seen in most SoS
publications.
6.2. Modelling and Architecture
One of the main challenges for modelling languages aimed at SoSs is to ensure that
they have well-founded semantics; as many of the modelling languages in the current
practice do not [Fitzgerald et al. 2012]. In order to strengthen the discipline of the SoS
engineering domain, creating modelling languages with a well-founded semantic basis
is essential. Such well-founded notations will enable the conceptual descriptions and
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:24 C. B. Nielsen et al.
evaluations of SoS Architectures. Only with the existence of such languages will it be
possible to conduct justifiable analysis of conceptual descriptions of an envisaged SoS.
As SoSs have a high degree of Distribution, the modelling languages must be able to
express the notion of multiple independent execution platforms that can be intercon-
nected. A particular focus should be the languages’ abilities to include Interoperability
challenges as part of modelling the individual constituent systems and their relation-
ships. Challenges connected to this would be how to define well-founded interfaces and
express desired policies between the systems. The high number of connections occur-
ring between the constituents in an SoS, calls for modelling languages that include
central aspects of Distribution through a strong focus on consistency and communi-
cation faults. Connections that also entail a high degree of Interoperability concerns,
that a modelling language must address as well. In order to handle these aspects it is
necessary to have ways of defining well-founded interfaces for each of the constituent
systems directly in models. Modelling languages that are effective for modelling dis-
tribution and those that can express clear interfaces need to be surveyed, in order to
provide the knowledge needed to establish a modelling notation with a stronger focus
at the SoS level.
In the current modelling efforts agent-based approaches are used to model the Au-
tonomy of SoS. A main challenge of doing SoS modelling is to embed this aspect into
more modelling languages by including constructs that enable users to describe au-
tonomous behaviour; in particular of how humans act in relation to the SoS. Therefore,
it is key that the modelling languages include constructs which enable the description
of such self-determining systems. Constructs that can express the capabilities and re-
sponsibilities of a system such that it can be self-controlling by using behavioural and
reasoning mechanisms and express self-coordinating behaviour based on observations.
Likewise, modelling languages need constructs that can describe the Dynamic Recon-
figuration in order for the models to express the dynamically changing infrastructures
which enables the constituent systems to initiate and break their interrelationships.
Modelling languages are needed that not only can express the dynamically changing
infrastructure, but also model the interruption of data exchange, lost messages and
error handling.
It is clear that modelling an SoS does raise high demands for the modelling language
itself. For instance, for a dimension such as Evolution it can be a challenge to include
all the necessary system aspects in the language initially. This means that the model
may not be capable of including important details as the language cannot express it.
An SoS modelling language should have features that enable well-founded extensions
of the language to be defined by super-users in a semantics preserving fashion. This is
by no means trivial, so reaching the scientific basis for this is a long-term goal. Equally,
Emergence of Behaviour is difficult to express in a model, as it is doubtful if an exactly
described behaviour can be considered truly emergent. It is a challenging SoS charac-
teristic to incorporate into a modelling language, as it cannot be accurately described.
Instead it is a characteristic that surfaces as a result of the other SoS characteristics.
In a modelling context the emergence itself can only be expressed as desirable or un-
desirable behaviours, while the behaviour itself only really becomes apparent through
simulation of the model.
The aspect of Independence between the different SoS stakeholders is another SoS
characteristic that can be challenging to include in a model. Not only can it be difficult
to express how the varied individual ownership affects the constituent systems and
the Interdependency between them, as seen in the E-commerce example in Section 4.4.
It can also be difficult to establish trust between stakeholders that may not want to
share their model or system details for confidentiality reasons, as will occur between
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:25
the competing stakeholders as for instance in the example of Smart Energy Grids given
in Section 4.2.
As the constituent systems in an SoS have independent ownership and will be sep-
arately developed it should be possible to work with divided models, such that each
constituent system can be defined by its own model with well-defined interfaces. Not
only will it allow for the constituents to be modelled independently, it will also facili-
tate the process of bringing the constituents together in the SoS. Having the Indepen-
dence and Interdependency dimensions of the constituents included directly into the
modelling language will improve the modelling effort. For instance by expressing the
Interdependency between systems could be achieved by having rely/guarantee policies
directly in the modelling languages.
Concerning architecture, Meilich [2006] focuses on net-centric environments to de-
scribe some of the challenges of SoS architecture for non-directed SoSs. These include
(1) dependency issues between the constituent systems due to Interoperability charac-
teristics; (2) Emergence of Behaviour of SoSs due to trade-offs between predictability
and composability and (3) collaboration between different Independent stakeholders of
the Distributed constituent systems.
Approaches are being proposed that shift focus from optimisation to run-time flexi-
bility and interoperability. Adaptive architectures are intended to enable system com-
position at run-time and to be more suited to the emergent behaviours found in SoSs.
Having such a flexible and dynamic architecture may come at the price of predictabil-
ity and potential lack of clarity on the system boundary. Model simulation is seen as a
response to these challenges and the way of engineering an SoS is to “experiment as
the system evolves”.
As we indicated in Section 1, model-based engineering for SoSs is a young subject,
and its limits have not yet been clearly mapped. The development of competent and
faithful SoS models is dependent on the ability to observe the state and operation of the
SoS itself with known levels of precision and accuracy, and remains an open research
topic.
6.3. Simulation
Being able to perform simulations of SoS by using executable subsets of models is es-
sential in the advancement towards a strengthened SoS engineering discipline. Simu-
lations can be a very powerful tool in analysing and understanding the complexity and
behaviour of an SoS.
The DoD guidelines recommends modelling and simulation as an effective means
for grasping the complexity and Emergence of Behaviour of SoS [OUSD(AT&L), DoD
2008]. In order for the simulation of an SoS model to be efficient it must be able to
show the characteristics of an SoS. If the main characteristics can be incorporated into
the model, the volatile characteristic emergent behaviour has a much better chance of
being detected through simulation of the model.
Simulation makes it possible to gain insight into the modelled system’s functional-
ity. One SoS characteristic that is particularly well suited for simulation is the aspect
of Autonomy. Simulating an SoS will show system behaviours and the constituents’
interactions, which would be very difficult to predict by merely analysing the models
or systems statically. An example of this would be the robot swarm systems described
in Section 4.3. Kilicay-Ergin and Dagli [2008] have proposed a framework named “Ar-
tificial Life” that is aimed at analysing the influence architectural changes have on
system behaviour. The framework consists of several layers that among others include;
computational intelligence tools, cognitive architecture and a multi-agent models level.
While Autonomy can be simulated with agent-based simulation tools, further advances
are needed for enabling users to interact with a simulation and reveal the autonomous
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:26 C. B. Nielsen et al.
behaviour occurring when humans act in relation to a specific SoS, for instance via
human-in-the-loop simulation.
An SoS simulation environment should also support the Independence of the con-
stituent systems, by allowing both stand–alone and combined simulation of models.
The constituent systems may be described in separate models that need to be simu-
lated both individually and in combination with models of other parts of the SoS. This
is particularly relevant in systems where competing concerns have to be balanced, such
with the Air traffic management systems used as an example in Section 4.1. Allowing
each simulation to be performed individually for each constituent, while at the same
time enabling a joint model to be simulated with other parts of the SoS would support
the Independence of the constituent systems. As mentioned above, confidentiality may
be important as owners of one constituent system participating in the SoS may not
wish to reveal the internals of their system through their model. In the same way the
simulators will be challenged by model languages that will need to evolve in order to
match the Evolving SoSs model.
An important outcome of doing simulations of SoS is to identify possible Emergence
of Behaviour. Simulations may be probabilistic, rather than definitive, so the aim
would be to demonstrate a low probability of undesirable emergent behaviour or a
high probability of desirable emergent behaviour. This could be determined by obser-
vation of the simulation result; however, a more powerful capability would be if such
behaviour could be flagged directly by the SoS simulation tool. The exact method for
doing this needs to be researched further, but one could focus on expressing the “desire
and expectation” policies of the SoS stakeholders directly in the modelling language
notations. The tool would then mark simulated variations from these policies as emer-
gent behaviour of the SoS. The Smart Energy Grids, described in Section 4.2, would be
a good initial starting point for such research as these systems aim for optimal energy
pricing and energy use already is policy-based.
Given the many relations and interactions that occur between the constituent sys-
tems in an SoS, a simulation must be able to encompass the challenges found in Dis-
tributed systems, such as concurrency, consistency and communication faults. Identi-
fying communication faults as well as newly initiated communication is necessary if
simulators are to be used for detecting aspects of Interdependency during simulation.
Because of the constant Evolution of an SoS, SoS simulation tool should enable ex-
tensions of the simulator to be made. For instance by enabling simulators to handle
extensions of the modelling language itself, or by enabling communication external to
the simulation environment, such as through remote procedure calls [Nielsen et al.
2012].
An SoS simulator should be capable of capturing the Dynamic Reconfiguration that
occurs as a result of the dynamically changing system topology, such that the dynamic
changes can be communicated to stakeholders. There already exist tools that have
a semantically well-founded foundation and are capable of performing simulation of
distributed systems. Some of these allow for the inclusion of independent execution
platform characteristics, with a focus on fault and data sharing. Fewer include the Dy-
namic Reconfiguration dimension, for which the altering system topologies entail in-
teroperability challenges, as well as overview challenges. A powerful simulator should
deliver mechanisms for keeping track of changes, for instance by visually indicating
the current topology. In the same way a strong SoS simulation tool needs to have a
focus on the interaction between the constituents. A simulation should show the In-
terdependence between constituents, data flow and failed links between incompatible
constituent.
Finally, as an SoS will consist of systems that have different data formats, archi-
tectures, and hardware, being able to include Interoperability aspects in the simula-
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:27
tion is a particularly interesting research topic. One such example could be to conduct
joint simulations with heterogeneous models that use different notations. Having this
ability could aid the construction of an SoS by allowing the integration of various het-
erogeneous models defined for different constituent systems that are to be joint in the
SoS.
6.4. Testing
The current challenges related to SoS testing can be classified according to the follow-
ing aspects:
(1) Complexity issues: For very large SoS, such as the Emergency Management and
Response example in Section 4.3, the size of the SoS state space will preclude, for ex-
ample, exhaustive testing [Chow 1978; Springintveld et al. 2001], as well as search-
based or exploratory testing [Spillner et al. 2006; Arcuri et al. 2010; C.Kaner 1999].
(2) Management issues: the multi-stakeholder situation which often is encountered in
SoS development, verification, and validation (V&V), such as the E-commerce ex-
ample given in Section 4.4, complicates capture of test cases for emergent SoS prop-
erties and the control of system integration test activities [Sledge 2006; Colombi
et al. 2008].
(3) Applicability of multiple standards: different standards impose different V&V and
tool qualification requirements [Brauer et al. 2012]. This will be seen in SoS that
are composed of independently owned and operated systems that are geographically
distributed, such as the Transport networks described in Section 4.1.
(4) Dynamic evolution of SoS configurations: complicates test automation techniques
and requires acceptance testing at run-time [Gonzalez et al. 2008].
[Gonzalez et al. 2008] point out that Dynamic Reconfiguration, is a crucial aspect
of SoS behaviour. From testing and model checking object-oriented systems, it is well
known that this conceptually unbounded state space of dynamically generated objects
represents additional challenges. It has to be determined which numbers of objects
considered in test configurations are meaningful to ensure that the SoS behaviour will
be appropriate in any configuration that may occur.
In order to mitigate the risks associated with the challenges listed above, the guide-
line [OUSD(AT&L), DoD 2008] recommends a model-based approach to SoS devel-
opment and V&V, since semantically well-defined models present a clearer view on
system capabilities than informal descriptions, and form the basis for automated de-
velopment and V&V activities. This view is backed up by various standards applica-
ble to complex, typically safety-critical systems: (1) The IEC standard [IEC61508-3
2010] classifies Model-based testing (MBT) as a highly recommended method for test-
ing software and systems of the highest safety integrity levels SIL-3 and SIL-4. (2) The
avionic standard [RTCA SC-205/EUROCAE WG-71 2011b] devotes a complete supple-
ment to model-based development and verification, the latter includes simulation and
testing [RTCA SC-205/EUROCAE WG-71 2011a]. (3) The automotive standard [In-
ternational Organization for Standardization 2009] acknowledges the capabilities of
model-based development and testing.
The benefits of MBT have been clearly identified [Baker et al. 2008], and case stud-
ies show the feasibility of the approach for industrial-size systems [Grieskamp 2010;
Peleska et al. 2011]. A particular benefit of the MBT approach consists in the possi-
bility to derive relevant test cases automatically from the model. Moreover, if model
elements are already linked to requirements, as supported, for example, by the SysML
modelling formalism [SysML1.2 2010], MBT can trace requirements to test cases and
procedures in an automated way. The feasibility proof, however, currently applies to
sub-systems only: as of today, there exists no accepted methodology for combining con-
stituent system test models into an SoS testing model. Model-based testing for small
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:28 C. B. Nielsen et al.
to medium size control systems may be fully automated, with potential additions of
user-defined test cases that may be specified, for example, already as parts of the test
model [Peleska et al. 2011].
Despite the commitment to MBT which is visible in international standards, and
despite the available evidence about MBT efficiency, the model-based testing approach
cannot be regarded today as an industrial best practice of SoS V&V. It is more the case
that companies are currently evaluating the potential benefits of the approach. The
change of paradigm required for introducing MBT is enforced rather slowly by the re-
sponsible management, since it may also require a change of skills in the testing teams
involved: the focus of testing activities is shifted from test procedure programming to
model development. As a consequence, teaching model-based testing methodology and
adapting V&V process models to incorporate the MBT approach is a major challenge
for SoS testing. Apart from these educational and managerial challenges, there re-
main some technical ones to be overcome for applying MBT successfully in SoS testing
campaigns. Typical success stories from the MBT application area emphasise the im-
portance of the availability of a complete test model [L¨
oding and Peleska 2010] as
starting point of the test automation tool chain. This may be a major obstacle: partly
due to the number of cooperating constituent systems for which sub-models have to
contribute to the SoS test model, and partly due to the complete SoS test model only
being available at the later stages of the development life cycle. As a consequence, the
benefits of automation are in danger of being nullified by the disadvantage of delayed
start of SoS system-level testing. We conclude that for these reasons it is mandatory to
support incremental development cycles for test models and test objectives, such that
each cycle may be concluded with a test generation and execution campaign.
The need for incremental and even partially automated test model development has
gained attention in the research communities, and [Vaandrager 2012] describes how
machine learning techniques can be applied to automatically construct test models by
incrementally deriving state machine models from test execution traces observed. This
approach is obviously attractive for MBT, since testing already starts while the model
is constructed.
Other research areas, however, emerge from the investigation of model-based SoS
testing and its specific needs. Extending the list of research foci shown at the beginning
of Section 6.4, we anticipate in-depth research on:
(5) SoS-specific formalisms for model-based testing: by abstracting constituent system
behaviour with the help of contracts, the complexity of SoS-level test models can be
considerably reduced [Coleman et al. 2012; Milius et al. 2011]. Relating require-
ments to model elements – as, for example, provided by SysML [Holt and Perry
2008] – enables automated tracing between requirements, test cases, procedures
and results [Hallerstede et al. 2012].
(6) Identification of test objectives for emergent properties: on SoS level, test objectives
can be identified by global mission threads or scenarios, and different constituent
system behaviours can be abstracted in equivalence classes [Colombi et al. 2008;
Kaner 2003].
(7) Methodology for incremental model-based SoS testing.
6.5. Verification
To realise the full potential of formal methods they must be integrated with other
techniques used in the development of SoSs. Verification technology complements si-
mulation and testing by providing exhaustive coverage; it can support requirements
analysis, design, testing, and code generation. Work with formal verification has been
performed in diverse areas, such as hardware platforms, concurrency, communication
protocols, and real-time systems. More research is needed on how the integration of
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:29
verification techniques into development processes would strengthen their use in an
SoS context. In order to advance the SoS engineering discipline by enabling proofs of
correctness for SoS, there is a need for both new and extended theories and tools.
It is worth noting that verification assumes that the specification of the system of
interest is well understood and is clearly or formally articulated, so as to form a basis
for verification judgements. SoSs often arise because reliance comes to be placed on
the collective functioning of the constituent systems, and they evolve during operation
as constituents change. The derivation and maintenance of SoS specifications that can
serve as a basis for verification is thus a significant research challenge in itself, in
particular to identify and avoid unwanted emergence.
Aside from the identification and maintenance of specification, other significant chal-
lenges for SoS verification include extending the range of techniques available, linking
them together, and finding ways to apply them effectively in the large. Creating new
theories, and providing extensions to existing theories in order to enable SoS verifica-
tion, would provide a very strong advance in SoS engineering. Not only will it enable
the proof of correctness for the critical system embodied by many SoSs, it will also lead
to a more systematic engineering process.
An SoS experiences a constant Evolution, the self-contained constituents are Inde-
pendent and Autonomous and there is a constant dynamic change of its configuration,
state, and operations to respond to spatial and temporal changes. Providing verifica-
tion support for all of these SoS characteristics is very challenging, as they can seem
boundless and indeterminate. Such change will for instance occur often in E-commerce,
given as an example in Section 4.4, because of the constant development these systems
face, for instance in terms of new suppliers or addition of new platforms on which the
virtual market place need to run.
The large degree of change and continuous development of the SoS topology and be-
haviour calls for new and extended theories and tools for verifying automatic, adaptive,
and self-awareness properties. These include modelling systems that are state-rich,
concurrent, distributed, autonomous, time-sensitive, mobile, hybrid, discrete and con-
tinuous, and probabilistic. Each of these areas has separate notations and modelling
techniques, and it is a challenge to unify them. The objective is to have a collection
of links that allow different aspects of SoSs and their component systems to be mod-
elled and therefore verified in the most suitable theoretical setting. Unification of these
modelling techniques would provide a coherent setting for understanding the SoS as
a whole. Unification of modelling techniques would address Independence,Autonomy,
and Distribution.
The key to scaling verification technology to realistic industrial SoSs is to model
and analyse them at different levels of abstraction and to use techniques that com-
pose cleanly. The link between different levels of abstraction is the use of refinement
techniques, and research is needed to develop suitable theories for the unified view dis-
cussed above. A significant challenge is to discover techniques that allow us to divide
and conquer the verification of large SoSs. As described with the Emergency Manage-
ment and Response example in Section 4.3, the special and uncommon circumstances
of a major emergency will involve a very large number of systems, which again will
lead to very large models. A compositional verification technique allows us to forget
the rest of the SoS when verifying a particular property or component system, and yet
know that any results we obtain can be safely extrapolated to the SoS as a whole. In-
troducing techniques for abstraction and compositionality would help with questions
of scale and Dynamic Reconfiguration and Emergence of Behaviour.
In the same way the challenges of Interoperability and Interdependence, require a
strengthening of the theories of compositionality, information flow, and confidentiality
as well as techniques for verifying compliance. The most challenging SoS characteristic
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:30 C. B. Nielsen et al.
for verification is the aspect of Emergence of Behaviour. There is a strong need for
continued research into the process of engineering emergent behaviour in general.
Research, which can then be used to establish support for the verification of such a
process.
It is clear that different tools are required for analysing different properties, such
as response time; data integrity; mutual exclusion of concurrent access, and these
too must be integrated. Each of these properties is best checked with a different tool
and an integrated tool-set is required. This integration could be quite tight with tools
cooperating in verification tasks. For example, a model checker could subcontract a
theorem prover to establish some of its results. It is a major theoretical and engineer-
ing challenge to integrate tools in this way.
Placing a sharp research focus on the development of strong tool support is impor-
tant, given that the engineers who work with SoS development come from very diverse
fields, and as such the expertise with verification techniques may be limited. Having
powerful and easy to use tools, for instance with automated proof-checking support,
will not only provide stronger results it will also ease the reception and approval of the
tool amongst its users.
7. CONCLUDING REMARKS
Our objective in this paper was to review the state of the art in model-based approaches
to the engineering of Systems of Systems. Faced with a large body of literature in
what is still a varied and lively research field within systems engineering, we ana-
lysed a wide range of sources offering definitions or taxonomic bases of SoS concepts
and identified eight “dimensions” that might allow one to place examples within the
SoS space. We think that the unification conducted in deriving these dimensions will
be useful in the future for classifying the focus of different SoS applications. We went
on to examine model-based techniques for system description, simulation, testing and
verification, relating these to the eight dimensions identified. We hope that these di-
rections of future challenges towards a strengthened discipline of SoS engineering will
be taken up by more researchers.
It should be noted that our survey did not address human or stochastic aspects of
SoS description, and we recognise that this means our story is necessarily partial.
However, it does reflect a relative paucity of research literature on “humans in the
SoS”.
The review identifies a need for a growing body of case studies of SoS engineering
on which research can draw, either to establish and test new methods and theories or
to verify existing ones. Getting access to various types of SoS can however prove to
be difficult. Much of the work on SoS engineering to date originates in military and
aerospace applications, and these typically have a degree of confidentiality that makes
it difficult to gain access. Having multiple owners and stakeholders of constituent sys-
tems make data collection and test difficult. Even for non-military SoS the sheer size
and geographical location of constituent systems and stakeholders can make it difficult
to gain access and overview for logistical reasons. As the SoS field is still in its infancy,
there is a need for case studies that show examples of SoS and show the effect that
the SoS dimensions have on the engineering effort. A series of challenging examples,
proposed by practitioners and placed in the public domain can galvanise competitive
research, as illustrated in some measure by the Verified Software Initiative [Hoare
et al. 2009]. This may also involve identifying systems that have not yet been seen as
SoS, but may express many of the SoS dimensions.
Although some large-scale SoSs are engineered using conventional techniques, hav-
ing evolved over a long period, and may not be identified explicitly as SoSs, the oppor-
tunities afforded by the convergence of embedded computing platforms and the Inter-
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:31
net mean that there may be many more SoSs engineered, and explicitly identified as
such, in the future.
ACKNOWLEDGMENTS
Our work is partially supported by the COMPASS project funded by the European Commission’s 7th Frame-
work Programme (Grant Agreement 287829), and the INTO-CPS project funded by the Horizon 2020 Pro-
gramme (Grant Agreement 644047). We are grateful to the anonymous reviewers, to Padraig Lyons, and to
many members of the COMPASS project team for valuable discussions during the preparation of this paper.
REFERENCES
ABBOTT, R. 2006. Open at the Top; Open at the Bottom; and Continually (but slowly) Evolving. In System
of Systems Engineering, 2006 IEEE/SMC International Conference on. IEEE.
ACKOFF, R. L. 1971. Towards A System of Systems Concept. Management Science 17, 11 (July), 661–671.
AGUSDINATA, D. B. AND DELAURENTIS, D. 2008. Specification of System-of-Systems for Policymaking in
the Energy Sector. Integrated Assessment Journal 8, 2, 1–24.
ARCURI , A., IQB AL, M . Z., AND BR IAN D, L. 2010. Black-box system testing of real-time embedded systems
using random and search-based testing. In 22nd IFIP international conference on Testing software and
systems. ICTSS’10. Springer-Verlag, 95–110.
AXELROD, R. 1997. The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration.
Princeton University Press.
BACH, J. AND SCHROEDER, P. 2004. Pairwise Testing – a Best Practice that isn’t. In 22nd Pacific Northwest
Software Quality Conference. 180–196.
BAKER, P., DAI , Z. R., GRABOWSKI, J., HAUGEN, O., SCHIEFERDECKER, I., AND WILLIAMS, C. 2008. Model
Driven Testing – Using the UML Testing Profile. Springer.
BALDWIN, W. AND SA USE R, B. 2009. Modeling the Characteristics of System of Systems. In System of
Systems Engineering, 2009. SoSE 2009. IEEE International Conference on. IEEE.
BALL SR, J. 1997. Free flight an air traffic management partnership. In Digital Avionics Systems Conference,
1997. 16th DASC., AIAA/IEEE. Vol. 2. IEEE, 9–1.
BAR-YAM , Y., ALL IS ON, M. A., BATDORF, R ., CHEN, H., GENERAZIO, H., SINGH, H., AND TUC KER, S.
2004. The Characteristics and Emerging Behaviors of System of Systems. NECSI: Complex Physical,
Biological and Social Systems Project, 1–16.
BEDAU, M. A. 1997. Weak emergence. In Philosophical Perspectives: Mind, Causation, and World. Vol. 11.
Blackwell, 375–399.
BERENJ I, H. AND JAMSHIDI, M. 2011. Fuzzy Reinforcement Learning for System of Systems (SoS). In Fuzzy
Systems (FUZZ), 2011 IEEE International Conference on. 1689 –1694.
BERRY, B. J. 1964. Cites as Systems within System of Cites. Papers and Proceedings of the Regional Science
Association 13, 1 (January), 149–163.
BEUGNARD, A., JEZEQUEL, J.-M., PLOUZEAU, N., AND WATKINS, D. 1999. Making Components Contract
Aware. IEEE Computer, 38–45.
BLOOMFIELD, R. AND GASHI, I. 2008. Evaluating the Resilience and Security of Boundaryless, Evolving
Socio-Technical Systems of Systems. Tech. rep., Centre for Software Reliability, City University. Sep.
BOARDM AN, J. AND SAUSER , B. 2006. System of Systems – the meaning of “of ”. In Proceedings of the 2006
IEEE/SMC International Conference on System of Systems Engineering. IEEE.
BOEHM, B. 2006. A view of 20th and 21st Century Software Engineering. In In Proc. ICSE‘06. ACM Press,
12–29.
BOULDING, K. E. 1956. General Systems Theory–The Skeleton of Science. Management Science 2, 3 (April).
BRAU ER, J., PELES KA, J., AND SCHULZE, U. 2012. Efficient and Trustworthy Tool Qualification for Model-
Based Testing Tools. In Testing Software and Systems. LNCS. Springer Berlin Heidelberg, 8–23.
BRYANS, J., FITZGERALD, J., PAYNE , R., AND KR IS TEN SE N, K. 2014. Maintaining emergence in systems
of systems integration: a contractual approach using sysml. In INCOSE International Symposium on
System Engineeering. INCOSE.
CAFFALL , D. S. AND MICHAEL, J. B. 2004. A new Paradigm for Requirements Specification and Analy-
sis of System-of-Systems. In Radical Innovations of Software and Systems Engineering in the Future,
9th International Workshop, RISSEF 2002. LNCS, vol. 2941. Springer, 108–121.
CALINESCU, R. AND KWIATKOWSKA, M. Z. 2009. Using Quantitative Analysis to Implement Autonomic IT
Systems. In 31st International Conference on Software Engineering, ICSE 2009, Vancouver. 100–110.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:32 C. B. Nielsen et al.
CALINE SCU, R. AND KWIATKOWSKA, M. Z. 2010. Software engineering techniques for the development of
Systems of Systems. In Foundations of Computer Software. Future Trends and Techniques for Develop-
ment, 15th Monterey Workshop 2008. LNCS, vol. 6028. Springer, 59–82.
CAMPBELL, J. E., LONGSINE, D. E ., SHIRAH, D., , AND ANDERSON, D. J. 2005. System of Systems Modeling
and Analysis. Tech. Rep. SAND2005-0020, Sandia National Laboratories. January.
CANTOT, P. AND LUZE AUX , D. 2009. Simulation et mod ´elisation des syst`emes de syst `emes: vers la maˆıtrise
de la complexit´e. Hermes Science Publications, ISBN: 978-2746221468.
CANTOT, P. AND LUZE AUX , D. 2011. Simulation and Modeling of Systems of Systems. Wiley, ISBN: 978-1-
84821-234-3.
CARLOCK, P. G. AND FENTON, R. E. 2001. System of Systems (SoS) Enterprise Systems Engineering for
Information-Intensive Organizations. Systems Engineering 4, 4, 242–261.
CARNEY, D., FASHER, D., AND PLA CE , P. 2005. Topics in Interoperability: System-of-Systems Evolution.
Tech. Rep. CMU/SEI-2005-TN-002, Software Engineering Institute, Carnegie Mellon University.
CHECKLAND, P. 1999. Systems Thinking, Systems Practice: Includes a 30-Year Retrospective. Wiley.
CHEN, P. AND CLOTHIER, J. 2003. Advancing systems engineering for systems-of-systems challenges. Sys-
tems Engineering 6, 3 (May), 170–183.
CHOW, T. S. 1978. Testing Software Design Modeled by Finite-State Machines. IEEE Transactions on Soft-
ware Engineering SE-4, 3 (Mar.), 178–186.
C.KANER, J.FAL K, H . 1999. Testing Computer Software. Wiley.
COCKS, D. 2006. How Should We Use the Term “System of Systems” and Why Should We Care? In Proceed-
ings of the 16th INCOSE International Symposium 2006. INCOSE.
COLEMAN, J. W., MALM OS, A. K., LA RS EN, P. G., PEL ES KA, J., HAI NS, R., ANDREWS, Z., PAYNE, R.,
FOSTER , S., MIYAZ AWA, A ., BERTOLINI, C., AND DIDIER, A . 2012. COMPASS Tool Vision for a System
of Systems Collaborative Development Environment. In Proceedings of the 7th International Conference
on System of System Engineering, IEEE SoSE 2012. 451–456.
COLOMBI, J., COHEE, B. C., AND TUR NE R, C. W. 2008. Interoperability Test and Evaluation: A Systems of
Systems Field Study. The Journal of Defense Software Engineering, 10–14.
COOK, S., LAWSON, E., AND ALLISO N, J. 1999. Towards a Unified Systems Methodology for Australian
Defence Systems-of-Systems. Proc. of the 9th INCOSE International Symposium.
COOK, S. C. 2001. On the Acquisition of Systems of Systems. 2001 INCOSE International Symposium.
COUTO, L. D. AND PAYNE , R. 2013. The COMPASS Proof Obligation Generator: A test case of Overture
Extensibility. In Proceedings of the 11th Overture Workshop.
CROSSLEY, W. 2004. System of Systems: An Introduction of Purdue University Schools of Engineering’s
Signature Area. In Engineering Systems Symposium,. MIT Engineering Systems Division.
DAHMANN, J. 2014. Systems of Systems Pain Points. In INCOSE International Symposium on Systems
Engineering 2014.
DAHMANN, J. AND BALDWIN, K. 2008. Understanding the Current State of US Defense Systems of Systems
and the Implications for Systems Engineering. In IEEE Systems Conference. IEEE.
DAHMANN, J. S., JR., G. R., AND LANE., J. A. 2008. Systems engineering for capabilities. CrossTalk Journal
(The Journal of Defense Software Engineering) 21, 11 (november), 4–9.
DANIEL, K., DUS ZA, B., LEWANDOWSKI, A., AND WIETFELD, C. 2009. AirShield: A System-of-Systems
MUAV Remote Sensing Architecture for Disaster Response. In Systems Conference, 2009 3rd Annual
IEEE. 196 –200.
DECANDIA, G., HASTORUN, D., JAM PAN I, M., KAKULAPATI, G., L AKSHMAN, A., PILCHIN, A., SI VASU B-
RAMAN IAN, S., VOSSHALL, P., AND VOGELS, W. 2007. Dynamo: Amazon’s Highly Available Key-value
Store. In Proceedings of Twenty-first ACM SIGOPS Symposium on Operating Systems Principles, SOSP
07. ACM, Reno, NV, USA, 205–220.
DELAURENTIS, D. A. 2005. Understanding Transportation as System-of-Systems Design Problem. In 43rd
AIAA Aerospace Sciences Meeting and Exhibit. AIAA.
DELAURENTIS, D. A. AND CROSSLEY, W. 2005. A taxonomy-based Perspective for Systems of Systems
Design Methods. In IEEE International Conference on Systems, Man and Cybernetics. IEEE, 86–91.
DEPARTMENT OF DEFE NSE . 2004. Defense Acquisition Guidebook. Chapter Ch. 4 System of Systems Engi-
neering. https://dag.dau.mil/Pages/Default.aspx.
DESPOTOU, G., ALEXANDER, R., AND HALL-MAY, M. 2003. Key Concepts and Characteristics of Systems
of Systems (SoS). DARP - HIRTS, University of York. February.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:33
DILLON, T., PO TD AR, V., SINGH, J., AND TALEVSKI, A. 2011. Cyber-physical systems: Providing Quality of
Service (QoS) in a Heterogeneous Systems-of-Systems Environment. In Digital Ecosystems and Tech-
nologies Conference (DEST), 2011, 5th IEEE International Conference on. 330 –335.
DRUSIN KSY, D. AND SHING, M.-T. 2005. Creation and evaluation of formal specifications for system-of-
systems development. In Systems, Man and Cybernetics, IEEE Intl. Conf. on. IEEE, 1864–1869.
DRUSIN SKY, D. 2003. Monitoring Temporal Rules combined with Time Series. In 15th Intl. Conf. on Com-
puter Aided Verification, ,CAV 2003, Boulder, July,. LNCS, vol. 2725. Springer, 114–117.
EDDY, J. P., ANDERSON, D. J., LAWT ON, C. R., AND CAMPBELL, J. E. 2007. Network and Adaptive System
of Systems Modeling and Analysis. Tech. Rep. SAND2007-2788, Sandia National Laboratories. May.
EISNER , H., MARCINIAK, J., AND MCMILLAN, R. 1991. Computer-Aided System of Systems (S2) Engineer-
ing. In Systems, Man, and Cybernetics, 1991. Proc, of IEEE Intl. Conf. on. 531 –537 vol.1.
EUROPEAN COMMISSION. 2012. Directions in Systems of Systems Engineering. Tech. rep., European Com-
mission, Communications Networks, Content and Technology Directorate- General Unit A3-DG CON-
NECT. July.
FISHER, D. A. 2006. An Emergent Perspective on Interoperation in Systems of Systems. Tech. rep., Software
Engineering Institute, Carnegie Mellon University. March. CMU/SEI-2006-TR-003.
FITZGERALD, J., LAR SE N, P. G., AND WOODCOCK, J. 2012. Modelling and Analysis Technology for Systems
of Systems Engineering: Research Challenges. In INCOSE. Rome, Italy.
GAMBLE, M. AND GAMBLE, R. 2008. Reasoning about Hybrid System of Systems Designs. In Composition-
Based Software Systems, 2008. ICCBSS 2008. Seventh International Conference on. 154 –163.
GEDDES, N., SMITH, D., AND LIZZA, C. 1998. Fostering Collaboration in Systems of Systems. In Systems,
Man, and Cybernetics, 1998. 1998 IEEE International Conference on. IEEE, 950 – 954 vol.1.
GONZAL EZ, A., PIEL, E., GROSS, H.-G., AND GLANDRUP, M. 2008. Testing Challenges of Maritime Safety
and Security Systems-of-Systems. In Proceedings of the Testing: Academic & Industrial Conference —
Practice and Research Techniques. IEEE Computer Society, 35–39.
GOROD, A., SAUS ER, B., AND BOAR DMAN, J. 2008. System-of-Systems Engineering Management: A Review
of Modern History and a Path Forward. Systems Journal, IEEE 2, 4 (December), 484–499.
GRIESK AMP, W. 2010. Microsoft’s protocol documentation program: A success story for model-based testing.
In Testing - Practice and Research Techniques, 5th International Academic and Industrial Conference,
TAIC PART 2010, Windsor, UK, September 3-5, 2010. Proceedings, L. Bottaci and G. Fraser, Eds. Lec-
ture Notes in Computer Science, vol. 6303. Springer, 7.
GUTIER REZ-GARCIA, J., RAMOS-CORCHADO, F., AND KONING, J.-L . 2009. Obligations as Constrainers,
Descriptors, and Linkers of Open System of Systems. In System of Systems Engineering, 2009. SoSE
2009. IEEE International Conference on. 1 –6.
HALLERSTEDE, S., HANS EN, F. O., HOLT, J., LAU RITSE N, R., LORENZEN, L., AND PE LESKA , J. 2012.
Technical Challenges of SoS Requirements Engineering. In 7th International Conference on System of
System Engineering, IEEE SoSE 2012. IEEE, 573–578.
HAREL, D. 1987. StateCharts: A Visual Formalism for Complex Systems. Science of Computer Program-
ming 8, 3, 231–274.
HARMON, K. 2005. The “systems” Nature of Enterprise Architecture. In Systems, Man and Cybernetics,
2005 IEEE International Conference on. IEEE, 78 – 85 vol. 1.
HAVEL UN D, K. AND RO SU, G. 2001. Monitoring Programs using Rewriting. In 16th IEEE International
Conference on Automated Software Engineering (ASE 2001), Coronado Island. 135–143.
HITCHINS, D. K . 2005. Systems methodology. In Conference on Systems Engineering Research.
HOARE, C. 1978. Communicating Sequential Processes. Communications of the ACM 21, 8 (August).
HOARE, C., MISRA, J., LEAVEN S, G. T., AND SHANKAR, N. 2009. The verified software initiative: A mani-
festo. ACM Comput. Surv. 41, 4 (Oct.), 22:1–22:8.
HOLT, J. 2012. Model-based Requirements Engineering for System of Systems. In 7th International Confer-
ence on System of System Engineering, IEEE SoSE 2012. IEEE.
HOLT, J. AND PERRY, S. 2008. SysML for Systems Engineering. IET.
IEC61508-3. 2010. IEC61508-3: Functional safety of electrical/electronic/programmable electronic safety-
related systems - Part 3: Software requirements. International Electrotechnical Commission.
IEEE1516 2010. IEEE Standard for Modeling and Simulation: High Level Architecture (HLA)– Framework
and Rules. IEEE Std 1516-2010 (Revision of IEEE Std 1516-2000), 1 –38.
ILI ´
C, M., XIE , L., KHAN, U., AND MOUR A, J. M. F. 2010. Modeling of Future Cyber-Physical Energy Sys-
tems for Distributed Sensing and Control. Systems, Man and Cybernetics, Part A: Systems and Humans,
IEEE Transactions on 40, 4, 825–838.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:34 C. B. Nielsen et al.
INCOSE. 2011. Systems Engineering Handbook. A Guide for System Life Cycle Processes and Activities,
Version 3.2.2. Tech. Rep. INCOSE-TP-2003-002-03.2.2, International Council on Systems Engineering
(INCOSE). October.
INCOSE. 2015. INCOSE Systems Engineering Handbook: A Guide for System Life Cycle Processes and
Activities, Version 3.2.2., 4th ed. Wiley.
INGRAM, C., PAY NE, R., PE RRY, S., HOLT, J., HA NS EN, F., AND COU TO, L. 2014. Modelling Patterns for
Systems of Systems Architectures. In International Systems Conference (SysCon2014). IEEE.
INTERNATIONAL ORGA NI ZAT ION F OR STANDARDIZATION. 2009. ISO 26262 - Road Vehicles - Functional
Safety - Part 6: Product development: software level. ICS 43.040.10.
INTERNATIONAL ORGANIZATION FOR STANDARDIZATION. 2015. ISO/IEC/IEEE 15288 - Systems and Soft-
ware Eningeering–System Life Cycle Porcesses.
JACKSO N, M. C. AND KEYS, P. 1984. Towards a System of Systems Methodologies. The Journal of the
Operational Research Society 35, 6 (June), 473–486.
JACOB, F. 1974. The Logic of Living Systems: a History of Heredity. Allen Lane.
JAMSHIDI, M. 2005. System-of-Systems Engineering – a Definition. In International Conference on Systems,
Man, and Cybernetics. IEEE.
JAMSHIDI, M. 2008. System of Systems Engineering – New Challenges for the 21st Century. Aerospace and
Electronic Systems Magazine, IEEE 23, 5 (may), 4 –19.
JIAN, X., BING-FEN G, G., X IA O-KE , Z., KE-WE I, Y., AND YING-WU, C. 2010. Evaluation Method of System-
of-Systems Architecture using Knowledge-based Executable Model. In Management Science and Engi-
neering (ICMSE), 2010 International Conference on. 141 –147.
JONES, C. 1983a. Specification and Design of (Parallel) Programs. In Proceedings of IFIP ’83. IFIP, North-
Holland, 321–332.
JONES, C. B. 1983b. Tentative Steps Toward a Development Method for Interfering Programs. ACM Trans-
actions on Programming Languages and Systems 5, 4 (October), 596–619.
KANER, C. 2003. Cem Kaner on Scenario Testing: The Power of “What If...” and Nine Ways to Fuel Your
Imagination. Testing & Quality Engineering Magazine.
KARCAN IAS, N. AND HESS AMI , A. 2010. Complexity and the Notion of System of Systems: Part (II): Defining
the Notion of System of Systems. In World Automation Congress (WAC), 2010. 1–7.
KEAT IN G, C. 2005. Research Foundations for System of Systems Engineering. In Systems, Man and Cyber-
netics, 2005 IEEE International Conference on. IEEE, 2720 – 2725.
KEAT IN G, C., ROGERS, R., UNAL, R., D RYER, D., SOUS A-POZ A, A ., SA FFO RD, R ., PETERSON, W., AND
RABADI, G. 2003. System of System Engineering. Engineering Management Journal 15, 3, 36–45.
KILICAY-ERGIN, N. AND DAG LI , C. 2008. Executable Modeling for System of Systems Architecting: An
Artificial Life Framework. In Systems Conference, 2008 2nd Annual IEEE. 1 –5.
KOT OV, V. 1997. Systems-of-Systems as Communicating Structures. Tech. Rep. HPL-97-124, Hewlett
Packard Computer Systems Laboratory Paper.
KRAMER , J. 2007. Is Abstraction the Key to Computing? Communications of the ACM 50, 4, 37–42.
KRYGIEL, A. J. 1999. Behind the Wizard’s Curtain, an Integration Environment for a System of Systems.
CCRP publication series.
LANE, J. A. AND VALERDI, R. 2007. Synthesizing SoS Concepts for use in Cost Modeling. Systems Engi-
neering 10, 4 (December), 297–308.
LEES, M., LOGAN, B., AND THEODOROPOULOS, G. 2007. Distributed Simulation of Agent-based Systems
with HLA. ACM Trans. Model. Comput. Simul. 17.
LEVESO N, N. 1995. SAFEWARE: System Safety and Computers. Addison-Wesley.
LIANG, Q. AND RUBIN, S. H. 2009. Randomization for Testing Systems of Systems. In IEEE IRI 2009:
International Conference on Information Reuse & Integration. IEEE, 110–114.
LIU, S. 2011. Employing System of Systems Engineering in China’s Emergency Management. Systems Con-
ference, 2010 4th Annual IEEE 5, 2 (april), 541 –546.
LOCK, R. AND SOMMERVILLE, I. 2010. Modelling and Analysis of Socio-Technical System of Systems. In
15th IEEE Intl. Conference on Engineering of Complex Computer Systems (ICECCS). 224 –232.
L¨
ODING, H. AND PEL ESK A, J. 2010. Timed Moore Automata: Test Data Generation and Model Checking. In
International Conference on Software Testing, Verification, and Validation, ICST2008. IEEE, 449–458.
LUCKE, C., KRELL, S., AND LECHNER, U. 2010. Critical Issues in Enterprise Architecting – A Literature
Review. In AMCIS 2010 Proceedings.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:35
LUDWIG, M., FARCET, N., BAB AU, J.-P., AND CHAMPEAU, J. 2011. Integrating Design and Runtime Vari-
ability Support into a System ADL. In 7th European Conference on Modelling Foundations and Appli-
cations. Springer-Verlag, 270–281.
LUKASI K, S. J. 1998. Systems, Systems of Systems, and the Education of Engineers. In AI EDAM. Vol. 12.
Cambridge University, 55–60.
MAHULKAR, V., MCKAY, S., ADAMS, D., AND CHATURVEDI, A. 2009. System-of-Systems Modeling and
Simulation of a Ship Environment With Wireless and Intelligent Maintenance Technologies. Systems,
Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 39, 6 (Nov), 1255 –1270.
MAIER, M. W. 1996. Architecting Principles for Systems-of-Systems. In INCOSE 1996 Sixth annual Inter-
national Symposium of the International Council on Systems Engineering. INCOSE.
MAIER, M . W. 1998a. Architecting Principles for Systems-of-Systems. Systems Engineering 1, 4, 267–284.
MAIER, M. W. 1998b. Architecting principles for systems-of-systems. The Information Architects Coopera-
tive (TIAC) whitepaper, www.infoed.com/Open/PAPERS/systems.htm.
MAIER, M. W. 2005. Research Challenges for Systems-of-Systems. In Systems, Man and Cybernetics, 2005
IEEE International Conference on. IEEE.
MAIER, M. W. AND RECHTIN, E . 1997. The Art of Systems Architecting. CRC Press LLC.
MALER, O. AND NICKOVIC, D. 2004. Monitoring Temporal Properties of Continuous Signals. In FORMATS
2004 and FTRTFT 2004. LNCS, vol. 3253. Springer, 152–166.
MANSOURI, M., GOROD, A., WAKEMAN, T. H., AND SAUSER, B. 2009. Maritime Transportation System of
Systems Management Framework: a System of Systems Engineering approach. International Journal
of Ocean Systems Management 1, 2, 200 – 226.
MANTHORPE, W. H. J. 1996. The Emerging Joint System of Systems: A Systems Engineering Challenge
and Opportunity for APL. John Hopkins APL Technical Digest 17, 3, 305–310.
MAU SS, F., VAL EN CIA , J., HATCHELL, B., SI LVER S, K., AND CROWELL, S. 2015. System of Systems Ap-
proaches for Mobile Source Transit Security. In Proceedings of 15th INCOSE International Symposium
(IS2015). INCOSE.
MEILICH, A. 2006. System of Systems (SoS) Engineering & Architecture Challenges in a Net Centric Envi-
ronment. In 2006 IEEE/SMC International Conference on System of Systems Engineering. IEEE.
MEYER, B. 1992. Applying Design by Contract. IEEE Computer 25, 10, 40–51.
MICHAEL, J., RIEHLE, R., AND SHING, M. -T. 2009. The verification and validation of software architecture
for systems of systems. In System of Systems Engineering, 2009. SoSE 2009. IEEE Intl. Conf. on. 1–6.
MILIUS, S., PELE SK A, J., AND SULZMANN, M. 2011. Contract Specification and Domain Specific Modeling
Language for GALS Systems An Approach to System Validation. Tech. rep., Technische Universit¨
at
Braunschweig. October.
MILLER, M., GRI ENDLI NG, K ., AND MAVR IS, D. 2012. Exploring human factors effects in the smart grid
system of systems demand response. In System of Systems Engineering (SoSE), 2012 7th International
Conference on. IEEE, 1–6.
MITTAL, S., ZEIGLER, B., MARTIN, J., SAHI N, F., AND JAMSHIDI, M. 2009. System of Systems – Innovations
for the 21st Centery. Wiley, Chapter 5. Modeling and Simulation for Systems of Systems Engineering,
101–149.
M¨
ULLER -MERB ACH , H. 1994. A System of Systems Approaches. Interfaces 24, 4, pp. 16–25.
NIELSE N, C. B. AND LARSEN, P. G. 2012. Extending VDM-RT to Enable the Formal Modelling of System of
Systems. In 7th Intl. Conf. on System of System Engineering, IEEE SoSE 2012. IEEE.
NIELSE N, C. B., LAUSDA HL, K., AND LARSEN, P. G. 2012. Combining VDM with Executable Code. In
Abstract State Machines, Alloy, B, VDM, and Z, J. Derrick, J. Fitzgerald, S. Gnesi, S. Khurshid,
M. Leuschel, S. Reeves, and E. Riccobene, Eds. Lecture Notes in Computer Science, vol. 7316. Springer-
Verlag, Berlin, Heidelberg, 266–279. ISBN 978-3-642-30884-0.
NOAM, E. M. 1994. Beyond Liberalization : From the Network of Networks to the System of Systems.
Telecommunications Policy 18, 4 (May), 286–294.
NORTHROP, L., FEILER, P., GABRIEL, R. P., GOODENOUGH, J., LINGER, R., L ON GST AFF, T., KAZMAN,
R., KLEIN, M., SCHMIDT, D., SU LLI VAN, K., AND WAL LNA U, K. 2006. Ultra-large-scale systems: The
software challenge of the future. Tech. rep., Software Engineering Institute, Carnegie Mellon University,
Pittsburgh, PA. June.
OUSD(AT&L), DOD. 2008. Systems and Software Engineering. Systems Engineering Guide for Systems of
Systems. Tech. Rep. Version 1.0., Office of the Deputy Under Secretary of Defense for Acquisition and
Technology, Department of Defense. August.
OWENS, A. W. A. 1995. Dominant Battlespace Knowledge. Institute for National Strategic Studies, Chapter
The Emerging U.S. System-of-Systems.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:36 C. B. Nielsen et al.
PAYNE , R., BRYANS, J., FITZGERALD, J., AND RIDDLE, S. 2012. Interface Specification for System-of-
Systems Architectures. In 7th Intl. Conf. on System of System Engineering, IEEE SoSE 2012. IEEE.
PAYNE , R. J. AND FITZGERALD, J. S. 2010. Evaluation of Architectural Frameworks Supporting Contract-
based Specification. Tech. Rep. CS-TR-1233, School of Computing Science, Newcastle University. Dec.
PEI, R. S. 2000. Systems of Systems Integration (SoSI)-A Smart Way of Acquiring Army C4I2WS Systems.
In Summer Computer Simulation Conference.
PELESK A, J. 2013. Industrial-Strength Model-Based Testing - State of the Art and Current Challenges.
Electronic Proceedings in Theoretical Computer Science abs/1303.1006, 3–28.
PELESK A, J., HONISC H, A., LAPSCHIES, F. , L ¨
ODING, H., SCHMID, H., SMUD A, P., VOROBEV, E., AND
ZAHLTEN, C. 2011. A Real-World Benchmark Model for Testing Concurrent Real-Time Systems in the
Automotive Domain. In Intl. Conf. on Testing Software and Systems. ICTSS 2011. LNCS, vol. 7019.
Springer, 146–161.
PHADKE, M. S. 1989. Quality Engineering Using Robust Design. Prentice Hall.
POLACK, F. AND STEPN EY, S. 2005. Emergent Properties do not Refine. Electr. Notes Theor. Comput.
Sci. 137, 2, 163–181.
RICKER, F. R. AND KALA KO TA, R. 1999. Order fulfillment: The hidden key to e-commerce success. Supply
Chain Management Review 11, 3 (Fall), 60–70.
RING, J. AND MADNI, A. 2005. Key challenges and opportunities in ’system of systems’ engineering. In
Systems, Man and Cybernetics, 2005 IEEE International Conference on. 973–978.
ROSCOE , A. W. 2010. Understanding Concurrent Systems. Springer.
RTCA SC-205/EUROCAE WG-71. 2011a. Model-Based Development and Verification Supplement to DO-
178C and DO-278A. Number RTCA/DO-331. RTCA, Inc., 1140 Connecticut Avenue, N.W., Suite 1020,
Washington, D.C. 20036.
RTCA SC-205/EUROCAE WG-71. 2011b. Software Considerations in Airborne Systems and Equipment
Certification. Number RTCA/DO-178C. RTCA, Inc., 1140 Connecticut Avenue, N.W., Suite 1020, Wash-
ington, D.C. 20036.
SAGE, A. P. AND CUPPAN, C. D. 2001. On the Systems Engineering and Management of Systems of Systems
and Federations of Systems. Inf. Knowl. Syst. Manag. 2, 4 (December), 325–345.
SAHIN, F., JAMSHIDI, M., AND SRIDHAR, P. 2007. A Discrete Event XML based Simulation Framework for
System of Systems Architectures. In IEEE Intl. Conf. on System of Systems Engineering 2007, SoSE ’07.
SANDERS, J. W. AND SMITH, G. 2012. Emergence and Refinement. Formal Aspects of Computing 24, 1,
45–65.
SCHNEIDER, D. AND TRA PP, M. 2009. Runtime Safety Models in Open Systems of Systems. Dependable,
Autonomic and Secure Computing, IEEE International Symposium on, 455–460.
SELBERG, S. A. AND AUSTIN, M. A. 2008. Toward an Evolutionary System of Systems Architecture. Tech.
rep., Institute for Systems Research., INCOSE.
SHARAWI , A., SAL A-DIAKANDA, S. N., QUIJADA, S., YOUSEF, N., RABELO, L., AND SEP ULVE DA , J. 2006.
A Distributed Simulation Approach for Modeling and Analyzing System of Systems. In 2006 Winter
Simulation Conference.
SHENHAR, A. J. 1994. A New Systems Engineering Taxonomy. In 4th Annual International Symposium of
The National Council on Systems Engineering. Vol. 2. 261–276.
SHENHAR, A. J. AND BONEN, Z. 1997. The New Taxonomy of Systems: Toward an Adaptive Systems Engi-
neering Framework. Systems, Man and Cybernetics, 2005 IEEE Intl. Conf. on 27, 2, 137–145.
SLEDGE, C. A. 2006. Army ASSIP Systems-of-Systems Test Metrics Task. Tech. rep., Software Engineering
Institute, Carnegie Mellon University. November. CMU/SEI-2006-SR-011.
SOMMERVILLE, I., CLI FF, D., CAL INE SC U, R., KEEN, J., KE LLY, T., KWIATKOWSKA, M., MCDERMI D, J.,
AND PAIGE, R. 2012. Large-scale complex IT systems. Communications of the ACM 55, 7.
SPILLNER, A., LINZ, T., AND SCHAEFER, H. 2006. Software Testing Foundations. dpunkt.verlag.
SPRING INTVE LD, J., VAANDRAGER, F., AND D’ARGEN IO, P. 2001. Testing Timed Automata. Theoretical
Computer Science 254, 1-2 (March), 225–257.
SysML1.2 2010. OMG Systems Modeling Language (OMG SysMLTM). Tech. Rep. Version 1.2, SysML Mod-
elling team. June. http://www.sysml.org/docs/specs/OMGSysML-v1.2-10-06-02.pdf.
TAGUCHI, G. 1987. System of Experimental Design, Volume 1 & 2. UNIPUB/Kraus Intl Publications.
TATS UM I, K . 1987. Test Case Design Support System. In International Conference on Quality Control
(ICQC), Tokyo. 615–620.
UNITED ST ATE S, CONGRESS, SENATE, COMMITTEE ON ARMED SERVICES. 1988. Restructuring of the
Strategic Defense Initiative (SDI) Program: US Senate. University of Michigan Library.
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:37
VAANDRAGER, F. 2012. Active Learning of Extended Finite State Machines. In 24th IFIP International
Conference on Testing Software and Systems.,ICTSS 2012. Number 7641 in LNCS. Springer, 5–7.
VALERDI, R., A XELBAND, E., THOMAS BAEHREN, B. B., DORENBOS, D., JA CKSON, S., MADNI, A.,
NADLER, G., ROBITAILLE, P., AND SETTLES, S. 2008. A Research Agenda for Systems of Systems
Architecting. Int. J. System of Systems Engineering 1, 1/2, 171–188.
WHITE, B. AND JEA N, P. 2011. Case study in System of Systems Engineering: NASA’s Advanced Com-
munications Technology Satellite. In System of Systems Engineering (SoSE), 2011 6th International
Conference on. 237 –244.
WICKRAMASINGHE, N., CHAL ASA NI , S., BOPPA NA , R., AND MADNI, A. 2007. Healthcare System of Sys-
tems. In System of Systems Engineering, 2007. SoSE ’07. IEEE International Conference on. 1 –6.
WOODCOCK, J. AND CAVALCA NT I, A . 2002. The Semantics of Circus. In 2nd International Conference of B
and Z Users on Formal Specification and Development in Z and B. ZB ’02. Springer-Verlag, 184–203.
WOODCOCK, J., CAVAL CANTI , A., FITZGERALD, J., LARSEN, P., MIYA ZAWA , A., AND PERR Y, S. 2012. Fea-
tures of CML: a Formal Modelling Language for Systems of Systems. In SoSE 2012. IEEE.
WOODCOCK, J., LA RSEN, P. G., BICARREGUI, J., AND FITZGERALD, J. 2009. Formal Methods: Practice and
Experience. ACM Computing Surveys 41, 4 (October), 1–36.
WOODCOCK, J. C. P. AND CAVALCAN TI, A. L. C. 2001. A Concurrent Language for Refinement. In
IWFM’01: 5th Irish Workshop in Formal Methods. BCS Electronic Workshops in Computing.
ZAVE, P. 1993. Feature Interactions and Formal Specifications in Telecommunications. Computer 26, 8, 20–
28.
Received .; revised .; accepted .
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:38 C. B. Nielsen et al.
A. OVERVIEW OF STATE OF PRACTICE
An overview of the current state of practice within SoS engineering and development
is supplied in Table II. The table lists the four focus areas with the eight dimensions
derived in Section 3 and for each combination the current state of practice is assessed
based on the literature surveyed in the current section.
Table II. Overview of State of Practice
Characteristics Modelling/Arch. Simulation Verification Testing
Autonomy Agent-based Agent-based Model checking Independent
DT&E, OT&E
Independence Independent
constituent
system model
Using stubs Compositionality Independent
DT&E, OT&E
Distribution HLA and
DEVS
Simulators
combined
using HLA
Theorem
proving, model
checking
Conformance
testing,
Interoperability
testing
Evolution Software
engineering
principles
No best
practices
available
Theorem
proving, model
checking
Repeated
interoperability
testing
Dynamic
Reconfiguration
Domain
specific
languages
Domain
specific
languages
Theorem
proving, model
checking
No best
practices
available
Emergence of
Behaviour
No best
practices
available
No best
practices
available
Refinement End-to-end test-
ing
Interdependence Interface-level Interface-level Compositionality End-to-end test-
ing
Interoperability HLA and
architectural
frameworks
HLA and
DEVS
Compositionality Conformance
testing and
interoperability
testing
strategies
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:39
B. OVERVIEW OF CHALLENGES IN SOS ENGINEERING
An overview of the research challenges in model-based techniques for SoS engineering
is supplied in Table III. The table lists the four focus areas with the eight dimensions
derived in Section 3 and for each combination the key research challenge is assessed
based on the literature surveyed in the current section.
Table III. Overview of challenges in SoS Engineering
Characteristics Modelling/Arch. Simulation Test Verification
Autonomy Higher
abstraction
level
Human
behaviour and
constituents’
interactions
No SoS-specific
challenges
Unification of
modelling
techniques
Independence Sharing and
trust
Combining
models and
distributed
simulation
No SoS-specific
challenges
Unification of
modelling
techniques
Distribution Describing
independent
constituent
platforms
Concurrency,
consistency
and
communication
faults
Complexity Unification of
modelling
techniques
Evolution Ease of
evolving
models
Handling
evolving
semantics
Incremental
test modelling
Evolving
verification
arguments
Dynamic Re-
configuration
Ability to
express
dynamic
evolution
Communicating
dynamic
scenarios
Dynamic object
management
Techniques for
abstraction
and composi-
tionality
Emergence of
Behaviour
Ability to
express desires
Detecting
emergent
properties
appearing
Specification of
SoS test
objectives
Techniques for
abstraction
and composi-
tionality
Interdependence Ability to
express
rely/guarantee
policies
Detecting
violations
during
simulation
Complexity,
specification of
SoS test
objectives
Unification of
verification
technologies
Interoperability Well-founded
interfaces and
desired policies
Including
aspects as data
formats,
architectures
and hardware
Complexity,
specification of
SoS test
objectives
Unification of
verification
technologies
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
XXXX:40 C. B. Nielsen et al.
C. OVERVIEW OF A STRENGTHENED SOS ENGINEERING DISCIPLINE
This section provides a summation of the areas for future research and improved de-
velopment methodologies, in order to establish a firmer foundation for model-based
SoS engineering. An overview of each discussed area is supplied in Table IV. The table
lists the four focus areas with the eight dimensions derived in Section 3 and for each
combination the areas of research interest are summarized.
Table IV. Overview of a Strengthened SoS Engineering Discipline
Characteristics Modelling/Arch. Simulation Test Verification
Autonomy Expressing
capabilities
and behaviour
formally
Human-in-the-
loop
simulation
“local” MBT Verification of
self-awareness
properties
Independence Divided models
and
well-founded
sharing rules
Individual and
joint model
simulation
“local” MBT Theories for
modelling
unification and
confidentiality
Distribution Modelling of
consistency
and
communication
faults
Simulating
platform
characteristics,
consistency
and faults
Contract-based
test models
Extending
existing
verification of
concurrency
and
distribution
Evolution Well-founded
language
extensibility
Simulator
extensibility
for language
extensions
Incremental
test modelling
Theories for
system
composition
Dynamic Re-
configuration
Expressing
dynamically
changing
infrastructures
and error
handling
Easy
communication
to
non-technical
stakeholders
Dynamic object
cre-
ation/deletion
in test models
Verification of
adaptive
properties and
system
composition
Emergence of
Behaviour
Expressing
desirable and
undesirable
behaviours
Identification
of emergence
on the basis of
policies
Deriving
verification
theories from
more universal
research in
emergent
behaviours
Contract-based
test models,
test objectives
for emergent
properties
Interdependence Well-founded
policy for
desired
properties
Detecting data
flow and
connection
policy
violations,
Test objectives
for emergent
properties
Techniques for
verifying
compliance
and
information
flow
Interoperability Well-founded
and detailed
constituent
interfaces
Co-simulation
to enable
heterogeneous
simulation
Test objectives
for emergent
properties
Unification of
verification
methods for
data integrity
and composi-
tionality
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
SoS Engineering: Basic Concepts, Model-based Techniques, and Research Directions XXXX:41
D. OVERVIEW OF SOS DEFINITIONS
Table V. Historical overview of publications defining SoS
Author(s) Main characteristics Section
Boulding 1956 Static structure of “open systems” from different disci-
plines
2.1
Ackoff 1971 System science organising systems into structured frame-
work
2.1
Eisner et al. 1991 Meta-systems engineering framework combining inde-
pendent systems
2.5
Shenhar et al. 1994 Taxonomy with technological uncertainty and scope 2.3
Noam94 Telecommunication infrastructure moving from “network
of networks” to an SoS
2.4
Manthorpe 1996 Focus on the jointness between C4I in a defence setting 2.4
Kotov 1997 Large-scale concurrent and distributed systems 2.4
Lukasik 1998 The importance of educating of engineers to deal with
evolving self-organizing systems
2.5
Maier 1998 Most influential paper defining the OMGEE characteris-
tics of SoS
2.2
Roe 1999 Systems engineering process for military SoS 2.5
Cook et al. 1999 SoS as a systems methodology for military systems with
concerns for hierarchy, emergence, and C2
2.7
Krygiel 1999 Focus on interoperability of information and data sharing 2.7
Pei 2000 SoS as a defining factor in future battlefield scenarios 2.4
Carlock et al. 2001 Enterprise Systems Engineering point of view 2.4
Sage et al. 2001 Use of the strategy “new federalism” for organisational
structuring
2.7
Chen et al. 2003 Focus on the SoS environment with a core in architecture
interoperability and dynamic behaviour
2.5
Keating et al. 2003 SoS as a meta-system of interrelated complex subsystems 2.5
Bar-Yam et al. 2004 Derives SoS characteristic from the fields military, biol-
ogy and sociology
2.7
Crossley 2004 SoS as a multidisciplinary research in interoperability,
individual behaviour and human behaviour
2.5
De Laurentis et al. 2005 Three dimension taxonomy for SoS analysis and design 2.3
Abbott 2006 Open at the top, open at the bottom and continually evolv-
ing, but slowly
2.2
Boardman et al. 2006 The alphabet characteristics: autonomy, belonging, con-
nectivity, diversity and emerging
2.2
Boehm 2006 Software-Intensive SoSs 2.4
Cocks 2006 An SoS may not be as much about the system mission,
but about the architecture of the selected solution
2.4
Fisher 2006 Composition of autonomous systems is an SoS 2.7
Sharawi et al. 2006 Independence, interoperability and global goal are essen-
tial SoS concepts for modelling and simulation
2.7
DOD-SoS Engineering 2008 SoS is an arrangement of independent and useful systems
that integrate to delivers unique capabilities
2.6
Karcanias 2010 Evolution of the notion of Composite Systems to include
automony and independence.
2.3
Cantot 2011 Independently acquired systems operated in order to
maximise the performance of the global operation
2.5
INCOSE 2015 Multiple heterogeneous distributed systems addressing
large-scale and interdisciplinary problems; entail am-
biguous requirements, unclear boundaries and interfaces
2.6
ACM Computing Surveys, Vol. V, No. N, Article XXXX, Publication date: 2015.
