ArticlePDF Available

Abstract and Figures

Combining system dynamics (SD) modelling with other research methods serves to articulate complex problems and explore potential solutions and policies. A growing number of studies draw on SD in combination with at least one other method, but there is hardly any knowledge about why, when, and how to make such combinations. We address this gap by conducting a systematic literature review of studies that have combined SD with at least one other method. Our findings are synthesised in an evidence-based framework that demonstrates why, when, and how SD is combined with other methods. This framework provides a point of reference for those who want to go beyond stand-alone SD modelling. In addition, this paper contributes to the multi-methodology literature by consolidating an area in which substantial experience in combining methods has been gained.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=tjsm20
Download by: [Eindhoven University of Technology] Date: 05 January 2018, At: 00:43
Journal of Simulation
ISSN: 1747-7778 (Print) 1747-7786 (Online) Journal homepage: http://www.tandfonline.com/loi/tjsm20
Why, when, and how to combine system dynamics
with other methods: Towards an evidence-based
framework
Mohammadreza Zolfagharian, A. Georges L. Romme & Bob Walrave
To cite this article: Mohammadreza Zolfagharian, A. Georges L. Romme & Bob Walrave (2018):
Why, when, and how to combine system dynamics with other methods: Towards an evidence-
based framework, Journal of Simulation
To link to this article: https://doi.org/10.1080/17477778.2017.1418639
Published online: 04 Jan 2018.
Submit your article to this journal
View related articles
View Crossmark data
JOURNAL OF SIMULATION, 2018
https://doi.org/10.1080/17477778.2017.1418639
KEYWORDS
System dynamics; multi-
methodology; systematic
review; modelling; evidence;
simulation
ARTICLE HISTORY
Received 5 December 2016
Revised5 December 2017
Accepted12 December 2017
© Operational Research Society 2018
CONTACT A. Georges L. Romme a.g.l.romme@tue.nl
ORIGINAL ARTICLE
Why, when, and how to combine system dynamics with other methods:
Towards an evidence-based framework
MohammadrezaZolfagharian, A. Georges L.Romme and BobWalrave
School of Industrial Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
ABSTRACT
Combining system dynamics (SD) modelling with other research methods serves to articulate
complex problems and explore potential solutions and policies. A growing number of studies
draw on SD in combination with at least one other method, but there is hardly any knowledge
about why, when, and how to make such combinations. We address this gap by conducting a
systematic literature review of studies that have combined SD with at least one other method.
Our ndings are synthesised in an evidence-based framework that demonstrates why, when,
and how SD is combined with other methods. This framework provides a point of reference for
those who want to go beyond stand-alone SD modelling. In addition, this paper contributes to
the multi-methodology literature by consolidating an area in which substantial experience in
combining methods has been gained.
1. Introduction
Combining two or more research methods may serve
to articulate complex problems and develop potential
solutions in more profound ways than a single-method
study can do. In this respect, each research method
has its strengths and weaknesses. For instance, system
dynamics (SD) models produce feedback-driven expla-
nations of complex system behaviour (Schwaninger,
2006) and are instrumental in investigating multiple
interacting processes and feedback loops, time delays,
and other non-linear eects (Romme, 2004; Rudolph &
Repenning, 2002; Walrave, Van Oorschot, & Romme,
2011). Moreover, SD models can be used to combine
qualitative and quantitative aspects of a dynamic phe-
nomenon (Sterman, 2000).
While SD modelling thus serves to understand a
dynamic phenomenon at a rather high abstraction
level, one weakness is its inability to model ‘low level’
interactions. More specically, SD is less equipped to
address and handle the highly divergent viewpoints
existing within complex networks of dierent actors,
especially when issues of power arise (Lane & Oliva,
1998). ese micro-interactions are oen found in
complex socio-economic phenomena characterised by
many agents with highly specic rules and behaviours
(Schieritz & Milling, 2003).
Moreover, the validation of SD models has been crit-
icised for relying too much on informal and subjective
procedures (Barlas, 1996; Barlas & Carpenter, 1990;
Zolfagharian, Akbari, & Fartookzadeh, 2014). Last, but
not least, while the non-predictive nature of SD models
and especially the concept of ‘policy insight’ distinguish
SD from other methods, problem owners and stakehold-
ers tend to expect objective policy solutions for their
problems (Lane, 2012). e nature of policy-making in
SD models is thus oen antagonistic to the expectations
of key stakeholders in a particular policy domain.
SD scholars increasingly draw on multi-method
approaches to overcome these limitations of stand-alone
SD modelling. In this respect, multi-method approaches
appear to result in more useful research outcomes, more
comprehensive models, and opportunities for generating
more valid inferences (Ivankova & Kawamura, 2010).
For example, Schwaninger (2004) combines SD with via-
ble systems modelling in studying a technology transfer
system, to better understand and model the behaviour of
various actors. Swinerd and McNaught (2014) combine
SD with agent-based modelling in a study of the diu-
sion of technological innovation to incorporate feedback
between modules in a continuous, uid process that
involves a large number of dierent agents. Yearworth
and White (2013) use grounded theory to improve the
process of qualitative data analysis and validation as
well as to provide a more rigorous approach towards
SD modelling. ese examples suggest that combin-
ing SD with other methods serves to more profoundly
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
2 M. ZOLFAGHARIAN ET AL.
articulate complex problems and, subsequently, analyse
and develop eective policies.
More broadly, new ways to mix and triangulate
research methods have been emerging in the social
and behavioural sciences (e.g., Mingers & Brocklesby,
1997; Tashakkori & Teddlie, 2010). Moreover, manage-
ment scientists have explored the challenges of ‘mul-
ti-methodology’ (Bennett, 1985; Flood, 1995a; Howick
& Ackermann, 2011; Jackson, 1997a, 1997b; Kotiadis &
Mingers, 2006; Midgley, 2000; Mingers & White, 2010;
Munro & Mingers, 2002; Pollack, 2009). In addition,
philosophers have explored the challenges arising from
methodological and theoretical pluralism (Biesta, 2010;
Greene & Hall, 2010; Johnson & Gray, 2010). However,
the body of (codied) knowledge arising from these
three literatures is limited. While some authors have
attempted to provide guidelines for combining SD with
one specic method, such as discrete event simulation
(e.g., Morgan, Howick, & Belton, 2017), there is no com-
prehensive overview of methods that have been com-
bined with SD. is lack of overview may demotivate
scholars to actually combine SD with other methods
– despite the advantages that can be gained from such
combinations.
In this paper, we review the literature to provide
an overview of which methods are combined with
SD, and why, when, and how these combinations are
made. To address these questions, we draw on a sys-
tematic literature review. e next section outlines the
review approach. Subsequently, we review the litera-
ture, classify the results, and synthesise these results in
an evidence-based framework. Finally, we discuss the
implications for SD modelling practice, and conclude
with recommendations for future work in this area.
2. Method
We adopt a systematic review approach to identify, ana-
lyse, and consolidate relevant sources of data. Originally
developed in medicine, a systematic review involves a
comprehensive, explicit, and replicable review of rele-
vant literature regarding a particular question of interest
(Traneld, Denyer, & Smart, 2003). Generally, systematic
reviews involve three main stages: planning, conducting,
and reporting the review (Traneld et al., 2003). In each
stage, the reviewer strives ‘to report as accurately as pos-
sible what is known and not known about the questions
addressed in the review’ (Briner & Denyer, 2012, p. 115).
In the planning stage, we set out to assess the most
important and inuential papers in which SD is com-
bined with other methods. e ISI Web of Science (WoS)
was selected as the database. We initially searched this
database with the query ‘System Dynamics’ (in title,
abstract, key words, and/or keywords that WoS sug-
gests for each record), which resulted in 15,008 arti-
cles covering the period from 1981 to January 1 2017.
Subsequently, we rened the results by focusing on the
domains of ‘Management’ and/or ‘Business’, the core
areas for which SD methodology was initially designed
(Forrester, 1961). According to the Scope Notes of
WoS, the ‘Management’ category covers management
science, organisation studies, strategic planning and
decision-making methods, leadership studies, and total
quality management. In addition, the ‘Business’ cate-
gory covers all aspects of business, including for example
marketing, forecasting, administration, compensation,
strategy, and retailing, but also business history and
business ethics. is resulted in 1581 papers (see also
Appendix 1).
Finally, to focus on the more inuential papers, we
selected papers from the following journals: System
Dynamics Review (342 papers identied), Journal
of the Operational Research Society (101 papers),
European Journal of Operational Research (91 papers),
Systems Research and Behavioural Science (81 papers),
Technological Forecasting and Social Change (50 papers),
and Systemic Practice and Action Research (17 papers).
ese six journals publish most articles drawing on SD
modelling and accounted for 73% of the relevant papers
in the category of management and business before 2017.
Other journals such as Management Science, Academy
of Management Journal, Administrative Science
Quarterly, and Journal of Simulation were excluded from
the list, because they have published no, or hardly any,
multi-method SD papers.
In total, 675 articles were selected for further review.
Subsequently, we planned to look for papers that delib-
erately combine SD with some other method. Notably,
there is no clear agreement in the literature on the
meaning of ‘method’: authors conceive of ‘methods’,
‘methodologies’ as well as ‘techniques’ dierently and,
thus, in some studies the term ‘method’ refers to some
specic technique (e.g., algorithm), whereas in other
papers the same term refers to methodology. Mingers
and Brocklesby (1997) therefore deliberately avoid any
attempt to dene ‘method’. Accordingly, we adopt a
more inductive approach in this paper, as Howick and
Ackermann (2011) also do in their review of mixing
operational research methods. is inductive approach
implies we select those papers that explicitly use terms
such as ‘integrating SD with, ‘combining SD with, ‘syn-
thesise SD with, ‘use SD with’, ‘supplement SD with,
and/or ‘enhance SD through’ (Table 1 provides a list of
phrases used in the papers). We searched for these and
similar clues in the title and abstract and/or keywords of
the articles, assuming that a key role of multi-method(ol-
ogy) in any paper would also be explicitly reected in
its title, abstract, or keywords. To make the claim that
SD is combined with something else, a denition of the
accepted body of SD modelling is needed. To avoid a
(possibly very complex) conceptual denition of a eld
that continues to evolve, we assumed that Sterman’s
(2000) widely used textbook adequately represents the
current body of knowledge on SD.
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 3
Table 1.Benefits arising from combining SD modelling with another method.*
*Note that different text phrases may implicate different multi-method designs; these phrases have only been used to determine whether the full text of the paper should be examined in detail or not.
Methods combined with SD Exemplary quotes that indicate combining SD with other methods in the papers reviewed
Increased con-
dence & rigour
in obtaining
and quantifying
variables
Inclusion of mul-
tiple attributes &
perspectives of
agents
Structures &
processes that sup-
port intervention &
implementation
Phase of SD
modelling that
is enriched
1. Fuzzy methods “A System Dynamics Simulation with Fuzzy Sets” (Sanatani, 1981, p. 313) Simulation
2. Control theory “integrate system dynamics and control theory” (Joglekar & Ford, 2005, p. 73) Simulation
3. Qualitative theory of non-linearity “application of qualitative analysis techniques to system dynamics” (Toro & Aracil, 1988, p. 56) Simulation
4. Genetic algorithm “a new optimization method for SD” (Duggan, 2008, p. 97). Simulation
5. Coevolutionary algorithm “provide an architecture to demonstrate the potential for the use of coevolution to explore the policy space in SD
models” (Liu et al., 2012, p. 363)
  Simulation
6. Artificial neural network “An Integrated Artificial Neural Network And System Dynamic Approach in Support of the Viable System Model”
(Azadeh et al., 2014, p. 236)
Simulation
7. Recurrent neural network “a policy design method for system dynamics models based on recurrent neural network” (Chen et al., 2011; : 369) Simulation
8. Qualitative simulation (QS) “integrating qualitative simulation (QS) procedures with system dynamics models” (Dolado, 1992, p. 55) Conceptualisation
9. Differential evolution “differential evolution is used to configure the model” (Swinerd & McNaught, 2015, p. 330) Simulation
10. Optimisation (linear program-
ming)
“methodologically synthesises optimization and system dynamics” (Olaya & Dyner, 2005, p. 1123) Simulation
11. Multiple-criteria decision analysis
(MCDA)
“integrated use of SD and MCDA” (Santos et al., 2008, p. 762) Simulation
12. Decision tree analysis “Evaluating system dynamics models of risky projects using decision trees” (Tan et al., 2010, p. 1) Simulation
13. Taguchi method “Taguchibased approach isapplied and tested in the context of a SD corporate model” (Hadjis, 2011, p. 374) Simulation
14. Conjoint analysis “develop a system dynamics model and combine it with” (Kopainsky et al., 2012, p. 575)   Simulation
15. Discrete event simulation “Combining discrete-event simulation and system dynamics” (Viana et al., 2014, p. 196) Simulation
16. Queuing theory “to supplement the strategic'macro' model [SD model] with a 'micro', OR-type model [queuing modelling]”
(Homer, 1999, p. 139)
Simulation
17. Soft systems methodology (SSM) “Combining Soft Systems Methodology (SSM) and System Dynamics (SD)” (Rodríguez-Ulloa & Paucar-Caceres,
2005, p. 303)
Conceptualisation
18. Viable systems modelling (VSM) “Using the Viable Systems Model to Structure a System Dynamics” (Haslett & Sarah, 2006, p. 273) Conceptualisation
19. Critical systems heuristics (CSH) “implementation of SSMand CSH to enhance SD” (Setianto et al., 2014, p. 642) Conceptualisation
20. Agent-based modelling “use system dynamics theory to model it from the agent-based modelling perspective” (Wu et al., 2010; p. 858)   Conceptualisation
& Simulation
21. (Evolutionary) Game theory “SD model based on an evolutionary game” (Zhao et al., 2016, p. 265)   Conceptualisation
& Simulation
22. Exploratory modelling and anal-
ysis (EMA)
“combination of EMA and SD” (Kwakkel & Pruyt, 2015, p. 360)   Simulation
23. Robust decision-making (RDM) “RDM in combination with SD” (Auping et al., 2015, p. 499)   Conceptualisation
& Simulation
24. Delphi method “guide SD model experimentation” (Chen et al., 2012, p. 1265)   Conceptualisation
25. Grounded theory “bridging betweengrounded theory and system dynamics” (Yearworth & White, 2013, p. 151) Conceptualisation
26. Social fabric matrix “An integrated social fabric matrix/system dynamics” (Gill, 1996, p. 167) Conceptualisation
27. Geomapping “combined geomapping and modelling approach” (Evenden et al., 2006, p. 1411) Simulation
28. Cognitive mapping “used to complement system dynamics” (Coyle et al., 1999, p. 373) Conceptualisation
29. Patent analysis, bibliometrics &
scenario analysis
“integrating multiple methodologies” (Daim et al., 2006, p. 1009) Conceptualisation
& Simulation
30. Scenario planning (A factor choice
method through textual analysis)
“System/scenario duality – a supporting equivalence” (Powell, 2014, p. 1344)   Conceptualisation
31. Event map of scenarios*“integration of) scenario maps and system dynamics and Event map of scenarios” (Howick et al., 2006, p. 122) Conceptualisation
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
4 M. ZOLFAGHARIAN ET AL.
econometric methods, agent-based modelling, and
multi-objective optimisation;
systems methodologies such as viable systems mod-
elling and so systems methodology;
other methods such as grounded theory develop-
ment and the social fabric matrix.
e variety of these methods demonstrates SDs func-
tionality as a multi-method platform (discussed earlier).
In this respect, 34 of the reviewed papers combine SD
with other methods in order to understand and model
a particular problem more deeply and eectively. e
other 21 articles explicitly present a specic combination
of SD with another method as their main contribution,
and then illustrate this in a particular application.
3.2. When is SD combined with another method?
We identied the key characteristics of the research
problems and questions addressed in each study, which
appear to have motivated the author(s) of that study to
combine SD with at least one other method. Appendix
4 outlines the coding process for identifying these char-
acteristics. e following problem characteristics were
identied:
(1) Nature of the problem. is refers to the distinc-
tive features of the research problem addressed,
which may pertain to the problems structure
or complexity, such as a complex queuing sit-
uation or an inter-organisational collaboration
challenge.
(2) Variables and their causal relationships.
Examples are (high levels of) uncertainty, fuzz-
iness, linguistic unclarity, and intangibility of
incumbent variables, which in turn create major
diculties in formulating causal relations.
(3) Diversity of agents involved. is problem
characteristic refers to the diversity of agents
in terms of their views, interests and behav-
iours. Examples are systems involving multi-
ple agents with highly dierent interests and
points of view. e term ‘agent’ here refers to
any (organisational constellation or group of)
human(s) who is involved in and aected by the
problem or phenomenon addressed.
(4) Context of the problem. Examples are settings
that involve power asymmetry, closely moni-
tored by governments, or exposed to pressing
public concerns.
Notably, a research problem or question may have one
or more of these key characteristics. Moreover, it is also
possible that the research problem is not particularly
pronounced in any of these characteristics, but neverthe-
less provides an opportunity to combine SD with other
methods.
Furthermore, we only included articles that address a
specic empirical problem; this serves to exclude papers
on combining SD with other methods which are entirely
explorative or conceptual in nature. By applying these
criteria, the initial sample of 675 was reduced to 55 arti-
cles. Appendix 2 provides examples of studies that were
excluded from, or included in, the nal set.
In the conducting stage, we adopted a qualitative cod-
ing procedure (e.g., Saldaña, 2015). e rst author read
each paper to identify and extract information about why,
when, and how SD is combined with other methods. is
information was collected in a tabular format created in
Microso Excel. In line with the inductive nature of the
review approach (outlined earlier), we avoided the use of
any predetermined categories. e categories were thus
allowed to emerge, through comparing and nding com-
mon features. is coding process was iterative in nature,
to assure reasonable relations between categories and data
to emerge. For example, when a new potential category
emerged, the researcher would return to the papers already
read to explore if there was any related evidence. is cod-
ing process of category determination through continual
and iterative comparison proceeded until no new catego-
ries emerged and a saturation point was reached. e main
coding was done by the rst author. Another author coded
a sample of 25% of all papers (including those papers that
the rst coder was uncertain about). e inter-coder reli-
ability of the results from the initial double coding eort
was 96%; the remaining disagreements were subsequently
discussed and resolved by the two coders.
e obtained set of categories was subsequently
assessed by all three authors of this study. In particu-
lar, the set of benets (arising from combining SD with
another method) was also theoretically validated by con-
sulting relevant publications on research methodology.
is approach helped to ensure that each category is
accurately labelled and unambiguous in its meaning.
In the last stage, we identied several relationships
among the categories, to synthesise our ndings in terms
of why, when, and how SD is combined with other meth-
ods. e outcome is an evidence-based framework that
will be explained in Section 4. In the next section, we
report the main ndings of the systematic review.
3. Findings
In this section, we describe the main ndings of the sys-
tematic review. Appendix 3 provides an overview of all
papers reviewed, including several key coding outcomes.
3.1. Which methods are combined with SD?
e various methods combined with SD can be cate-
gorised in three main, somewhat overlapping, groups:
quantitative methods including statistical, op-
timisation and simulation approaches such as
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 5
Sterman, 2000; Wolstenholme, 1990). e conceptual-
isation phase involves the initial problem articulation,
with ‘words’ (verbal language) as the primary tool,
possibly extended with visual tools (such as causal
loop diagrams). Simulation refers to all activities in SD
modelling that include numbers and equations as the
primary vehicles. Table 1 lists the benets of combin-
ing SD modelling with other methods, as distilled from
the papers reviewed (see also Appendix 6 for a related
overview).
4. Towards an evidence-based framework
An evidence-based framework can now be distilled from
the review (see Figure 1 for an overview). e rst step
in this framework is to articulate the initial problem,
which also serves to establish whether or not SD can be
used to address the problem (Morecro, 2007; Sterman,
2000). e second step involves assessing the problem in
terms of (a) its nature, (b) key variables and their causal
relationships, (c) diversity of the agents involved, and (d)
its context. By and large, the rst two steps address the
question ‘when’ SD is combined with another method.
e third step explores why a particular combina-
tion would be appropriate. A candidate method can
be selected, based on the following criteria. First, the
mere number of key problem characteristics appears to
provide a good indicator of the level of (multi-method)
complexity. As such, candidate methods can be identi-
ed in one of the sections of Table 2(a–d) (with com-
plexity increasing from 2(a) to (d)). e conditions
outlined below each method in Table 2(a–d) serve to
identify methods that are likely to be appropriate. Once
a candidate method has been specied, one can check
the potential benets of the method in Table 1. In addi-
tion, the exemplary studies in Table 2(a–d) may inform
scholars about the experiences that others have obtained
in combining a particular method with SD.
e fourth step in the cycle is about ‘how’ to actually
get the intended combination implemented. As such,
the process of combining a particular method with SD
3.3. Why are combinations made?
We also reviewed all papers with regard to the bene-
ts arising from combining SD with other methods.
Appendix 5 outlines the coding procedure adopted to
identify and synthesise the ndings in general categories.
e following three generic benets were identied:
(1) Increased condence and rigour in obtaining and
quantifying variables. For example, Seth (1994)
combined SD with fuzzy set theory to more eas-
ily incorporate subjective beliefs and percep-
tions in the model. Kim and Andersen (2012)
used grounded theory to identify problems,
key variables, and their structural relationships
from purposive text data. Duggan (2008) used
genetic algorithms in simulation experiments to
discover the best strategies and solutions.
(2) Inclusion of multiple attributes and perspectives
of agents. For example, Coyle, Exelby, and Holt
(1999) used cognitive mapping to identify sev-
eral key processes arising from multiple agents
and their dierent points of view. Wu, Kefan,
Hua, Shi, and Olson (2010) adopted an agent-
based modelling perspective to better under-
stand the interaction of agents in a system.
(3) Structures and processes that support inter-
vention and implementation. For example,
Rodríguez-Ulloa, Montbrun, and Martínez-
Vicente (2011) used so systems methodol-
ogy to orchestrate and implement change in
social systems, based on a multi-paradigmatic
approach. Haslett and Sarah (2006) employed
viable systems modelling to develop the formal
organisational and political structures and pro-
cesses necessary to support a SD intervention
in a large bureaucracy.
In order to better understand these benets, they can
be assigned to two main phases in SD modelling: the
conceptualisation and simulation phase (Randers,
1980; Roberts, Andersen, Deal, Grant, & Shaer, 1983;
1. Define the initial problem
to be studied and modelled
2. Assess the problem
in terms of (a) its nature, (b) key
variables and their causal
relationships, (c) diversity of
agents involved, and (d) its
context
3. Select candidate method(s)
to be combined with SD, based on
its potential benefits, number of
key characteristics, and conditions
for using it
4. Implement
the selected method, by combining
it with SD
Figure 1.Four steps in combining SD with other methods.
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
6 M. ZOLFAGHARIAN ET AL.
Table 2a.Methods combined with SD for problems with only one key characteristic.
Key characteristic Methods and exemplary studies
Nature of the problem Control theory/methods
Resource management and allocation (Anderson et al., 2005); non-linear dynamic behaviour of supply chains (Joglekar & Ford,
2005; Spiegler et al., 2016)
Co-evolutionary algorithm
An optimization framework to explore policy options for inter-organisational models: in particular, where the model comprises
the interaction of sectors with distinct, intendedly rational decision rules (Liu et al., 2012)
Queuing modelling
If the problem involves questions about ‘how much and what configuration of service capacity will best meet randomly arriving
service demand’ (Homer, 1999, p. 149)
Discrete event simulation
If the problem involves stochasticity in waiting/queuing processes (Viana et al., 2014)
Variables and their
causal relationships
Fuzzy methods (fuzzy set theory, techniques, and logic)
If the problem includes fuzzy (instead of crisp) variables that can take on linguistic rather than precise numerical values (Kunsch
& Springael, 2008; Liu et al., 2011; Sanatani, 1981; Seth, 1994).
Optimisation
In case of high uncertainty in variables (Dangerfield & Roberts, 1999; Georgiadis & Athanasiou, 2013) and multi-objective optimi-
sation (Adamides et al., 2009)
Social Fabric Matrix
The collective wisdom collected and generated from this matrix is quantified, so that it can be used in SD models (Gill, 1996)
Patent Analysis, Bibliometrics, and Scenario Analysis
When no historical data are available and many different (political, cultural, etc.) factors appear to be relevant, but technical
trend analysis alone cannot capture the various organisational and political scenarios (Daim et al., 2006)
Diversity of agents
involved
Event map of scenarios
Valuing the client group by enabling them to visualise the links between scenarios (Howick et al., 2006)
Multiple-criteria decision analysis
Wide range of views by multiple stakeholders (Santos et al., 2008)
Cognitive mapping
Involving agents and the outcomes of human decision-making and sense-making (Coyle et al., 1999)
Context of the
problem
Geomapping
Especially useful when clustering of data (patterns) for separate geographical regions is required (Evenden et al., 2006)
Table 2b.Methods combined with SD for problems with two key characteristics.
Key characteristic Methods and exemplary studies
Nature of the problem
Variables and their causal relationships
Optimisation (linear programming)
Each component has its own specific features, but when analysed as a whole, a synthesised modelling
approach is needed – possibly also because of the large number of actors and transactions that require
optimisation of a linear objective function, subject to linear constraints. (Olaya & Dyner, 2005)
Decision tree analysis
If the problem involves sequential decision processes and requires backwards induction (Tan et al., 2010)
Nature of the problem
Diversity of agents involved
Genetic algorithms
Best suited for modelling sets of interacting agents using SD, where each agent has access to a number of
heuristics or decision rules that they use to control their system state (Duggan, 2008)
Nature of the problem
Context of the problem
(This combination of problem characteristics is not present in any of the papers reviewed)
Variables and their causal relationships
Diversity of agents involved
(Choice-based) Conjoint analysis
For multi-attribute choice problems: when behavioural policies of decision-makers include trade-offs
among multiple attributes (Schmidt & Gary, 2002) and/or tangible and intangible attributes are available
in addition to social dynamics (Kopainsky et al., 2012)
In case of divergent preferences of agents (Wang et al., 2016)
Scenario planning (factor choice method through textual analysis)
If the problem addressed is about creating possible futures of the system, by drawing on the ideas of the
key agents (Powell, 2014)
Agent-based modelling
In case of uncertain and conflicting information as well as conflicting objectives; when multiple agents
need to realise their goals subject to limited perception and behavioural capacity (Wu et al., 2010)
If the concerned problem involves decision-making by multiple individual agents and their interaction
with each other (Kolominsky-Rabas et al., 2015; Lamberson, 2016; Pasaoglu et al., 2016)
Agent-based modelling and dierential evolution method
If patterns of behaviour are arising from national decision-making within a social system of nations (Swin-
erd & McNaught, 2015)
Agent-based modelling using game theory concepts
If the problem involves time-dependent behaviour of a physical system and there is complex relationship
between the sequential (conflicting) actions of agents (Paez-Perez & Sanchez-Silva, 2016)
Variables and their causal relationships
Context of the problem
(This combination of problem characteristics is not present in any of the papers reviewed.)
Diversity of agents involved
Context of the problem
Soft Systems Methodology
For social issues that extend, beyond the purely technical, to both practical and emancipatory spheres of
interest of key actors involved; when there are multiple views that make the problem highly ambiguous;
and when the problem is publicly sensitive (Adamides et al., 2009; Rodríguez-Ulloa & Paucar-Caceres,
2005; Rodríguez-Ulloa et al., 2011).
Delphi method
In order to tap into the expertise and experience of many experts (Chen et al., 2012).
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 7
of SD modelling, especially when other methods (com-
bined or stand-alone) appear to be more appropriate
and promising.
As an illustration, assume you are interested in
a problem involving a high level of uncertainty and
ambiguity with regard to the key variables aecting this
problem, as well as highly conicting interests of the
stakeholders involved. If these are the two key character-
istics of the problem, the corresponding section in Table
2(b) labelled ‘variables and their causal relationships’
and ‘diversity of agents involved’ suggests three sets
of methods: conjoint analysis, scenario planning, and
agent-based modelling. e conditions outlined for each
of these methods suggest that agent-based modelling
might be a suitable approach to overcome the deciency
of SD in dealing with the problem at hand. According
to Table 2(b), all three methods are likely to increase the
condence and rigour in obtaining and quantifying var-
iables, and also serve to embrace multiple attributes and
perspectives of agents. However, while conjoint analy-
sis and scenario planning only enrich the simulation or
conceptualisation phase of SD modelling, agent-based
modelling may enhance both phases. Subsequently, one
can decide to study exemplary papers on combining
agent-based modelling with SD, such as the study by
Wu et al. (2010) listed in Table 2(b), to learn how others
have applied this specic combination of methods.
Notably, xed positions cannot be assumed for any
method. As Table 2(a–d) illustrates, some methods (e.g.,
optimisation) can be used for a variety of problems with
dierent characteristics. is observation also serves to
avoid the ‘imprisoning’ of any method (Midgley, 2000).
modelling can be customised in view of the specic
characteristics of the problem addressed. Table 2(a–d)
here provides examples that may provide learnings and
insights.
Figure 1 suggests that any eort to combine SD
with another method is likely to be highly iterative, as
reected in the clockwise as well as counter-clockwise
arrows. For example, aer the initial selection of a can-
didate method in step 3 and studying some of the exem-
plary studies in this area, the researcher might become
more familiar with some of the implications of the initial
choice, and thus decide to return to step 2 in order to
reassess the framing of the problem. Another example
is that scholars, while implementing their choice for a
particular method in combination with SD, discover new
characteristics that lead them to re-assess and redene
the problem addressed. e iterative nature of the frame-
work may even imply the decision to abandon the use
Table 2c.Methods combined with SD for problems with three key characteristics.
Key characteristic Methods and exemplary studies
Nature of the problem
Variables and their causal relationships
Diversity of agents involved
Exploratory modelling and analysis
In case of parametric uncertainties, orders of time delays, non-linear lookups, and profoundly divergent
views in the problem
If the problem is characterised by both dynamic complexity and deep uncertainty, and therefore divergent
hypotheses (e.g., about causal relations, parameter values, and table functions) exist (Kwakkel & Pruyt,
2013, 2015; Kwakkel et al., 2013; Pruyt & Kwakkel, 2014)
Robust decision-making
If the problem is messy and involves deep uncertainty as well as diverging stakeholder views regarding the
desirability of policy measures (Auping et al., 2015).
Evolutionary game theory
If the problem involves interactions and conflicts among players whose strategic behaviours are influenced
by their expected payoffs (Zhao et al., 2016)
Nature of the problem
Variables and their causal relationships
Context of the problem
(This combination of problem characteristics is not present in any of the papers reviewed.)
Nature of the problem
Diversity of agents involved
Context of the problem
Viable systems modelling (VSM)
Long-term capability development process, with client involvement and key emphasis on the context, struc-
ture and process of organisational change (Haslett & Sarah, 2006; Schwaninger, 2004)
Soft systems methodology and critical systems heuristics
If a complex system involves not only biophysical but also social, ecological, political, and economic dimen-
sions and engages a broad set of actors with different interests and power asymmetry (Setianto et al., 2014)
Variables and their causal relationships
Diversity of agents involved
Context of the problem
(This combination of problem characteristics is not present in any of the papers reviewed.)
Table 2d.Methods combined with SD for problems with four
key characteristics.
Key characteristic
Methods and exemplary
studies
Nature of the problem
Variables and their causal relationships
Diversity of agents involved
Context of the problem
Delphi technique and VSM
If the system is composed
of numerous subsystems
exposed to varying factors
and relationships (in-
volving many oscillations
and turbulences) and the
system’s performance and
future is very important for
policy-makers (Azadeh et
al., 2014)
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
8 M. ZOLFAGHARIAN ET AL.
and directs how research studies are conducted, and
the antithesis that paradigms are unimportant to how
research is actually conducted, especially in applied
elds (Tashakkori & Teddlie, 2010). In the sample of
studies reviewed in the previous section, only seven
papers include any comments or notes on the challenges
of multi-methodology. is does not necessarily imply
that the other studies draw on an aparadigmatic stance.
However, these results suggest that many SD modellers
may need to become more aware of the theoretical chal-
lenges of multi-method research, irrespective of whether
these challenges can be resolved or not. Some of the key
challenges arise from questions about paradigm incom-
mensurability and cultural tensions, especially the extent
to which organisational and academic cultures might
resist multi-paradigm work; and psychological barriers
that scholars may face when moving from one paradigm
to another (Mingers & Brocklesby, 1997; Tashakkori &
Teddlie, 2010).
e main limitation of this study arises from the set of
articles reviewed. Any attempt to review all studies that
combine SD with other methods, in various domains
and disciplines, is clearly not feasible within the con-
straints of a journal article. erefore, we focused on a
sample of papers published in a relatively small number
of journals in the management and business domain.
Future work in this area may serve to extend the scope
of the framework proposed.
Combining SD with other methods serves to articu-
late complex problems and explore potential solutions
and policies for these problems. We conducted a system-
atic literature review of studies that combine SD with at
least one other method, and then synthesised the nd-
ings in an evidence-based framework. is framework
serves to inform other scholars about why, when, and
how SD is combined with other methods.
Statement of contribution
A growing number of studies combines system dynam-
ics (SD) modelling with other methods, but there is a
lack of knowledge about why, when, and how to make
such combinations. We address this gap by means of a
systematic literature review of studies that have com-
bined SD with at least one other method. is results
in an evidence-based framework that synthesises the
fragmented literature on SD multi-methodology and
codies the experiences of scholars who have combined
multiple methods. is evidence-based framework may
thus help other scholars to more eectively articulate
complex problems and explore potential solutions and
policies. As such, this framework provides a point of
reference for those who wish to go beyond stand-alone
SD modelling. In addition, because there is hardly any
codied knowledge on why, when, and how (SD) simu-
lation modelling is combined with other methods, this
study contributes to the multi-methodology literature
5. Discussion
Our analysis shows that combining SD with other meth-
ods is becoming increasingly popular (78% of the papers
reviewed are published aer 2000; see also Appendix
7). Previous reviews in this area involve eorts to pro-
duce guidelines or frameworks for combining SD with
just one other method, such as so systems method-
ology (e.g., Lane & Oliva, 1998), discrete event simu-
lation (e.g., Chahal, Eldabi, & Young, 2013; Morgan et
al., 2017; Venkateswaran & Son, 2005) or agent-based
modelling (e.g., Lättilä, Hilletoh, & Lin, 2010; Swinerd
& McNaught, 2012). us, there is hardly any codied
knowledge on why, when, and how researchers combine
SD with any particular method chosen from a large port-
folio of possible methods. In this paper, we developed
an evidence-based framework that addresses these chal-
lenges. As such, this study synthesises the fragmented
literature on SD multi-methodology by codifying the
experiences of scholars who have combined multiple
methods. In this respect, the framework proposed in
the previous section is essentially descriptive (rather than
prescriptive) in nature, based on an inductive analysis of
a sample of studies. Future work may serve to translate
this descriptive framework into a robust prescriptive
tool.
e wide range of methods identied in our review
suggests that SD can serve as a platform for combining
various methods, that is an ‘umbrella for integrating
problem structuring methods’ (Kaempf & Ninios, 1998,
p. 1). ese multi-method platforms are needed because
many scholars are strongly inclined to adopt the method
they are most comfortable with, given their expertise
and experience (Flood, 1995b). To help scholars exploit
the full potential of SD as a multi-method platform, our
framework provides an overview of the opportunities
arising from other methods. However, this framework is
somewhat constrained by the limited number of papers
reviewed. In this respect, our ndings are preliminary
in nature, to be extended in future work.
Notably, the methods included in Table 2(a–d) are
not necessarily the most feasible and desirable ones,
but are potentially complementary to SD. erefore, the
choice to adopt a combined or stand-alone approach
to SD largely depends on the initial research question,
which in turn is likely to be inuenced by the back-
ground, skills and expertise of the researcher, interests
of the stakeholders, and available time, data, and other
resources (Brannen, 2005; Flood, 1995b; Marsland,
Wilson, Abeyasekera, & Kleih, 2000). Accordingly, for
many research questions a stand-alone SD study, when
skilfully conducted, will be the most eective and con-
sistent approach.
In the multi-methodology literature, there is ongoing
controversy about the paradigmatic and conceptual sta-
tus of mixing methods. ese debates can be placed on
a spectrum between the thesis that a paradigm guides
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 9
models. Systems Research and Behavioral Science, 28(4),
369–390.
Chen, H., Wakeland, W., & Yu, J. (2012). A two-stage
technology foresight model with system dynamics
simulation and its application in the Chinese ICT industry.
Technological Forecasting and Social Change, 79(7), 1254–
1267.
Coyle, R. G. (1985). e use of optimization methods
for policy design in a system dynamics model. System
Dynamics Review, 1(1), 81–91.
Coyle, J. M., Exelby, D., & Holt, J. (1999). System dynamics
in defence analysis: Some case studies. Journal of the
Operational Research Society, 50(4), 372–382.
Daim, T. U., Rueda, G., Martin, H., & Gerdsri, P. (2006).
Forecasting emerging technologies: Use of bibliometrics
and patent analysis. Technological Forecasting and Social
Change, 73(8), 981–1012.
Dangereld, B., & Roberts, C. (1999). Optimisation as a
statistical estimation tool an example in estimating the
AIDS treatment-free incubation period distribution.
System Dynamics Review, 15(3), 273–291.
Dolado, J. J. (1992). Qualitative simulation and system
dynamics. System Dynamics Review, 8(1), 55–81.
Duggan, J. (2008). Equation-based policy optimization for
agent-oriented system dynamics models. System Dynamics
Review, 24(1), 97–118.
Evenden, D., Harper, P. R., Brailsford, S. C., & Harindra, V.
(2006). Improving the cost-eectiveness of Chlamydia
screening with targeted screening strategies. Journal of the
Operational Research Society, 57(12), 1400–1412.
Fiddaman, T. S. (2002). Exploring policy options with a
behavioral climate–economy model. System Dynamics
Review, 18(2), 243–267.
Flood, R. L. (1995a). Solving Problem Solving: A Potent Force
for Eective Management. Chichester, UK: Wiley.
Flood, R. L. (1995b). Total Systems Intervention (TSI): A
Reconstitution. Journal of the Operational Research Society,
46(2), 174–191.
Forrester, J. W. (1961). Industrial Dynamics. Cambridge, MA:
MIT Press.
Georgiadis, P., & Athanasiou, E. (2013). Flexible long-
term capacity planning in closed-loop supply chains
with remanufacturing. European Journal of Operational
Research, 225(1), 44–58.
Gill, R. (1996). An integrated social fabric matrix/system
dynamics approach to policy analysis. System Dynamics
Review, 12(3), 167–181.
Greene, J. C., & Hall, J. (2010). Dialectics and pragmatism:
Being of consequence. In A. Tashakkori & C. Teddlie
(Eds.), Sage handbook of mixed methods in social &
behavioral research (2nd ed., pp. 119–143). ousand
Oaks, CA: Sage.
Hadjis, A. (2011). Bringing economy and robustness in
parameter testing: A Taguchi methods-based approach to
model validation. System Dynamics Review, 27(4), 374–
391.
Haslett, T., & Sarah, R. (2006). Using the viable systems model
to structure a system dynamics mapping and modeling
project for the Australian taxation oce. Systemic Practice
and Action Research, 19(3), 273–290.
Homer, J. B. (1999). Macro- and micro-modeling of eld
service dynamics. System Dynamics Review, 15(2), 139–
162.
Howick, S., & Ackermann, F. (2011). Mixing OR methods
in practice: Past, present and future directions. European
Journal of Operational Research, 215(3), 503–511.
by consolidating an area where substantial experience
in combining methods has been obtained.
Disclosure statement
No potential conict of interest was reported by the authors.
ORCID
A. Georges L. Romme http://orcid.org/0000-0002-3997-1192
References
Adamides, E. D., Mitropoulos, P., Giannikos, I., &
Mitropoulos, I. (2009). A multi methodological approach
to the development of a regional solid waste management
system. Journal of the Operational Research Society, 60(6),
758–770.
Anderson, E. G. (2011). A dynamic model of counterinsurgency
policy including the eects of intelligence, public security,
popular support, and insurgent experience. System
Dynamics Review, 27(2), 111–141.
Anderson, E. G., Morrice, D. J., & Lundeen, G. (2005).
e “physics” of capacity and backlog management in
service and custom manufacturing supply chains. System
Dynamics Review, 21(3), 217–247.
Auping, W. L., Pruyt, E., & Kwakkel, J. H. (2015). Societal
ageing in the Netherlands: A robust system dynamics
approach. Systems Research and Behavioral Science, 32(4),
485–501.
Azadeh, A., Darivandi Shoushtari, K., Saberi, M., & Teimoury,
E. (2014). An Integrated articial neural network and
system dynamics approach in support of the viable system
model to enhance industrial intelligence: e case of a
large broiler industry. Systems Research and Behavioral
Science, 31(2), 236–257.
Barlas, Y. (1996). Formal aspects of model validity and
validation in system dynamics. System Dynamics Review,
12(3), 183–210.
Barlas, Y., & Carpenter, S. (1990). Philosophical roots of
model validation: Two paradigms. System Dynamics
Review, 6(2), 148–166.
Bennett, P. G. (1985). On linking approaches to decision-
aiding: Issues and prospects. Journal of the Operational
Research Society, 36(8), 659–669.
Biesta, G. (2010). Pragmatism and the philosophical
foundations of mixed methods research. In A. Tashakkori
& C. Teddlie (Eds.), Sage handbook of mixed methods
in social & behavioral research (2nd ed., pp. 95–118).
ousand Oaks, CA: Sage.
Brannen, J. (2005). Mixed methods research: A discussion
paper. (NCRM Methods Review Papers NCRM/005).
Southampton, UK: ESRC National Centre for Research
Methods.
Briner, R. B., & Denyer, D. (2012). Systematic review and
evidence synthesis as a practice and scholarship tool. In
e Oxford (Ed.), Rousseau DM (pp. 112–129). Oxford,
UK: Handbook of Evidence-Based Management. Oxford
University Press.
Chahal, K., Eldabi, T., & Young, T. (2013). A conceptual
framework for hybrid system dynamics and discrete event
simulation for healthcare. Journal of Enterpr ise Information
Management, 26(1/2), 50–74.
Chen, Y., Tu, Y., & Jeng, B. (2011). A machine learning
approach to policy optimization in system dynamics
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
10 M. ZOLFAGHARIAN ET AL.
Kopainsky, B., Tröger, K., Derwisch, S., & Ulli-Beer, S. (2012).
Designing sustainable food security policies in sub-
Saharan African countries: How social dynamics over-ride
utility evaluations for good and bad. Systems Research and
Behavioral Science, 29(6), 575–589.
Lättilä, L., Hilletoh, P., & Lin, B. (2010). Hybrid simulation
models – When, why, how? Expert Systems with
Applications, 37(12), 7969–7975.
Lee, D. H., Park, S. Y., Kim, J. W., & Lee, S. K. (2013). Analysis
on the feedback eect for the diusion of innovative
technologies focusing on the green car. Technological
Forecasting and Social Change, 80(3), 498–509.
Liu, H., Howley, E., & Duggan, J. (2012). Co-evolutionary
analysis: A policy exploration method for system dynamics
models. System Dynamics Review, 28(4), 361–369.
Liu, S., Triantis, K. P., & Sarangi, S. (2011). Representing
qualitative variables and their interactions with fuzzy
logic in system dynamics modeling. Systems Research and
Behavioral Science, 28(3), 245–263.
Macedo, J. (1989). A reference approach for policy
optimization in system dynamics models. System
Dynamics Review, 5(2), 148–175.
Marsland, N., Wilson, I. M., Abeyasekera, S., & Kleih, U.
K. (2000). A methodological framework for combining
quantitative and qualitative survey methods. Reading, UK:
Statistical Services Centre, University of Reading.
Midgley, G. (2000). Systemic intervention: Philosophy,
methodology, and practice. New York, NY: Kluwer
Academic/Plenum.
Miller, B., & Clarke, J. (2007). e hidden value of air
transportation infrastructure. Technological Forecasting
and Social Change, 74(1), 18–35.
Mingers, J., & Brocklesby, J. (1997). Multimethodology:
Towards a framework for mixing methodologies. Omega,
25(5), 489–509.
Mingers, J., & White, L. (2010). A review of the recent
contribution of systems thinking to operational research
and management science. European Journal of Operational
Research, 207(3), 1147–1161.
Mohapatra, P. K. J., & Sharma, S. K. (1985). Synthetic design
of policy decisions in system dynamics models: A modal
control theoretical approach. System Dynamics Review,
1(1), 63–80.
Morecro, J. D. W. (2007). Strategic modeling and business
dynamics: A feedback systems approach. Chichester, UK: Wiley.
Morgan, J. S., Howick, S., & Belton, V. (2017). A toolkit of
designs for mixing discrete event simulation and system
dynamics. European Journal of Operational Research,
257(3), 907–918.
Munro, I., & Mingers, J. (2002). e use of multimethodology
in practice – Results of a survey of practitioners. Journal of
the Operational Research Society, 53(4), 369–378.
Olaya, Y., & Dyner, I. (2005). Modeling for policy assessment
in the natural gas industry. Journal of the Operational
Research Society, 56(10), 1122–1131.
Paez-Perez, D., & Sanchez-Silva, M. (2016). A dynamic
principal-agent framework for modeling the performance
of infrastructure. European Journal of Operational
Research, 254(2), 576–594.
Pasaoglu, G., Harrison, G., Jones, L., Hill, A., Beaudet, A., &
iel, C. (2016). A system dynamics based market agent
model simulating future powertrain technology transition:
Scenarios in the EU light duty vehicle road transport sector.
Technological Forecasting and Social Change, 104, 133–146.
Pollack, J. (2009). Multimethodology in series and parallel:
Strategic planning using hard and so OR. Journal of the
Operational Research Society, 60(2), 156–167.
Howick, S., Ackermann, F., & Andersen, D. (2006). Linking
event thinking with structural thinking: Methods to
improve client value in projects. System Dynamics Review,
22(2), 113–140.
Ivankova, N. V., & Kawamura, Y. (2010). Emerging trends
in the utilization of integrated designs in the social,
behavioral, and health sciences. In A. Tashakkori & C.
Teddlie (Eds.) Sage handbook of mixed methods in social
& behavioral research (2nd ed., pp. 581–611). ousand
Oaks, CA: Sage.
Jackson, M. C. (1997a). Critical systems thinking and
information systems research. In J. Mingers & F. Stowell
(Eds.), Information systems: An emerging discipline (pp.
201–230). London: McGraw-Hill.
Jackson, M. C. (1997b). Pluralism in systems thinking and
practice. In J. Mingers & A. Gill (Eds.), Multimethodology:
eory and practice of combining management science
methodologies (pp. 345–378). Chichester, UK: Wiley.
Joglekar, N. R., & Ford, D. N. (2005). Product development
resource allocation with foresight. European Journal of
Operational Research, 160(1), 72–87.
Johnson, R. B., & Gray, R. (2010). A history of philosophical
and theoretical issues for mixed methods research. In A.
Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed
methods in social & behavioral research (2nd ed., pp. 69–
94). ousand Oaks, CA: Sage.
Kaempf, A., & Ninios, P. (1998). System dynamics: An
umbrella for integrated problem solving. In Proceedings
of the 1998 International System Dynamics Conference.
Québec, Canada: System Dynamics Society.
Kim, H., & Andersen, D. F. (2012). Building condence in
causal maps generated from purposive text data: Mapping
transcripts of the Federal Reserve. System Dynamics
Review, 28(4), 311–328.
Kolominsky-Rabas, P. L., Djanatliev, A., Wahlster, P.,
Gantner-Bär, M., Hofmann, B., German, R., … Kriza,
C. (2015). Technology foresight for medical device
development through hybrid simulation: e ProHTA
Project. Technological Forecasting and Social Change, 97,
105–114.
Kotiadis, K., & Mingers, J. (2006). Combining PSMs with
hard OR methods: e philosophical and practical
challenges. Journal of the Operational Research Society,
57(7), 856–867.
Kunsch, P., & Springael, J. (2008). Simulation with system
dynamics and fuzzy reasoning of a tax policy to reduce
CO2 emissions in the residential sector. European Journal
of Operational Research, 185(3), 1285–1299.
Kwakkel, J. H., Auping, W. L., & Pruyt, E. (2013). Dynamic
scenario discovery under deep uncertainty: e future of
copper. Technological Forecasting and Social Change, 80(4),
789–800.
Kwakkel, J. H., & Pruyt, E. (2013). Exploratory modeling and
analysis, an approach for model–based foresight under
deep uncertainty. Technological Forecasting and Social
Change, 80(3), 419–431.
Kwakkel, J. H., & Pruyt, E. (2015). Using system dynamics for
grand challenges: e ESDMA approach. Systems Research
and Behavioral Science, 32(3), 358–375.
Lamberson, P. J. (2016). Winner-take-all or long tail?: A
behavioral model of markets with increasing returns.
System Dynamics Review, 32(3–4), 233–260.
Lane, D. C. (2012). What is a ‘policy insight’? Systems Research
and Behavioral Science, 29(6), 590–595.
Lane, D. C., & Oliva, R. (1998). e greater whole: Towards a
synthesis of system dynamics and so systems methodology.
European Journal of Operational Research, 107(1), 214–235.
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 11
Swinerd, C., & McNaught, K. R. (2012). Design classes for
hybrid simulations involving agent-based and system
dynamics models. Simulation Modelling Practice and
eory, 25, 118–133.
Swinerd, C., & McNaught, K. R. (2014). Simulating the
diusion of technological innovation with an integrated
hybrid agent-based system dynamics model. Journal of
Simulation, 8(3), 231–240.
Swinerd, C., & McNaught, K. R. (2015). Comparing a
simulation model with various analytic models of
the international diusion of consumer technology.
Technological Forecasting and Social Change, 100, 330–343.
Tan, B., Anderson, E. G., Dyer, J. S., & Parker, G. G. (2010).
Evaluating system dynamics models of risky projects using
decision trees: Alternative energy projects as an illustrative
example. System Dynamics Review, 26(1), 1–17.
Tashakkori, A., & Teddlie, C. (2010). Sage handbook of
mixed methods in social & behavioral research (2nd ed.).
ousand Oaks, CA: Sage.
Toro, M., & Aracil, J. (1988). Qualitative analysis of system
dynamics ecological models. System Dynamics Review,
4(1–2), 56–80.
Traneld, D., Denyer, D., & Smart, P. (2003). Towards
a methodology for developing evidence-informed
management knowledge by means of systematic review.
British Journal of Management, 14(3), 207–222.
Venkateswaran, J., & Son, Y. J. (2005). Hybrid system
dynamic-discrete event simulation-based architecture for
hierarchical production planning. International Journal of
Production Research, 43(20), 4397–4429.
Vennix, J., Akkermans, H. A., & Rouwette, E. A. (1996).
Group model-building to facilitate organizational change:
An exploratory study. System Dynamics Review, 12(1),
39–58.
Viana, J., Brailsford, S. C., Harindra, V., & Harper, P. R.
(2014). Combining discrete-event simulation and system
dynamics in a healthcare setting: A composite model for
Chlamydia infection. European Journal of Operational
Research, 237(1), 196–206.
Walrave, B., Van Oorschot, K. E., & Romme, A. G. L. (2011).
Getting trapped in the suppression of exploration: A
simulation model. Journal of Management Studies, 48(8),
1727–1751.
Wang, J., Lai, J.-Y., & Chang, C.-H. (2016). Modeling and
analysis for mobile application services: e perspective of
mobile network operators. Technological Forecasting and
Social Change, 111, 146–163.
Wolstenholme, E. (1990). System enquiry: A system dynamics
approach. Chichester, UK: Wiley.
Wu, D. D., Kefan, X., Hua, L., Shi, Z., & Olson, D. L.
(2010). Modeling technological innovation risks of an
entrepreneurial team using system dynamics: An agent-
based perspective. Technological Forecasting and Social
Change, 77(6), 857–869.
Yearworth, M., & White, L. (2013). e uses of qualitative
data in multimethodology: Developing causal loop
diagrams during the coding process. European Journal of
Operational Research, 231(1), 151–161.
Zhao, R., Zhou, X., Han, J., & Liu, C. (2016). For the sustainable
performance of the carbon reduction labeling policies
under an evolutionary game simulation. Technological
Forecasting and Social Change, 112, 262–274.
Zolfagharian, M., Akbari, R., & Fartookzadeh, H. (2014).
eory of knowledge in system dynamics models.
Foundations of Science, 19(2), 189–207.
Powell, J. H. (2014). System/scenario duality–A supporting
equivalence. Journal of the Operational Research Society,
65(9), 1344–1360.
Pruyt, E., & Kwakkel, J. H. (2014). Radicalization under
deep uncertainty: A multi-model exploration of activism,
extremism, and terrorism. System Dynamics Review, 30(1–
2), 1–28.
Randers, J. (1980). Guidelines for model conceptualization.
In Randers J. (Ed.), Elements of the system dynamics
method (pp. 117–139). Cambridge, MA: MIT Press.
Roberts, N. H., Andersen, D. F., Deal, R. M., Grant, M. S., &
Shaer, W. A. (1983). Introduction to computer simulation:
e system dynamics modeling approach. Reading, MA:
Addison-Wesley.
Rodríguez-Ulloa, R. A., Montbrun, A., & Martínez-Vicente, S.
(2011). So system dynamics methodology in action: A study
of the problem of citizen insecurity in an Argentinean province.
Systemic Practice and Action Research, 24(4), 275–323.
Rodríguez-Ulloa, R. A., & Paucar-Caceres, A. (2005). So
system dynamics methodology (SSDM): Combining so
systems methodology (SSM) and system dynamics (SD).
Systemic Practice and Action Research, 18(3), 303–334.
Romme, A. G. L. (2004). Unanimity rule and organizational
decision making: A simulation model. Organization
Science, 15(6), 704–718.
Rudolph, J. W., & Repenning, N. P. (2002). Disaster dynamics:
Understanding the role of quantity in organizational
collapse. Administrative Science Quarterly, 47(1), 1–30.
Saldaña, J. (2015). e coding manual for qualitative
researchers. London: Sage.
Sanatani, S. (1981). Market penetration of new products in
segmented populations: A system dynamics simulation
with fuzzy sets. Technological Forecasting and Social
Change, 19(4), 313–329.
Santos, S. P., Belton, V., & Howick, S. (2008). Enhanced
performance measurement using OR: A case study. Journal
of the Operational Research Society, 59(6), 762–775.
Schieritz, N., & Milling, P. M. (2003). Modeling the forest or
modeling the trees. In Proceedings of the 2003 International
System Dynamics Conference. New York, NY: System
Dynamics Society.
Schmidt, M. J., & Gary, M. S. (2002). Combining system
dynamics and conjoint analysis for strategic decision
making with an automotive high-tech SME. System
Dynamics Review, 18(3), 359–379.
Schwaninger, M. (2004). Methodologies in conict: Achieving
synergies between system dynamics and organizational
cybernetics. Systems Research and Behavioral Science,
21(4), 411–431.
Schwaninger, M. (2006). System dynamics and the evolution
of the systems movement. Systems Research and Behavioral
Science, 23(5), 583–594.
Seth, K. (1994). A fuzzy set theoretic approach to qualitative
analysis of causal loops in system dynamics. European
Journal of Operational Research, 78(3), 380–393.
Setianto, N. A., Cameron, D., & Gaughan, J. B. (2014).
Identifying archetypes of an enhanced system dynamics
causal loop diagram in pursuit of strategies to improve
smallholder beef farming in Java, Indonesia. Systems
Research and Behavioral Science, 31(5), 642–654.
Spiegler, V. L., Naim, M. M., Towill, D. R., & Wikner, J. (2016).
A technique to develop simplied and linearised models of
complex dynamic supply chain systems. European Journal
of Operational Research, 251(3), 888–903.
Sterman, J. D. (2000). Business dynamics: Systems thinking
and modeling for a complex world. Boston, MA: Irwin/
McGraw-Hill.
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
12 M. ZOLFAGHARIAN ET AL.
Appendix 1. Steps in selecting papers for review
a) TOPIC:(‘system dynamics’)
Timespan=All years<2017 (searched conducted at June 16, 2017)
Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH.
Result: 15,008 papers
b) Refined by: WEB OF SCIENCE CATEGORIES=( MANAGEMENT OR BUSINESS )
Result: 1581 papers
c) Refined by: SOURCE TITLES=( SYSTEM DYNAMICS REVIEW OR JOURNAL OF THE OPERATIONAL RESEARCH SOCIETY OR EUROPEAN JOURNAL OF OP-
ERATIONAL RESEARCH OR SYSTEMS RESEARCH AND BEHAVIOURAL SCIENCE OR TECHNOLOGICAL FORECASTING AND SOCIAL CHANGE OR SYSTEMIC
PRACTICE AND ACTION RESEARCH )
Result: 675 papers
d) Further selection by following two inclusion/exclusion criteria: the study has to
(1) deliberately combine SD with some other method
Key 1: The explicit use of related wordings in the title, abstract, and/or keywords of the article
Key 2: ‘Other method’=any method that is not part of the accepted body of knowledge in SD, represented by Sterman’s (2000) textbook
(2) address a specific empirical problem
Result: 55 papers
Appendix 2. Examples of excluded and included papers
e tables below provide several illustrations of papers that were excluded from and included in the nal set of papers reviewed.
Examples of papers excluded from the review:
Author(s) Reason for exclusion
Lane and Oliva (1998) A synthesis of system dynamics and soft systems methodology without an application to a specific problem
Fiddaman (2002), Miller and Clarke
(2007)
Monte Carlo simulation used in SD modelling process; this combination of methods is explained in detail by Ster-
man (2000, pp. 885–887)
Anderson (2011) Hill-climbing optimisation heuristics are used for sensitivity and policy testing (which is explained in detail in
Sterman, 2000, pp. 537–544)
Vennix, Akkermans, and Rouwette
(1996)
This study draws on a series of group model-building (GMB) sessions facilitated by the authors. GMB is considered
to be part of the accepted body of knowledge. In this respect, the method has been widely applied by SD schol-
ars (Sterman, 2000)
Examples of papers included in the review:
Author(s) Reason for inclusion
Schwaninger (2004) SD combined with viable systems modelling (VSM) to study a regional innovation and technology transfer system.
Sterman (2000) does not mention VSM
Rodríguez-Ulloa and Paucar-Caceres
(2005)
SD complemented with soft systems methodology (SSM) for a small Peruvian company dedicated to commercialise
national and imported steel products. SSM is not included in Sterman (2000)
Yearworth and White (2013) Grounded theory development is embedded in the SD modelling process regarding three case studies of organisa-
tional change and entrepreneurship. Sterman (2000) does not mention grounded theory development
Kwakkel, Auping, and Pruyt (2013) SD is integrated in Exploratory Modelling and Analysis (EMA) to analyse the problem of copper scarcity. There is no
reference to EMA in Sterman (2000)
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 13
Appendix 3. Key coding results
Author(s) Area of the study Most important benets of the method combined with SD Enriched phase of SD
1 Sanatani (1981) Information and knowledge Increased confidence and rigour in obtaining and quantifying variables Simulation
2 Coyle (1985) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
3 Mohapatra and Sharma (1985) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
4 Toro and Aracil (1988) Resources Increased confidence and rigour in obtaining and quantifying variables Simulation
5 Macedo (1989) Public policy Increased confidence and rigour in obtaining and quantifying variables Simulation
6 Dolado (1992) Economics Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
7 Seth (1994) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
8 Gill (1996) Resources Inclusion of multiple attributes and perspectives of agents SD is added to another (central) method
9 Coyle et al. (1999) Security Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
10 Dangerfield and Roberts (1999) Health Increased confidence and rigour in obtaining and quantifying variables Simulation
11 Homer (1999) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
12 Schmidt and Gary (2002) Information and knowledge Increased confidence and rigour in obtaining and quantifying variables Simulation
13 Schwaninger (2004) Public policy Inclusion of multiple attributes and perspectives of agents Conceptualisation
Structures and processes that support intervention and implementation.
14 Anderson, Morrice, and Lundeen (2005) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
15 Haslett and Sarah (2006) Economics Structures and processes that support intervention and implementation Conceptualisation
16 Joglekar and Ford (2005) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
17 Olaya and Dyner (2005) Resources Increased confidence and rigour in obtaining and quantifying variables Simulation
18 Rodríguez-Ulloa and Paucar-Caceres (2005) Resources Inclusion of multiple attributes and perspectives of agents Conceptualisation
19 Daim, Rueda, Martin, and Gerdsri (2006) Resources Increased confidence and rigour in obtaining and quantifying variables Both phases
20 Evenden, Harper, Brailsford, and Harindra (2006) Health Increased confidence and rigour in obtaining and quantifying variables Both phases
21 Howick, Ackermann, and Andersen (2006) Resources Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
22 Duggan (2008) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
23 Kunsch and Springael (2008) Economics Increased confidence and rigour in obtaining and quantifying variables Simulation
24 Santos, Belton, and Howick (2008) Health Increased confidence and rigour in obtaining and quantifying variables Simulation
25 Adamides, Mitropoulos, Giannikos, and Mitropoulos (2009) Public policy Increased confidence and rigour in obtaining and quantifying variables Both phases
26 Tan, Anderson, Dyer, and Parker (2010) Resources Increased confidence and rigour in obtaining and quantifying variables Simulation
27 Wu et al. (2010) Information and knowledge Inclusion of multiple attributes and perspectives of agents Both phases
Increased confidence and rigour in obtaining and quantifying variables
28 Chen, Tu, and Jeng (2011) Public policy Increased confidence and rigour in obtaining and quantifying variables Simulation
29 Hadjis (2011) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
30 Liu, Triantis, and Sarangi (2011) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
31 Rodríguez-Ulloa et al. (2011) Security Structures and processes that support intervention and implementation Conceptualisation
32 Chen, Wakeland, and Yu (2012) Public policy Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
33 Kim and Andersen (2012) Economics Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
34 Kopainsky, Tröger, Derwisch, and Ulli-Beer (2012) Resources Inclusion of multiple attributes and perspectives of agents Simulation
Increased confidence and rigour in obtaining and quantifying variables
35 Liu, Howley, and Duggan (2012) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
36 Georgiadis and Athanasiou (2013) Business and operations Increased confidence and rigour in obtaining and quantifying variables Simulation
37 Kwakkel and Pruyt (2013) Resources Increased confidence and rigour in obtaining and quantifying variables SD is added to another (central) method
38 Kwakkel et al. (2013) Resources Increased confidence and rigour in obtaining and quantifying variables Simulation
39 Lee, Park, Kim, and Lee (2013) Public policy Increased confidence and rigour in obtaining and quantifying variables Both phases
40 Yearworth and White (2013) Information and knowledge Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
41 Paez-Perez and Sanchez-Silva (2016) Public Policy Inclusion of multiple attributes and perspectives of agents SD is added to another (central) method
Increased confidence and rigour in obtaining and quantifying variables
42 Spiegler, Naim, Towill, and Wikner (2016) Business Increased confidence and rigour in obtaining and quantifying variables Simulation
(Continued)
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
14 M. ZOLFAGHARIAN ET AL.
Author(s) Area of the study Most important benets of the method combined with SD Enriched phase of SD
43 Viana, Brailsford, Harindra, and Harper (2014) Health Increased confidence and rigour in obtaining and quantifying variables Simulation
44 Powell (2014) Learning and teaching Increased confidence and rigour in obtaining and quantifying variables Conceptualisation
Inclusion of multiple attributes and perspectives of agents
45 Auping, Pruyt, and Kwakkel (2015) Public Policy & Health Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
46 Azadeh, Darivandi Shoushtari, Saberi, and Teimoury (2014) Business Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
Structures and processes that support intervention and implementation
47 Kwakkel and Pruyt (2015) Resources Increased confidence and rigour in obtaining and quantifying variables Simulation
48 Setianto, Cameron, and Gaughan (2014) Resources Inclusion of multiple attributes and perspectives of agents Conceptualisation
Structures and processes that support intervention and implementation
49 Lamberson (2016) Business Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
50 Pruyt and Kwakkel (2014) Security Increased confidence and rigour in obtaining and quantifying variables Simulation
Inclusion of multiple attributes and perspectives of agents
51 Kolominsky-Rabas et al. (2015) Health Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
52 Pasaoglu et al. (2016) Public policy Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
53 Swinerd and McNaught (2015) Information and Knowledge Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
54 Wang, Lai, and Chang (2016) Business Increased confidence and rigour in obtaining and quantifying variables Simulation
Inclusion of multiple attributes and perspectives of agents
55 Zhao, Zhou, Han, and Liu (2016) Business Increased confidence and rigour in obtaining and quantifying variables Both phases
Inclusion of multiple attributes and perspectives of agents
Appendix 3.(Continued)
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 15
Appendix 4. Identifying key characteristics of the research problem
For each paper, we rst identied the main characteristics of the problem which motivated the authors of the paper reviewed to
combine SD with one or more other methods (as outlined in supplementary data), and then coded and synthesised the results.
is resulted in four general categories of problem characteristics: see table below. is table also demonstrates that three char-
acteristics are frequently mentioned as an explicit motivation for combining SD with another method: nature of the problem
(20), variables and their causal relationships (24), and diversity of agents involved (24). e context as a problem characteristic
occurs in a few studies (5). Notably, some studies refer to multiple or similar characteristics, which explains why the total num-
ber of text phrases in (each section of) the following table diers from the number of studies reviewed.
Problem characteristics that motivate the combination of SD with other methods
a) Nature of the problem
(In papers reviewed: 20)
Multi-level, multi-aspect
Multi-disciplinary; several organisational factors – political, cultural, etc.
Complexity (involving not only biophysical but also social, ecological, political, and economic elements)
Dynamic and structural aspects of the problem
High ambiguity of the problem
Queueing nature of problem
Continuous flows of timevarying commodities interrelated by complex non-linear feedback and coupling mechanisms
Specific features of each component that, when analysed as a single whole, imply a synthesised modelling approach
Dependent on complex computer systems
Cyclic behaviour that can be periodic
Long-term capability developmental process
Wide range of operational circumstances
The structure and process dimensions of an organisational change
Non-linearities in system
Stochasticity
Messy problem
Deep uncertainty
b) Variables and their
causal relationships
(In papers reviewed: 24)
Fuzziness of some variables
Linguistic and soft variables in the model
Uncertainty in variables
Varying variables (destabilised and changing variables) and information sharing
Uncertainties and conflicting information
Uncertainties on time-dependent key parameters or exogenous variables
Tangible and intangible attributes, besides social dynamic factors such as trust
Behavioural policies of decision-makers include trade-offs among multiple attributes
Parametric uncertainties, orders of time delays, non-linear lookups
Availability of data from Social Fabric Matrix (based on collective wisdom)
‘Purposive’ text data as a source of causal structures
Qualitative data collection and analysis
Time-dependent behaviour of system
c) Diversity of agents
involved
(In papers reviewed: 24)
Multiple stakeholders
Including (groups of ) agents in the system definition
Client involvement, client centre
Involving a large number of experts from government, industry, academia, and research agencies
Multiple independent actors who coordinate a diverse set of decision policies, and whose decisions are intendedly rational
Uncertainty derived from profoundly divergent views
Wide range of views
Social issue that extends, beyond the purely technical, to both practical and emancipatory spheres of interest
Social dynamic factors such as trust
Network of N agents, a set of M agent strategies, and a set of agent parameters
Multiple, subjective views
Relevant and conflicting objectives between upper and lower layers
Many ideas and relationships are obscured in agents’ intuitions about dynamics
Relevance of human decision-making
Various preferences of users and developers
Agents’ individual decision-making within a social system
The heterogeneity in agent ideal points and feasible sets, and the interaction between consumer purchase decisions
Microscopic level with agents, traversing various workflows
Multiple individual agent's individual decisions and their interaction with each other
Individual national decision-making within a social system of nations
Complex relationship between the sequential actions of players (public and private) that are inherently conflictive
Diverging stakeholder views regarding the desirability of policy measures
Interactions among conflicted players whose strategic behaviours are influenced by their payoffs
A wide variety of actors whose interests are varied
Conflicting goals
d) Context of the
problem
(In papers reviewed: 5)
Problem mostly under the auspices of central government
High managerial flexibility
Public sensitivity of the problem addressed
Major public health concern
An important (significant) issue for country and officials
Power asymmetry
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
16 M. ZOLFAGHARIAN ET AL.
Appendix 5. The benets of combining SD modelling with other methods
e table below reports the text phrases regarding benets of combining SD with other methods, taken from the papers re-
viewed. By codifying the (perceived) benets of each method combined with SD and synthesising these in higher-level notions,
three generic benets of combining SD with other methods were identied. e table demonstrates that the most frequently
mentioned benet is the increased condence and rigour in obtaining and quantifying variables (mentioned in 49 studies). e
inclusion of multiple attributes and perspectives of agents is mentioned in 17 studies, whereas only 5 studies refer to benets
arising from structures and processes that support intervention and implementation. Notably, some studies refer to multiple
benets and/or similar ones, which explains why the total number of text phrases in the table below (71) is higher than the total
number of studies reviewed (55).
Increased condence and rigour in obtaining and quantifying variables Occurs in 49 papers reviewed
Incorporating subjective beliefs and perceptions
Quantifying fuzzy concepts and parameters
Keeping all available information and merging available data sets according to their respective credibility factors
More holistic way in considering socio-economic factors
Providing an alternative approach to incorporate linguistic variables in SD modelling
Identify problems, key variables, and their structural relationships from purposive text data
Adding more confidence, rigour, and flexibility in modelling
Better analysis performance of SD
Systematic procedure for qualitative analysis
Improving the formulation and validation of SD models and also, its policy analysis
Identifying the processes in the system (using cognitive mapping)
Improving dynamic sensibility in the process of qualitative data analysis, providing a more rigorous approach to develop CLDs in formation stage of SD
modelling
Improve and widen the understanding/perspectives of analysts and policy-makers
Identifying highly uncertain and influential relationships and parameters, and testing these for sensitivity analysis in an efficient and effective way
Improving the performance of hypothetical or generic models
Efficiently handling multiple sources of uncertainty with less risk to give in to ‘the curse of dimensionality’
More robust policies
More rigorous method of policy design
Useful insights into possible future scenarios
Designing realistic policies
Visualising the links between the scenarios and over-time dynamics, stronger analysis, and more robust results
More complete forecasting methodology, using SD as a platform for integrating different methods
Allowing for varying of policy equations, to discover the best strategies for a given problem, its scalability, in terms of the ease of adding new agents
Firm policy conclusion
More robust policy analysis
Policy-makers’ need for creating desired reference modes and running the algorithm to search for appropriate model(s) that can fit it without heuristic
objective functions or eigenvalues
Providing more feasible and flexible policies as improved alternatives
Extending the scenario discovery approach conceptually, technically, and practically
Specifying a foresighted policy, tested with an SD model
Offering an additional, powerful dimension to policy exploration that can be viewed as a computational extension of the ideas of partial model testing
Providing an intuitive approach in modelling managerial flexibility and discrete approximations of project uncertainty
Exploring the search space to discover the best combination of parameters and equation-based strategies for a given SD problem
Embracing the complexity in which the process is embedded
It can monitor the dynamic evolution of all the components within a game realisation
Capturing the essential detailed individual variability within parts of that system
Identifying key factors, which together capture the essential characteristics of the system-in-focus, particularly with regard to its future extrapolation
Provides a straightforward way of testing policy robustness across a multitude of plausible futures
Improved prediction of the behaviour of critical environmental elements
Systematic exploration of plausible models and scenarios
Assessed uncertainties present in the cases have been explicitly accounted for and their consequences
Facilitates production of more effective strategy solutions
Gives insight into a micro-level choice process that gives rise to the macro-level pattern
Generate an ensemble of many computational experiments using various radicalisation models
Explore and analyse the ensemble of computational experiments
Robust adaptive policies against undesirable radicalisation
Allows the parameter space to be explored in an efficient manner, without bias or subjective disagreement
Inclusion of multiple attributes and perspectives of agents Occurs in 17 papers reviewed
Bringing the multiple actors together and help actors at different levels to achieve the requisite variety
Active participation of system players in policy development
Introducing explicitly the observer’s weltanschauung and the observer’s role in SD studies
New planning or decision-making ideas that improves the basic function of each agent and understanding the system from Agents' interaction, facilitates
to organise the system by use of Modular Style sheets
Allows to elicit choice preferences of stakeholders in detail and to add precision to the structure of the model
Identifying and evaluating key factors (variables) by nationwide experts
A better representation of the player’s payoffs that result from their chosen strategies
Capturing different stakeholder perspectives
Capturing the ideas of experts
Helping document and resolve conflicting or even contradictory views and expectations with respect to the boundary issues
Determination of values that are important to related individuals
Incorporating the major market agent roles and interactions with each other
Improved understanding of the ‘values system’ or the preferences of individual customers and developers
Makes a unique prediction from the possible strategic actions that each player may choose
Capturing influence of individual adopting agents on the adoption in others
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
JOURNAL OF SIMULATION 17
Structures and processes that support intervention and implementation Occurs in 5 papers reviewed
Developing the formal organisational and political structures and processes necessary to support a SD intervention in a large bureaucracy, rather than
modelling per se
Simultaneous operations at the content as well as context level
Orchestrate and implant change in social systems, based on a multi-methodological and multi-paradigmatic approach
Provide required concepts and framework for analysing a complex system
A conceptual model of the problematic situation that was accepted by all stakeholders
Appendix 6. Enriched phases of SD modelling
e table below shows the numbers of studies in which another method enhanced either the conceptualisation phase or the
simulation phase in SD, or both. In this respect, the number of papers in which the simulation phase is enriched appears to be
much larger than the number of papers with an enriched conceptualisation phase. Two of the studies reviewed are not includ-
ed in this table, because they used SD as an additional method to enrich another method (at the core of that study), such as
exploratory modelling.
Enriched phases of SD modelling in the reviewed papers
Conceptualisation Simulation Conceptualisation & Simulation Not Relevant
Number of articles 12 28 12 3
Percentage of articles
reviewed
22% 51% 22% 5%
Appendix 7. Number of papers combining SD with other methods (over time)
Appendix 5.(Continued)
Downloaded by [Eindhoven University of Technology] at 00:43 05 January 2018
... In other words, real-world challenges are most complicated, and various research methods should be employed to explore different facets of a case (Morgan et al., 2017). On account of this, adopting mixed research methods provides the opportunity to resolve most of the limitations and drawbacks of individual inquiry methods (Morgan et al., 2017) and offer possible solutions to complex problems that are more in-depth than a single research method do (Zolfagharian et al., 2018). Therefore, this study responds to this research agenda by focusing on mixing two methods of inquiry, which differentiate in nature and each has its own contributions On the one hand, the System Dynamics method has been significantly used by many scholars and practitioners to model complex socio-economic systems over time and analyze the effectiveness of the intervening policies (Sterman, 2000). ...
... On account of this, adopting mixed research methods provides the opportunity to resolve most of the limitations and drawbacks of individual inquiry methods (Howick & Ackermann, 2011;Morgan et al., 2017) and offer possible solutions to complex problems that are more in-depth than a single research method do (Zolfagharian et al., 2018). ...
... The first complementary characteristic of CLA and SD pertains to their abilities for deeply investigating societal issues. While almost all of the SD practices conceptualize and model the complex systems only at the most obvious and macro levels and lack critical power to delve much more than the existing structures and stereotypes (Ding et al., 2018;Morgan et al., 2017;Zolfagharian et al., 2018), CLA is capable of moving the exploration of an issue beyond the superficial and obvious to the deeper and marginal (Inayatullah, 2017). Therefore, linking up these two methods allows for systematic and deep analysis and modelling of an issue. ...
Article
The brain drain crisis is among the most challenging issues of many developing countries, attributed to the compounding effects of multiple factors. As communities and their challenges become more complex, new sets of mixed research tools that critically investigate these issues as whole systems rather than isolated parts are crucial for future decision makings. In light of this, mixed research approaches offer the chance to overcome most of the limitations of individual methods and provide potential solutions to complicated problems that are more in-depth than a single form of research method. In this paper, by drawing on the complementary characteristics of Causal Layered Analysis and Participatory System Dynamics, a mixed approach for investigating the future behaviors of complex socio-economic systems and exploring the effectiveness of various policy interventions over time is proposed. Moreover, Iran’s brain drain case study is presented for further assessment of the proposed approach. The implication of the proposed approach to skills mobility or the so-called brain drain phenomenon is the undisputable role of human capital efficiency in the future developments of the nation. Through this approach, in the first step, an issue is critically analyzed and unpacked into four layers, namely as litany, structure, worldview, and metaphor. In the second step, a participatory approach is implemented to systematically model the issue based on the results of the previous step via five stages. The utility of this approach is its ability to critically examine the future behaviors of complex systems under the effect of different policy interventions and scenarios. Therefore, the proposed approach will generate theoretical value for researchers in the field of futures studies and harbor practical value for policymakers who seek a better mechanism to critically investigate complex socio-economic systems.
... Since simulation lacks inherent optimisation capabilities, lately, it has been combined in many ways with optimisation methods in various application settings (Linn eusson et al., 2018;Zolfagharian et al., 2018), including health systems management (Osorio et al., 2017). Optimisation can be an "extension" of a simulation model, processing its output data so that becomes more meaningful and useful to decision makers, for instance, by balancing optimally tradeoffs in system models with multiple objectives (Linn eusson et al., 2018), or even for calibrating a simulation model (Parra et al., 2018) (optimisation serves simulation model). ...
... The GC of emerging technologies is influenced by technological factors, market environment factors, enterprise innovation environment factors, industrial factors, and regional macro-environmental factors [4], [81]- [84]. The technological factors shown in Table I refer to the characteristics of the technology itself; the market environment factors refer to the demand and competitiveness of the emerging technology market; the enterprise innovation environment factors refer to the ability of an enterprise to research, develop, and invest in emerging technologies; the industrial factors refer to the supply of external resources required for the development of emerging technologies; and the regional macroenvironmental factors mainly refer to the level of regional economic development. ...
Article
Full-text available
As China’s economy becomes more robust, the government has emphasized the need to increase the civil aviation industry’s rapid development and reduce civil aviation’s technological gap compared to other advanced aircraft manufacturers. The use of the appropriate technologies in the aviation industry can help achieve the government’s plan of creating more jobs and increasing its revenue through the civil aviation industry. However, emerging technologies’ uncertainty and dynamic nature have led to significant challenges when these technologies are applied to the aviation manufacturing industry. This article applies the principles and methods of system dynamics based on the gradual-change factors of emerging technologies as identified through expert interviews to construct conceptual models depicting the gradual-change of emerging technologies in the aviation manufacturing field. Historical data from Shaanxi province were used to simulate and analyze the gradual development of emerging technologies. An analysis of the simulation results revealed that the rapid growth of competitive technologies in the growth process inhibits the gradual-change of emerging technologies. Also, increasing enterprise innovation capabilities, market demand expansion, and technology optimization significantly promote emerging technologies’ development and application. Furthermore, the research result shows that competitive products and technologies influence the gradual development of emerging technologies. Finally, the results indicate that the civil aviation industry should optimize the performance and function of emerging technologies during the early development stage instead of at the market application stage.
... Rather than building a System Dynamics model, the CLD can be also used for defining and developing connections between models [176]. Pathways to integrating models can include a portfolio approach (organize a family of independent models without attempting to link them mathematically), loosely coupled models (where the output from one model is used as the input to the next), fully coupled models (combine multiple large-scale models where information is transferred at each time step), and metamodels (a large holistic and fully-integrated model that simulates details within all systems). ...
Article
Although great progress has been made to advance the scientific understanding of oil spills, tools for integrated assessment modeling of the long-term impacts on ecosystems, socioeconomics and human health are lacking. The objective of this study was to develop a conceptual framework that could be used to answer stakeholder questions about oil spill impacts and to identify knowledge gaps and future integration priorities. The framework was initially separated into four knowledge domains (ocean environment, biological ecosystems, socioeconomics, and human health) whose interactions were explored by gathering stakeholder questions through public engagement, assimilating expert input about existing models, and consolidating information through a system dynamics approach. This synthesis resulted in a causal loop diagram from which the interconnectivity of the system could be visualized. Results of this analysis indicate that the system naturally separates into two tiers, ocean environment and biological ecosystems versus socioeconomics and human health. As a result, ocean environment and ecosystem models could be used to provide input to explore human health and socioeconomic variables in hypothetical scenarios. At decadal-plus time scales, the analysis emphasized that human domains influence the natural domains through changes in oil-spill related laws and regulations. Although data gaps were identified in all four model domains, the socioeconomics and human health domains are the least established. Considerable future work is needed to address research gaps and to create fully coupled quantitative integrative assessment models that can be used in strategic decision-making that will optimize recoveries from future large oil spills.
... According to Zolfagharian et al. (2018), SD scholars are increasingly drawing to multi-method approaches to overcome the limitations of this approach. Therefore, they combine SD with one or more research methods to analyze complex problems and develop deeper solutions than a singlemethod study can do. ...
Article
Full-text available
Systems Dynamics (SD) is an efficient method that allows to analyze systems with dynamic complexity and policy resistance. The holistic approach of SD is suitable to analyze the current transportation problems. Among the applications of SD in the transportation sector, the use of this tool in the development and implementation of sustainable transport policies stands out. In this context, this paper aims to conduct a systematic literature review to assess the use of SD in the development and implementation of urban policies focused on sustainable transportation. The results show that most studies analyze policies focused on reducing the negative externalities of transportation, highlighting the efforts to reduce air pollutant emissions and traffic congestion. However, we did not find a study that analyzes non-motorized and motorized modes by economic, environmental, social, spatial and traffic variables. At the end of this study, the gaps found in the literature are pointed out, being available to further exploration in future studies.
... Second, we have acknowledged various characteristics of adopters in the model but have not allowed for any interactions between these individual consumers. Future work in this area may therefore seek to extend our multi-level model with an agent-based module, for example, [130]. Third, we assumed that consumers only have one characteristic (e.g., age, income) in conducting simulation runs. ...
Article
Full-text available
Transportation systems are transitioning to e-mobility, but scholars and policymakers are struggling to understand how to accomplish this transition effectively. In response, we draw on the technological innovation systems perspective and the unified theory of acceptance and use of technology to develop a theory-guided and entity-based simulation model to better understand, among others, electric vehicle (EV) adoption processes as a specific yet core element driving business innovation. By doing so, our model is among the first to capture and combine the macro-and microlevel interactions associated with the EV transition process. Our simulation results shed light on the impact of alternative innovation policies, notably by explaining relations between EV market dynamics and changes in e-mobility policies, such as EV-related subsidies and resource mobilization. As such, the simulation modeling approach adopted in this paper enables a more in-depth study of transition problems related to e-mobility. Notably, the resulting modular model can be adjusted to other e-mobility transition problems by changing the specified entities.
... In particular, the proportions of quantitative and qualitative approaches may offer the opportunity to reduce this asymmetry. While qualitative methods may imply relying on subjective data and high investments for gathering and analyzing information, future research may consider following this type of approach to shed more light on the dynamics that quantitative studies may not capture (Zolfagharian et al. 2018). In addition, considering the complex nature of this research domain, a combination of qualitative and quantitative studies may allow us to produce a more precise and reliable comprehension of the dynamics studied. ...
Article
Full-text available
The interplay between internationalization and digitalization (IDI) is a relatively young research domain that is finding its way into numerous academic journals, which motivates us to systematically portray the proceedings and evolutions of this interplay over time, analyze its methodological and theoretical background and, more importantly, provide insights to guide future research on these phenomena. Combining qualitative and quantitative approaches, we analyze 68 peer-reviewed international business and management papers to explore the structure of knowledge production and identify the emerging trends. Our research reveals four key stages of the debate: (a) how IDI influences interorganizational aspects (1996–2004), (b) how digitalization influences the balance between perceived risk and returns in internationalization (2005–2009), (c) how digital advances and available internationalization channel alternatives influence IDI (2010–2014), and (d) how IDI influences intraorganizational aspects (2015–2019). Moreover, our analysis reveals heterogeneity across theoretical perspectives, methods, and research themes over the years. Finally, we identify theoretical and methodological challenges related to IDI research.
Article
Full-text available
Resilience management of farming systems requires building an understanding of the underlying drivers of the adaptive capacity of the system. In this paper, we use the concept of resilience as a framework to understand how bovine livestock farming systems may adjust to challenging environmental, social, and political conditions. Using an interactive simulation model (microworld), we explored potential developments for livestock farmers in Bourbonnais, France, to the effect of simultaneous changes in the socioeconomic landscape and unpredictable weather conditions resulting from climate change. The results offer insights into the potential trade-offs between systems scale and long-term sustainability by suggesting that sacrificing socioeconomic performance in the short and medium term may increase long-term sustainability and resilience. ARTICLE HISTORY
Conference Paper
The review established the state-of-the-art in hybrid simulation modelling and mixed-use methodologies focusing on their utility in the abatement of Greenhouse Gas Emissions efforts and the realisation of a circular green economy in Ireland by 2050. Ireland missed its EU 2020 climate emissions target and is not on the right trajectory towards decarbonisation in the longer 2030 and 2050 challenges (Environmental Protection Agency, 2019). Ireland's transition to a low-carbon, climate-resilient environment, society, and economy while meeting national and international emission targets is a priority for decision-makers. Treating waste as a resource and moving towards a more resource-efficient and circular green economy is also an objective. Agriculture is on course, to rise by 3% from current levels and will account for 38% of carbon emissions by 2030 (Environmental Protection Agency, 2018), mainly as a result of plans to increase the national herd (Department of Agriculture, Food and the Marine, 2015). To reduce carbon emissions, the Climate Change Advisory Council in Ireland has pointed to a need for an increase in carbon tax. Key action nine of the Government's Climate Action Plan commits to the implementation of a carbon tax rate of at least €80 per tonne by 2030, currently the cost is €20 per tonne (Government of Ireland, 2019). Policymakers can use mathematical decision support models which can help design climate policies and select relevant climate change strategies. A simulation model can be used to represent the system, a simplified abstraction of reality which allows for the study of the system at a low cost. A system is a collection of entities (people, parts, messages, machines, servers,) that act and interacts together towards some end (Schmidt.Taylor, 1970). Problems in the real world usually depend on the influence of many variables. Intelligent decision making requires the appropriate use of many different models designed for specific purposes and not be reliant on a single comprehensive model of the world (Sterman, 1991). By setting up a range of "what-if" queries into the model, it is possible to elaborate which possible solution is the best compared to others (Hari & Taha, 2008). For a model to be useful to decision-makers, it must provide some view on future behavior. (Meadows, et al., 1974) provides a valuable classification of the types of outputs models can provide. Multivariant relationships are introduced by having a computer model handle computation in practical applications. Models are classified as Simple or Complex. Simple models include Static, Deterministic, Linear and Continuous. Complex models include Dynamic, Stochastic, Nonlinear, and Discrete. Discrete Event Simulation (DES) is an operational tool designed for the optimisation of the system performance at a very detailed level. System Dynamics (SD) is an approach to understanding nonlinear behavior of complex systems over time that uses stocks, flows, feedback loops, table functions and time delays. Agents Based Modelling (AGM) has autonomous agents which can make their own decisions acting within their environment. Hybrid mixed-method simulation is designed for both continuous and discrete parameters and can be found in the computer science literature since (Mušič & Matko, 1999). The contribution addressed how to combine developing hybrid simulation techniques with established and matured practices such as Lean Six Sigma, which originates from Operations Research (OR). It applied the synthetic mode of thought which, when mapped to a system problem is called the systems approach. In this approach, a problem is not solved by taking it apart but by viewing it as a part of a whole of a more significant problem. Putting things together, synthesis is the key to systems thinking just as analysis or taking them apart was to Machine-Age thinking. The differences between Systems-Age and Machine-Age thinking derives not from the fact that one synthesises and the other analyses, but from the fact that systems thinking combines the two in a new way (Ackoff, 1981). The systems thinking approach can be applied with (Senge, 1990) and (Checkland, 2000) in the field of Operations Research (OR). The research investigated the use of System Dynamics (SD) applied with Lean Six Sigma and the use of the DMAIC (Define, Measure, Analyse, Improve, Control) cycle. (Cardiel-Ortega, Baeza-Serrato, & Lizarraga-Morales, 2017) results show an improvement in the process performance by increasing the level of Sigma, allowing the validation of the proposed approach. While (Hari & Taha, 2008) maintain that Six Sigma needs more development to meet requirements. There is little literature on the use of SD with the Lean Six Sigma DMAIC cycle, which provides for further research opportunities. The review analysed environmentally-focused models utilised in Ireland within the European Union such as the GAINS (Air pollution Interactions and Synergies) model used for air pollutants, the JRC-EU-TIMES, (Joint Research Council-European Union-The Integrated MARKAL-EFOM System) and the Irish TIMES model used for energy, the integrated modelling project Ireland (GAINS & TIMES), and the environmental, economic model ENV- Linkages. It is useful for the future, (Kelly, Chiodi, Fu, Deane, & O'Gallachóir, 2013) to have experience of reconciling the energy pathway recommendations from the Irish TIMES model with the GAINS Ireland modelling system. This technique will allow those pathways to be assessed in the familiar GAINS format by the European Commission or other international policymaking bodies during the negotiation and review processes for global climate and air policy. The teams showed a soft link is possible as there is no automated solution to link the two models. This enables assessments of what such climate optimisations imply for legally binding air pollutant targets for Ireland, and it further allows the estimation of air pollutant- related impact costs by using their marginal damage valuation methodology. Supplementing this modelling with tailored policy research will help to identify practical means of delivering better policy decisions for Ireland in the future. (Kelly, A., 2013) recommends further research is needed to precisely assess the practicality of any increased biomass use in Ireland consistent with appropriate policies and technologies. Further research should assess the directly corresponding health and environmental impacts that could otherwise occur due to associated increases in the levels of air pollutants across Ireland. The results highlight the importance of integrated policy analysis which needs to take a broader perspective with national decision- making which considers a policy change from multiple angles such as air, climate, economy, and health. Often, a lack of understanding of the systems assessed is the biggest weakness in an assessment. One example of this is in understanding the decision-making process (Parson, 1997), (European Environment Agency, 1998). Is Ireland a model state? While there is a lot of work in simulation modelling in areas such as computer science, healthcare, environment, and Operations Research (OR), different facilities look at simulation independently. The literature established that simulation modelling is becoming an interdisciplinary field with hybrid models of simulation used in addressing real-world problems in all disciplines and provides the most opportunities for further research. For this project SD could be applied to model the overall system (carbon emissions), SIM to model the parts within the system (production process of the raw materials) and AGM to model the agents acting within the system (uptake of farmers growing hemp. SD modelers should explore the progress made in the rapidly evolving area of hybrid modelling and collaborate with others who have developed successful hybrid modelling methods (Sterman, D, J., 2019). Part of the state of the art will be the expertise to draw from the various tools available and pick the ones that are most appropriate for the task and problem at hand. The field need to have a multi-faceted approach binding under an organised framework, a Nexus.
Chapter
Full-text available
This chapter discusses the contribution that critical systems thinking can make to the emerging discipline of information systems and, consequently, to the practice of information systems development. It begins by arguing that systems thinking provides a natural underpinning for work in information systems research and a theoretical platform which should ease the growth of the emerging discipline towards maturity.
Chapter
Full-text available
Pluralism, interpreted in the broadest sense of the use of different methodologies, methods and/or techniques in combination, is a topic of considerable interest in the systems community these days. This chapter contains a thorough review of the origins and nature of pluralism in systems thinking and the management sciences.
Article
This paper develops a model of consumer choice that demonstrates why some markets with increasing returns converge to a winner-take-all outcome while many others have a power law market share distribution with a “long tail” of small-share products. The model takes the standard winner-take-all model of increasing returns and adds a simple behavioral assumption: when faced with complex choices, decision makers first quickly eliminate many of the available options using a simple heuristic before selecting from the remaining feasible set. We examine the market-level consequences of this model using an agent-based simulation. Under a wide range of parameters the model produces a power law share distribution. But when consumers have very large feasible sets the market converges to a winner-take-all outcome, and when consumers have very small feasible sets the model produces an evenly split market. Copyright © 2017 System Dynamics Society.
Article
In recent years there has been significant interest in multimethodology and the mixing of OR/MS methods, including Discrete Event Simulation (DES) with System Dynamics (SD). Several examples of mixing DES and SD are described in the literature but there is no overarching framework which characterises the spectrum of options available to modellers. This paper draws on a sample of published case studies, in conjunction with the theoretical literature on mixing methods, to propose a toolkit of designs for mixing DES and SD which can be implemented as a set of questions which a modeller should ask in order to guide the choice of design and inform the associated project methodology. The impetus for this work was the perceived need to transfer insight from reported practice in order to formalise how the two methods can be and have been mixed.
Article
The success of Apple's App Store has motivated many mobile network operators (MNOs) to embrace the application store. However, finding a successful strategy that provides meaningful differentiation in an increasingly competitive mobile application market is a great challenge for MNOs. This research treats the mobile application service as a two-sided market and develops a system dynamics model to understand the diffusion behavior of mobile application services. Choice-based conjoint analysis is used to collect multi-attribute preference data of end-users and developers in order to understand how they select application store service offerings in Taiwan's mobile application market. We found that under the great pressure of application stores offered by mobile platform providers, MNOs have to provide services that directly attract adoption by end-users and developers, because the cross-side network effect is not significant. MNOs need to apply a localization strategy based on their strengths while improving their store service quality by collaborating with third-party IT vendors to persuade developers and end-users to adopt their stores as another “home base”. We conclude that MNOs need to not only fully exploit their resources to sense and seize emerging opportunities, but also reconfigure their resources with respect to market dynamics to rebuild their core competences for sustaining their competitive advantages.
Article
This paper presents a novel approach to the problem of infrastructure development by integrating technical, economic and operational aspects, as well as the interactions between the entities who jointly carry out the project. The problem is defined within the context of a Public Private Partnership (PPP), where a public entity delegates the design, construction and maintenance of an infrastructure system to a private entity. Despite the benefits of this procurement method, the relationship between the two entities is inherently conflictive. Three main factors give rise to such conflict: the goals of the public and private party do not coincide, there is information asymmetry between them and their interaction unfolds in environments under uncertainty. The theory of contracts refers to this problem as a principal-agent problem; however, due to the complexity of the problem, it is necessary to recreate a dynamic interaction between the principal (i.e., the public entity) and the agent (i.e., the private entity) while including the monitoring of the infrastructure performance as an essential part of the interaction. The complex relationship between the sequential actions of players and the time-dependent behavior of a physical system is explored using a hybrid agent-based simulation model. The model is illustrated with several examples that show the versatility of the approach and its ability to accommodate the different decision strategies of the players (i.e., principal, agent) and the model of a physical infrastructure system.