MODERN AGILE LEARNING ENVIRONMENT

Abstract

The skills required by modern organizations and their experts to remain competitive change faster and faster in today’s world. In order to foster complexity, the learning and eLearning environment of the future is agile in all aspects, starting from the technical infrastructure and the managerial approach to implementation, to the learning content and learning itself. This paper proposes an integrated holistic formal concept, the Agile Learning Environment. It presents the conceptual processes, functional architecture and technology platform for an eLearning and collaboration platform able to support it, and approaches to collaborative content creation and knowledge building. The approach starts from rigorous stakeholder management, based on the concept of Circles of Knowledge – overlapping communities of interest and expertise. The border between trainer and trainee is no longer clearly defined in modern knowledge-building environments, hence stakeholder management must be particularly agile. The complexity of today’s organizations requires special complexity management methods and skills, but more than this, it also offers significant opportunities, through such characteristics associated with dynamic complexity such as emergence and innovation. The paper uses qualitative research, based on actual project cases. Keywords: eLearning, collaborative learning, agile learning, e-Learning, knowledge building, Circles of Knowledge.

Full text

MODERN AGILE LEARNING ENVIRONMENT
S. Morcov
Katholieke Universiteit Leuven (BELGIUM)
Abstract
The skills required by modern organizations and their experts to remain competitive change faster and
faster in today’s world. In order to foster complexity, the learning and eLearning environment of the
future is agile in all aspects, starting from the technical infrastructure and the managerial approach to
implementation, to the learning content and learning itself. This paper proposes an integrated holistic
formal concept, the Agile Learning Environment. It presents the conceptual processes, functional
architecture and technology platform for an eLearning and collaboration platform able to support it,
and approaches to collaborative content creation and knowledge building. The approach starts from
rigorous stakeholder management, based on the concept of Circles of Knowledge – overlapping
communities of interest and expertise. The border between trainer and trainee is no longer clearly
defined in modern knowledge-building environments, hence stakeholder management must be
particularly agile. The complexity of today’s organizations requires special complexity management
methods and skills, but more than this, it also offers significant opportunities, through such
characteristics associated with dynamic complexity such as emergence and innovation. The paper
uses qualitative research, based on actual project cases.
Keywords: eLearning, collaborative learning, agile learning, e-Learning, knowledge building, Circles of
Knowledge.
1 INTRODUCTION
The skills required by modern organizations and experts to remain competitive change faster and
faster. The internal learning curve is always shorter: content is created and becomes obsolete in short
cycles, sometimes in just a few months. Staff moves horizontally as well as vertically through the
organizational charts, to acquire new insights and competences, to foster innovation and exchange of
information across organizations. New technology such as Robotic Process Automation (RPA) and
Artificial Intelligence (AI makes yesterday’s jobs obsolete, while new job types are created daily.
In modern organizations, stakeholders switch roles frequently between trainer and trainee.
Experts contribute to training content as much as they benefit from it. The barrier between learner and
trainer is no longer clear. To support today’s organizations’ needs, a new agile approach to training is
required; a new paradigm that supports collaborative learning and knowledge creation, flexible
learning styles, short learning and induction cycles; in order to drive organizational innovation,
competitiveness and growth.
This paper proposes a formalization of the modern Agile Learning Environment, based on a
constructivist pedagogical foundation and on using agility to foster complexity. It takes into account
various aspects of complexity, including structural and dynamic, technological and organizational [1]
[2] [3] [4] [5].
The pedagogy/andragogy foundation is grounded on constructivism [6] [7]. This is operationalized as
knowledge building [8], namely collaborative learning and information sharing in groups of expertise
– overlapping knowledge communities - Circles of Knowledge [9]. The main aspect is that the learning
paradigm changes from a traditional unidirectional, top-down, trainer-to-trainees approach, to a
community-driven, bottom-up approach. This strategy is particularly well adapted to suit the needs of
the modern adult professionals:
• highly motivated;
• has a high level of expertise;
• learns, innovates and creates knowhow continuously, at work, on-the-job. In fact, 68% of
employees prefer now to learn at work [10].
• changes frequently role and required competencies;
Proceedings of INTED2020 Conference
2nd-4th March 2020, Valencia, Spain
ISBN: 978-84-09-17939-8
9336

• knows more about certain topics than trainers;
• have limited time available [11].
The mid-2000s introduced the eLearning 2.0 paradigm, based on virtual collaboration and
communication environments, web 2.0 tools, delivery of personalized multimedia-rich content, project-
based learning, and collaborative learning environments, developed according to scientific pedagogy
principles [12] [13] [14]. This paradigm already embraced the principle that computer education is still
education, and eLearning is learning. But more than this, modern learning and education is also
necessarily eLearning and digital learning. Thus, the modern Agile Learning Environment is an
eLearning environment.
2 METHODOLOGY
The approach for this research was qualitative, exploratory and descriptive. It is based on cross-
sectional study of recent actual project cases. Conclusions are drawn from analyzing common
approaches between these projects, evaluating new modern tendencies and through generalization
and abstraction. The cases represent large complex programs for training, eLearning, communication
and capacity building, at international level, covering large user bases, in numerous varied geographic
regions and countries, in many different languages. This matches generally accepted definitions of
complex projects [15] [16]. Qualitative research supports understanding new scientific areas [17].
Multiple case studies support building new theories and their external validation. Research based on
case studies allows for emergent strategies, being among the most impactful approaches [18].
2.1 Case-study: Epale - Electronic Platform for Adult Learning in Europe
The program aims to support the European Union’s (EU) policy on adult education, being
implemented by the European Commission - Education, Audiovisual and Culture Executive Agency
(EACEA), together with DG Education, DG Employment and supported by DG Digit. The overall
objectives are to support the implementation of the European Pillar of Social Rights and the Council
Resolution on a Renewed European Agenda for Adult Learning and its successor texts; contributing to
the development of an active online community of adult learning professionals in all participating
countries; making available on the internet high quality and accurate information and advice for adult
learning professionals in Europe about adult learning policy, provision and practice; and fostering the
development of the skills of adult education professionals, including by facilitating their participation in
continuous professional development. It covers all 36 countries participating in the Erasmus+
Programme, being supported by autonomous National Support Services in each participating country.
The platform has 50,000 registered users and a catalogue of 9,000 educational resources [19].
2.2 Case-study: B-Train - EU-wide training programme in the field of customs
and taxation
B-Train, implemented by the European Commission – Directorate-General Taxation and Customs
Union (DG Taxud), is the integrated capacity building program for supporting experts, officials in
customs and taxation and economic operators throughout Europe, to be informed, learn, be certified,
communicate and exchange best practice, to increase efficiency in the implementation of the
European Customs Union regulations and of the European Customs and Taxation Competency
Framework. It supports the implementation of Customs 2020 and Fiscalis 2020 programmes. It
includes development and deployment of eLearning modules, platform and tools, capacity building,
EU-wide communication and consultancy. It covers 35 Participating Countries and all their languages
[20].
2.3 Case-study: Consumer Champion
The project was implemented by CHAFEA – the Consumers, Health and Food Executive Agency of
the European Commission, together with DG Justice (initially DG Sanco). Its objective was to build the
capacity and effectiveness of consumer organizations, by training, a collaboration web portal and
virtual community, development of multimedia eLearning courses and an eLearning portal, translation
in 11 languages. It targeted cca. 5,000 users from multiple stakeholder organizations in all EU
countries.
9337

2.4 Case-study: Consumer Classroom
The project was implemented by CHAFEA – the Consumers, Health and Food Executive Agency of
the European Commission, together with DG Justice (initially DG Sanco). Its objective was to promote
consumer education and encourage its teaching in European secondary schools, by development of
an eLearning portal and virtual community, multimedia eLearning content, a content repository, and
translation and dissemination in all 24 EU languages and 28 countries [21].
3 RESULTS AND DISCUSSION
3.1 Agile stakeholder approach
Stakeholder management must be particularly agile in today’s complex organizational environments,
characterized by multiple, ambiguous, uncertain and conflicting objectives, as well as methods to
reach these objectives [22] [3] [23]. In modern organizations, stakeholders switch roles frequently
between trainer and trainees. They contribute to the eLearning content as much as they benefit from
it. Staff needs to learn and adapt fast. They cannot afford to take the long time-offs required for
traditional training [10]; instead they need to find exactly the right piece of content needed on the spot.
Using operations management terminology, this is Just-in-time Learning (JIT Learning) [24].
Traditionally, complex projects and programs are managed through systematic stakeholder
management processes. At the same time, innovation is born out of loose, self-organized, complex
teams, that exhibit chaos characteristics and experience emergence and black-swan phenomena [25]
[26]. In fact, in order to be innovative, creative and changeable, a system must be taken away from
equilibrium, and should make use of disorder, irregularity and difference for driving change [27].
Complex knowledge-building organizations work with organized centers of expertise, or communities
of practice, which create and disseminate information and culture. Some of these groups are formal or
semi-formal. They are emergent. They are often distributed and spread across different units and
organizations.
The Circle of Knowledge concept represents formal or semi-formal communities of interest and
expertise, with common interests, that exchange information and support building knowledge. They
are moderated rather than managed; they function based on positive bottom-up incentives, reward-
based rather than task-based.
The members of these communities act both as trainers and trainees. The stakeholder relations in a
Circle of Knowledge are varied and not necessarily strict. The main directions of influence between
stakeholders are backwards and inwards (based on historical knowhow and constant feedback), as
well as forwards (anticipating needs and trends) and outwards [28]. The Circles of Knowledge
communities act thus as collaborative authoring teams of learning content.
Figure 1. Circles of Knowledge concept
9338

The Circles of Knowledge concept is presented in the figure above, and is illustrated by several
examples from the project cases below.
Figure 2. Circles of Knowledge: application to the project case EPALE
Figure 3. Circles of Knowledge: the Stakeholders Map of the B-Train project
3.2 Agile content authoring and knowledge-building process
Learning content has 2 main sources:
- Authoring specialized, professional eLearning courses;
- Building semi-structured knowledge bases.
Authoring specialized eLearning content is done using dedicated methodologies and tools. The
Successive Approximation Model (SAM) Instructional Design Approach [29] [30] is more suitable to
today’s agile learning environments. SAM is more agile compared to the traditional ADDIE (Analysis
Design Development Implementation Evaluation). Agile content authoring makes extensive use of fast
prototyping, rapid delivery cycles and Design Thinking [31]. Modern collaborative authoring such as
Articulate 360 or Adobe Captivate support both development of rich interactive courses as well as
minimalistic courses, allow for interactive collaborative reviews and recording screencasts.
9339

Generic knowledge-bases support building indexed semi-structured or non-organized searchable
databases, from many small heterogenous pieces of information, in any form or format, including basic
text, graphic and video content – mostly small bits of information, also known as nano-learnings or
micro-learnings. Ad-hoc pieces of information form dynamic emergent knowledge bases. These adapt
and grow continuously. Content is thus created using wiki, blogs, incident databases, translation tools,
shared glossaries to support common language and terminology.
The collaborative content authoring process and platform are built and deployed on agile principles,
and use agile principles for all processes and products. Deployment of an agile learning environment
requires agile project management and planning methodologies, agile release management tools such
as Jira, CI/CD platforms to support an overall agile development process to all content authoring sub-
processes; including design, storyboarding, development of complex multimedia eLearning content,
translation, localization, maintenance, continuous build, testing, integration and deployment.
Collaboration, planning, supporting agile development, management of agile sprints and releases for
authoring content collaboratively can be based on the Kanban board principle, with a multi-
dimensional matrix of dependencies between deliverables and stakeholders involved in the process.
Figure 4. Agile Learning Environment concept
A conceptual architecture for the modern agile eLearning environment is presented in the figure
above, and illustrated for several project cases below.
9340

Figure 5. The vision for the European Learning Environment (ELE) architectural concept,
of the EU customs and taxation – the B-Train programme [20]
Figure 6. Cyprus DIAS project - conceptual architecture [32]
9341

3.3 Agile eLearning and learning content
Learning agility goes well beyond content authoring or platform deployment processes. End-
deliverables are formed of agile content and agile learning, supporting the delivery of granular pieces
of information that are proven to be more easily digestible by today’s users, thereby maximizing the
effectiveness of the platform and learning process. Agile learning therefore requires moving from the
traditional hours long courses, to small pieces of information, accessible when needed (JIT learning).
eLearning has long been dominated by the Reusable Learning Objects (RLO) philosophy to enhance
agility [33]. SCORM [34] with its RLOs is exceptionally powerful and respected for its capacity for
standardization and for its versatility. Standardization remains a requirement for content and platforms,
including SCORM compliance, as well as other interoperability and packaging standards, such as
IMS, x-api for LRS integration, ChemML, MathML etc. SCORM and similar standards delivered on
their promise of interoperability, and partially delivered reusability. But the RLO paradigm has shown
significant limitations and did not deliver on its promise of agility. The RLO is intrinsically simply not
agile enough for today’s learning needs, both in terms of content authoring as well as delivery. Adult
learning requires micro- or nano-bits of information, which are created, delivered and digested on-the-
spot, Just-in-time.
Digital learning offers numerous possibilities as regards the form and delivery of content - especially in
terms of multimedia and interactivity. eLearning has long been associated with highly interactive and
rich-multimedia content. Still, the basic pedagogic principle should remain that content should not be
only fashionable and appealing, but it needs to be mostly effective. It is time to revert back to basics:
eLearning is learning [14]. Use of multimedia for the sake of multimedia is a waste of resources - the
modern user experience is designed as fit-for-purpose.
eLearning thus reverts to basic principles: Covey’s “begin with the end in mind” [35], as well as
Occam’s razor principle of simplicity, in order to design, develop and deploy whatever type of content
that supports learning and knowledge building, in whatever form and format best suitable.
3.4 Delivery: Agile Learning
Learning delivery was traditionally a unidirectional process, from trainer to trainee, with feedback loops
for monitoring, evaluation and improvement. In today’s organizations, the barrier between trainer and
trainee is not always clear. Agile stakeholder management means that trainers become trainees, and
trainees become trainers, depending on the learning context, in an agile collaborative learning
environment.
• The key stakeholders, i.e. the Who’s, are thus agile.
• The delivery, i.e. the When, is agile: the platform allows for trainees to access content and ask
for support whenever they need it.
• The location of delivery, i.e. the Where, is agile. Trainers and trainees access the content and
platform from anywhere, with particular emphasis on mobile platforms. 75% of employees use
mobile learning and 99% of mobile learners believe the mobile format enhances their learning
[36] [37] [38].
Today’s eLearning solutions such as Moodle are suitable to serve as agile learning delivery platforms.
Modern eLearning platforms include video/audio conferencing, screen sharing, app sharing and
whiteboarding, such as Slack, Skype and Google Hangouts, productivity tools such as Menti.com for
quick, live surveys; Workflowy.com, Microsoft OneNote, Google Docs or Minutes.io for collaborative
notes and minutes of meetings.
3.5 Agile project implementation, management, support and improvement
According to systems theory, complex products are implemented in complex organizations through
complex projects and programs. Agile, incremental and iterative processes are valid solutions for
managing project complexity. Agile implementation principles are supported by frameworks and tools
such as Scrum, Kanban or Rational Unified Process (RUP).
Agile projects adapt and respond better to environmental changes, and are able to exploit
opportunities associated with complexity, such as emergence, innovation and creativity. Examples of
positive complexity effects are the appearance of ad-hoc learning groups by mixing team-members
with different expertise (organizational complexity, process-related), solutions or ideas emerging from
9342

analyzing new perspectives in cross-disciplinary teams, products or content being used by users
differently than intended (technological complexity, product-related), a product going viral
unexpectedly (external environmental complexity, product-related) [39].
Implementing complex projects as programs has significant advantages: strategic importance and
focus, access to power sponsors, agility [40] [41]. Individual projects part of larger programs are
smaller and shorter, hence more agile, with clearer results, easier to measure, more transparent. In
fact, all project cases analyzed during this research were implemented as programs, with multiple
phases and iterations, with sequential or parallel project streams, following a common global strategy.
4 CONCLUSIONS
This paper presented a new holistic integrated approach to modern Agile Learning Environments and
to stakeholder management in complex learning programs, projects and organizations. These support
collaborative knowledge building in a world more and more dynamic and complex.
The modern eLearning environment is grounded on an overall agile paradigm. Agility is a powerful tool
to tackle the increasing complexity that manifests in all aspects of society. A holistic approach is
required, combining system thinking elements, such as the correlation between project complexity,
product, process and organizational complexity.
Today’s complex projects, programs and organizations are characterized by multiple, ambiguous,
uncertain and conflicting objectives, as well as multiple methods to reach these objectives. While
complexity poses significant challenges, complexity is often associated with innovation and
emergence [25] [42]. The complexity of today’s technological and learning environments creates
opportunities that must be exploited.
REFERENCES
[1] D. Baccarini, “The concept of project complexity, a review,” International Journal of Project
Management, vol. 14, no. 4, pp. 201-204, 1996.
[2] J. D. McKeen, T. Guimaraes and J. C. Wetherbe, “The Relationship between User Participation
and User Satisfaction: An Investigation of Four Contingency Factors,” MIS Quarterly, vol. 18, no.
4, pp. 427-451, Dec 1994.
[3] T. M. Williams, “The need for new paradigms for complex projects,” International Journal of
Project Management, vol. 17, no. 5, pp. 269-273, 1999.
[4] W. Xia and G. Lee, “Complexity of Information Systems Development Projects: Conceptualization
and Measurement Development,” Journal of Management Information Systems, vol. 22, no. 1, pp.
45-83, 2005.
[5] S. J. Whitty and H. Maylor, "And then came Complex Project Management (revised),"
International Journal of Project Management, no. 27, p. 304–310, 2009.
[6] J. Piaget, Psychology and Epistemology: Towards a Theory of Knowledge, New York: Grossman,
1971.
[7] M. D. Eddy, “Fallible or Inerrant? A Belated review of the "Constructivist Bible,” British Journal for
the History of Science, vol. 37, p. 93–8, 2004.
[8] M. Scardamalia and C. Bereiter, “Knowledge Building,” in Encyclopedia of Education. 2nd edition,
New York, Macmillan Reference, 2003.
[9] S. Morcov, “Collaborative knowledge building,” 18 Oct 2019. [Online]. Available:
https://blog.tremend.com/collaborative-knowledge-building/.
[10] B. Spar and R. Lefkowitz, “2018 Workplace Learning Report,” LinkedIn, 2018. [Online]. Available:
https://learning.linkedin.com/resources/workplace-learning-report-2018. [Accessed 23 Dec 2019].
[11] S. Morcov, “Welcome to TALE: Tremend’s Agile Learning Environment,” 16 Oct 2019. [Online].
Available: https://blog.tremend.com/welcome-to-tale-tremends-agile-learning-environment/.
[12] S. Downes, “E-learning 2.0,” ACM, Oct 2005. [Online]. Available: http://www.elearnmag.org/
subpage.cfm?section=articles&article=29-1. [Accessed 5 Jan 2020].
9343

[13] T. Karrer, 10 Feb 2006. [Online]. Available: http://elearningtech.blogspot.com/2006/02/what-is-
elearning-20.html. [Accessed 05 Jan 2020].
[14] S. Morcov, “Complex IT Projects in Education: The Challenge,” International Journal of Computer
Science Research and Application, vol. 2, no. 1, pp. 115-125, 2012.
[15] J. Geraldi, H. Maylor and T. Williams, “Now, let's make it really complex (complicated): A
systematic review of the complexities of projects,” International Journal of Operations &
Production Management, vol. 31, no. 9, pp. 966-990, 2011.
[16] F. Marle and L.-A. Vidal, Managing complex, high risk projects - a guide to basic and advanced
project management, London: Springer-Verlag, 2016.
[17] H. Levitt, M. M. Bamberg, J. W. Creswell, D. M. Frost, R. Josselson and C. Suárez-Orozco,
“Journal Article Reporting Standards for Qualitative Primary, Qualitative Meta-Analytic, and Mixed
Methods Research in Psychology: The APA Publications and Communications Board Task Force
Report,” American Psychologist, vol. 73, no. 1, pp. 26-46, 2018.
[18] M. Gibbert and W. Ruigrok, “The ‘‘What’’ and ‘‘How’’ of Case Study Rigor Three Strategies Based
on Published Work,” Organizational Research Methods, 2010.
[19] European Commission - EACEA, “Procurement documents - EACEA/2019/01 - EPALE
(Electronic Platform for Adult Learning in Europe): Central Support Service,” 3 May 2019. [Online].
Available: https://etendering.ted.europa.eu/cft/cft-display.html?cftId=4700.
[20] European Commission - DG Taxud, “Procurement documents - TAXUD/2019/OP/0002 -
Framework Contract for the Provision of Expert Services Supporting Multifaceted EU Training and
Staff Development in Customs and Taxation (B-TRAIN4),” 3 Jun 2019. [Online]. Available:
https://etendering.ted.europa.eu/cft/cft-display.html?cftId=4752&locale=en.
[21] SIVECO Romania, “How to be a well-informed consumer,” 18 Dec 2014. [Online]. Available:
http://www.siveco.ro/en/about-siveco-romania/press/press-releases/how-to-be-a-well-informed-
consumer.
[22] J. Turner and R. Cochrane, "Goals-and-methods matrix: coping with projects with ill defined goals
and/or methods of achieving them," International Journal of Project Management, no. 11, pp. 93-
102, 1993.
[23] College of Complex Project Managers And Defence Materiel Organisation, Competency Standard
for Complex Project Managers, Version 2.0 ed., 2006.
[24] M. Riel, “Education in the 21st Century: Just-in-Time Learning or Learning Communities,” in
Challenges of the Next Millennium: Education & Development of Human Resources. The Fourth
Annual Conference of the Emirates Center for Strategic Studies and Research, May 24–26, 1998.
[25] S. Floricel, J. L. Michel and S. Piperca, “Complexity, uncertainty-reduction strategies, and project
performance,” International Journal of Project Management, vol. 34, no. 7, pp. 1360-1383, Oct
2016.
[26] N. N. Taleb, The Black Swan: The Impact of the Highly Improbable, Random House, 2007.
[27] R. D. Stacey, “The Science of Complexity: An Alternative Perspective for Strategic,” Strategic
Management Journal, vol. 16, no. 6, p. 477–495, 1995.
[28] L. Bourne, “Project relationships and the Stakeholder Circle,” Montréal, Québec, Canada, 2006.
[29] M. Allen and R. Sites, Leaving ADDIE for SAM: An Agile Model for Developing the Best Learning
Experiences, American Society for Training and Development, 2012.
[30] W. Sirui, “The Instructional Design Model That Best Fits – SAM Model,” 18 Aug 2018. [Online].
Available: https://learningbyclick.wordpress.com/2018/08/18/the-instructional-design-model-that-
best-fits-sam-model/. [Accessed 23 Dec 2019].
[31] J. E. Arnold, Creative Engineering: Promoting Innovation by Thinking Differently, 1959.
[32] S. Morcov, L. Pintelon and R. J. Kusters, “Challenges of implementing a large-scale eLearning
and collaboration platform - the DIAS project, Cyprus,” in Conference proceedings of "eLearning
and Software for Education" ELSE, 2012.
9344

[33] Cisco Systems, “Reusable information object strategy,” 30 May 2013. [Online]. Available:
https://web.archive.org/web/20130530223125/http://www.cisco.com/warp/public/779/ibs/solutions/
learning/whitepapers/el_cisco_rio.pdf .
[34] ADLnet, “Advanced Distributed Learning Initiative,” 03 Jan 2020. [Online]. Available:
https://adlnet.gov/.
[35] S. R. Covey, The Seven Habits of Highly Effective People: Restoring the Character Ethic, New
York: Simon and Schuster, 1989.
[36] R. Skilbeck, “Top 11 learning trends for 2019,” 01 Feb 2019. [Online]. Available:
https://www.pageuppeople.com/resource/top-11-learning-trends-for-2019/.
[37] T. Tauber, T. Smolen, C. Probst and M. Huang, “How the workforce learns in 2016,” 2016.
[Online]. Available:
https://get.degreed.com/hubfs/Degreed_How_the_Workforce_Learns_in_2016.pdf.
[38] S. Johnson, “Top 5 Corporate Learning Trends for 2016 (Infographic),” 24 Feb 2016. [Online].
Available: https://elearninginfographics.com/top-5-corporate-learning-trends-2016-infographic/13/.
[39] S. Giles, The New Science of Radical Innovation: The Six Competencies Leaders Need to Win in
a Complex World, Dallas, TX: BenBella Books, 2018.
[40] P. Ribbers and K. Schoo, “Program Management and Complexity of ERP Implementations,”
Engineering Management Journal, vol. 14, no. 2, pp. 45-52, 2002.
[41] K. Remington and J. Pollack, Tools for complex projects, London: Gower Publishing Ltd, 2007.
[42] M. Eriksson, J. Lillieskoeld, N. Jonsson and D. Novosel, “How to manage complex, multinational
R&D projects successfully,” Engineering Management Journal, vol. 14, no. 2, pp. 53-60, 2002.
[43] T. Williams, “The Nature of Risk in Complex Projects,” Project Management Journal, vol. 48, no.
4, p. 55–66, 2017.
[44] N. N. Taleb, D. G. Goldstein and M. W. Spitznagel, “The Six Mistakes Executives Make in Risk
Management,” Harvard Business Review, 2009.
9345