Conference PaperPDF Available

How to Adopt Continuous Delivery? A Research Proposal



Continuous delivery is a software development discipline in which software can be released to production at any time. The proposed dissertation aims to understand the problems that emerge when adopting continuous delivery and find solutions to those problems. The goal is reached by performing a systematic literature review followed by case studies. Root cause analysis is utilized as a data collection method in some of the case studies. The contribution of the dissertation will be a theory on how continuous delivery can be adopted.
How to Adopt Continuous Delivery? A Research
Eero Laukkanen1and Casper Lassenius1,2(supervisor)
1Aalto University, P.O. BOX 19210, FI-00076, Aalto, Finland,
2Massachusetts Institute of Technology, Sloan School of Management
Abstract. Continuous delivery is a software development discipline in
which software can be released to production at any time. The proposed
dissertation aims to understand the problems that emerge when adopting
continuous delivery and find solutions to those problems. The goal is
reached by performing a systematic literature review followed by case
studies. Root cause analysis is utilized as a data collection method in
some of the case studies. The contribution of the dissertation will be a
theory on how continuous delivery can be adopted.
Key words: continuous integration, continuous delivery, continuous de-
1 Introduction
Continuous delivery (CD) is a software development discipline in which software
can be released to production at any time [1]. The discipline is achieved through
optimization, automatization and utilization of the build, deploy, test and release
process [2]. The proposed benefits of CD are increased visibility, faster feedback
and empowerment of stakeholders [2]. However, when trying to adopt CD, orga-
nizations have faced numerous challenges and problems [3, 4]. Even continuous
integration (CI), which is a prerequisite for CD [2], has not been adopted in
some cases [5].
The dissertation aims to gain deep understanding of the problems when
adopting CD and build solutions to those problems. Since the already found
problems have been numerous [3, 4], we believe that simply finding the prob-
lems is not enough, but we need to analyze the causal structure of the problems
in order to solve them. In general, software engineering project failures tend to
be caused by complex causal structure of different failure factors [6]. Our hy-
pothesis is that the adoption problems of CD are similar and they cannot be
solved in isolation from each other.
To understand the adoption problems of CD, the following research questions
are asked:
1. What problems are faced when adopting CD for a software product? With
this question, we explore what kind of problems emerge when adopting CD.
2 Eero Laukkanen, Casper Lassenius
2. What are the causes for those problems? With this question, we explain why
the problems happen when adopting CD.
3. What solutions can be used for solving those problems? With this question,
we explore available solutions for CD adoption problems.
Answering these questions builds the basis for knowledge about adopting CD
and should be valuable for organizations pursuing CD. The focus of the research
questions is on individual software products instead of whole organizations, be-
cause it is possible that problem causes when adopting CD can vary between
2 Literature review
Studies on continuous integration and continuous deployment are introduced
here, because the adoption problems of those practices often cause problems for
CD too. The difference between the practices is discussed in [1].
Downs et al. [7] recognized the importance of communication when prac-
ticing continuous integration. With increased communication, they discovered
quantitative and qualitative improvements in a case study. Thus, insufficient
communication can be problematic when adopting CD.
St˚ahl and Bosch [5] studied the experienced positive effects of continuous
integration and found that the effects differ between cases. They also noticed
that some cases do not perceive the positive effects at all. Afterwards, they
constructed a model for describing automated integration flows and found that
different cases have very different flows, which might cause the difference between
positive effects. Our research aims to understand how distinct differences affect
the outcomes.
Eck et al. [8] studied how organizations assimilate continuous integration
practice. They propose that the assimilation happens through three stages and
found organizational implications for each stage. Olsson et al. [9] studied the
usual path organizations go through when they advance software development
practices. In their stairway to heaven model, organizations evolve from agile
R&D department to continuous integration and finally to continuous deploy-
ment. In this study, we are interested in the transition from the agile R&D
department to continuous deployment.
Bellomo et al. [10] studied how design decisions affected the deployment of
the software artifacts. Their study indicates that while the automation of the
release process is important, the practice of CD requires making correct design
decisions too.
While there exists increasing interest studying CD, none of the previous
studies try to analyze the causal structure of adoption problems. This gap is to
be filled with the proposed dissertation.
How to Adopt Continuous Delivery? 3
Table 1. Studies and Schedule of the Dissertation
Subject Method Schedule
1. CD adoption problems and solutions SLR Spring 2015
2. Individual views on the adoption of CD CS Fall 2015
3. Deep understanding of CD adoption problems with root
cause analysis
CS Spring 2016
4. Impact of the corrective actions on adopting CD CS Fall 2016
5. Generalization of CD adoption problems and solutions Multi-CS Spring 2017
Summary for the dissertation Spring 2018
3 Research methodology
The dissertation will consist of five empirical qualitative studies (Table 1). First,
a systematic literature review (SLR) [11] is executed to summarize existing re-
search and experience reports on the subject. The SLR will include empirical
studies and experience reports which are qualitatively synthesized into a theory
of adoption problems and solutions. Second, the theory is refined through a sin-
gle case study [12] focusing on problems perceived by individuals in a project
adopting CD. Individual analysis is done, because it is believed that the per-
ceptions between individuals differ. Third, a single case study will be performed
using ARCA root cause analysis method [13] to study adoption problems in a
single case and to innovate solutions to the problems. Fourth, the impact of these
solutions is investigated in a follow-up study. Fifth, the root cause analysis is
performed in other cases too, and the results are synthesized into a generalized
Data sources for the case studies are Finnish software companies in the Digile
Need for Speed program that continues until 2017. Thus, there is an opportunity
to conduct long-term longitudinal case studies of CD adoption. The selection of
data sources for the case studies is done partly based on convenience, but also
according to the research goals. The subject of the research program is tightly
related to CD, so it is expected that the case companies are also valid subjects
for the studies.
4 Results and future agenda
The systematic literature review is in writing and will complete in near future.
Data collection for the single case study has been executed partially. The design
for the first root cause analysis case study has been done and the data collection
will begin in the near future.
The initial results of systematic literature review indicate that numerous
problems when adopting CD have been identified and some of them could be
solved. However, we found that there is a lack of causal analysis done on the
problems. This indicates that our future agenda on root cause analysis will pro-
vide improvement on the current knowledge.
4 Eero Laukkanen, Casper Lassenius
1. Fowler, M.: ContinuousDelivery (May 2013),
2. Humble, J., Farley, D.: Continuous Delivery: Reliable Software Releases Through
Build, Test, and Deployment Automation. Addison-Wesley Professional, 1st edn.
3. Claps, G.G., Svensson, R.B., Aurum, A.: On the journey to continuous deploy-
ment: Technical and social challenges along the way. Information and Software
Technology 57(0), 21 – 31 (2015)
4. Debbiche, A., Dien´er, M., Berntsson Svensson, R.: Challenges When Adopting
Continuous Integration: A Case Study. In: Product-Focused Software Process Im-
provement. Lecture Notes in Computer Science, vol. 8892, pp. 17–32. Springer
International Publishing (2014)
5. St˚ahl, D., Bosch, J.: Automated Software Integration Flows in Industry: A
Multiple-case Study. In: Companion Proceedings of the 36th International Confer-
ence on Software Engineering. pp. 54–63. New York, NY, USA (2014)
6. Lehtinen, T.O., M¨antyl¨a, M.V., Vanhanen, J., Itkonen, J., Lassenius, C.: Perceived
Causes of Software Project Failures–An Analysis of their Relationships. Informa-
tion and Software Technology 56(6), 623–643 (2014)
7. Downs, J., Plimmer, B., Hosking, J.: Ambient awareness of build status in collo-
cated software teams. In: 34th International Conference on Software Engineering.
pp. 507–517 (Jun 2012)
8. Eck, A., Uebernickel, F., Brenner, W.: Fit for Continuous Integration: How Or-
ganizations Assimilate an Agile Practice. In: Twentieth Americas Conference on
Information Systems. Savannah, Georgia, USA (2014)
9. Olsson, H., Bosch, J.: Towards agile and beyond: An empirical account on the
challenges involved when advancing software development practices. Lecture Notes
in Business Information Processing 179 LNBIP, 327–335 (2014)
10. Bellomo, S., Ernst, N., Nord, R., Kazman, R.: Toward Design Decisions to Enable
Deployability: Empirical Study of Three Projects Reaching for the Continuous
Delivery Holy Grail. In: 2014 44th Annual IEEE/IFIP International Conference
on Dependable Systems and Networks (DSN). pp. 702–707 (Jun 2014)
11. Kitchenham, B.: Guidelines for performing systematic literature reviews in software
engineering. Tech. rep., Keele University Technical Report (2007)
12. Runeson, P., H¨ost, M.: Guidelines for conducting and reporting case study research
in software engineering. Empirical Software Engineering 14(2), 131–164 (2009)
13. Lehtinen, T.O., M¨antyl¨a, M.V., Vanhanen, J.: Development and evaluation of a
lightweight root cause analysis method (ARCA method)–field studies at four soft-
ware companies. Information and Software Technology 53(10), 1045–1061 (2011)
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
The objective of this report is to propose comprehensive guidelines for systematic literature reviews appropriate for software engineering researchers, including PhD students. A systematic literature review is a means of evaluating and interpreting all available research relevant to a particular research question, topic area, or phenomenon of interest. Systematic reviews aim to present a fair evaluation of a research topic by using a trustworthy, rigorous, and auditable methodology. The guidelines presented in this report were derived from three existing guidelines used by medical researchers, two books produced by researchers with social science backgrounds and discussions with researchers from other disciplines who are involved in evidence-based practice. The guidelines have been adapted to reflect the specific problems of software engineering research. The guidelines cover three phases of a systematic literature review: planning the review, conducting the review and reporting the review. They provide a relatively high level description. They do not consider the impact of the research questions on the review procedures, nor do they specify in detail the mechanisms needed to perform meta-analysis.
Conference Paper
Full-text available
During the last decade, the vast majority of software companies have adopted agile development practices. Now companies are looking to move beyond agile and further advance their practices. In this paper, we report on the experiences of a company in the embedded systems domain that is adopting agile practices with the intention to move beyond agile and towards continuous deployment of software. Based on case study research involving group interviews and a web-based survey, we identify challenges in relation to (1) the adoption of agile practices, (2) testing practices, (3) continuous deployment, and (4) customer validation.
Conference Paper
Full-text available
Despite being a cornerstone of agile development, surprisingly little is known about how organizations assimilate continuous integration (CI) and what organizational changes the practice implies. Through a systematic literature review complemented by case study research we address this gap and develop a conceptual framework describing organizational implications of CI assimilation. We employ adoption theory of technology-induced innovation as theoretical lens and argue that as organizations transform through the assimilation stages, ambiguity increases, forcing them to think in alternatives and take trade-offs. To the existing body of knowledge on agile development assimilation we add an in-depth study of a distinct agile practice.
Full-text available
There is a steadily increasing interest in the agile practice of continuous integration. Consequently, there is great diversity in how it is interpreted and implemented, and a need to study, document and analyze how automated software integration flows are implemented in the industry today. In this paper we study five separate cases, using a descriptive model developed to address the variation points in continuous integration practice discovered in literature. Each case is discussed and evaluated individually, whereupon six guidelines for the design and implementation of automated software integration are presented. Furthermore, the descriptive model used to document the cases is evaluated and evolved.
Conference Paper
There is growing interest in continuous delivery practices to enable rapid and reliable deployment. While practices are important, we suggest architectural design decisions are equally important for projects to achieve goals such continuous integration (CI) build, automated testing and reduced deployment-cycle time. Architectural design decisions that conflict with deploy ability goals can impede the team's ability to achieve the desired state of deployment and may result in substantial technical debt. To explore this assertion, we interviewed three project teams striving to practicing continuous delivery. In this paper, we summarize examples of the deploy ability goals for each project as well as the architectural decisions that they have made to enable deploy ability. We present the deploy ability goals, design decisions, and deploy ability tactics collected and summarize the design tactics derived from the interviews in the form of an initial draft version hierarchical deploy ability tactic tree.
Context Continuous deployment (CD) is an emerging software development process with organisations such as Facebook, Microsoft, and IBM successfully implementing and using the process. The CD process aims to immediately deploy software to customers as soon as new code is developed, and can result in a number of benefits for organizations, such as: new business opportunities, reduced risk for each release, and prevent development of wasted software. There is little academic literature on the challenges organisations face when adopting the CD process, however there are many anecdotal challenges that organisations have voiced on their online blogs. Objective The aim of this research is to examine the challenges faced by organisations when adopting CD as well as the strategies to mitigate these challenges. Method An explorative case study technique that involves in-depth interviews with software practitioners in an organisation that has adopted CD was conducted to identify these challenges. Results This study found a total of 20 technical and social adoption challenges that organisations may face when adopting the CD process. The results are discussed to gain a deeper understanding of the strategies employed by organisations to mitigate the impacts of these challenges. Conclusion While a number of individual technical and social adoption challenges were uncovered by the case study in this research, most challenges were not faced in isolation. The severity of these challenges were reduced by a number of mitigation strategies employed by the case study organization. It is concluded that organisations need to be well prepared to handle technical and social adoption challenges with their existing expertise, processes and tools before adopting the CD process. For practitioners, knowing how to address the challenges an organization may face when adopting the CD process provides a level of awareness that they previously may not have had.
Context Software project failures are common. Even though the reasons for failures have been widely studied, the analysis of their causal relationships is lacking. This creates an illusion that the causes of project failures are unrelated. Objective The aim of this study is to conduct in-depth analysis of software project failures in four software product companies in order to understand the causes of failures and their relationships. For each failure, we want to understand which causes, so called bridge causes, interconnect different process areas, and which causes were perceived as the most promising targets for process improvement. Method The causes of failures were detected by conducting root cause analysis. For each cause, we classified its type, process area, and interconnectedness to other causes. We quantitatively analyzed which type, process area, and interconnectedness categories (bridge, local) were common among the causes selected as the most feasible targets for process improvement activities. Finally, we qualitatively analyzed the bridge causes in order to find common denominators for the causal relationships interconnecting the process areas. Results For each failure, our method identified causal relationships diagrams including 130 to 185 causes each. All four cases were unique, albeit some similarities occurred. On average, 50% of the causes were bridge causes. Lack of cooperation, weak task backlog, and lack of software testing resources were common bridge causes. Bridge causes, and causes related to tasks, people, and methods were common among the causes perceived as the most feasible targets for process improvement. The causes related to the project environment were frequent, but seldom perceived as feasible targets for process improvement. Conclusion Prevention of a software project failure requires a case-specific analysis and controlling causes outside the process area where the failure surfaces. This calls for collaboration between the individuals and managers responsible for different process areas.
Conference Paper
We describe the evaluation of a build awareness system that assists agile software development teams to understand current build status and who is responsible for any build breakages. The system uses ambient awareness technologies, providing a separate, easily perceived communication channel distinct from standard team workflow. Multiple system configurations and behaviours were evaluated. An evaluation of the system showed that, while there was no significant change in the proportion of build breakages, the overall number of builds increased substantially, and the duration of broken builds decreased. Team members also reported an increased sense of awareness of, and responsibility for, broken builds and some noted the system dramatically changed their perception of the build process making them more cognisant of broken builds.
Context: The key for effective problem prevention is detecting the causes of a problem that has occurred. Root cause analysis (RCA) is a structured investigation of the problem to identify which underlying causes need to be fixed. The RCA method consists of three steps: target problem detection, root cause detection, and corrective action innovation. Its results can help with process improvement. Objective: This paper presents a lightweight RCA method, named the ARCA method, and its empirical evaluation. In the ARCA method, the target problem detection is based on a focus group meeting. This is in contrast to prior RCA methods, where the target problem detection is based on problem sampling, requiring heavy startup investments. Method: The ARCA method was created with the framework of design science. We evaluated it through field studies at four medium-sized software companies using interviews and query forms to collect feedback from the case attendees. A total of five key representatives of the companies were interviewed, and 30 case participants answered the query forms. The output of the ARCA method was also evaluated by the case attendees, i.e., a total 757 target problem causes and 124 related corrective actions. Results: The case attendees considered the ARCA method useful and easy to use, which indicates that it is beneficial for process improvement and problem prevention. In each case, 24-77 target problem root causes were processed and 13-40 corrective actions were developed. The effort of applying the method was 89 man-hours, on average. Conclusion: The ARCA method required an acceptable level of effort and resulted in numerous high-quality corrective actions. In contrast to the current company practices, the method is an efficient method to detect new process improvement opportunities and develop new process improvement ideas. Additionally, it is easy to use.