Conference PaperPDF Available

On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study

Authors:

Abstract and Figures

Due to its enormous benefits, the research and industry communities have shown an increasing interest in the Microservices Architecture (MSA) style over the last few years. Despite this, there is a limited evidence-based and thorough understanding of the types of issues (e.g., faults, errors, failures, mistakes) faced by microservices system developers and causes that trigger the issues. Such evidence-based understanding of issues and causes is vital for long-term, impactful, and quality research and practice in the MSA style. To that end, we conducted an empirical study on 1,345 issue discussions extracted from five open source microservices systems hosted on GitHub. Our analysis led to the first of its kind taxonomy of the types of issues in open source microservices systems, informing that the problems originating from Technical debt (321, 23.86%), Build (145, 10.78%), Security (137, 10.18%), and Service execution and communication (119, 8.84%) are prominent. We identified that “General programming errors”, “Poor security management”, “Invalid configuration and communication", and “Legacy versions, compatibility and dependency" are the predominant causes for the leading four issue categories. Study results streamline a taxonomy of issues, their mapping with underlying causes, and present empirical findings that could facilitate research and development on emerging and next-generation microservices systems.
Content may be subject to copyright.
On the Nature of Issues in Five Open Source Microservices
Systems: An Empirical Study
Muhammad Waseem1, Peng Liang1, Mojtaba Shahin2, Aakash Ahmad3, Ali Rezaei Nasab4
1School of Computer Science, Wuhan University, Wuhan, China
2Faculty of Information Technology, Monash University, Melbourne, Australia
3College of Computer Science and Engineering, University of Ha’il, Ha’il, Saudi Arabia
4Department of Engineering, Computer Science and Information Technology, Shiraz University, Shiraz, Iran
{m.waseem, liangp}@whu.edu.cn, mojtaba.shahin@monash.edu, a.abbasi@uoh.edu.sa, rezaei.ali.nasab@gmail.com
ABSTRACT
Due to its enormous benets, the research and industry communi-
ties have shown an increasing interest in the Microservices Archi-
tecture (MSA) style over the last few years. Despite this, there is a
limited evidence-based and thorough understanding of the types
of issues (e.g., faults, errors, failures, mistakes) faced by microser-
vices system developers and causes that trigger the issues. Such
evidence-based understanding of issues and causes is vital for long-
term, impactful, and quality research and practice in the MSA style.
To that end, we conducted an empirical study on 1,345 issue discus-
sions extracted from ve open source microservices systems hosted
on GitHub. Our analysis led to the rst of its kind taxonomy of the
types of issues in open source microservices systems, informing
that the problems originating from Technical debt (321, 23.86%),
Build (145, 10.78%), Security (137, 10.18%), and Service execution
and communication (119, 8.84%) are prominent. We identied that
“General programming errors”, “Poor security management”, “In-
valid conguration and communication", and “Legacy versions,
compatibility and dependency" are the predominant causes for the
leading four issue categories. Study results streamline a taxonomy
of issues, their mapping with underlying causes, and present em-
pirical ndings that could facilitate research and development on
emerging and next-generation microservices systems.
CCS CONCEPTS
Software and its engineering Designing software
;
Gen-
eral and reference Empirical studies.
KEYWORDS
Microservice, Open Source Software, Microservices Architecture,
Issue, Empirical Study
ACM Reference Format:
Muhammad Waseem
1
, Peng Liang
1
, Mojtaba Shahin
2
, Aakash Ahmad
3
,
Ali Rezaei Nasab
4
. 2021. On the Nature of Issues in Five Open Source
Microservices Systems: An Empirical Study. In Evaluation and Assessment
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
EASE 2021, June 21–23, 2021, Trondheim, Norway
©2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-9053-8/21/06. . . $15.00
https://doi.org/10.1145/3463274.3463337
in Software Engineering (EASE 2021), June 21–23, 2021, Trondheim, Norway.
ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3463274.3463337
1 INTRODUCTION
The software industry has recently witnessed a growing popularity
of microservices architecture (MSA) style as a promising design
approach to develop applications that consist of multiple small,
manageable, and independently deployable services [
12
,
14
]. Soft-
ware development organizations may have adopted or plan to use
the MSA style for various reasons. Specically, some want to in-
crease the scalability of applications using the MSA style, while
others use it to quickly release new products and services to the cus-
tomer, whereas it is argued that the MSA style can also help build
autonomous development teams [
16
,
27
]. From an architectural
perspective, a microservices system (system that adopts the MSA
style) entails a signicant degree of complexity both at the design
phase as well as at runtime conguration [
23
]. This implies that
the MSA style brings unique challenges for software organizations,
and many quality attributes may be (positively or negatively) inu-
enced [
12
,
29
]. For example, service level security may be impacted
because microservices are developed and deployed by various tech-
nologies (e.g., Docker containers [
7
]) and tools that are potentially
vulnerable to security attacks [
23
,
33
]. Data management is also
inuenced because each microservice needs to own its domain data
and logic [
9
]. This can, for example, challenge reaching consistency
across multiple microservices.
Zimmermann argues that MSA is not entirely new from Service-
Oriented Architecture (SOA) (e.g., “microservices constitute one par-
ticular implementation approach to SOA – service development and
deployment”) [
35
]. Similarly, Márquez and Astudillo discovered
that some existing design rationale and patterns from SOA hold
for MSAs [
19
]. However, an important body of literature (e.g.,
[
12
,
13
,
16
,
32
]) has concluded that there are overwhelming dif-
ferences between microservices systems and monolithic systems
and traditional service-oriented systems in terms of design, imple-
mentation, test, and deployment. Gupta and Palvankar indicated
that even having SOA experience and background can lead to sub-
optimal decisions (e.g., excessive service calls) in microservices
systems development [
15
]. Hence, microservices systems may have
an additional and specic set of
issues
. Borrowing the idea from
[
12
,
25
], we dene
issues
in this study as errors, faults, failures,
and bugs that occur in a microservices system and consequently
impact its quality and functionality. Hence, there is a need for new
practices, techniques, and tools or revising old ones to address the
specic and additional issues for microservices systems.
EASE 2021, June 21–23, 2021, Trondheim, Norway M. Waseem et al.
A few studies have recently investigated particular issues (e.g.,
performance issues) in microservices systems [
26
,
28
,
31
,
34
]. De-
spite these eorts, there is no in-depth and comprehensive study on
the nature of dierent types of issues that microservices developers
face and the potential causes of these issues. Jamshidi et al. believe
that this can be partially attributed to the fact that researchers have
limited access to industry scale microservices systems [
16
]. The em-
pirical knowledge on the nature of issues occurring in microservices
systems can be useful from the following perspectives: (i) experi-
enced microservices developers can be allocated to address the most
frequent and challenging issues, (ii) novice microservices develop-
ers can quickly be informed of empirically-justied issues and avoid
common mistakes, and (iii) the industry and academic communi-
ties can invest resources to develop tools and techniques for the
frequently reported issues in microservices systems.
This work
aims to
systematically and comprehensively study and classify the
issues that developers face in developing microservices systems
and the causes that trigger these issues. To this end, we conducted
an empirical study on 1,345 issue discussions collected from ve
Open Source Software (OSS) projects in GitHub. We started with
identifying ve OSS projects in GitHub designed by following the
MSA style (open source microservices systems). Then, we extracted
and manually analyzed 1,345 issues from these ve open source
microservices systems.
Our key ndings
are based on an issue taxonomy that indicates:
(1) the issues in open-source microservices systems are classied
into 17 categories, in which Technical debt (321, 23.86%), Build (145,
10.78%), Security (137, 10.18%), and Service execution and commu-
nication (119, 8.84%) issues are most frequently reported; and (2)
the leading
causes
behind these issues are “General programming
errors”, “Poor security management”, “Invalid conguration and
communication”, and “Legacy versions, compatibility and depen-
dency”.
Our core contributions
are: (1) we develop the rst of its kind
taxonomy of issues occurring in open-source microservices systems
based on a qualitative analysis of 1,345 issue discussions among
developers, (2) we identify a comprehensive list of causes for identi-
ed issues and map them to identied issues, (3) we provide several
potential research topics on microservices systems that require
more attention, and (4) we publicly release a dataset to enable re-
searchers and practitioners to access all collected data and replicate
and validate our study [30].
Paper organization
: Section 2 reviews the related work. Sec-
tion 3 explains our research method. Section 4 presents the results
followed by a discussion on the key ndings in Section 5. Section 6
reports validity threats. Section 7 concludes the paper.
2 RELATED WORK
2.1 Mining Knowledge from Open-Source
Microservices Systems
Architectural patterns and tactics for MSA
: Architectural pat-
terns and tactics represent generic and reuse-driven knowledge
that enables developers and architects to rely on best practices dur-
ing design and development of MSA-based systems [
21
]. Several
studies, such as [
19
,
22
], focused on mining software repositories
(e.g., analyzing version control, change logs, and design documents)
to conduct postmortem analysis of historical data of open-source
MSA-based systems. The postmortem analysis of repositories such
as GitHub or logs of source code supports the empirical discovery
of patterns to address various reuse-driven development [
19
,
21
]
and evolution issues [
1
] of MSAs. Specically, the studies [
19
,
21
]
investigated artifacts of MSA-based projects deployed on GitHub,
such as developers’ discussion and source code les, to discover
17 architectural patterns that address reusability [
19
] and 5 archi-
tectural tactics that resolve scalability issues of open-source MSA-
based systems. In contrast to the design and development phase, the
study in [
1
] investigated the modernization processes of legacy soft-
ware to discover and document 15 patterns for architecture-centric
refactoring and migration of MSA-based systems. To complement
repository mining approaches [
19
,
21
], the study in [
2
] focuses on
analyzing developers’ discussion – investigating human-centric
view – regarding MSA development on SO to analyze the topics
(e.g., processes, tools, issues, solutions, etc.) that developers nd
most exciting or challenging.
Survey and analysis for issues of service evolvability
: In
addition to repository mining approaches, several studies have
focused on practitioners’ perspectives and system analysis to ad-
dress issues specic to the maintenance and evolvability of MSA.
For example, Bogner et al. [
3
] conducted 17 structured interviews
involving 14 practitioners from 10 MSA-based projects to investi-
gate the role of tools, metrics, and patterns that manage issues like
technical debt and enhance software evolvability. The surveyed
practitioners recommended many techniques, such as code review
or service cutting to address issues like granularity, composition,
coupling, and service cohesion in MSA-based applications. Analysis
of MSA-based systems, i.e., investigating source code and monitor-
ing application execution, helps to identify the issues that could
hinder or enhance the evolvability of microservices.
2.2 Detecting Code Smells and Performance
Issues in Microservices Systems
Architectural smells for MSA
: Code smells, also referred to as
architectural smells in MSA-based systems, indicate issues of poor
coding and development practices that hinder source code under-
standability and ultimately impacts service maintenance and evo-
lution [
24
,
26
]. The research in [
26
] represents an empirical study
that interviews 72 MSA developers to develop a catalog of 11 micro
service-specic architectural smells that are frequently considered
harmful by practitioners. For example, an architectural smell named
‘Wrong Cuts’ corresponds to a bad practice of microservices being
split based on technical layers (presentation, business, and data lay-
ers) instead of business capabilities. The wrong cuts for MSA result
in a wrong separation of concerns that can increase data splitting
and computation tasks for microservices. The solution presented
in [
24
] complements survey ndings on identifying architectural
smells in a semi-automated way by detecting three bad smells from
5 MSA-based systems. The studies [
24
,
26
] also pinpoint that in the
absence of design strategies or architectural tactics, code smells
could lead to functional and non-functional faults that impact the
functionality and quality of MSA.
On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study EASE 2021, June 21–23, 2021, Trondheim, Norway
Faults and performance issues in MSA
: A recently conducted
industrial study in [
34
] identies and analyzes typical faults of mi-
croservices systems, the current practice of service debugging, and
the challenges faced by developers in practice. For example, a fault
such as ‘transactional service failure’ is caused due to overloaded
requests to a third-party (payment gateway) service, ultimately
leading to a denial of service issues. In an attempt towards system-
atically addressing MSA faults, the authors in [
20
] examined the
source code and documentation of 17 open source microservices
systems to identify and document 5 architectural tactics of service
availability. The study systematically identies and characterizes ar-
chitectural tactics in microservices-based systems to prevent faults
relating to service availability and integrity. Other studies, such
as [
6
,
31
], considered performance issues such as response time,
throughput, and eciency as runtime faults for MSA-based systems.
For example, the study [
31
] contextualizes some faults from existing
literature and datasets to present a solution named MicroService
Root Cause Analysis (MicroRCA). MicroRCA identies the root
causes of performance-related faults by considering factors like
resource utilization and throughput of the services.
2.3 Conclusive Summary
To conclude, we categorize the most relevant existing research as
(a) repository mining approaches such as [
6
,
19
,
21
] and (b) survey-
based studies including [
3
,
26
,
34
] to investigate MSA-based systems
and their underlying issues. Specically, the repository mining ap-
proaches exploit GitHub data (e.g., source code, design documents,
docker les, and conguration scripts) to pinpoint a number of
issues related to reusability and architectural quality [
19
], service
scalability [
21
], and evolvability [
6
]. In comparison, our proposed
study analyzes the real issues and their discussion by MSA devel-
opers on GitHub, to investigate what are the issues and why such
issues are caused throughout the life-cycle of microservices systems.
On the other hand, existing survey-based studies primarily focus
on engaging MSA practitioners via structured questionnaire to
identify issues related to service evolvability [
3
], system faults [
34
],
and bad smells [
26
]. Such survey-based studies provide insights
about various issues in the form of practitioners’ (e.g., MSA archi-
tects) opinions and recommendations. In contrast to the existing
survey-based studies (e.g., [
3
,
26
,
34
]), our study leverages repos-
itory mining process to (i) extract issues, and (ii) taxonomically
classify the issues with their root-cause analysis.
3 RESEARCH METHOD
3.1 Phase I - Specify Research Questions (RQs)
We formulated two RQs that guide subsequent phases of the method-
ology as shown in Fig. 1.
RQ1
:What types of issues do developers face in open-source mi-
croservices systems?
Objective
: To systematically identify and taxonomically classify
the types of issues that are faced by developers and impact open
source microservices systems.
RQ2
:What are the causes of issues that occur in open-source
microservices systems?
Objective
: To investigate the causes and map them with identi-
ed issues that represent root causes of identied issues.
3.2 Phase II - Identify and Select Open-Source
MSA Projects
Identifying open-source MSA projects includes two steps: Keyword-
based Search and Conrming with Contributors.
Keyword-based Search
: To search the open source microser-
vices systems in GitHub, Phase II in Fig. 1, we followed the guide-
lines of systematic search from [
5
] to compose search strings with
format [keyword-1 <OR logic> keyword-2, . . . , keyword-N], where
keywords = [‘micro service’,‘micro-service’,‘microservice’,‘Micro
service’,‘Micro-service’,‘Microservice’]. The search strings were
searched on the title and the description of the projects on the
GHTorrent dump hosted on Google Cloud version 1/4/2018. The
search string returned 2,690 repositories. We then applied multi-
criteria ltering with reference to [
11
], focused on popularity or
perceived signicance in the community (total stars), adoption by
or interests of developers (total forks), and the number of devel-
opers (total contributors) for the projects [
4
]. We selected those
repositories whose stars and forks were more than 10, the language
was English, and contributors were more than 3. This led to 167
potential open source microservices projects.
Conrming with Contributors
: We further conrmed with
the core contributors whether these 167 projects follow the MSA
style by asking them three questions:
(1) Can you please conrm if our interpretation is correct that
this project (URL of the project) is designed by following the MSA
style? Note that if the projects (e.g., frameworks or tools) support
the development of microservices systems but are not designed
based on the MSA style, please clarify it.
(2) If our interpretation is correct, what features or characteristics
of the architecture of this open-source project show that the MSA
style has been used?
(3) Optional question: Do you know any other open-source
projects that are designed by following the MSA style?
We considered the top three contributors as core contributors of
a project who have the most commits, and their email addresses are
publicly available. Only 39 core contributors (9% of 426 core con-
tributors) responded to our inquiry. Based on the core contributors’
responses, we eventually concluded that ve projects are designed
based on the MSA style, and we selected them for data collection
and synthesis. The synopsis of these ve projects is presented in
Table 1.
Table 1: List of Identied open source microservices systems
Project Name Closed Issue Contributors Forks Stars
goa1799 82 447 4K
eShopOnContainers2904 119 6.7K 16K
light-4j3523 28 492 2.9K
Moleculer4436 76 377 3.9K
Microservices-demo5280 45 1.5K 2.5K
1https://github.com/goadesign/goa
2https://github.com/dotnet-architecture/eShopOnContainers
3https://github.com/networknt/light-4j
4https://github.com/moleculerjs/moleculer
5https://github.com/microservices-demo/microservices- demo
EASE 2021, June 21–23, 2021, Trondheim, Norway M. Waseem et al.
RQ1
Issues Faced by
Developers
RQ2
Causes of
Issues
Project Search Strings (OR logic between keywords)
OSS Project
Repository
[GitHub]
Stars > 10
Forks >10
Contributors >3
Language = 'EN'
microservice
micro service
micro-service
...
Project Selection Criteria (AND logic between criterion)
Identified
Projects
[167]
Project
Contributors
[426]
search
retrieve contact
Shortlisted
Projects
[5]
select
Phase I
Specify Research Questions
Phase II
Identify and Select MSA-based OSS Projects
Phase III
Extract and Synthesize Extracted Issues
root causes
for issues
Results
(RQ Answers)
Build Issues (BI)
Container
Fail (CF)
'DB' not
found
(Sub-theme -- Sub-issues)
Issue Classification Cause Mapping
Contributors' Discussions (developers' perspectives)
mapped to
"Container does not build when
running `docker-compose up ..."
CF
npm
cache
BI
(Issues)
...
... (Cause)
Mapping of Causes
Issues
Sub-Issues
Causes
Sub-Causes
Figure 1: Overview of the research method
3.3 Phase III- Extract and Synthesize Issues
Issues screening
: After selecting ve open-source microservices
systems, the rst author manually retrieved the background in-
formation (e.g., issue label, URL) about issues of each system (see
Table 2). Only closed issues potentially provide answers to ques-
tions about the types (RQ1) and causes (RQ2). Therefore, we only
extracted the information about closed issues. The total number of
initially retrieved issues was 2,942. During this step, the rst author
thoroughly analyzed each of the 2,942 issues and excluded all those
issues that consist of (i) general questions, opinions, feedback, and
ideas, (ii) feature requests (e.g., enhancements, proposals), (iii) an-
nouncements, (iv) duplicated issues, and (v) issues without detailed
description. There were several issues that the rst author could
not decide whether or not to include for further analysis. In such a
situation, the rst author discussed those issues with the second
and third authors for their opinions about inclusion or exclusion.
Any disagreements about the results of screening issues were dis-
cussed among all the authors to get a consensus. After completion
of this step, we got 1,345 issues.
Table 2: Data items extracted
# Data item Description
D1 Index The ID of the issue
D2 Issue title The title of the issue
D3 Issue link The weblink of the issue
D4 Number of participants
The number of practitioners who participate in
the discussion
D5 Issue (key points)
The identied key point(s) of the issue from
practitioners’ discussion
D6 Issue subcategory The derived subcategory for the issue
D7 Issue category The derived category for the issue
D8 Cause (key points)
The identied key point(s) for the cause from
practitioners’ discussion
D9 Cause subcategory The derived subcategory for the cause
D10 Cause category The derived category for the cause
Data extraction
: To answer the RQs formulated in Section 3.1,
we dened a set of data items (see Table 2) to be extracted from the
discussion of selected issues. To check the viability of the dened
data items, the rst author conducted a pilot data extraction with
200 issues, and the rest of the authors evaluated the extracted data.
After that, the rst author used a revised set of data items for
formal data extraction from the selected issues. All the authors then
checked the extracted data to reduce potential bias and ambiguity.
Data items (D1-D4) are used to provide the issue ID, issue title,
issue link, and a who participated in the discussion about issues,
whereas data items (D5-D10) are used to extract the data to answer
the RQ1 and RQ2. Finally, we used spreadsheets to structure the
data for further computation and synthesis of the extracted data.
Data synthesis
: First, we applied thematic analysis [
8
] to iden-
tify the categories of the issues, which is a method for systematically
identifying, organizing, and oering insight into meaning (e.g., cat-
egories) of a dataset. It is composed of ve steps. (i) Familiarizing
with data: The rst author repeatedly read the contributors discus-
sion about issues and documented all points regarding issues and
their causes discussed by the contributors. (ii) Generating initial
codes: after data familiarization, rst author produced an initial
list of codes from the extracted data about microservices issues
and their causes (see sheet Extracted Data in the dataset [
30
]). (iii)
Searching for the types of issues: We analyzed the initially gen-
erated codes and brought them under the specic types of issues.
For instance, “Code Smell and service dependency” (see Fig. 2). (iv)
Reviewing types of issues: all the authors reviewed and rened the
coding results with the corresponding types of issues. We separated,
merged, and dropped several issues based on mutual discussion
between all authors during this step. (v) Dening and naming cate-
gories: During this step, we dened and further rened all the types
of issues under precise and clear subcategories and categories (see
Fig. 2). We introduced two levels of categories for managing the
issues identied. First, we dened the subcategories to organize the
On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study EASE 2021, June 21–23, 2021, Trondheim, Norway
types of issues under the specic subcategory (e.g., code debt for
code smell). Then we arranged the subcategories under the precise
categories (e.g., Code Debt in Technical debt). Furthermore, we
also applied thematic analysis to identify, analyze, and classify the
causes of issues (RQ2).
4 RESULTS
This section reports the study results through analyzing and syn-
thesizing the extracted data from the issues. We report the types of
issues in Section 4.1 and the causes of the leading issue categories in
Section 4.2. The results of the issue types and causes are organized
into three levels, including categories (e.g.,
Technical debt
), sub-
categories (e.g., code debt), and types (e.g., code smell). We present
categories in
boldface
, subcategories in italic, and types in small
capitals.
4.1 Types of Issues (RQ1)
Fig. 2 presents the taxonomy of issues in open-source microser-
vices systems. The results show that Technical debt (23.86%), Build
(10.78%), Security (10.18%), and Service execution and communi-
cation (8.84%) issues are most frequently discussed. Number and
percentage of the issues are shown around the corners of each
category. Due to the space limit, we only present the top four is-
sue categories, and the complete list of categories, subcategories,
and types of issues can be found in the dataset [
30
]. We provide
examples in italic phrase with the “#” sign, which denotes the “issue
number” in the “Master data sheet” in the dataset.
Finding 1
: We derived a taxonomy of open source microservices
systems issues consisting of 17 categories, 46 subcategories, and
138 types, indicating the diversity of the issues in microservices
systems.
4.1.1
Technical debt (TD)
.is identied as a type of critical issues
in software development [
17
]. We found 321 out of 1345 (23.86%)
issues related to code debt and service design debt. Below is an
overview of each TD subcategory.
Code debt refers to the source code’s issues, which could ad-
versely impact the code’s quality. Most of the issues are related
to code smells,code refactoring, and code formatting. For
instance, goa developers found code smells in which “attributes
does not inherit properties if a list of attributes is specied in the
Body function, #143”. Similarly, developers refactored the code of
Microservices-demo project by “adding more domain logic and rules
to the “Order Aggregate-Root” module and its methods, #11”. In ad-
dition, derived class errors (e.g., “property changes on derived
classes do not occur in the mobile application, #267”), cyclomatic
complexity (e.g., “goagen rendering code does not handle the cyclic
dependency and innity loops properly, #260”), and wrong headers
(e.g., “using dial conguration header instead of Dial peer header in
generated code, #255”) also negatively aect the code legibility of
microservices systems.
Service design debt refers to the violation of adopting successful
practices (e.g., MSA patterns) for designing open source microser-
vices systems. We identied several types of issues related to ser-
vice design debt that can be further classied into business logic
separation (e.g., “separation of business logic from user registra-
tion, #245”), service dependencies (e.g., “tests of service is failed
in case of service have dependencies, #290”), and missing function-
ality (e.g., “missing proper le upload format to a generated client,
#281”). Besides, we also observed design patterns implementa-
tion (e.g., “current interface does not support the implementation of
multi-part/data decoder pattern, #276”), and orphan responses (e.g.,
orphan response is received, #282”) as service design debt.
Finding 2
: Most (i.e., 23.86%, 321 of 1345) of open source mi-
croservices systems issues are related to
technical debt
, including
Code debt (e.g., code smell) and service design debt (e.g., service
dependencies).
4.1.2
Build issue
.Build is a process in which source code is con-
verted into executable les (e.g., Jar, War, Ear, or Apk) for staging
and production environments. We found that 10.75% (i.e., 145 out of
1,345) issues are related to build issues, which are further classied
into build errors,broken and missing artifacts, and others subcate-
gories (see Fig. 2).
Build errors: This category reports the errors related to build
script,Docker build failure,build pipeline,module resolu-
tion,plugin compatibility, and source file loading.
A build script is an input artifact to build systems written in the
scripting language and carry the developer’s instruction to perform
dierent build tasks. We identied several issues in which devel-
opers discuss build script errors (e.g.,failed at the node-sass4.5.0
postinstall script ‘node scripts/build.js, #366”). The Docker build com-
mand builds Docker images from a Docker le and a “context”,
which is a set of les located in the specied PATH or URL. The
Docker build failure occurs when any context les are missing
or conguration settings are incorrect. Developers received errors
like “build error while connecting to Docker, #431. Another type
of build issue that we identied is regarding module resolution
errors in which project modules are not correctly recognized or
reloaded in the build system or dierent operating systems. For
example, developers of Moleculer project discussed “Build module
will not be reloaded when mod.js is changed, #391”, and developers
of the light-4j project mentioned “cong module build failing on
windows environment, #397”. We also identied that developers
reported a small ratio (i.e., 0.8% issues) of source file loading er-
rors. These errors mainly occurred when build systems are unable
to load specied les for compilation. For instance, Microservices-
demo developers were “unable to load the service index le for the
compilation, #425”. It is common to use many third-party plugins
or packages in open source microservices systems, and build er-
rors occur when existing code or used plugins are not compatible
with each other. Our results indicate that developers face plugin
compatibility errors during the build process of microservices
systems. For instance, Microservices-demo developers mentioned
this type of error “NU1202 Package Newtonsoft.Json 11.0.2 is not
compatible with netcoreapp 2.0, #398”.
Broken and missing artifacts: This subcategory covers the issues
related to missing artifacts (i.e., properties, packages, les), ille-
gal symbols, and broken files. This kind of error usually occurs
in the build process’s parsing stage, in which build systems ver-
ify the required information in the build script before executing
build tasks. missing properties or file errors occur when the
required les or packages are not placed on a designated location or
not imported into the required directory. For example, developers
EASE 2021, June 21–23, 2021, Trondheim, Norway M. Waseem et al.
Visual Studio
Installation Issue
Security Issue
Authentication and
Authorization (41)
Access
Control (46)
Shared Authentication
Handling Authorization
Header
Security Policy
API Key Security
Managing Credential
Setup
Encryption and
Decryption (6)
Configuration
Decryption
Data Encryption
Secure Certificate and
Connection (27)
Transport Layer
Security (TLS)
CertificatIon
Secure Socket
Layer (SSL) Connection
Expired Certificate
JSON Web
Token (JWT)
Others (17)
Cross-Origin
Resource Sharing
(CORS) header
HTTP Cookie
Build Issue
Broken or Missing
Artifacts (51)
Build Pipeline
Module Resolution
Build Script
Source File Loading
Plugin Compatibility
Broken File
Illegal Symbol
Missing
Property,
Package,or
File
Others (9)
Docker Build Failure
Obsolete
API
Wrong use of
Universally Unique
Identifier
(UUID)
Service Execution and Communication Issue
Service
Execution (17)
Service
Communication (102)
RPC Connection
gRPC Connection
Broken URLs
Server Connection
Error
Centralized
Transporter Error
Broken Connection
String
HTTP Connection
Service Messaging
Service Discovery
Failure
Web Socket Protocol
Asynchronous
Communication
Dynamic Port
Binding
RabbitMQ
Message
Service Broker
Exception Handling Issue
Resource Not Found
Exception (61)
Communication
Exception (30)
Attribute Donot
Exist
Class Not
Found
Variables Not
Declared
File Not Found
Object Reference
Not Set
Callback exception
in http2client
HTTP Request
Internenal
Server Error
JSON Reader
Exception
Others (27)
Null
Exception
Null Pointer
Exception
IO
Exception
Login
Exception
Database Issue
Database
Query (12)
Database
Connectivity (13)
Query
Parameters
Wrong Query
Database Server
Response
SQL Transient
Connection Failure
SQL Container Failure
Others (25)
Database
Migration
Database
Configuration
JSON Schema
Error
Technical Debt
Code Debt
(270)
Service Design Debt
(51)
Duplicate Value
Design Pattern
Implementation
Code Formatting
Code Refactoring
Code Smell
Derived Class
Error
Wrong Header
Service
Dependencies
Missing
Functionality
Middleware
Compatibility
Regression
Testing Error
Conflict on
Config File
Names
Orphan
Response
Configuration Issue
Configuration
Setting Error (22)
Configuration
File Error (15)
AKS Configuration
Error
Configuration
Mismatch
Docker Cinfiguration
Error
Server Configuration
Error
Unable to Find
Configuration
File Name
Conflicts
Incorrect File
Path
Overridden
Settings
Code and Component
Test (19)
Application Test
(9)
Naming
Conflict
API Test Error
Component Test
Error
Code Debug
Issue
Parameter
Validation Error
Missing Metadata
End to End
Testing Error
Payload
Validation Security Testing
Errors
Test Case
(29)
Faulty
Test Case
Missing Design
Test
Missing
Test Case
Non Buildable
Test Case
Typecasting Issue
Type
Conversion (25)
Boxing
Conversion
Enum
Conversion
Identity
Conversion
Narrow/Wide
Conversion (15)
Narrowing Reference
Conversion
Narrowing Primitive
Conversion
Widening Primitive
Conversion
Others (10)
No Control Over
Typecasting
Deployment Issue
Platform and
Technology (40)
Docker Deployment
Error
Virtual Machine
(VM) Error
Kubernetes
Deployment Error
Amazon Container
Service Error
Others (10)
CD Pipeline
Bottleneck
Mesos
Deployment
Error
Deployment
Script Error
Monitoring Issue
Health Check API Error (4)
Observational
Error (6)
Tracing and
Logging
Management
(9)
Storage Issue
Data Size and
Restore (13)
Storage Size
Constraints
(22)
Large Image
Size
Package
Restore
Low Buffer
Size
Heap Out of
Memory
Cache
Miss
Lack of Main
Memory
Stack
Overflow
UDP Discover
Error
Documentation Issue
Insufficient
Document (37)
Readability
Issue (24)
Broken
Images
Inappropriate
Examples
Outdated
Document
Updates and Installation Issue
Installation
Errors (15)
Update Errors
(22)
Node Package
Manage (NPM)
Installation Error
Poor
Readability
Missing
Readme File
Languages Package
Installation Error
Angular
Update
Version
Inconsistency
Networking Issue
Service
Accessibility (5)
Hosting and
Protocols (4)
DNS and
URL Errors
Firewall and
Proxy Settings
Localhost
Exception
Performance Issue
Front-end Service
Hangs (1)
High CPU Usage (2)
Service
Delay (6)
Compilation Issue
Syntax
Error (69)
Unexpected End-
of-File (3)
Wrong Method
Call (18)
Wrong Pointer
Type (3)
Testing Issue
Category
Taxonomy Legend
Type of Issues
Taxonomy
137 (10.18%)
9 (0.65%) 93 (6.91%)
119 (8.84%) 118 (8.77%)
321 (23.86%) 37 (2.75%) 57 (4.23%)
Taxonomy of Issues in MSA-based OSS Systems
50 (3.71%)
50 (3.71%)
19 (1.41%)
35 (2.6%)
9 (0.65%)
61 (4.53%)
37 (2.75%)
Number of Issues
(Percentage)
Inconsistent
Code
Syntax Errors
Datatype
Mismatch
Fix Validation
Error
Data Decryption
Business Logic
Separation
Excessive Literals
Cyclomatic
Complexity
145 (10.78%)
50 (3.71%)
Build
Error (85)
(Number of Issues)
Subcategory
Figure 2: A taxonomy of issues in ve open source microservices systems
On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study EASE 2021, June 21–23, 2021, Trondheim, Norway
discussed that “Catalog.WebForms missing almost all les cant be
compiled, #319” for Microservices-demo build. We identied several
posts regarding illegal symbols found during the build process,
which were triggered when the build system fails to resolve charac-
ters in the build script (e.g., “typo in the code, #342”, “invalid attribute
type, #390”). We identied broken file issues that generally occur
during the assembly stage of the build process. The assembly stage
will not proceed further when there are duplicated or broken les.
For instance, the Moleculer project’s build is failed “when KubeFed
event broadcast is broken, #293”.
Finding 3
: Issues appearing in the build stage of open source
microservices systems account for 10.78% (145 of 1345) of the is-
sues, mainly related to build errors,Docker build failure, and
broken or missing artifacts, and obsolete APIs.
4.1.3
Security issue
.Microservices provide public interfaces, use
network-exposed APIs for communicating with other services, and
are developed by using polyglot technologies and toolsets that may
be insecure. This makes microservices a potential target for cyber-
attacks; therefore, security in open source microservices systems
demands serious attention. Based on 132 Security issues, we dene
ve subcategories (see Fig. 2)
Authentication and authorization: Authentication is the process
of identifying a user, whereas authorization determines the ac-
cess rights of the specied user to system resources. We found
issues related to establishing secure authentication and authoriza-
tion mechanisms for open source microservices systems. Most of
these subcategory issues are related to handling authorization
header and shared authentication issues. The issues associated
with handling authorization header (e.g., Basic Auth, OAuth,
OAuth 2.0) are generally used to implement authorization mecha-
nisms. For instance, OAuth 2.0 lets the APIs authenticate and access
the required resources. We found several issues related to Basic
Auth, OAuth, or OAuth 2.0 header failure or non-availability. For
example, the developers of goa project discussed the issue about
gRPC authentication elds missing, #461”, and the developers of the
light-4j project mentioned that “OAuth helper needs to handle the er-
ror status from OAuth 2.0 provider, #470”. We also found many issues
about an improper implementation of shared authentication
techniqes, e.g., “shared authentication methods are not generated,
#464” in the goa project.
Security certicate and standard: We found several issues about
implementing security certicates and standards for open source
microservices systems. Security certicates (e.g., SSL, TSL) and stan-
dards (e.g., JWT) are used to secure connection and communication
between two parties (e.g., client-server, service to service). For ex-
ample, we found an issue “an SSL connection could not be established
for Docker 2.3.0.3 with WSL2 support” in the Microservices-demo
project. Another issue is that “TLS certicates and OAuth2 certi-
cates have conicts, #578” over authorization and authentication
in the light-4j project. Similarly, we found several issues in which
the developers discussed errors, violations, and failures in imple-
menting the jwt token. For instance, “JWT token fails, #509” in
the light-4j project because light rest API expects JWT to be in the
authorization module.
Encryption and decryption refer to converting plain text into ci-
phertext and ciphertext into plain text to secure the information.
We found several issues related to data encryption,data decryp-
tion, and configuration decryption of microservices systems.
For example, the Microservices-demo project developers discussed
that “The anti-forgery token could not be decrypted when the apps
are hosted at the same location, #494”. As another example, “The
values were not encrypted in a secret.yml le containing condential
information of the light-4j project, #495”.
We also found several Security issues about Access control (e.g.,
managing credential setups,API key security) and Others (e.g.,
cors header,http cookies). However, due to space limitation, we
can not discuss all the
Security issues
. For details, see Fig. 2 and
the dataset [30].
Finding 4
: 10.18% (135 of 1345) of the issues are identied as
Security issues in open source microservices systems, mainly re-
lated to authentication and authorization,access control,encryption
and decryption, and secure certicate and connection.
4.1.4
Service execution and communication issue
.Commu-
nication problems are deceptive when services communicate across
multiple servers and hosts in a distributed environment. Services
interact using e.g., HTTP, AMQP, and TCP protocols depending on
the nature of services. Our results show that 8.84% (119 out of 1,345)
issues are related to the execution and service communication of
microservices with two subcategories.
Service communication: This subcategory reports the issues, mainly
related to connection strings,communication protocols (e.g.,
gRPC, HTTP, Web Socket, RPC), centralized transporters (e.g.,
MQTT, NATS, Kafka, Moleculer), server connection,service dis-
covery, and broken URLs. The result reveals that most developers
face service discovery failure,HTTP protocol implementa-
tion, and centralized transporters issues. For instance, API
service discovery failed in light-4j project because of using “https
connect to Web console, #692”.
Service execution: This subcategory reports the issues regarding
asynchronous communication,dynamic port binding,Rab-
bitMQ messaging, and service broker during service execution.
These issues occur due to dierent reasons. For example, a de-
pendency issue between microservices occurred when “integration
commands were sent asynchronously, #598. We also found several
issues regarding dynamic port binding. For instance, a server
module of light-4j project could not dynamically allocate a “port on
the same host with a given range, #616”. Similarly, some developers
face a situation in which “old services could not be replaced with
new services because Service broker could not properly destroy the old
services, #690”.
Finding 5
: 8.84% (119 out of 1345) of the issues are related to
service execution (e.g., RPC connection) and service communication
(e.g., asynchronous communication) of open source microser-
vices systems.
4.2 Causes of Issues (RQ2)
We now report the causes of identied issues. For space reasons, we
report and exemplify causes corresponding to the subcategories of
the top four issue categories only. Illustrative details in Table 3 guide
the interpretation for the remaining categories and subcategories
of causes. To support such interpretations for extended details, a
complete list of cause categories, cause subcategories, and types
EASE 2021, June 21–23, 2021, Trondheim, Norway M. Waseem et al.
of causes can be found in the dataset [
30
]. For the top four issue
categories (see Fig.2), we identied a total of 94 types of causes that
can be classied into 7 categories and 21 subcategories. Overall, Ta-
ble 3 maps the cause subcategories of leading issue categories (i.e.,
TD
,
Build
,
Security
,
Service execution and communication
)
to their cause categories, which are
General programming error
(GPE)
,
Poor security management (PSM)
,
Invalid congura-
tion and communication (ICC)
,
Legacy version, compatibil-
ity, and dependency problem (LCD)
. The other three categories
are
Design level anomaly (DLA)
,
Missing features and arti-
facts (MFA)
, and
Insucient resource (IR)
. The numbers (e.g.,
158) in Table 3 denote the number of issues in a given issue sub-
category that stems from a certain cause category. Note that not
all the causes behind the issues are discussed. For instance, in the
Others subcategory of the
build issues
, we were not able to locate
any cause for obsolete API and wrong use of UUID issues. Note
that We identied 1007 (out of 1345) issues in which developers
discuss the causes for the issues (see sheet Causes for All Issues
in the dataset [
30
]), 584 issues out of 1007 issues are belong to the
leading four categories (see sheet Causes for Leading Issues in the
dataset [
30
]). Below, we report the top four cause categories with
the issues caused by them.
Finding 6
: Overall, we derived 7 categories and 26 subcategories
of causes from the 109 type of causes for all the issue categories,
indicating the variety of the reasons behind issues in open source
microservices systems.
Table 3: Distribution of issue subcategories (vertical) and
cause categories (horizontal)
GPE PSM ICC LCD DLA MFA IR
Code debt 158 0 1 1 33 9 0
Service design debt 0 0 0 0 19 0 0
Build errors 18 0 12 27 15 3 16
Broken and missing artifacts 8 0 0 0 0 31 0
Build: Others 0 0 0 0 0 0 0
Access control 1 23 12 0 4 0 0
Authentication and authorization: 1 32 8 2 0 2 0
Encryption and decryption 0 6 1 0 0 0 0
Security certicate and standard 0 23 1 5 0 0 0
Security: Others 0 8 1 0 0 1 0
Service communication 34 2 34 6 4 0 3
Service execution 3 0 10 2 0 0 0
4.2.1
General programming errors
.This category consists of
5 subcategories and 25 types of causes. The leading cause sub-
categories are erroneous method denition and execution,incorrect
naming and types, and compile time error (see sheet Causes for All
Issues in dataset [
30
]). Table 3 shows the distribution of leading
issue categories and GPE causes. Our ndings reveal that most GPE
causes have become the source for
code debt
,
service commu-
nication
, and
build error
issues. We identied 158 causes that
become the source for dierent types of code debt issues. We also
found several types of causes that become the reasons for more
than one issue. For example,
compile time errors
(e.g., semantics
errors) become the reasons for service communication,build errors,
and broken and missing artifacts issues.
Finding 7
:
General programming errors
identied as the ma-
jor cause category for microservices issues. Erroneous method de-
nition and execution, and incorrect naming and types, and compile
time error found as the major subcategories.
4.2.2
Poor security management
.This category consists of 3
subcategories and 17 types of causes mainly related to implementa-
tion of security control and policy at coding,communication , and
application levels (see sheet Causes for Leading Issues in the dataset
[
30
]). Our results show that most of PSM causes have become the
source for authentication and authorization,access control, and se-
curity certicates and standard issues. For instance, we identied
several authentication and authorization issues that occur because
of writing unsafe code, using expired authentication token,
and providing malformed input.
Finding 8
: Causes appearing in
poor security management
are account for 87 issues, all belong to coding (e.g., unsafe code,
communication (e.g., decryption failure), and application (e.g.,
missing security features) level subcategories.
4.2.3
Invalid configuration and communication
.We identi-
ed 2 subcategories (i.e., conguration problems,communication
problems) and 13 types of causes that become the source for 87
microservices issues. Incorrect configuration,service reg-
istry errors, and incorrect reqest handling are recognized
as leading
ICC
causes (see sheet Causes for Leading Issues in the
dataset [
30
]). Our result shows that most
ICC
causes have become
the source for issues from service communication,service execu-
tion,access control, and build error subcategories (see Table 3). For
instance, we identied several service communication issues that
ensued incorrect configuration,limitation of communica-
tion protocols, and deprecated localhost. Similarly, we also
nd causes to (build error) that occurred because of incorrect
configuration and firewall blocking. For example, Docker
build failure due to incorrect configuration.
Finding 9
: The types of causes (e.g., incorrect configuration)
from the
invalid conguration and communication
category
raise most issues in the leading issue categories (e.g., service com-
munication,build error,access control).
4.2.4
Legacy versions, compatibility, and dependency
.This
category covers the causes for issues that occur when develop-
ers use old versions of the software.
LCD
category consists of 3
subcategories (i.e., development and deployment,client and server,
compatibility and dependency), and 14 types of causes that become
the source for 49 microservices issues of top four categories. For
example, several build errors occur due to old dev branch, and Ku-
bernetes version. Similarly, causes (e.g., compatibility problem)
from the compatibility and dependency subcategory also become the
reason of build error,authorization and authentication, and secure
certicate and connection issues.
Finding 10
: 8.4% (49 out of 584) of identied causes are related
to the use of the old version of the software (e.g., identity server),
languages (e.g., Go), and deployment platforms (e.g., Kubernetes)
that give rise to build error,service communication, and access control
issues in open source microservices systems.
On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study EASE 2021, June 21–23, 2021, Trondheim, Norway
5 DISCUSSION
We now discuss (i) the top four categories of issues, (ii) their causes,
along with (iii) implications for research and practices.
Management of TD in microservices systems
: Around one-
fourth (23.62%) of the issues are related to TD, and the signs of
the TD can be found in various activities of microservices system
development (e.g., design, coding, refactoring, conguration). Our
ndings reveal that the primary causes behind TD issues are several
types of General programming errors, such as lack of cohesion, long
message chains, and semantic errors. We observed that TD issues in
microservices systems are growing at a higher rate than other types
of issues identied in this study. For instance, issues identied for
other categories are far less than TD (see Figure 2). Several studies
(e.g., [
10
,
26
]) have conrmed the existence of TD in microservices
systems from dierent perspectives (e.g., architectural TD, code
smells). This study results (i.e., the taxonomy of issues, causes of
issues) can raise awareness of architects and developers to avoid
the accumulation of TD issues before becoming too costly). Despite
this, we assert that it could be valuable to investigate some other
aspects of TD in microservices systems, such as i) how to design
microservices systems to control TD, ii) how TD grows at dierent
levels of microservices systems over time (e.g., design, code, com-
munication), and iii) tools and techniques used to identify, measure,
prioritize, monitor, and prevent TD in microservices systems.
Build process in microservices systems
: The build process
consists of several activities depending on programming languages,
build scripting languages, operating systems, and development pro-
cesses (e.g., CI/CD, DevOps).10.78% of the issues are related to the
microservices systems’ build process, mainly due to missing fea-
tures and artifacts, legacy versions, compatibility and dependency,
and insucient resources (see Fig. 2 and Table 3). The build is-
sues and their causes indicate that most build problems occur in
issue-triggering phases (e.g., parsing) due to developers’ mistakes
(e.g., adding dependencies) in build script le. The types of build
issues and their causes identied in our study can help practitioners
to avoid various types of build issues. For instance, practitioners
might avoid using unnecessary dependencies in Docker build les
and older versions of Kubernetes while establishing the build and
deployment pipeline for microservices systems. Many studies (e.g.,
[
18
]) investigated the build issues from monolithic systems in which
they claim that developers make non-trivial eorts to address the
build issues. However, we did not nd any literature that has at-
tempted to investigate what eorts are required to x the build
issues in microservices systems. It would be interesting to com-
pare the types of issues and the required eorts and cost to x
the build issues in microservices systems with monolithic systems.
The ndings of such studies can help software organizations in the
migration of monolithic systems to microservices systems.
Securing microservices systems
: The distributed nature of mi-
croservices systems makes them a potential target for cyber-attacks.
10.18% of the issues belong to the security issue category, mostly
because of poor security management of coding, communication,
and application-level (see the dataset [
30
]). The issues related to
JWT, TSL, SSL, and HTTP cookies type indicate that most of the
security issues occurred when microservices are communicating.
Similarly, identied causes are also pointing that the security of
microservices is poorly managed on a communication level. These
results conrm that microservices systems have a much larger at-
tack surface area than traditional systems. The identied security
issues and their causes can help practitioners to understand better
why and where specic security issues occur in microservices sys-
tems. For instance, practitioners might want to avoid writing unsafe
code to prevent access control issues. Our ndings suggest that se-
curity issues are multi-faceted, meaning that security problems can
be raised at dierent levels of microservices systems, such as data
centers, cloud providers, virtualization, communication, and orches-
tration. Therefore, it is valuable to i) explore security valunaribality
and related risks at various levels, and ii) propose multi-layered
security solutions for ne-grained security management.
Executing and communicating microservices
: Microservices
systems are distributed systems running on multiple servers or
hosts. Microservices communicate through inter-process communi-
cation protocols, such as HTTP and AMQ, depending on the type
of microservices. We identied that 8.87% of the issues interrupt
execution and communication of microservices mainly because of
incorrect conguration and general programming errors. This cate-
gory of issues and their causes can help practitioners to i) identify
the problem areas and ii) make strategies for avoiding the prob-
lems of execution and communication of microservices systems
(see the dataset [
30
]). Microservices systems may have hundreds
of instances that frequently communicate with each other. Ser-
vice execution and communication of microservices systems can
also exacerbate the issue of resiliency, load balancing, distributed
tracing, high coupling, and complexity. With these issues in mind,
researchers and practitioners can i) propose design techniques for
microservices architecture with a particular focus on highly re-
silient and low coupled microservices systems and ii) develop eec-
tive solutions to trace and isolate communication faults.
6 THREATS TO VALIDITY
Internal validity
corresponds to methodological rigor to ensure
minimal bias in data collection and, ultimately, its impact on study
results. As per the methodology in Fig. 1, potential threats to in-
ternal validity can be (i) improper selection of OSS projects and
data collection and (ii) researchers’ bias in extracting and synthe-
sizing the qualitative data. To minimize internal validity threats,
we adopted a multi-step approach to minimize the possible threat
stemming from subject system selection (Phase I in Fig. 1). A step-
wise and criteria-driven approach for the project selection helped
us identify the most appropriate projects (see Table 1) and elimi-
nate several false positives that may impact internal validity and
its impacts on the study results. To mitigate the risk of researchers’
bias in extracting and synthesizing the qualitative data (Phase III,
Fig. 1), we dened the explicit criteria (e.g., excluding all those is-
sues that consist of general questions), designed the data extraction
forms (see Table 2, and applied the thematic analysis approach by
involving all the authors. Any disagreements about the results of
screening, data extraction, and data synthesis process were dis-
cussed among all the authors to get a consensus for its resolution.
External validity
of the research refers to the generalization
and applicability of the results beyond this study. To maximize the
external validity, we started with a large number of issues (2,942
EASE 2021, June 21–23, 2021, Trondheim, Norway M. Waseem et al.
issues) and systematically selected 1,345 issues from the ve mi-
croservices OSS projects. Besides, to ensure the presentation (types
of identied issues) and hierarchical organization of the taxonomy
(i.e., types, subcategories, categories), we created an initial version
of the taxonomy for evaluation by domain experts. A total of four
experts (OSS and MSA researchers and developers) provided us
with their feedback to rene and nalize the taxonomy (Fig. 2).
However, the constructed taxonomy of issues (RQ1) and the iden-
tied categories of causes (RQ2) are exclusive to GitHub and may
not be comprehensive. We acknowledge that analyzing other open-
source microservices systems or proprietary microservice systems
may lead to dierent and/or more comprehensive taxonomies of is-
sues and causes. We also acknowledge that the ve studied projects
cannot represent all OSS MSA projects in GitHub, and we may
have missed some important open source microservices projects.
Moreover, we provided the dataset of this study (containing issues
and causes) online to verify and replicate this study [30].
7 CONCLUSIONS
In this work, we conducted an empirical study on the nature of
issues (e.g., bugs, faults, errors, failures, mistakes) in open-source
microservices systems with a taxonomy of issues based on a man-
ual analysis of 1,345 issue discussions from ve OSS projects. The
taxonomy is composed of 17 categories containing 46 subcategories
and 137 types of issues. We identied that the leading issues stem
from technical debt, build, security, and service execution and com-
munication. We also identied and classied the causes for the
issues. Overall, this study equips researchers and practitioners with
a deep insight into the development and deployment of open source
microservices systems.
As part of future work, we aim for (1) validation of the proposed
taxonomy of issues using an industrial survey as well as their
potential causes and solutions from the practitioners’ perspective,
and (2) investigation of the diculty and priority levels of the
identied issues when dealing with them in practice.
ACKNOWLEDGMENTS
This work has been partially supported by the National Key R&D
Program of China with Grant No. 2018YFB1402800.
REFERENCES
[1]
A. Balalaie, A. Heydarnoori, P. Jamshidi, D. Tamburri, and T. Lynn. 2018. Mi-
croservices migration patterns. Software: Practice and Experience 48, 11 (2018),
2019–2042.
[2]
A. Bandeira, C. A. Medeiros, M. Paixao, and P. H. Maia. 2019. We need to talk
about microservices: An analysis from the discussions on StackOverow. In Proc.
of the 16th Int. Conf. on Mining Software Repositories (MSR). IEEE, 255–259.
[3]
J. Bogner, J. Fritzsch, S. Wagner, and A. Zimmermann. 2019. Assuring the evolv-
ability of microservices: Insights into industry practices and challenges. In Proc.
of the 35th IEEE Int. Conf. on Software Maintenance and Evolution (ICSME). IEEE.
[4]
H. Borges and M. T. Valente. 2018. What’s in a GitHub star? Understanding
repository starring practices in a social coding platform. Journal of Systems and
Software 146 (2018), 112–129.
[5]
J. Brings, M. Daun, M. Kempe, and T. Weyer. 2018. On dierent search methods
for systematic literature reviews and maps: Experiences from a literature search
on validation and verication of emergent behavior. In Proc. of the 22nd Int. Conf.
on Evaluation and Assessment in Software Engineering (EASE). ACM, 35–45.
[6]
J. Cito, G. Schermann, J. Wittern, P. Leitner, S. Zumberi, and H. C. Gall. 2017. An
empirical analysis of the docker container ecosystem on GitHub. In Proc. of the
14th Int. Conf. on Mining Software Repositories (MSR). IEEE, 323–333.
[7]
T. Combe, A. Martin, and R. Di Pietro. 2016. To docker or not to docker: A security
perspective. IEEE Cloud Computing 3, 5 (2016), 54–62.
[8]
D. S. Cruzes and T. Dyba. 2011. Recommended steps for thematic synthesis
in software engineering. In Proc. of the 5th ACM/IEEE Int. Symp. on Empirical
Software Engineering and Measurement (ESEM). IEEE, 275–284.
[9]
C. de la Torre, B. Wagner, and M. Rousos. 2020. .NET Microservices: Architecture
for Containerized .NET Applications. Microsoft Corporation.
[10]
S. S. De Toledo, A. Martini, A. Przybyszewska, and D.I. Sjøberg. 2019. Architec-
tural technical debt in microservices: A case study in a large company. In Proc. of
the 2nd Int. Conf. on Technical Debt (TechDebt). IEEE, 78–87.
[11]
J. Di Rocco, D. Di Ruscio, C. Di Sipio, P. Nguyen, and R. Rubei. 2020. TopFilter:
An approach to recommend relevant GitHub topics. In Proc. of the 14th ACM/IEEE
Int. Symp. on Empirical Software Engineering and Measurement (ESEM). ACM,
1–11.
[12]
N. Dragoni, S. Giallorenzo, A. L. Lafuente, M. Mazzara, F. Montesi, R. Mustan,
and L. Sana. 2017. Microservices: Yesterday, Today, and Tomorrow. In Present
and Ulterior Software Engineering. Springer, 195–216.
[13]
C. Esposito, A. Castiglione, and K. Choo. 2016. Challenges in delivering software
in the cloud as microservices. IEEE Cloud Computing 3, 5 (2016), 10–14.
[14]
M. Fowler and J. Lewis. 2014. Microservices: A denition of this new architectural
term. http://martinfowler.com/articles/microservices.html
[15]
D. Gupta and M. Palvankar. 2020. Pitfalls & challenges faced during a microservices
architecture implementation. https://tinyurl.com/49n54h4p
[16]
P. Jamshidi, C. Pahl, N. C. Mendonça, J. Lewis, and S. Tilkov. 2018. Microservices:
The journey so far and challenges ahead. IEEE Software 35, 3 (2018), 24–35.
[17]
Z. Li, P. Avgeriou, and P. Liang. 2015. A systematic mapping study on technical
debt and its management. Journal of Systems and Software 101 (2015), 193–220.
[18]
Y. Lou, Z. Chen, Y. Cao, D. Hao, and L. Zhang. 2020. Understanding build issue
resolution in practice: symptoms and x patterns. In Proc. of the 28th ACM Joint
Meeting on European Software Engineering Conf. and Symp. on the Foundations of
Software Engineering (ESEC/FSE). ACM, 617–628.
[19]
G. Márquez and H. Astudillo. 2018. Actual use of architectural patterns in
microservices-based open source projects. In Proc. of the 25th Asia-Pacic Software
Engineering Conf. (APSEC). IEEE, 31–40.
[20]
G. Márquez and H. Astudillo. 2019. Identifying availability tactics to support
security architectural design of microservice-based systems. In Proc. of the 13th
European Conf. on Software Architecture (ECSA) Companion. Springer, 123–129.
[21]
G. Márquez, M. Villegas, and H. Astudillo. 2018. An empirical study of scalability
frameworks in open source microservices-based systems. In Proc. of the 37th Int.
Conf. of the Chilean Computer Science Society (SCCC). IEEE, 1–8.
[22]
G. Muntoni, J. Soldani, and A. Brogi. 2020. Mining the architecture of microservice-
based applications from their Kubernetes deployment. In Proc. of the 16th Int.
Workshop on Engineering Service-Oriented Applications and Cloud Services (WE-
SOACS). Springer, 1–12.
[23]
S. Newman. 2020. Building Microservices: Designing Fine-grained Systems (second
ed.). O’Reilly Media, Inc.
[24]
I. Pigazzini, F. A. Fontana, V. Lenarduzzi, and D. Taibi.2020. Towards microservice
smells detection. In Proc. of the 3rd Int. Conf. on Technical Debt (TechDebt). ACM,
92–97.
[25]
Y. Ren, G. Gay, C. Kästner, and P. Jamshidi. 2020. Understanding the nature of
system-related issues in machine learning frameworks: An exploratory study.
arXiv abs/2005.06091 (2020).
[26]
D. Taibi and V. Lenarduzzi. 2018. On the denition of microservice bad smells.
IEEE Software 35, 3 (2018), 56–62.
[27]
D. Taibi, V. Lenarduzzi, and C. Pahl. 2017. Processes, motivations, and issues for
migrating to microservices architectures: An empirical investigation. IEEE Cloud
Computing 4, 5 (2017), 22–32.
[28]
A. Walker, D. Das, and T. Cerny. 2020. Automated code-smell detection in
microservices through static analysis: A case study. Applied Sciences 10, 21 (2020),
7800.
[29]
M. Waseem, P. Liang, and M. Shahin. 2020. A systematic mapping study on
microservices architecture in devops. Journal of Systems and Software 170 (2020),
110798.
[30]
M. Waseem, P. Liang, M. Shahin, A. Ahmed, and A. Razaei Nasab. 2021. Dataset
of the Paper “On the Nature of Issues in Five Open Source Microservices Systems:
An Empirical Study”. https://doi.org/10.5281/zenodo.4602870
[31]
L. Wu, J. Tordsson, E. Elmroth, and O. Kao. 2020. MicroRCA: Root cause local-
ization of performance issues in microservices. In Proc. of the IEEE/IFIP Network
Operations and Management Symp. (NOMS). IEEE, 1–9.
[32]
T. Yarygina and A. H. Bagge. 2018. Overcoming security challenges in microser-
vice architectures. In Proc. of the 12th IEEE Int. Conf. on Service-Oriented System
Engineering (SOSE). IEEE, 11–20.
[33]
D. Yu, Y. Jin, Y. Zhang, and X. Zheng. 2019. A survey on security issues in
services communication of Microservices-enabled fog applications. Concurrency
and Computation: Practice and Experience 31, 22 (2019), e4436.
[34]
X. Zhou, X. Peng, T. Xie, J. Sun, C. Ji, W. Li, and D. Ding. 2018. Fault analysis and
debugging of microservice systems: Industrial survey, benchmark system, and
empirical study. IEEE Transactions on Software Engineering (2018).
[35]
O. Zimmermann. 2017. Microservices tenets. Computer Science-Research and
Development 32, 3-4 (2017), 301–310.
... Since the advent of the MSA style, securing microservices systems has been a challenge for software practitioners and organizations [4], [12], [13], [14], [15], [16], [17]. The potential security challenges associated with microservices systems may compel software organizations to revisit their decision to adopt or migrate to microservices [18], [19]. ...
... Waseem et al. [14] empirically investigated the issues reported in five open source microservices systems hosted on GitHub. They found that 10.18% of these issues can be attributed to security. ...
Preprint
Full-text available
Despite the numerous benefits of microservices systems, security has been a critical issue in such systems. Several factors explain this difficulty, including a knowledge gap among microservices practitioners on properly securing a microservices system. To (partially) bridge this gap, we conducted an empirical study to manually analyze 861 security points collected from 10 GitHub open-source microservices systems and Stack Overflow posts concerning security of microservices systems, leading to a catalog of 28 microservices security practices. We then ran a survey with 63 microservices practitioners to evaluate the usefulness of these 28 practices. Our findings demonstrate that the survey respondents affirmed the usefulness of the 28 practices. These 28 security practices are further classified into six categories based on their topics: Authorization and Authentication, Token and Credentials, Internal and External Microservices, Microservices Communications, Private Microservices, and Database and Environments. We believe that the catalog of microservices security practices can serve as a valuable resource for microservices practitioners to more effectively address security issues in microservices systems. It can also inform the research community of the required or less explored areas to develop microservices-specific security practices and tools.
... For instance, Ponce et al. [108] developed a taxonomy of smells and refactorings for microservices security by reviewing the grey and peer review literature. In our recent work (i.e., [109]), security issues are also identified as one of the top issues (10.18%) in microservices systems. The study result indicates that practitioners use patterns (e.g., Access token), cloud solutions (e.g., Role-Based Access Control), and development practices (e.g., DevSecOps) as solutions to address security challenges (see Section 4.2.5). ...
Preprint
Full-text available
Context: Microservices Architecture (MSA) has received significant attention in the software industry. However, little empirical evidence exists on design, monitoring, and testing of microservices systems. Objective: This research aims to gain a deep understanding of how microservices systems are designed, monitored, and tested in the industry. Method: A mixed-methods study was conducted with 106 survey responses and 6 interviews from microservices practitioners. Results: The main findings are: (1) a combination of domain-driven design and business capability is the most used strategy to decompose an application into microservices, (2) over half of the participants used architecture evaluation and architecture implementation when designing microservices systems, (3) API gateway and Backend for frontend patterns are the most used MSA patterns, (4) resource usage and load balancing as monitoring metrics, log management and exception tracking as monitoring practices are widely used, (5) unit and end-to-end testing are the most used testing strategies, and (6) the complexity of microservices systems poses challenges for their design, monitoring, and testing, for which there are no dedicated solutions. Conclusions: Our findings reveal that more research is needed to (1) deal with microservices complexity at the design level, (2) handle security in microservices systems, and (3) address the monitoring and testing challenges through dedicated solutions.
... For instance, Ponce et al. [108] developed a taxonomy of smells and refactorings for microservices security by reviewing the grey and peer review literature. In our recent work (i.e., [109]), security issues are also identified as one of the top issues (10.18%) in microservices systems. The study result indicates that practitioners use patterns (e.g., Access token), cloud solutions (e.g., Role-Based Access Control), and development practices (e.g., DevSecOps) as solutions to address security challenges (see Section 4.2.5). ...
Article
Full-text available
Context: Microservices Architecture (MSA) has received significant attention in the software industry. However, little empirical evidence exists on design, monitoring, and testing of microservices systems. Objective: This research aims to gain a deep understanding of how microservices systems are designed, monitored, and tested in the industry. Method: A mixed-methods study was conducted with 106 survey responses and 6 interviews from microservices practitioners. Results: The main findings are: (1) a combination of domain-driven design and business capability is the most used strategy to decompose an application into microservices, (2) over half of the participants used architecture evaluation and architecture implementation when designing microservices systems, (3) API gateway and Backend for frontend patterns are the most used MSA patterns, (4) resource usage and load balancing as monitoring metrics, log management and exception tracking as monitoring practices are widely used, (5) unit and end-to-end testing are the most used testing strategies, and (6) the complexity of microservices systems poses challenges for their design, monitoring, and testing, for which there are no dedicated solutions. Conclusions: Our findings reveal that more research is needed to (1) deal with microservices complexity at the design level, (2) handle security in microservices systems, and (3) address the monitoring and testing challenges through dedicated solutions.
Conference Paper
Full-text available
Researchers and practitioners have recently proposed many Microservices Architecture (MSA) patterns and strategies covering various aspects of microservices system life cycle, such as service design and security. However, selecting and implementing these patterns and strategies can entail various challenges for microservices practitioners. To this end, this study proposes decision models for selecting patterns and strategies covering four MSA design areas: application decomposition into microservices, microservices security, microservices communication, and service discovery. We used peer-reviewed and grey literature to identify the patterns, strategies, and quality attributes for creating these decision models. To evaluate the familiarity, understandability, completeness, and usefulness of the decision models, we conducted semi-structured interviews with 24 microservices practitioners from 12 countries across five continents. Our evaluation results show that the practitioners found the decision models as an effective guide to select microservices patterns and strategies.
Chapter
Full-text available
Microservices Architecture (MSA) style is a promising design approach to develop software applications consisting of multiple small and independently deployable services. Over the past few years, researchers and practitioners have proposed many MSA patterns and strategies covering various aspects of microservices design, such as application decomposition. However, selecting appropriate patterns and strategies can entail various challenges for practitioners. To this end, this study proposes a decision model for selecting patterns and strategies to decompose applications into microservices. We used peer-reviewed and grey literature to collect the patterns, strategies, and quality attributes for creating this decision model.
Article
Full-text available
Microservice Architecture (MSA) is becoming the predominant direction of new cloud-based applications. There are many advantages to using microservices, but also downsides to using a more complex architecture than a typical monolithic enterprise application. Beyond the normal poor coding practices and code smells of a typical application, microservice-specific code smells are difficult to discover within a distributed application setup. There are many static code analysis tools for monolithic applications, but tools to offer code-smell detection for microservice-based applications are lacking. This paper proposes a new approach to detect code smells in distributed applications based on microservices. We develop an MSANose tool to detect up to eleven different microservice specific code smells and share it as open-source. We demonstrate our tool through a case study on two robust benchmark microservice applications and verify its accuracy. Our results show that it is possible to detect code smells within microservice applications using bytecode and/or source code analysis throughout the development process or even before its deployment to production.
Conference Paper
Full-text available
Software architecture is undergoing a transition from monolithic architectures to microservices to achieve resilience , agility and scalability in software development. However, with microservices it is difficult to diagnose performance issues due to technology heterogeneity, large number of microservices, and frequent updates to both software features and infrastructure. This paper presents MicroRCA, a system to locate root causes of performance issues in microservices. MicroRCA infers root causes in real time by correlating application performance symptoms with corresponding system resource utilization, without any application instrumentation. The root cause localization is based on an attributed graph that model anomaly propagation across services and machines. Our experimental evaluation where common anomalies are injected to a microservice benchmark running in a Kubernetes cluster shows that MicroRCA locates root causes well, with 89% precision and 97% mean average precision, outperforming several state-of-the-art methods.
Conference Paper
Full-text available
While Microservices promise several beneficial characteristics for sustainable long-term software evolution, little empirical research covers what concrete activities industry applies for the evolvability assurance of Microservices and how technical debt is handled in such systems. Since insights into the current state of practice are very important for researchers, we performed a qualitative interview study to explore applied evolvability assurance processes, the usage of tools, metrics, and patterns, as well as participants' reflections on the topic. In 17 semi-structured interviews, we discussed 14 different Microservice-based systems with software professionals from 10 companies and how the sustainable evolution of these systems was ensured. Interview transcripts were analyzed with a detailed coding system and the constant comparison method. We found that especially systems for external customers relied on central governance for the assurance. Participants saw guidelines like architectural principles as important to ensure a base consistency for evolvability. Interviewees also valued manual activities like code review, even though automation and tool support was described as very important. Source code quality was the primary target for the usage of tools and metrics. Despite most reported issues being related to Architectural Technical Debt (ATD), our participants did not apply any architectural or service-oriented tools and metrics. While participants generally saw their Microservices as evolvable, service cutting and finding an appropriate service granularity with low coupling and high cohesion were reported as challenging. Future Microservices research in the areas of evolution and technical debt should take these findings and industry sentiments into account.
Conference Paper
Full-text available
Microservices are a new and rapidly growing architectural model aimed at developing highly scalable software solutions based on independently deployable and evolvable components. Due to its novelty, microservice-related discussions are increasing in Q&A websites, such as StackOverflow (SO). In order to understand what is being discussed by the microservice community, this work has applied mining techniques and topic modelling to a manually-curated dataset of 1,043 microservice-related posts from StackOverflow. As a result, we found that 13.68% of microservice technical posts on SO discuss a single technology: Netflix Eureka. Moreover, buzzwords in the microser-vice ecosystem, e.g., blue/green deployment, were not identified as relevant subjects of discussion on SO. Finally, we show how a high discussion rate on SO may not reflect the popularity of a certain subject within the microservice community.
Conference Paper
Full-text available
Microservices is an architectural style that considers systems as modular, costumer-centric, independent, and scalable suite of services. In order to address security requirements in microservices-based systems, architects often must focus on critical quality attributes, such as availability, aiming at employing architectural solutions that provide design decisions that address key security concerns (also known as architectural tactics). Although current architectural tactics for availability offer an extensive catalog of alternatives to improve availability and security factors, new availability concerns (emerging from security microservices requirements) demand new or improved architectural tactics. In this article, we examined the source code and documentation of 17 open source microservices-based systems, identified 5 uses of availability tactics, and characterized them using a newly introduced descriptive template. We found that almost all (4 out of 5) tactics did focus on preventing faults rather than detecting, mitigating or recovering from them (which are the traditional tactics taxonomies' branches). This approach can be further used to systematically identify and characterize architectural tactics in existing microservices-based systems in other critical quality attributes concerning security, such as confidentiality and integrity.
Article
Full-text available
Microservice-based systems (MBS) are distributed systems consistent of smaller components (possibly developed by separate teams using different technologies) that use lightweight mechanisms to communicate with each other. One of the main quality attributes that motivate MBS is scalability, and there are several available frameworks to face scalability challenges in MBS. However, there is still little empirical data about the architectural concerns of using these frameworks to support the scalability dimensions. This article builds upon a previous study that defined a pattern language for MBS scalability, and aims to identify and analyze which MBS frameworks are actually used in open source MBS projects to address the scalability dimensions. Results show that (1) nine common frameworks satisfy the scalability dimensions defined in the pattern language, (2) frameworks produce trade-offs among scalability dimensions, and (3) few frameworks address several scalability dimensions at once. Finally, we identify five reusable design decisions to address scalability requirements and propose them as microservices architectural tactics.
Conference Paper
Full-text available
Introduction: Software companies aim to achieve continuous delivery to constantly provide value to their customers. A popular strategy is to use microservices architecture. However, such an architecture is also subject to debt, which hinders the continuous delivery process and thus negatively affects the software released to the customers. Objectives: The aim of this study is to identify issues, solutions and risks related to Architecture Technical Debt in microservices. Method: We conducted an exploratory case study of a real life project with about 1000 services in a large, international company. Through qualitative analysis of documents and interviews, we investigated Architecture Technical Debt in the communication layer of a system with microservices architecture. Results: Our main contributions are a list of Architecture Technical Debt issues specific for the communication layer in a system with microservices architecture, as well as their associated negative impact (interest), a solution to repay the debt and the its cost (principal). Among the found Architecture Technical Debt issues were the existence of business logic in the communication layer and a high amount of point-to-point connections between services. The studied solution consists of the implementation of different canonical models specific to different domains, the removal of business logic from the communication layer, and migration from services to use the communication layer correctly. We also contributed with a list of possible risks that can affect the payment of the debt, as lack of funding and inadequate prioritization. Conclusion: We found issues, solutions and possible risks that are specific for microservices architectures not yet encountered in the current literature. Our results may be useful for practitioners that want to avoid or repay Technical Debt in their microservices architecture.