Variability debt in opportunistic reuse: A multi-project field study

Full text

Variability Debt in Opportunistic Reuse: A Multi-Project Field Study
Daniele Wolfarta, Jabier Martinezb, Wesley K. G. Assun¸c˜aoc,d,∗
, Thelma E. Colanzie, Alexander Egyedf
aPPGComp, Western Paran´a State University (UNIOESTE), Cascavel, Brazil.
bTecnalia, Basque Research and Technology Alliance (BRTA), Derio, Spain.
cCSC, North Carolina State University (NCSU), Raleigh, USA.
dOPUS, Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Rio de Janeiro, Brazil.
eDIN, State University of Maring´a (UEM), Maring´a, Brazil.
fISSE, Johannes Kepler University Linz (JKU), Linz, Austria.
Abstract
Technical debt is a metaphor to guide the identification, measurement, and general management of decisions that are
appropriate in the short term but create obstacles in the future evolution and maintenance of systems. Variability
management, which is the ability to create system variants to satisfy different business or technical needs, is a potential
source of technical debt. Variability debt, recently characterized in a systematic literature review we conducted, is
caused by suboptimal solutions in the implementation of variability management in software systems. In this work, we
present a field study in which we report quantitative and qualitative analysis of variability debt through artifact analysis
(e.g., requirements, source code, and tests) and a survey with stakeholders (e.g., analysts, developers, managers, and a
user). The context is a large company with three different systems, where opportunistic reuse (a.k.a., copy-and-paste
or clone-and-own reuse) of almost all project artifacts was performed to create variants for each system. We analyze
the variability debt phenomenon related to opportunistic reuse, and we assess the validity of the metaphor to create
awareness to stakeholders and guide technical debt management research related to variability aspects. The results of
the field study show evidences of factors that complicate the evolution of the variants, such as code duplication and non-
synchronized artifacts. Time pressure is identified as the main cause for not considering other options than opportunistic
reuse. Technical practitioners mostly agree on the creation of usability problems and complex maintenance of multiple
independent variants. However, this is not fully perceived by managerial practitioners.
Keywords: technical debt, variability management, clone and own, software reuse
1. Introduction
The technical debt metaphor emerged as an explana-
tion of the consequences of technical issues to nontechnical
product stakeholders [1]. “Not quite right code which we
postpone making it right” [1, 2] is how technical debt was
originally described. Since then, the technical debt con-
cept is already well-known in the software engineering cul-
ture, and it has been discussed and investigated from many
perspectives [3, 4] such as how to identify, measure, pri-
oritize, transform, or monitoring it across the system life
cycle [5]. Technical debt management is now considered
a discipline by itself, responsible to handle this important
aspect in systems and software engineering [6].
Recently, Avgeriou et al. [7] framed the term technical
debt as “delayed tasks and immature artifacts that consti-
tute a ‘debt’ because they incur extra costs in the future in
the form of increased cost of change during evolution and
maintenance”. This definition of technical debt describes
well the issues observed when opportunistic reuse practices
∗Corresponding author
Email address: wguezas@ncsu.edu (Wesley K. G. Assun¸c˜ao)
(i.e., non-systematic reuse of parts of an existing product
to create new ones—a.k.a., copy-and-paste or clone-and-
own) are used recurrently across projects without a proper
management of commonality and variability [8, 9]. This is
the scenario we focus on this work. Notably, for tailoring
products for different markets according to diverse regula-
tions, software that needs to work for diverse target hard-
ware, or just the need to allow end-user customization to
satisfy their specific needs, software companies commonly
adopt opportunistic reuse practices [10]. This practice of
copying and adapting similar products or features from ex-
isting projects lead to achieve short term benefits such as
creating new product variants quickly. However, it leads to
technical drawbacks in a medium- and long-term, like du-
plicate code and bugs, challenging the propagation of im-
provements and fixes across all the variants [9]. As we pre-
sented above, this problem traces back to what is known
as technical debt. The complexity in the evolution of the
family of software product variants as a whole is dramat-
ically increased [6]. This is a context frequently observed
in industry, which demonstrated the practical relevance of
understanding this phenomenon [11].
Despite the wide discussion of both technical debt [3, 4]
Preprint submitted to Journal of Systems and Software January 10, 2024

and engineering of families of software products with vari-
ability [12, 13, 14], little is known about how they are
related. To fill this gap, in a previous work we conducted
an investigation of variability management from a techni-
cal debt perspective [15]. Based on a systematic literature
review, we provided an initial definition of what we re-
ferred to as “Variability Debt”. Variability Debt refers to
technical debt caused by inadequate and/or sub-optimal
solutions in the implementation of variability management
in software systems [15]. In this initial investigation, we
distilled the causes, consequences, and artifacts impacted
by variability debt. However, a thorough study on how
variability debt manifests in practice is still missing.
The goal of this paper is to describe how variability
debt incurs in a real scenario and how practitioners in
this scenario perceive this phenomenon. For that, we con-
ducted a multi-project exploratory investigation following
the field study method [16]. A field study focus on data
collection in a natural setting with minimal intrusion of the
researcher to not disturb realism. The field study reported
in this paper relies on three projects in which opportunistic
reuse was used to create product variants. Thus, the focus
of this paper is on variability debt exclusively in the con-
text of opportunistic reuse as a variability management
approach, causing issues in evolution and maintenance.
The projects are developed and maintained in a Brazil-
ian multinational industry1leader in its market segment.
This industry follows a business strategy of producing all
goods needed in its product chain, so software variants
are created to different industry branches or departments.
We collected and analyzed data from artifacts of different
product life-cycle phases (e.g., requirements, source code,
and test/validation cases) from a total of seven system
variants, from three projects. Additionally, we asked 22
stakeholders (e.g., developers, analysts, testers, managers,
and a user) about their perception of technical debt in op-
portunistic reuse as well as about their own observations
on our findings.
The results of the artifact analysis showed evidences
of commonality and variability in the artifacts analyzed,
and inconsistencies among product variants, challenging
their future synchronization. We also observed a lack of
proper test cases for new variants. The survey with differ-
ent stakeholder roles showed agreements and divergences
with the causes and consequences identified in our previous
study [15]. For the variability debt definition and interest
of future research and development roadmap, stakeholders
agreed on the relevance of looking at variability manage-
ment issues from the technical debt management perspec-
tive.
The main contributions of this paper are:
•For practitioners, awareness about the importance
of Systematic Variability Management : despite the
1The name of the industry, its market segment, and the name of
the subject systems are omitted to keep confidentiality.
large number of studies on variability management
and engineering of system families, the use of system-
atic approaches (e.g., Software Product Lines [17]) is
still not widely adopted in the industry. Our findings
contribute to bring the attention of practitioners to
problems generated by using opportunistic and non-
systematic practices for software reuse and variabil-
ity management.
•For researchers, opportunities for new studies: the
survey included questions in which participants pri-
oritized the variability debt management activities
indicating, to some extent, the aspects with more
potential interest or impact.
•For tool builders, source of information for new tool-
ing support: similarly to the opportunities for new
studies, tooling and visualization support for vari-
ability debt management is desired, mainly because
of the complexity of dealing with families of product
variants.
The remainder of this paper is structured as follows:
Section 2 presents background and Section 3 describes re-
lated works. Then, Section 4 presents the study design,
followed by the results and the discussion in Section 5.
Section 6 discusses the threats to validity. Finally, Sec-
tion 7 presents the conclusions and outlines future work.
2. Background on Variability Debt
Variability Debt [15] occurs when practitioners choose
to implement variability with an approach that would bring
short-term benefits (e.g., using opportunistic reuse) re-
gardless of technical and architectural drawbacks in the
medium- and long-term. This leads to maintenance and
evolution difficulties to manage families of systems com-
posed of multiple individual variants [17].
In our previous study, we characterize the variabil-
ity debt from the perspective of its causes, consequences,
and artifacts [15]. Among the types of impacted artifacts,
source code is the most common, followed by requirements,
architecture, and models. Test cases, database schemas,
and build scripts were referenced in few occasions. Vari-
ability debt is observed in different types of artifacts cre-
ated along the whole software development life-cycle, and
it is not exclusive to the software implementation phase.
This is aligned with the current understanding of technical
debt management [6].
Regarding the context in which variability debt occurs,
we list below the main causes:
•Business aspects: to decrease time to market and
reduce development costs;
•Technical aspects: poor design decisions of variabil-
ity implementation, lack of traceability among arti-
facts, and over-engineering; and,
•Operational aspects: to integrate geographically dis-
tributed teams, merge different companies, and the
lack of knowledge of the system domain.
2

Companies are faced with several problems due to vari-
ability debt. The consequences observed are: complex
maintenance of multiple independent variants, inability to
systematically deal with customization, inability to create
complex systems, poor overall internal quality, complex
test cycles, creating usability problems, portfolio manage-
ment problems, and repeated roles in similar projects.
We conceived a catalog of variability debt by crossing
information on technical debt reported in the 52 case stud-
ies considered in our systematic literature review [15]. Be-
low, we briefly present the entries of this catalog. These
entries are inspired on technical debt subtypes as defined
in Li et al. [6], but from the perspective of variability debt
concerns.
•Out-of-date, incomplete, and duplicate docu-
mentation: The implementation of variability us-
ing opportunistic reuse usually leads to duplicate
documentation. The maintenance of multiple doc-
uments becomes complex and ends up being a tech-
nical debt. The team might tend to implement the
variability and not update the documentation in the
variants.
•Code duplication: Practitioners copy and paste
code of features or entire products, leading to com-
plex maintenance of multiple independent product
variants. For example, when a bug occurs, it must
be fixed in all product variants. Similarly, enhance-
ments are difficult to synchronize among variants as
it requires a change propagation effort. This is one
of the most frequent type of technical debt when im-
plementing variability with opportunistic reuse.
•Multi-version support: When variability is imple-
mented in a non-systematic way, it brings short-term
benefits, such as time-to-market, without requiring
knowledge of variability management. However, op-
portunistic reuse lead to multiple product versions
that can be upgraded at different times. For exam-
ple, a variant A is upgraded while a variant B is not,
on the other hand it is possible to have an upgrade
for variant C and not for A and B. When source code
and documentation must be synchronized, support
becomes even more complex.
•Architectural anti-patterns: The implementation
of variability adopting opportunistic reuse can lead
to undesired anti-patterns, such as the replication of
a complex integration each time a product is cloned,
requiring architecture refactoring and evolution.
•Lack of testing, expensive tests, and poor test
of feature interactions: For releasing a new prod-
uct variant quickly, the testing phase might be skipped
or partially conducted. With the assumption that
the main product works, practitioners might decide
that there is no need to test the new product vari-
ant, or the testing phase is underestimated, then
the test is reduced to save time. The main conse-
quence of this debt is the potential lack of quality in
the variants. Elaborating complex and even repeti-
tive test scenarios to cover all system variants might
be a time- and resource-consuming activity, mainly
when the development team have limited knowledge
in variability management and testing techniques, or
just have limited time to put the testing infrastruc-
ture in place. Possible feature interactions should
be tested when creating new variants with different
feature configurations because the interaction of fea-
tures can change variants’ behavior.
•Old technology in use: This variability debt oc-
curs when the company has multiple variants of a
legacy system, or multiple variants were created to
meet different requirements with the original legacy
technology and architecture. When there is a need
to update the underlying technology, it becomes a
difficult task.
In this present work, we relied on our catalog of vari-
ability debts to carry out the field study.
3. Related Work
Li, Avgeriou, and Liang [6] presented a systematic re-
view to propose a technical debt catalog that includes 10
types of technical debt and eight activities for technical
debt management. However, they do not discuss the tech-
nical debts caused by the non-systematic implementation
of variability. In our previous work [15], we proposed a cat-
alog of variability debts in the light of technical debt found
in the literature [6] and we added new debts found in our
study. Our catalog, presented in Section 2, includes the
technical debts that occurs when implementing variability
by means of opportunistic reuse, its causes, consequence
and the main artifacts involved. In this present work, we
relied on our catalog of variability debts to carry out the
field study.
The state of the art on technical debt has not culmi-
nated in rigorous analysis models yet. In this sense, Siebra
et al. [18] investigated decisions on technical debt and re-
lated events in an industrial large scale project with the
goal of characterizing the technical debt existence as well
as its parameters’ evolution. The study was based on only
one system (SMB - Samsung Mobile Business) and in three
different scenarios: persistence layer, user interface layer,
and during the inclusion of a new programming language.
However, characteristics related to the system variabilities
were not investigated.
Some studies investigated strategies to manage techni-
cal debt and prioritization criteria to pay off the debts [19,
20, 21]. Ribeiro et al. [19] presented 14 criteria to support
the development team in deciding which debts should be
prioritized. They also presented a list of debt types related
to each criterion. In a study with 60 software engineers of
11 software companies from nine different countries, Ar-
vanitou et al. [20] found that stakeholders need different
3

views of the quality dashboard on metrics related to techni-
cal debt because they usually have different interests. For
instance, managers are more interested in financial con-
cepts, whereas developers are more interested in the na-
ture of the problems that exist in the code. This difference
was also observed in our study (see Section 5). Lenarduzzi
et al. [21] performed a systematic literature review to in-
vestigate the body of knowledge on approaches proposed
in both academy and industry to prioritize technical debts.
The main finding is that there is no consensus on which
are the most important factors and how they should be
measured in the context of technical debt prioritization.
Our work mainly differs from those because we investigate
the technical debt related to variability, and we are not
discussing when and how to pay off the technical debts in
this context.
Fenske and Schulze [22] introduced the notion code
smells related to variability, proposing an initial catalog
of four variability-aware code smells. Their findings point
out that the proposed smells (i) have been observed in
existing product lines and (ii) are considered to be prob-
lematic for common software development activities, such
as program comprehension, maintenance, and evolution.
Our work complements this notion because we are inter-
ested in improving the understanding of variability debt.
Digkas et al. [23] investigated the effects of software
reuse on technical debt. They carried out an empirical
study on the relation between the existence of reusing code
retrieved from StackOverflow and the technical debt of the
target system. Their empirical results provide insights to
the potential impact of small-scale code reuse on technical
debt and highlight the benefits of assessing code quality
before performing the reuse of pieces of code. Despite
addressing software reuse and technical debt, their study
did not take into account families of systems, thus, they
did not study variants created using opportunistic reuse.
Verdecchia et al. [24] focused on architectural techni-
cal debt in software-intensive systems. This is the debt
incurred in the architectural level of software design (e.g.,
choices regarding structure, frameworks, technologies, and
languages) that, while being suitable or even optimal when
made, significantly hinder progress in the future. The goal
of the study is understanding how software development
organizations conceptualize architectural debt, and how
they deal with it. Differently from their study, our work
addresses systems that implement variability in a non-
systematic way, observing any level of technical debt (code
or architectural).
Recently, Capilla et al. [25] carried out an exploratory
study to investigate to what extent reusing components
opportunistically negatively affects the quality of systems,
focusing on the impact of opportunistic reuse on technical
debt. They were concerned with the difficulties and impact
of reusing third-party components, while we are concerned
with the debt related to variability management.
Given the aforementioned context, at the time of the
writing, we have not found any investigation about the re-
lationship between implementing variability using oppor-
tunistic reuse and technical debt. Thus, the purpose of
our study is to investigate this relationship and assess its
importance in an industrial context.
During the review process of this manuscript, we orga-
nized a first workshop in the software and systems product
line conference (SPLC) on the subject of technical debt
for variability-intensive systems (TD4ViS) [26]. The dis-
cussions were fruitful, and the subject was discussed from
different perspectives.2However, because of the time lim-
itations, we did not manage to solve the issue of charac-
terizing the variability debt phenomenon.
4. Study Design
Our work relies on a field study. As defined by Stol
and Fitzgerald [16], a field study has a “natural setting
that exists before the researcher enters it with a minimal
intrusion of the setting so as not to disturb realism, only to
facilitate data collection.” The authors also point out that
it “facilitates study of phenomena and actors and their
behavior in natural contexts.” The exploratory nature of
a field study helps “to understand what’s going on, how
things work, or to generate hypotheses.” Based on this
definition of a field study, the goal of our work is:
Goal Understanding variability debt in the natural set-
tings of a company that uses opportunistic reuse to
create families of product variants.
To achieve this goal through our field study, we focus
on answering two research questions regarding variability
debt in opportunistic reuse:
RQ1 Which artifacts are affected by variability debt and
how?
RQ2 How do practitioners from the company perceive
variability debt?
Technical debt can occur at any stage or activity of a
software development process [6]. The RQ1 of our field
study serves as a confirmatory study regarding which and
how artifacts from different stages of the development pro-
cess are affected by opportunistic reuse. Our ob jective
with RQ1 is to discuss the impact of opportunistic reuse
from a technical debt perspective. For RQ2, we focused
on studying three main aspects of the industry: (i) char-
acterization of the practitioners; (ii) artifacts, causes, and
consequences related to the technical debt because of op-
portunistic reuse; and (iii) definition and perspectives re-
garding the variability debt concept and its management.
To answer the research questions and achieve our goal,
in Section 4.1 we describe in detail the industrial scenario,
and in Section 4.2 we present the data collection and anal-
ysis process. Despite the advantages of a field study, it
also has intrinsic threats to validity regarding generaliza-
tion, which are further discussed in Section 6.
2https://td4vis.github.io/2022/
4

4.1. Characteristics of the Industrial Scenario
The field study reported in this paper was conducted in
a Brazilian multinational industry leader in its market seg-
ment in Latin America. This industry produces all goods
needed in its product chain, having an Information Tech-
nology (IT) department in charge of providing software
solutions for all branches or departments within the com-
pany. Based on that, the team of the IT department have
employed opportunistic reuse when a customized variant
of a software system is needed. These practitioners have
also been responsible for maintaining the variants of the
systems in operation.
4.1.1. Subject Systems and Variants
Table 1 presents a characterization of the three systems
(names omitted, see previous footnote 1), in terms of their
variants, the total number of features, and the number of
common features among the variants.
SysX is a software system for the inventory manage-
ment of finished goods, having three product variants. The
Original variant was developed for stock management at
headquarters. After a few years, a branch of the group, re-
sponsible for delivering and packaging, grew up substan-
tially and had the need to control its own stock, then,
Variant1 was created. Recently, the company created a
branch that is a distribution center, reusing again the Orig-
inal variant to create Variant2. During the opportunistic
reuse, some features cease to exist in variant systems (e.g.,
Check Remittance). This happened because Variants 1
and 2 were customized for branches of the organizational
group that do not carry out sales, but only store prod-
ucts, unlike the variant Original. This way, remittance
was not needed. On the other hand, a new feature ap-
peared in Variant 1, namely Remove Blocked Fractional,
because, contrary to the headquarters that uses the vari-
ant Original and the branch that uses Variant2, Variant1
was deployed in a branch that deals with products that can
be fractionated (e.g., delivering part of a batch to another
branch of the industry).
SysY manages documents and controls their printing in
the company. The difference between the variant Original
and Variant1 is that some features were removed during
Table 1: Subject Systems and Variants
System Variant Year #F #CF
SysX
Original 2010 14
7Variant1 2018 10
Variant2 2020 11
SysY Original 2012 34 24
Variant1 2016 24
SysZ Original 2018 26 24
Variant1 2018 26
Year = year in which the variant was created, #F = number
of features, #CF = number of common features
the reuse. The variant Original was implemented for the
headquarters, which uses manufacturing orders to gener-
ate the finished product. In this variant, there are specific
features that associate documents with these manufactur-
ing orders and with the materials. Differently, Variant1
was implemented to a branch that does not have focus on
product manufacturing, but that needed to control docu-
mentation, thus, some unnecessary features were removed.
SysZ manages non-conformity, that is, actions or situa-
tions that fail to comply with a pre-established procedure.
The system controls the registration, evaluation, investiga-
tion, creation of the action plan related to non-conformity.
The differences between the Original and Variant1 is that
two features were removed (Supplier/Manufacturer Non-
Conformity Registration and CAC Non-Conformity Reg-
istration) and two features were added (Classification Of-
ficer Report and Investigation Officer Report). Thus, the
IT team adapted the Original to create Variant1, given
that more reports were needed to submit for inspection.
4.1.2. Stakeholders and Current practices
We had access to reused artifacts and stakeholders that
took part during the use of opportunistic reuse for creating
the variants of the subject systems. For the field study, we
collected information (data collection procedure described
in the next section) from several stakeholders. Most of
them are from the IT department, such as managers, de-
velopers, analysts, and testers. Additionally, for the sake
of representativeness, we also collected information from
managers and a user, which allowed us to enrich the anal-
ysis of the causes and consequences of variability debt.
As the characteristics of the stakeholders were collected
through the survey, we delegate the presentation of the 22
participants to Section 5.
4.2. Data Collection and Analysis
We describe below the sources of information for col-
lecting the data, analysis conducted, and tools used for
our field study.
4.2.1. Source of information
For the data collection of our field study, we have
two main sources of information: (i) artifacts such as
source code, system requirements, feature specifications,
and test/validation cases; and (ii) survey with stakehold-
ers for collecting characteristics and opinions of people
involved with the subject systems and their variants. For
collecting the data, according to Lethbridge et al. classi-
fication [27], we used a technique of the second and third
level. The survey is in the second level, as we collected the
answers (raw data) directly from the stakeholders using
an online form, namely Google Forms.3The data col-
lected from artifacts is considered in the third level, since
we analyzed artifacts already available.
3https://www.google.com/forms/about/
5

4.2.2. Survey
We present all survey questions in Appendix A. It
contains five groups of questions: (i) about the participant
profile (e.g., experience), (ii) technical questions about the
variability debt incurred because of clone-and-own prac-
tices, (iii) information about their perception of the causes,
(iv) perception of the consequences, (v) questions aiming
to explore the suitability of the variability debt metaphor
for their case and the prioritization of variability debt man-
agement activities that they envision interesting for their
scenario. The survey questions expect different types of
answers, namely open text boxes, predefined alternatives,
and five-point Likert scales.
Prior to answering the survey, we introduced the con-
cepts of variability and opportunistic reuse to the partici-
pants as follows:
•Variability : Variability is the ability of a software
system or software artifact to be extended, cus-
tomized, or configured for use and reuse in specific
contexts. Thus, a system variant is one of the system
versions created for a specific context.
•Opportunistic reuse: a common practice in devel-
opment companies is opportunistic (unsystematic)
reuse, a.k.a., copy-and-paste or clone-and-own. In
this practice, when there is a demand for a new prod-
uct variant or feature, parts or complete artifacts of
existing an existing product are copied/cloned and
modified/adapted to satisfy the requirements.
The definition of variability debt was only presented
during the survey when responding to the Definitions part
of the survey.
4.2.3. Tools
For basic analysis of the source code such as files
and packages structure/nomenclatures, we relied on the
Eclipse IDE.4For the comparison between variants of each
system, we used the framework Bottom-Up Technologies
for Reuse (BUT4Reuse) [17]. BUT4Reuse5provides sev-
eral tools for mining software artifact variants, allowing
overlap analysis (n-way comparison of variants for com-
monality and variability analysis), among many other fea-
tures for re-engineering variants that are out of the scope
of this work. For the quantitative and qualitative anal-
ysis of the survey with stakeholders, we adopted online
collaborative spreadsheets, namely Google Sheets.6
5. Results and discussions
The results and discussions are presented according to
the research questions presented in Section 4.
4https://www.eclipse.org/ide/
5https://but4reuse.github.io/
6https://www.google.com/sheets/about/
Table 2: Available artifacts existing for each system
Artifact SysX SysY SysZ
Orig Var1 Var2 Orig Var1 Orig Var1
Source code ✓ ✓ ✓ ✓ ✓ ✓ ✓
Requirements ✓ ✓ ✓ ✓ ✓ ✓
Specifications ✓ ✓ ✓ ✓ ✓ ✓ ✓
Validation Tests ✓ ✓ ✓ ✓ ✓
5.1. RQ1: Artifacts affected by variability debt
In order to respond to which and how artifacts are af-
fected by variability debt, we had to search for evidences of
opportunistic reuse and analyze its consequences for evo-
lution or maintenance. The results of artifact analysis of
our field study are presented firstly for the requirements,
specifications, and tests, and secondly for the source code.
5.1.1. Requirements, specifications, and tests
Table 2 presents the artifacts available in each variant.
For the Original variant of SysX there are no requirements.
The explanation for this is that this variant was developed
in 2010, when the IT team of the company was not required
to create and maintain a formal document of requirements.
Then, as part of the development of Variants 1 and 2, the
system analysts created the requirements.
As a further refinement of the high-level requirements,
aspecification in the case of our target company consists
in a detailed description of the use cases, business rules,
even fine-grained indications such as the fields that the
user will have in the forms of the system. These specifica-
tions are the main asset for guiding the implementation.
Artifacts of type specification are present in all variants of
the three systems. Table 3 presents the number of specifi-
cation documents for each variant. For SysX, the number
of specifications is similar to the number of features, as ex-
pected. However, for SysY and SysZ, the number of speci-
fications is lower than the number of features. In the anal-
ysis, we observed that this happens because only features
that would be validated according to existing regulation
had the specification created. For SysY, different variant
features would be validated for regulations, leading to dif-
ferent number of specifications. In the case of SysZ, the
same number of specifications was observed for Original
and Variant1, however, the specifications are not totally
similar. We closely analyzed the specifications, identifying
several changes. Thus, we found evidences of the variabil-
ity debt of duplicate documentation with modifications.
The validation tests are artifacts designed mainly to
comply with regulation of national agencies. They are doc-
Table 3: Number of specifications for each variant
SysX SysY SysZ
Orig Var1 Var2 Orig Var1 Orig Var1
14 10 11 26 17 7 7
6

Table 4: Technology stack
Technology SysX SysY SysZ
Platform Mobile Web Web
Application server Server JSE 1.6 EJB Java EE6 EJB Java EE7
Database Management Systems PostgreSQL 9.4 PostgreSQL 9.1 Oracle 11g
Frameworks JSE 1.3 Hibernate 3 and Primefaces 2.2.1 Hibernate 5 and Spring Data JPA
Graphical User Interface AWT JSF 2.1 JSF 2.2
Reporting Engine - JasperReport 4.5.1 JasperReport 6.5.1
Connection with the ERP RFC RFC RFC
Authentication User Database LDAP Active Directory
uments for performing system and acceptance test, con-
taining checkboxes to indicate that everything is function-
ing as expected. These tests were performed manually
by a team of around four persons, creating screenshots as
evidences. In this case, two variants, in SysX and SysZ,
did not have this artifact, since the branch in which these
variants operate does not require following any regulation.
This lack of testing in variants created using opportunistic
reuse, even if for validation purposes, is one of the variabil-
ity debt items described in the literature (see Section 2)
and observed in our field study. Besides these validation
tests, automatic unit tests are subject of opportunistic
reuse for the creation of variants in the field study. This
was confirmed in the survey results when asking about the
type of artifacts that were cloned (see Table 8 that will be
introduced later, where we can see the mention to unit test
cases). In the case of RQ1, we did not distinguish between
source code and unit tests (which are also source code)
when performing the analysis. Therefore, as we will see in
the next subsection, unit tests also have the code duplica-
tion and synchronization issues for multi-version support.
5.1.2. Technology stack and source code
We analyzed the technology stack used in the imple-
mentation of the systems, presented in Table 4. During our
analysis, we observed that the variants of the same system
use the same technology stack. This helps in the mainte-
nance and evolution of the system variants as a whole, so
apparently no issues were identified regarding this.
The next analysis relied on the source code of the vari-
ants. The three systems have Java-based source code. We
used BUT4Reuse that allows the analysis of source code
through reverse engineering techniques, including the com-
parison of several variants (n-way comparison). Table 5
shows some metrics obtained with the tool to provide an
overview of the size of the systems. SysY is the largest
one, but both SysX and SysZ are also complex systems of
relatively large size.
To provide evidences and measures of commonality
and variability within these large systems, we used the
variants’ comparison techniques of BUT4Reuse. Fig-
ure 1 shows the bars and stripes visualization [28] with
the results of the comparisons between variants for the
three systems. For this analysis, we used the Java JDT
(Eclipse Java development tools) source code adapter of
Table 5: Source code size of the variants (packages = Java Packages,
.java = Java files, methods = Java methods)
Sys. Var. #packages #.java #methods
SysX
Orig 4 91 1497
Var1 4 90 1517
Var2 5 91 1495
SysY Orig 82 605 6270
Var1 63 475 4971
SysZ Orig 23 128 2846
Var1 23 132 2838
BUT4Reuse. Thus, each bar is a variant, and the stripes
(blocks) with different colors are the results of the static
commonality and variability analysis. A shared block color
among variants means that these elements are shared. The
size of the blocks correspond to the number of Java ele-
ments (e.g., compilation units, types, methods, and fields).
Each of the five subfigures in Figure 1 are independent
comparison analyses. That means that the colors of the
blocks do not mean that they have any relation with the
other subfigures. The colors and numbers of the block are
determined according to the identified intersections of the
variants during the comparison of the elements. Table 6
will present later information about the identified blocks
for the most interesting figures, which are Figures 1b, 1c,
and 1e.
For SysX and SysZ in Figure 1, we show two
cases, namely “before preparatory refactorings” and “after
preparatory refactorings”. When we analyzed the source
code of the systems without any modification, we found
a lack of common source code between variants that were
supposed to be created through clone-and-own. As we
can observe in Figures 1a and 1d, the results incorrectly
suggested that the variants were unique or almost unique
(e.g., just one shared Java file in SysX variants). How-
ever, by the specifications and requirements, we knew that
these variants have most of their implementation similar
to each other. Then, we realized that after cloning the
variant Original, many packages and Java types were re-
named, making inconsistent the file structure between the
variants.
These non-propagated changes might already challenge
the future evolution of the system, as it complicates the
7

(a) SysX before preparatory refactorings (b) SysX after preparatory refactorings
(c) SysY, no preparatory refactor-
ings needed
(d) SysZ before preparatory refac-
torings
(e) SysZ after preparatory refactor-
ings
Figure 1: Differences in the comparison of variants when comparing the source code using the “Java JDT” adapter of BUT4Reuse. All
sub-figures are independent comparison analyses even if the used colors are the same. The figures “before” show the results as defined by the
developers, while the “after” were fixed by the first author of the paper to recover the needed consistency for the analysis.
manual propagation of bug fixes and the future consoli-
dation of variants [29, 30]. A characteristic found in the
three systems is to rename packages based on the indus-
try branch that was targeting the variant. Other changes
seemed more with the intention to improve the file struc-
ture quality. SysY, where the source code was consistent
between the two variants, shows that it is possible to cre-
ate a variant removing ten features (see Table 1) while
keeping the source code structure consistent. There will
be technical debt for propagating changes, but at least the
propagation will require less effort as the developers will
not need to reason in the refactorings.
To continue with our variability analysis, we needed
to perform what has been defined as preparatory refac-
torings for analyzing cloned variants [29]. This practice
has been already reported analyzing a real set of variants
called Apo-Games [29, 31]. After a manual analysis of
the source code, the first author of this paper refactored
the variants through several packages renaming to make
them consistent. Finally, after rerunning BUT4Reuse to
analyze the commonality and variability, we obtained the
result presented in Figures 1b and 1e. We can clearly see
how similar are the variants “after” the preparatory refac-
torings in comparison to the variants “before” the renam-
ing. The preparatory refactoring was needed for the au-
tomated static commonality and variability analysis tech-
nique to identify the resulting blocks. As mentioned be-
fore, all the subfigures in Figure 1 represent independent
analysis. This applies also for the “before” and “after”
analysis. For example, it must not be considered that the
Table 6: Source code size of the blocks (pack. = Java Packages, .java
= Java files, me. = unique implementations of Java methods). The
rows correspond to Figures 1b, 1c, and 1e.
Sys. Blocks Var. presence #pack. #.java #me.
SysX
Common Orig,Var1,Var2 3 23 208
Block 1 Orig,Var2 0 26 599
Block 2 Orig,Var1 1 40 520
Block 3 Orig 0 1 41
Block 4 Var1 0 25 596
Block 5 Var2 0 1 52
SysY
Common Orig,Var1 61 466 4622
Block 1 Orig 21 139 1648
Block 2 Var1 0 2 272
SysZ
Common Orig, ar1 23 126 2431
Block 1 Orig 0 2 415
Block 2 Var1 0 6 407
block with green color in Figure 1a (the block present only
in Variant 2) has relation with the same block color of Fig-
ure 1b.
Table 6 provides details on the distinguishable blocks
shown in Figures 1b, 1c, and 1e. As discussed above, mod-
ifications in the source code of the blocks corresponding to
more than one variant (i.e., Common, Block 1 and 2 for
SysX, and Common for SysY and SysZ) will suggest the
need for changes propagation. We can observe that these
are the larger blocks, except for SysX Bock 4 where Vari-
ant1 has significant source code specific for this variant,
namely 25 Java files and 596 unique implementations of
8

methods in these Java files or in other Java files from the
Common or Block 2. Notice that certain numbers between
Table 5 and Table 6 does not necessarily need to match.
For example, in SysX Variant2 we have five packages while
the total number of the SysX blocks is four. This is because
of the preparatory refactorings, as Table 5 represented the
version of the developers and one package was renamed.
Also, the number of methods in Table 6 does not repre-
sent the number of method declarations, but the number
of unique implementations of a method. That means that,
for example, the same method declaration can have one
implementation in one block and another one in another
(i.e., at least one source code statement inside the method
were different between variants).
Based on our catalog of variability debt items and the
commonality and variability evidences in the source code,
we can observe that code duplication is a prevalent issue.
Moreover, the fact that refactorings were performed in the
variants challenges the synchronization of the variants for
multi-version support.
Answering RQ1: Based on the analysis of how vari-
ability debt happens in practice, we observe that source
code is the artifact more cloned for implementing vari-
ability in an opportunistic reuse way. This created is-
sues like the replication of business rules, duplicate bugs
and the need to correct them. The complexity in mainte-
nance is aggravated because of non-synchronized refac-
torings. Also, specification documents are also replicated
for software variants.
5.2. RQ2: Survey with stakeholders to analyze their per-
ception of technical debt related to opportunistic reuse
This section describes the results and analysis of the
survey with stakeholders to investigate the phenomenon
of variability debt from the point of view of practitioners.
To answer RQ2, based on the survey in Appendix A,
we ask stakeholders about the context, causes, and con-
sequences, they faced about opportunistic reuse without
mentioning the variability debt concept. Then, after these
questions parts were answered, we introduced to them our
definition of variability debt as a way to name the phe-
nomenon. This last part aims to understand if the practi-
tioners agree with this category of technical debt, indepen-
dently whether they considered that they had it in their
systems or not. In the following analysis, we present the
results without making the distinction between variabil-
ity debt and the issues derived from opportunistic reuse
regarding evolution and maintenance of the family as a
whole.
During the study design, we identified 25 stakeholders
that could be part of the survey. We invited all of them,
and the survey was responded by 22 practitioners (88% of
participation rate). The following subsections present the
survey results of our field study.
Table 7: Survey participants and basic information
ID Role Prof. Exp. System
P1 Developer ≥10 SysY
P2 Developer ≥10 SysX
P3 Developer 5 to 10 SysX
P4 Business manager ≥10 SysY
P5 Help-desk analyst 5 to 10 SysY
P6 Test analyst ≥10 SysX
P7 System analyst ≥10 SysX
P8 Developer ≥10 SysZ
P9 System analyst ≥10 SysY
P10 End user ≥10 SysY
P11 Business manager ≥10 SysY
P12 Test analyst 5 to 10 SysX
P13 IT manager ≥10 SysX
P14 System analyst 5 to 10 SysY
P15 Developer ≥10 SysY
P16 Developer ≥10 SysX
P17 Business manager 5 to 10 SysZ
P18 Infrastructure analyst ≥10 SysY
P19 Developer 5 to 10 SysX
P20 IT manager ≥10 SysY
P21 Test analyst 5 to 10 SysZ
P22 System analyst 2 to 5 SysZ
Prof. Exp. = professional experience in years,
options: 0 to 2, 2 to 5, 5 to 10, ≥10
5.2.1. Characterization
Table 7 presents the survey participants. For the anal-
ysis, we identified the participants of the survey from P1 to
P22. From the data in the table, we did different analysis.
The pie-charts presented in Figures 2, 3, and 4, show that
most of the participants had more than 10 years of expe-
rience, the three systems are well represented with more
participants in SysY and SysX, and Developer is the most
represented role, but a diversity of roles is observed.
More than 10 years
14
From 5 to 10 years
7From 2 to 5 years
1
Figure 2: Years of professional experience
SysX
8
SysY
10
SysZ
4
Figure 3: Number of participants per system
As part of the survey, we asked the participants if the
use of opportunistic reuse in the system they know led to
9

Developer
7
System analyst
4
Test analyst
3
Business manager
3
IT manager
2
Infrastructure analyst
1Help-desk analyst
1End user
1
Figure 4: Participant’s roles
initial benefit to the business, but incurred in maintenance
and evolution problems afterward. Figure 5 presents the
percentage of answers according to the Likert scale. This
figure does not consider the answer of P10, an end user
that selected “not answer”. Most of the participants (12
= 57%) agreed or strongly agreed that there were problems
related to the maintenance and evolution in the systems
they worked on, which were created with opportunistic
reuse. Five participants disagree and only one strongly
disagree (summing 29%) of the existence of such problems
in the system. Finally, three participants do not agree or
disagree (neutral = 14%).
29% 57%14%
Existence of
variability debt
in the system
100 50 0 50 100
Percentage
Response Strongly Disagree Disagree Neutral Agree Strongly Agree
Figure 5: Opinion of practitioners regarding the existence of vari-
ability debt in the systems/variants they interact with.
For a deeper understanding of the practitioners’ opin-
ions, we analyzed the answers based on the role of the
participants. We found that: (i) 66.6% of the business
managers does not agree that the systems created with
opportunistic reuse had maintenance and evolution prob-
lems, whereas IT managers have split opinions; (ii) among
system and infrastructure analysts, 80% (strongly) agree
that the systems have technical debt related to variability;
(iii) the majority of developers (57%) also (strongly) agree
with the existence of maintenance and evolution problems
in the systems they worked on, whereas 28% disagree and
14% responded as neutral.
Based on the results, we can see a tendency that prac-
titioners in managerial roles not perceive variability debt.
On the other hand, practitioners in technical roles, mostly
agreed that the opportunistic reuse practices led the sys-
tems to incur in technical debt related to variability. How-
ever, we can observe some exceptions. For example, a
business manager (P17) commented about the Variant1 of
SysZ:
‘The system [Variant1] did not reflect our real-
ity, the non-conformity classifications were not
in accordance to our needs.’
Interestingly, this is related to the external quality of the
systems, which is not directly related to technical debt.
However, the quality of the variant created with oppor-
tunistic reuse was not satisfactory, and the manager could
see issues. Variability debt related to system-level struc-
ture quality issues,lack of testing, or multi-version support
might be the source of problems for this external quality
of the variant. This finding is supported by the answers
from developers. P1 reported:
‘Some bugs for automatic job execution were
fixed on the main system [SysY, Original].
However, they were not fixed in the other sys-
tem [SysY, Variant1], which was a copy. Also
some improvements in the document reprint
rules applied during the evolution of the main
system [SysY, Original] were not reproduced in
the other system Y [SysY, Variant1].’
Similarly, P15 (for SysY) and P16 (for SysX) also re-
ported that they had to change features of variants created
with opportunistic reuse because they had been copied
the whole original variants during the reuse. However,
changed were needed in Variant1 because they did not
fully fit the context of the system (variant Original). This
statement relates to multi-version support. Additionally,
P3 provided further comments about documentation:
‘In some parts of the documentation of simi-
lar features [of SysX], some small details that
were not changed in the copy [Variant1] com-
promised the result.’
This is the occurrence of the variability debt of duplicate
documentation. The developer P2 described a different
situation:
‘the copied parts [from SysX Original] (com-
munication structures and integration with
legacy system) were quite stable and had little
structural difference.’
This was the justification of the developer for answering
that opportunistic reuse in SysX did not lead to technical
debt. But as we can see, this developer considered a very
specific point, perhaps not taking part in issues raised by
the other developers.
Answering RQ2 - Characterization: Based on the
survey responses, we can observe that managerial prac-
titioners usually do not perceive variability debt, where
technical practitioners clearly agree with the variability
debt existence. Also, in the projects analyzed, we could
see instances of variability debt described in the litera-
ture, namely system-level structure quality issues, lack of
testing, multi-version support, and duplicate documenta-
tion.
10

5.2.2. Artifacts, Causes, and Consequences
To investigate how variability debt incurs in artifacts,
we inquired which types of artifacts were part of the oppor-
tunistic reuse when creating the variants. Table 8 presents
the list of artifacts and the number of mentions from dif-
ferent participants, sort by the latter.
We observe a large diversity of mentioned artifacts. As
the field study took place in an industry that must fol-
low regulations, several artifacts of the software develop-
ment process are mandatory. However, some variants (e.g.,
SysX - Variant1 and SysZ - Variant1) were deployed in
unregulated group branches, which reduced the concerns
of reusing some artifacts. We can see that the company
pays important attention to requirements that describes
customer needs, as requirements is the artifact type men-
tioned the most. Requirement was mentioned by all practi-
tioners from all roles. As expected, 71% of the mentions to
source code artifact came from software developers. The
artifact specifications had a considerable division, being
50% indicated by developers, 25% by system analysts and
the rest by the infrastructure analyst and the test analyst.
Some artifacts are only mentioned once or few times. How-
ever, identifying them through the survey was valuable to
understand the diversity of artifacts.
Considering that most of the practitioners pointed the
existence of maintenance and evolution problems because
of opportunistic reuse, we asked about the causes that lead
the practitioners to incur in such a type of debt. The basis
for the predefined options of answer were the findings of
our previous study [15] (see Section 2). Using the Likert
scale, the participant could answer about each cause.
Figure 6 presents a summary of the answers. Time
pressure is convincingly pointed as one of the main causes
for variability debt, in which all participants agree (only P3
chose “not answer”). This highlights the phenomenon of
technical debt, because when delivering in a short period
of time we have some benefits, reduced time-to-market,
or for regulatory reasons, satisfy customer pressure; but
ends up generating a mid- long-term loss due to lack of
planning, poor quality, and maintenance difficulties. Some
comments from the stakeholders about time pressure are
Table 8: Artifacts mentioned in the survey
Artifact # mentions
Requirements 13
Specifications 8
Source code 7
Technical Manuals 4
Screen prototypes 4
Docummented architecture 3
UML diagrams 3
Entity-relationship diagram 2
Unit test cases 1
Widget test cases 1
DB schemas 1
0%
55%
33%
32%
100%
41%
33%
27%
0%
5%
33%
41%
Time pressure
Lack of knowledge
on variability
management
Constantly
changing
scenarios
Operational
constraints
100 50 0 50 100
Percentage
Response Strongly Disagree Disagree Neutral Agree Strongly Agree
Figure 6: Evaluation of the participants about variability debt causes
presented in what follows:
‘P2: Time pressure and the use of the same
communication architecture were reasons for
using the solution that was already working.’
‘P4: When the development time is not
enough, you end up choosing to use ready-made
or already known functionalities.’
‘P6: The time to execute the project was
strictly short, and it was not possible to imple-
ment a new tool because it is a process with par-
ticularities and with many changes. The com-
pany and the process is extremely dynamic.’
‘P9: Due to lack of time, it is always faster to
copy and paste, and only make changes where
necessary.’
Other nine participants also reported their experience with
the projects and the time pressure they had faced to build
and deploy the variants.
For the other causes, namely Constantly changing sce-
narios, Operational constraints, and Lack of knowledge on
variability management, we can observe, generally, a dis-
agreement or neutral position.
The survey results of the consequences are presented
in Figure 7. The consequence list were also obtained from
our previous study [15]. We can observe that creating
usability problems and complex maintenance of multiple
independent variants are the ones with more agreement.
This complex maintenance is comprehensible, as observed
in our results of the artifacts’ analysis.
Examples of comments about usability problems are:
‘P12: Users do not understand the usability of
the system and tend to be resistant to chang-
ing the process to use the system without cus-
tomization.’
‘P21: There can be usability problems when the
same user executes processes within two cloned
systems, which can be different on each sys-
tem.’
11

23%
32%
25%
33%
18%
41%
43%
57%
55%
55%
50%
43%
41%
41%
33%
10%
23%
14%
25%
24%
41%
18%
24%
33%
Complex
maintenance of
multiple
independent
variants
Inability to
systematically
deal with
customization
Inability to
create complex
systems
Poor overall
internal quality
Complex test
cycles
Creating
usability
problems
Portfolio
management
problems
Repeated roles in
similar projects
100 50 0 50 100
Percentage
Response Strongly Disagree Disagree Neutral Agree Strongly Agree
Figure 7: Consequences of variability debt
And examples of participants discussing the complex
maintenance and inability to systematically deal with cus-
tomization.
‘P17: Any change that was needed was ex-
tremely time consuming.’
‘P20: Problems generated by the conflict of dif-
ferent business rules for each operation, were
causing unplanned downtime and greater com-
plexity in test cycles.’
‘P21: Similar improvements might be neces-
sary in cloned systems, but this can be diffi-
cult due to the specific customizations of each
project.’
Answering RQ2 - Artifacts, Causes, and Conse-
quences: Based on the characteristics collected in the
literature on variability debt (definition, causes, artifacts,
and consequences) practitioners with different roles were
asked about their opinion regarding these characteristics.
In general, most agree with what was found in the litera-
ture and added the causes of “financial constraints” and
“lack of adequate software development process”. Addi-
tionally, they mentioned the consequences of “increased
business risk for the client”, “increased rework” and “pro-
cess improvements are lost”.
5.2.3. Definition and Perspectives
In the last part of the survey, we presented our defini-
tion of variability debt and asked the participants about
the clarity of such definition. Concretely, to which extent
do they agree that the variability debt definition is clearly
described in relation to its usage context and purpose for
0% 91%9%
Definition of
variability debt
100 50 0 50 100
Percentage
Response Strongly Disagree Disagree Neutral Agree Strongly Agree
Figure 8: Clarity of the variability debt definition and concept
creating awareness in the company. The response is shown
in Figure 8 evidencing an agreement.
One participant made an interesting observation:
‘P1: I think that the term technical debt is
enough. Technical debt occurs for several rea-
sons, and variability is one dimension among
them.’
We agree with P1. As already mentioned, the research line
of variability debt is to investigate the role of variability
as source of technical debt. In this work we focus on op-
portunistic reuse as source of technical debt, but technical
debt can also be incurred in a software product line, as
also pointed out by another participant:
‘P15: I agree that initially clone-and-own
proves to be simpler, but as time goes by, main-
taining these architectures becomes much more
complex. I also understand that maintaining
highly configurable software is extremely com-
plex and that if it is poorly applied, it can
cause even greater maintenance or impair sys-
tem performance, mainly due to the large oc-
currence of requirements changes during its life
cycle.’
Then we asked participants about perspectives with regard
to variability debt management. Notably, to understand
to which extent do they agree that supporting certain ac-
tivities is important. We ask for the five key activities in
technical debt management as reported in Li et al. [5, 6].
Following Li et al. definitions of the activities, adapted
to variability debt, they are: (i) Identification: Detecting
technical debt items associated with variability, (ii) Mea-
surement: Estimating the cost and benefit associated with
a variability debt item, (iii) Prioritization: Identify which
variability debt item should be addressed first, (iv) Repay-
ment: Changes and transformation helpers to eliminate
the negative effect of a variability debt item, and (v) Mon-
itoring: Check the changes of the costs and benefits of
unresolved variability debt items over time. Despite their
importance, we did not include, technical debt prevention,
representation/documentation, and communication which
were later added in another work of Li et al. [6]. We
excluded them to shorten the survey to the key ones as
included in Li et al. [5] and reduce the participant fatigue.
The result is presented in Figure 9, showing a general
agreement of the importance of all activities. Full agree-
12

ment corresponds to the Identification activity, followed
by Prioritization and Measurement.
0%
0%
0%
5%
5%
100%
91%
90%
85%
67%
0%
9%
10%
10%
29%
Identification
Measurement
Prioritization
Repayment
Monitoring
100 50 0 50 100
Percentage
Response Strongly Disagree Disagree Neutral Agree Strongly Agree
Figure 9: Perspectives for the activities of variability debt manage-
ment
One participant emphasized the quantitative measure-
ment of the debt as a relevant input for the business. This
will also include its identification.
‘P20,: The biggest challenge is to quantify this
Debt for the business before developing the so-
lutions, as the short-term need ends up taking
the “easy” path.’
Answering RQ2 - Definition and Perspectives:
Variability debt is perceived as a valuable metaphor to
create awareness within the company about the concerns
of opportunistic reuse in the creation of variants. Meth-
ods and functionalities will be desired for variability debt
management activities.
6. Threats to validity
As mentioned by Stol and Fitzgerald [16], known prob-
lems of field studies are the lack of statistical generalizabil-
ity, no control over events, and the low precision of mea-
surement. Every company will have their own scenario
and characteristics, but the insights can be generalized
to companies with similar settings. Therefore, applying
this study to a company and three projects might not be
enough to generalize the answers of our RQs. However,
considering that we are dealing with an industrial field
study, we are focusing on the scope of this company and
its settings.
For the preparatory refactorings needed for the source
code analysis, the systems are complex and of significant
size, thus, it is possible that this effort was not complete
(e.g., we did not investigate renaming at method level).
However, we consider that for the purpose of showing ev-
idence and size of commonality and variability, the refac-
torings identified and reverted for the variants’ comparison
purpose were already valid. For an actual re-engineering
of the variants to a software product line, this effort will
require much more fine-grained analysis, but this was out
of the scope of the study.
There are also threats to the validity regarding the sur-
vey. The total number of participants can be considered a
small number compared with other more general surveys
in software engineering research. Being a field study, as
mentioned, we tried to cover most of the involved stake-
holders, and we consider that we succeed in covering the
diversity of involved roles, namely, developers, system an-
alysts, test analysts, business managers, IT managers, in-
frastructure analyst, help-desk analyst, and even one end
user. In fact, the majority of participants (except for the
end-user) represent stakeholders with knowledge on the in-
ternal and the development of the targeted systems, so for
a field study, our sample of participants can be considered
as representative. However, there are other known intrin-
sic threats to validity regarding surveys, as for example,
user fatigue while responding, or the interpretation of Lik-
ert scales. We mitigated this issue by limiting the amount
of questions to an acceptable number such that it did not
take longer than half-an-hour, and by interacting among
the authors to define clear questions.
7. Conclusions and future work
The focus of the present work is to analyze and un-
derstand variability debt in a real scenario of a company
that uses opportunistic reuse. To do so, we performed
a field study in a Brazilian multinational industry leader
in its market segment in Latin America. Our field study
relied on three subject systems and their seven variants.
Our study also included a survey with 22 stakeholders of
the three systems to observe their perception of variability
debt.
We can synthesize our findings as follows: (i) the most
cloned artifact for implementing variability in an oppor-
tunistic reuse way is the source code, which creates fu-
ture issues like the replication of business rules, duplicate
bugs, and the need to correct them independently; (ii) dif-
ferent from the managerial practitioners, technical prac-
titioners clearly perceived variability debt; (iii) we could
see instances of variability debt described in the literature,
namely system-level structure quality issues, lack of test-
ing, multi-version support, and duplicate documentation;
(iv) the participants of the survey confirmed the causes and
consequences of variability debt of our previous study [15]
and added few ones; and (v) the variability debt metaphor
was considered useful to highlight the concerns of oppor-
tunistic reuse in the creation of variants.
Our findings contribute to bringing the attention of
practitioners to problems that can be generated by using
opportunistic reuse practices for software reuse and vari-
ability management. Furthermore, the results of our field
study generate opportunities for new studies and tooling
for variability debt management in the opportunistic reuse
scenario including more advanced visualization support for
variants evolution [32] that could improve, for example,
13

the automation and quality of the analyses described in
Section 5.1.
For further work, we consider that more work and com-
munity consensus is needed on the understanding of the
variability debt phenomenon. Notably, as confirmed in
the TD4ViS workshop mentioned in Section 3, we desire
to have a better understanding in other scenarios not re-
lated to opportunistic reuse. For instance, limitations in
the design of a software product line architecture. We can
consider that defects in (or the absence of) software prod-
uct line engineering processes can create maintenance or
evolution issues of the product family as a whole. This
way, variability debt can be investigated in the mentioned
case of opportunistic reuse, but also, for instance in the
case of implementing variability management with fea-
ture toggles, creating variants with component-based soft-
ware engineering, feature-based SPL with an annotative
approach, feature selection for components composition,
feature-based manual creation of the variants (feature se-
lection is available, but there is no derivation mechanism
and the variant need to be composed almost in a manual
way). Also, we consider that field studies, similar to the
one present in this work, are a good direction to achieve
this goal. This way, research works can investigate the
subject from a bottom-up perspective (as the presented
field study), while others, more oriented to a theoretical
and conceptual framework can, in parallel, investigate it
in a top-down approach.
Finally, besides the conceptual characterization of vari-
ability debt, concrete tools for variability debt man-
agement are to be designed, implemented, and evalu-
ated. This way, identification, measurement, prioriti-
zation, repayment, monitoring, prevention, representa-
tion/documentation, and communication of variability
debt can be possible. Some already existing tools can be
integrated in the conceptual framework of variability debt
management, however, we consider that new ones are to
be developed leveraging the conceptual framework and the
advances in the general technical debt management field.
Acknowledgment
The research reported in this paper has been funded
by CNPq Grant No. 428994/2018-0; FAPERJ PDR-10
program under Grant No. 202073/2020; BMK, BMDW,
and the State of Upper Austria in the frame of SCCH,
part of the COMET Programme managed by FFG; the
LIT Secure and Correct System Lab funded by the State
of Upper Austria; and the Austrian Science Fund (FWF),
grand no. P31989.
References
[1] W. Cunningham, The wycash portfolio management system,
ACM SIGPLAN OOPS Messenger 4 (2) (1992) 29–30. doi:
10.1145/157709.157715.
[2] P. Kruchten, R. L. Nord, I. Ozkaya, Technical debt: From
metaphor to theory and practice, IEEE software 29 (6) (2012)
18–21. doi:10.1109/MS.2012.167.
[3] N. Rios, M. G. de Mendon¸ca Neto, R. O. Sp´ınola, A tertiary
study on technical debt: Types, management strategies, re-
search trends, and base information for practitioners, Infor-
mation and Software Technology 102 (2018) 117–145. doi:
10.1016/j.infsof.2018.05.010.
[4] H. J. Junior, G. H. Travassos, Consolidating a common per-
spective on Technical Debt and its management through a
tertiary study, Inf. Softw. Technol. 149 (2022) 106964. doi:
10.1016/j.infsof.2022.106964.
[5] Z. Li, P. Liang, P. Avgeriou, Architectural debt management
in value-oriented architecting, in: Economics-Driven Software
Architecture, Morgan Kaufmann / Academic Press / Else-
vier, 2013, pp. 183–204. doi:10.1016/b978- 0- 12-410464- 8.
00009-x.
[6] Z. Li, P. Avgeriou, P. Liang, A systematic mapping study on
technical debt and its management, Journal of Systems and
Software 101 (2015) 193–220. doi:10.1016/j.jss.2014.12.
027.
[7] P. Avgeriou, P. Kruchten, I. Ozkaya, C. Seaman, Manag-
ing Technical Debt in Software Engineering (Dagstuhl Sem-
inar 16162), Dagstuhl Reports 6 (4) (2016) 110–138. doi:
10.4230/DagRep.6.4.110.
[8] S. Fischer, L. Linsbauer, R. E. Lopez-Herrejon, A. Egyed, En-
hancing clone-and-own with systematic reuse for developing
software variants, in: IEEE International Conference on Soft-
ware Maintenance and Evolution, 2014, pp. 391–400. doi:
10.1109/ICSME.2014.61.
[9] W. K. Assun¸c˜ao, R. E. Lopez-Herrejon, L. Linsbauer, S. R.
Vergilio, A. Egyed, Reengineering legacy applications into soft-
ware product lines: a systematic mapping, Empirical Soft-
ware Engineering 22 (6) (2017) 2972–3016. doi:10.1007/
s10664-017- 9499-z.
[10] J. Echeverr´ıa, F. P´erez, J. I. Panach, C. Cetina, An empirical
study of performance using clone & own and software prod-
uct lines in an industrial context, Information and Software
Technology 130 (2021) 106444. doi:10.1016/j.infsof.2020.
106444.
[11] J. Martinez, W. K. G. Assun¸c˜ao, T. Ziadi, ESPLA: A catalog
of extractive SPL adoption case studies, in: 21st International
Systems and Software Product Line Conference (SPLC), ACM,
2017, pp. 38–41. doi:10.1145/3109729.3109748.
[12] R. Capilla, J. Bosch, K. Kang, Systems and Software Variability
Management: Concepts, Tools and Experiences, Springer, 2013.
doi:10.1007/978-3- 642-36583- 6.
[13] F. van der Linden, K. Schmid, E. Rommes, Software product
lines in action - the best industrial practice in product line en-
gineering, Springer, 2007. doi:10.1007/978- 3-540-71437- 8.
[14] K. Pohl, G. B¨ockle, F. van der Linden, Software Product
Line Engineering - Foundations, Principles, and Techniques,
Springer, 2005. doi:10.1007/3- 540-28901-1.
[15] D. Wolfart, W. K. G. Assun¸c˜ao, J. Martinez, Variability debt:
Characterization, causes and consequences, in: SBQS 2021:
20th Brazilian Software Quality Symposium, 2021, pp. 1–10.
doi:10.1145/3493244.3493250.
[16] K.-J. Stol, B. Fitzgerald, The ABC of Software Engineering Re-
search, ACM Trans. Softw. Eng. Methodol. 27 (3) (Sep. 2018).
doi:10.1145/3241743.
[17] J. Martinez, T. Ziadi, T. F. Bissyand´e, J. Klein, Y. L.
Traon, Bottom-up technologies for reuse: A framework to
support extractive software product line adoption activities,
in: R. E. Lopez-Herrejon, J. Martinez, W. K. G. Assun¸c˜ao,
T. Ziadi, M. Acher, S. R. Vergilio (Eds.), Handbook of Re-
Engineering Software Intensive Systems into Software Product
Lines, Springer International Publishing, 2023, pp. 355–377.
doi:10.1007/978-3- 031-11686- 5\_14.
[18] C. A. Siebra, G. S. Tonin, F. Q. B. da Silva, R. G. Oliveira, L. C.
Antonio, R. C. G. Miranda, A. L. M. Santos, Managing tech-
nical debt in practice: An industrial report, in: Proceedings
14

of the 2012 ACM-IEEE International Symposium on Empiri-
cal Software Engineering and Measurement, 2012, pp. 247–250.
doi:10.1145/2372251.2372297.
[19] L. Ribeiro, M. Farias, M. Mendon¸ca, R. Sp´ınola, Decision cri-
teria for the payment of technical debt in software projects: A
systematic mapping study, in: Proceedings of the 18th Interna-
tional Conference on Enterprise Information Systems - Volume
1: ICEIS, 2016, pp. 572–579. doi:10.5220/0005914605720579.
[20] E.-M. Arvanitou, A. Ampatzoglou, S. Bibi, A. Chatzigeorgiou,
I. Stamelos, Monitoring technical debt in an industrial setting,
in: Proceedings of the Evaluation and Assessment on Software
Engineering, EASE ’19, Association for Computing Machinery,
New York, NY, USA, 2019, p. 123–132. doi:10.1145/3319008.
3319019.
[21] V. Lenarduzzi, T. Besker, D. Taibi, A. Martini, F. Arcelli
Fontana, A systematic literature review on technical debt pri-
oritization: Strategies, processes, factors, and tools, Journal of
Systems and Software 171 (2021) 110827. doi:10.1016/j.jss.
2020.110827.
[22] W. Fenske, S. Schulze, Code smells revisited: A variability
perspective, in: Proceedings of the Ninth International Work-
shop on Variability Modelling of Software-Intensive Systems,
VaMoS ’15, Association for Computing Machinery, New York,
NY, USA, 2015, p. 3–10. doi:10.1145/2701319.2701321.
[23] G. Digkas, N. Nikolaidis, A. Ampatzoglou, A. Chatzigeorgiou,
Reusing code from stackoverflow: The effect on technical debt,
in: 2019 45th Euromicro Conference on Software Engineering
and Advanced Applications (SEAA), 2019, pp. 87–91. doi:
10.1109/SEAA.2019.00022.
[24] R. Verdecchia, P. Kruchten, P. Lago, I. Malavolta, Building and
evaluating a theory of architectural technical debt in software-
intensive systems, Journal of Systems and Software 176 (2021)
110925. doi:10.1016/j.jss.2021.110925.
[25] R. Capilla, T. Mikkonen, C. Carrillo, F. A. Fontana, I. Pigazz-
ini, V. Lenarduzzi, Impact of opportunistic reuse practices
to technical debt, in: 2021 IEEE/ACM International Confer-
ence on Technical Debt (TechDebt), 2021, pp. 16–25. doi:
10.1109/TechDebt52882.2021.00011.
[26] J. Martinez, W. K. G. Assun¸c˜ao, D. Wolfart, K. Schmid, A. Am-
patzoglou, TD4ViS 2022: 1st International Workshop on Tech-
nical Debt for Variability-Intensive Systems, in: 26th ACM
International Systems and Software Product Line Conference
(SPLC), ACM, 2022, p. 267. doi:10.1145/3546932.3547017.
[27] T. Lethbridge, S. Sim, J. Singer, Studying software engineers:
Data collection techniques for software field studies, Empir-
ical Software Engineering 10 (2005) 311–341. doi:10.1007/
s10664-005- 1290-x.
[28] J. Martinez, T. Ziadi, J. Klein, Y. L. Traon, Identifying and
visualising commonality and variability in model variants, in:
10th European Conference on Modelling Foundations and Ap-
plications (ECMFA@STAF), Vol. 8569 of Lecture Notes in
Computer Science, Springer, 2014, pp. 117–131. doi:10.1007/
978-3- 319-09195- 2\_8.
[29] W. Fenske, J. Meinicke, S. Schulze, S. Schulze, G. Saake,
Variant-preserving refactorings for migrating cloned products to
a product line, in: IEEE 24th International Conference on Soft-
ware Analysis, Evolution and Reengineering (SANER), IEEE
Computer Society, 2017, pp. 316–326. doi:10.1109/SANER.
2017.7884632.
[30] A. Benmerzoug, L. Yessad, T. Ziadi, Analyzing the impact of
refactoring variants on feature location, in: 19th International
Conference on Software and Systems Reuse (ICSR), Vol. 12541
of Lecture Notes in Computer Science, Springer, 2020, pp. 279–
291. doi:10.1007/978-3- 030-64694- 3\_17.
[31] C. Lima, W. K. G. Assun¸c˜ao, J. Martinez, W. Mendon¸ca,
I. do Carmo Machado, C. von Flach G. Chavez, Product line
architecture recovery with outlier filtering in software families:
the apo-games case study, J. Braz. Comput. Soc. 25 (1) (2019)
7:1–7:17. doi:10.1186/s13173-019- 0088-4.
[32] R. Medeiros, J. Martinez, O. D´ıaz, J. Falleri, Visualizations for
the evolution of variant-rich systems: A systematic mapping
study, Inf. Softw. Technol. 154 (2023) 107084. doi:10.1016/j.
infsof.2022.107084.
URL https://doi.org/10.1016/j.infsof.2022.107084
Appendix A. Questionnaire
Table A.9 presents the complete information about the
questionnaire used for the survey (see Section 5.2).
15

Table A.9: Question of the survey with the stakeholders
Group ID Question Mand. Type
1 Characterization
1 What is your name? No Open
2 How long is your professional experience? Options (in years): <2, 2-5, 5-10, >10 Yes Alt
3 Indicate which system you will answer the questions. Options: SysX, Sysy, SysZ Yes Alt
All the questions below are related to the system indicated above
4 How long have you been involved with the system? Options (in months): <6, 6-12, 12-24, >24 Yes Alt
5 What was your role in the system or department when you participated in the development of the system?
(If you had more than one function, please indicate)
Yes Open
6 How much experience did you have in this role when dealing with the system? Options (in months): <6,
6-24, 24-48, >48
Yes Alt
2 Context
7 Did the use of copy and paste practice in creating different variants of this system have an initial benefit,
but did it incur maintenance and evolution problems of the variants (systems) later?
Yes Likert
8 If you agreed with the above question, could you report any issues you faced? No Open
9 For the system variants that you came across, what artifacts did you deal with during the software devel-
opment and system deployment process? (e.g. requirements, design templates, manuals, source code, etc.)
No Open
3 Causes
10 To what extent do you agree that the following factors are reasons for using an unsystematic practice (e.g.,
copying and pasting) to create different software variants?
- Time Pressure Yes Likert
- Lack of knowledge Yes Likert
- Constant changes in requirements Yes Likert
- Operational constraints Yes Likert
11 For the reasons you agree with above, could you provide more information by citing examples, or describing
how this process took place?
No Open
12 Are there additional reasons that justified the choice of a non-systematic practice in creating the variants? No Open
4 Consequences
13 Are the following problems consequences of using a non-systematic approach to managing product family
variability?
- Complex maintenance Yes Likert
- Inability to systematically deal with customization Yes Likert
- Inability to create complex systems Yes Likert
- Poor internal software quality Yes Likert
- Complex test cycles Yes Likert
- Usability issues Yes Likert
- Portfolio management issues Yes Likert
- Repeated roles in similar projects Yes Likert
14 For the consequences you agree with above, could you provide more information by citing examples, or
describing how this impact occurred?
No Open
15 Can you indicate other consequences in the system due to the use of non-systematic practice (copy-paste
use)
No Open
5 Definitions
16 A new term was created during the development of this work. ”Variability Debt represents Technical Debt
caused by suboptimal problems and solutions in implementing variability management in software systems.
That is, companies may choose to implement variability with an approach that would bring short-term
benefits, for example, using opportunistic reuse, regardless of medium to long-term technical/architectural
disadvantages. The debt of variability leads to maintenance and evolution difficulties to manage families
of systems or highly configurable systems.”
- To what extent do you agree that the metaphor is clearly described with respect to its context of use and
purpose for creating awareness in companies?;
Yes Likert
17 Do you have any suggestions/comments on the definition of the term Variability Debt? No Open
18 To what extent do you agree that supporting the following activities is important?
- Identification: Detection of technical debt items associated with variability Yes Likert
- Measurement: estimating the cost and benefit associated with a variability debt item Yes Likert
- Prioritization: Identify which variability debt item should be addressed first Yes Likert
- Repayment: Changes and transformations help eliminate the negative effect of a debt item of variability Yes Likert
- Monitoring: Watch for changes in the costs and benefits of unresolved variability debt items over time Yes Likert
Mand.: Mandatory = Yes/No. Type: Open = open-ended question, Alt = alternative selection, Likert = five points Likert scale
16