Trade-off-Oriented Development: Making Quality Attribute Trade-offs First-Class

Abstract

Implementing a solution for a design decision that precisely satisfies the trade-off between quality attributes can be extremely challenging. Further, typically trade-offs are not represented as first-class entities in development artifacts. Hence, decisions might be suboptimal and lack requirements trace-ability as well as changeability. We propose Trade-off-oriented Development (ToD), a new concept to automate the selection and integration of reusable implementations for a given design decision based on quality attribute trade-offs. Implementations that vary in quality attributes and that solve reoccurring design decisions are collected in a design decision library. Developers declaratively specify the quality attribute trade-off, which is then used to automatically select the best fitting implementation. We argue that thereby, software satisfies the trade-offs more precisely , requirements are traceable and changeable, and advances in implementations automatically improve existing software.

Full text

Trade-off-Oriented Development:
Making Quality Attribute Trade-offs First-Class
Tobias D¨
urschmid
Institute for Software Research
Carnegie Mellon University
duerschmid@cmu.edu
Eunsuk Kang
Institute for Software Research
Carnegie Mellon University
eskang@cmu.edu
David Garlan
Institute for Software Research
Carnegie Mellon University
garlan@cs.cmu.edu
Abstract—Implementing a solution for a design decision that
precisely satisfies the trade-off between quality attributes can
be extremely challenging. Further, typically trade-offs are not
represented as first-class entities in development artifacts. Hence,
decisions might be suboptimal and lack requirements trace-
ability as well as changeability. We propose Trade-off-oriented
Development (ToD), a new concept to automate the selection
and integration of reusable implementations for a given design
decision based on quality attribute trade-offs. Implementations
that vary in quality attributes and that solve reoccurring design
decisions are collected in a design decision library. Developers
declaratively specify the quality attribute trade-off, which is then
used to automatically select the best fitting implementation. We
argue that thereby, software satisfies the trade-offs more pre-
cisely, requirements are traceable and changeable, and advances
in implementations automatically improve existing software.
Index Terms—Design decision, software architecture, quality
attribute, non-functional property, reuse, software design
I. INTRODUCTION
Software developers have to make trade-offs between qual-
ity attributes, such as security, performance, power consump-
tion, reliability, and availability that are implicit in the imple-
mented code [1]. For example, there are many considerations
in implementing a pipe [2, pp. 53–70] to connect two com-
ponents: Should the data stream be encrypted and signed to
ensure confidentiality while decreasing performance, and if
yes, which algorithms to use? Should a ping / echo tactic or
heartbeat tactic [1, pp. 87–89] be used to ensure availability
of both components while reducing the bandwidth, and if
yes, how often should the ping or heartbeat occur, and which
implementation should be used? Should the data stream be
compressed to increase the bandwidth while adding a small
performance overhead at both components?
To make optimal design decisions, developers have to be
experts in all of these domains. However, nowadays, soft-
ware is often developed by non-domain-experts. Studies in
the security domain have shown that developers often lack
knowledge, do not know which library to use, or how to use
APIs [3], [4]. Therefore, they demand higher-level abstractions
regarding security [3]. Furthermore, since these trade-offs
have to be made at various places of the software, quality
attributes are scattered over the code [5]. Additionally, design
decisions that involve quality attribute trade-offs are typically
implicit and not directly observable in the code. So it is
often difficult to trace non-functional requirements and quality
attribute trade-offs to implementations or adjust the software
if those requirements change, which often happens during
software development [5], [6]. Cross-module changes, for
example, are especially error-prone [7].
To address these problems, we propose a new development
concept called ToD that makes trade-offs first class entities of
programming by making them explicitly part of the code and
design. ToD reuses all existing implementations to a reoccur-
ring design decision in a design decision library, lets the de-
veloper declaratively specify quality attributes, optionally with
a priority, for each design decision, and automatically selects
the best-suited implementation from those in the library.
We argue that this concept has three main advantages: (1)
developers need less domain knowledge to create software
with the demanded quality attributes, because ToD simplifies
the reuse of expert solutions; (2) the resulting system more
easily responds to changing requirements, because develop-
ers need only to adjust the declaratively specified quality
attributes; and (3) if a new implementation that better suits the
defined requirements is added to the design decision library,
the software automatically uses the new implementation and,
therefore, more precisely satisfies the requirements.
II. RE LATED WORK
Feature-Oriented Programming [8] is an approach for flexi-
ble composition of objects and systems from a set of features.
It provides variability in the functional dimension of the
requirements by activating and deactivating features. While
quality attributes can be affected by the activation of a feature,
the focus is on changing the functionality. In contrast, the
proposed concept provides variability in the non-functional
dimension while keeping the functionality fixed. By changing
the implementation of each feature, the intent is to specifically
control the quality attributes of the system.
The modeling technique Concern-Oriented Software De-
sign [9] supports developers in selecting a reusable imple-
mentation by offering model-driven tools for exploring a hier-
archical design space. Similarly, End-user architecting [10]
is a concept that enables non-professional programmers to
compose functionality to create programs. In these approaches,
developers explicitly make each design decision by select-
ing one solution for each variation point without having to

Banking Application
q[] : Quality Attributes
Design Decision Library
1: findSatisfying(c, q[])
2: i
3: init(sink, source)
sink : Component
source : Component
: Pipe Contract
i: DeflatedStream
PipeImplementation : CheckedStream
PipeImplementation
: Control
Message
Dependency
Generalization
c : Application Context
: Implementation Selector <Pipe>
: CipherStream
PipeImplementation
Fig. 1. Overview of ToD applied to the pipe example. The colors of the concepts are used in all following figures. Quality attributes are green, contracts are
yellow, and implementations are blue. Other ToD concepts are red, while classes belonging to the example are white. The control needs a pipe implementation.
The control forwards the quality attribute trade-off and the application context to the implementation selector (findSatisfying), which determines the best
matching implementation. Afterwards the control initializes (init) the returned pipe implementation to connect the source component with the sink component.
know the details of the low-level implementation. This guides
developers in creating high quality software. However, the
software does not express the trade-offs behind each decision.
Therefore, it lacks requirements traceability. Furthermore,
since the developers select one implementation, they can build
upon implicit assumptions. Hence, later it is hard to change a
decision if these requirements change. Our proposed approach
targets these challenges by explicitly stating the required
quality attributes while keeping the decision transparent.
TradeMaker [11] is an approach that automatically analyses
trade-off spaces of design decisions and synthesizes an imple-
mentation from an abstract specification of object-relational
mappers. It is focused on one domain and its performance-
related quality attributes. So ToD generalizes this approach.
Further, it builds on reuse rather than program synthesis.
III. TRA DE -OFF -ORIENTED DEVELO PM EN T
This section introduces the generic concepts of the ToD
approach, as visualized in Figure 1. As an example for ToD,
suppose developer Alice wants to implement a pipe for con-
necting two components of a distributed banking application.
She searches the design decision library for an abstract func-
tional contract (explained below) of the pipe implementations.
Because sensitive data is transfered, she prioritizes the quality
attributes of data confidentiality, data integrity, and reliability
with the highest priority; availability with medium priority;
and performance with the lowest priority. She states the
contract to use and the trade-off between quality attributes
(findSatisfying in Figure 1). Then the implementation that best
satisfies this trade-off is automatically selected and used. Alice
does not need to know about the details of the implementation.
Further, the trade-off of the required quality attributes is first-
class represented in the code and easily changeable. Additional
details of the use case can be found in Section IV.
A. Contract
Acontract is a functional interface for design decision
implementations that vary in quality attributes. Each contract
can be implemented by a set of implementations (e.g., en-
cryption algorithms or pipe implementations). However, to
provide changeability, developers should not be able to make
Ciper
Banking Application
Deflated Checked
RSA AES
Key Exchange Contract
Diffie-Hellman Implementation
Contract
satisfies
Legend
Encryption Contract
Pipe Contract
Fig. 2. Hierarchical design decisions. Implementations of contracts can
delegate responsibilities to other contracts. Since quality attribute trade-offs
are forwarded to the children, the peripheral application controls all decisions.
assumptions on a concrete implementation of a design de-
cision. Instead, the best matching implementation is auto-
matically selected according to the required quality attributes
with associated priorities as defined in the source code. Thus,
developers can explicitly weight the requirements, so that the
design decision implementation that is optimal in this context
will be selected and the trade-off becomes first class.
B. Design Decision Implementation
Adesign decision implementation is a concrete implemen-
tation of a corresponding contract. It is characterized by the
degree to which it supports certain quality attributes.
A contract can be implemented by many different im-
plementations. Each implementation offers quality attributes
across various dimensions (e.g., performance, memory con-
sumption, security). An implementation provides some of
these properties to a higher degree than other properties.
Hence a utility function is used to assess the design decision’s
suitability for a concrete context.
Implementations can delegate responsibilities to other con-
tracts created by other domain experts, as shown in Figure 2.
The quality attribute trade-offs are forwarded to the children
to tailor them to the peripheral application. Thereby, imple-
mentations can be reused while hiding the hierarchy from the
user and delaying all inner decisions to the target application.
Hence, a hierarchical tree of design decision can be created
that provides multiple levels of abstraction and a larger amount
of tailored reuse.

Contract Use
Legend
Module
Quality Attribute
Scope
Module A Module B Module C Module D
Fig. 3. The scope of quality attributes can cross-cut multiple modules, be
limited to one single module (Module C), or influence only a singe contract
(Module D). Quality attribute scopes can overlap with each other.
C. Quality Attributes
Aquality attribute is one dimension for characterizing
an implementation of a contract. Examples are performance,
memory consumption, security, or domain-specific properties
such as the softness of a shadowing algorithm.
The application developer then specifies the trade-off be-
tween quality attributes by creating a utility function that
weights the quality attributes. This could be a linear combina-
tion of the quality attributes, or quality attribute scenarios [1],
or it could also include conditional logic (e.g, as long as the
computation is faster than one second, I do not care about
performance). By explicitly specifying the utility function that
evaluates the implementations, the trade-off becomes fist class.
Each quality attribute has a scope to which it applies, as
shown in Figure 3. This can be the use of a single contract,
but it can also be a whole module (such as the networking
component), or a cross-cutting feature of the system (such as
all messages sent to one specific receiver). For example, if
one part of the system becomes a bottleneck, the performance
requirements can be prioritized higher for this specific part
of the system. So there is no need to consider performance
for other system parts that do not constitute a bottleneck.
To implement scopes, quality attributes can be expressed as
separate artifacts that, similarly to pointcuts in aspect-oriented
programming, abstractly specify the contracts, modules, or
features to which they bind to.
D. Application Context
The application context is additional application-wide in-
formation used for selecting an implementation. Information
about the hardware on which the software is expected to
run (e.g., memory, CPU speed, amount of cores, and GPU
properties) and the environment (e.g., speed, reliability and
security of the network), can influence the decision about
which implement is best suited. Therefore, developers can
optionally specify information about the application context
in a key-value schema (e.g., #CPU cores →2–4). This
information usually is uniform across the whole application
and will be used for all trade-offs.
E. Implementation Selector
The implementation selector automatically chooses an im-
plementation for a contract. To select the implementation
that best satisfies the trade-off, the selector evaluates each
implementation according to the priorities of quality attributes
and the context. The implementation selector uses, if possible,
automated analysis, such as performance metrics as carried
out by TradeMaker [11] and security metrics, or otherwise
assessment functions q:context ×qualityAttributes →R
manually defined by domain experts, of the implementations to
determine which best satisfies the requirements. The optional
information of the application context can be used to tailor the
evaluation to the execution environment of the software.
IV. USE CA SE SCE NAR IOS
ToD mainly targets these scenarios: (A) implementing a
feature without being an expert in the domain, (B) adapting the
software according to changing non-functional requirements,
and (C) updating to new advancements in the domain.
A. Implementation Scenario
Developer Alice needs a pipe implementation for her dis-
tributed banking system. She browses the design decision
library for the pipe contract by searching for the term “pipe”
in the index. Because sensitive data is transfered, she prior-
itizes data confidentiality, data integrity, and reliability with
the highest priority. She thinks availability deserves medium
priority, because it is more important to ensure that the
data transfer is correct. Performance is assigned the lowest
priority, since delays are acceptable. So she creates a utility
function that weights these quality attributes according to
these priorities and passes this to the implementation selector.
Since she knows the hardware properties and the network
capacity she adds this information to the application context,
as recommended in the documentation of the pipe contract.
B. Changing Requirements Scenario
After some time of testing, Alice recognizes that there
happens some communication between the two components
that does not involve sensitive data, which became a perfor-
mance bottleneck because of the encryption overhead. Since
the quality attribute trade-off is represented in the code as
first-class entity, she can directly trace the location where
this decision was made. So she limits the scope of data
confidentiality to sensitive data exchange only and increases
the priority of performance at the bottleneck. Internally, differ-
ent implementations are used for the corresponding types of
communication and the system is now tailored to the changed
requirements.
C. Implementation Update Scenario
A security researcher just published a new encryption algo-
rithm that provides a higher level of security while still being
faster to compute than existing algorithms. This attributes
are determined by automated tests that are attached to the
implementation and the assessment of other domain experts
that know all encryption algorithms of the library. Alice
does not know about this new advancement, but since the
implementation became part of the design decision library, her
application automatically selects the improved version. After
the next deployment, her banking application runs faster and
provides a higher level of security.

V. RESEARCH CHALLENGES
To make ToD usable in practice, some further research
questions have to be addressed:
How to deal with implementations that do not match the
interface of the contract? ToD is limited to implementations
that share a common interface, the contract. If an imple-
mentation requires other inputs, it is incompatible with the
general contract. A contract with the additional inputs can be
created that automatically includes the general contract using
a wrapping implementation. However, since this workaround
works for a limited amount of input sets only, it remains
to be explored how to deal with large varieties of input
sets. However, the general problem of adaptation to interface
mismatch is still an open problem [12].
How to evaluate quality attributes? ToD relies on an ob-
jective and comparable assessment of quality attributes of im-
plementations. In the optimal case, this is done automatically.
This is already possible for performance [11] and partially for
specific network security attacks [13], but hardly for quality
attributes such as reliability and availability. So future work
should research techniques to evaluate all quality attributes.
How to specify quality attribute trade-offs? In ToD quality
attribute trade-offs need to be specified using a precise, formal
notation to enable automatic implementation evaluation. We
outlined some ideas in Subsection III-C. Most of the prior
works on interface specification have focused on functional
requirements, although there are recent works on specification
of certain quality attributes such as reliability (e.g.,[14]).
However, further research is needed to devise specification
mechanisms to express a wide range of quality attributes, their
relative priorities, and their relations to each other.
How to optimize the quality attributes globally? In ToD
each decision point locally optimizes quality attribute trade-
offs. However, the overall goal of software architecture is to
optimize the quality attributes of the whole system. So when
applying ToD, one still needs an architect who maintains an
overview of the whole system and defines constraints for each
component. For example, to ensure that the overall system
does not consume more than xmemory, the implementation
for this contract should not consume more than ymemory,
where ydepends on x. This directly results in the research
questions of how to globally optimize the decisions of the
contracts. Since this exploration space is potentially exponen-
tial, it is not simple to solve this efficiently.
VI. DISCUSSION AND CONCLUSIONS
We propose ToD, a new concept to make quality attribute
trade-offs first class. We argue that thereby, requirements are
more easily traceable and changeable. Furthermore, developers
might concentrate more on quality attributes. ToD might
change the way that software is developed by facilitating rapid
cycles of development, testing, and adjusting requirements.
This in turn will enable developers to deliberately control
the quality attributes of each part of the system separately.
For example, performance optimizations would be limited
to bottlenecks and security optimizations would be limited
to sensitive parts of the system. Moreover, newly developed
solutions that more precisely satisfy the trade-offs will auto-
matically be integrated into the software due to the decou-
pling of trade-off specifications from the solutions. Thereby,
advances in research and development of new solutions might
have more direct and immediate impact on practice. Finally,
since less expertise is needed to develop high quality systems,
this concept is one step towards the goal of making software
development more accessible to non-experts.
ToD can be applied to, for example, architectural connec-
tors, resource management, persistence, computer graphics,
data structure, security, and other domains with unified con-
tracts and varieties of implementations. However, ToD will
not work well for domains that are not yet well understood
and thats community has ongoing controversial discussions
about problem definitions. They might not agree on common
standard implementations and unified contracts.
Nonetheless, ToD might dramatically change the way that
code is reused. Currently either a small, negligible portion
of software is reused, or a larger portion that includes many
decisions that might not fit the context of the reusing software
and, therefore, limits the applicability of the reused code. ToD
allows one to combine both advantages to delay all decisions
that are made in reused code to the client that uses it. So
developers would not decide whether to use security library
A or B, but state the decisions that are made during the
development of the security library. Thereby it is in one steps
towards the vision of mass customized modules.
REFERENCES
[1] L. Bass, P. Clements, and R. Kazman, Software Architecture in
Practice, 3rd ed.
[2] F. Buschmann, R. Meunier, H. Rohnert, S. Peter, and M. Stal, Pattern-
oriented Software Architecture: A System of Patterns, vol. 1.
[3] S. Nadi, S. Kr¨
uger, M. Mezini, and E. Bodden, “Jumping Through
Hoops: Why Do Java Developers Struggle with Cryptography APIs?”
In Proc. ICSE 2016.
[4] S. Indela, M. Kulkarni, K. Nayak, and T. Dumitras
,, “Helping Johnny
Encrypt: Toward Semantic Interfaces for Cryptographic Frameworks,”
in Proc. Onward! 2016.
[5] A. Jansen and J. Bosch, “Software architecture as a set of architectural
design decisions,” in Proc. WICSA 2005.
[6] M. P. Robillard, “Sustainable Software Design,” in Proc. FSE 2016.
[7] E. Kouroshfar, M. Mirakhorli, H. Bagheri, L. Xiao, S. Malek, and
Y. Cai, “A study on the role of software architecture in the evolution
and quality of software,” in Proc. MSR 2015.
[8] C. Prehofer, “Feature-oriented programming: A fresh look at objects,”
in Proc. ECOOP 1997, M. Aks¸it and S. Matsuoka, Eds.
[9] O. Alam, J. Kienzle, and G. Mussbacher, “Concern-oriented software
design,” in Proc. MODELS 2013.
[10] D. Garlan, V. Dwivedi, I. Ruchkin, and B. Schmerl, “Foundations and
tools for end-user architecting,” in Large-Scale Complex IT Systems.
Development, Operation and Management, ser. LNCS 7539. 2012.
[11] H. Bagheri, C. Tang, and K. Sullivan, “TradeMaker: Automated
Dynamic Analysis of Synthesized Tradespaces,” in Proc. ICSE 2014.
[12] C. Canal, J. M. Murillo, and P. Poizat, “Software Adaptation,” L’Objet,
vol. 12, no. 1, 2006.
[13] A. Ramos, M. Lazar, R. H. Filho, and J. J. P. C. Rodrigues, “Model-
Based Quantitative Network Security Metrics: A Survey,” IEEE Com-
munications Surveys Tutorials, vol. 19, no. 4, 2017.
[14] S. Misailovic, M. Carbin, S. Achour, Z. Qi, and M. Rinard,
“Reliability-aware optimization of approximate computational kernels
with rely,” Tech. Rep., 2014.