Content uploaded by Ken Power
Author content
All content in this area was uploaded by Ken Power on Oct 30, 2018
Content may be subject to copyright.
Understanding Architecture Decisions in Context
An industry case study of architects’ decision-making context
Ken Power1[0000 -0002-37 82-9539] and Rebecca Wirfs-Brock2
1 Cisco Systems, Inc., Ireland
ken.power@gmail.com
2 Wirfs-Brock Associates, USA
rebecca@wirfs-brock.com
Abstract. Many organizations struggle with efficient architecture decision-mak-
ing approaches. Often, the decision-making approaches are not articulated or un-
derstood. This problem is particularly evident in large, globally distributed or-
ganizations with multiple large products and systems. The significant architec-
ture decisions of a system are a critical organization knowledge asset, as well as
a determinant of success. However, the environment in which decisions get
made, recorded, and followed-up on often confounds rather than helps articula-
tion and execution of architecture decisions. This paper looks at aspects of archi-
tecture decision-making, drawing from an industry-based case study. The data
represents findings from a qualitative case study involving a survey and three
focus groups across multiple organizations in a global technology company. Ar-
chitects in this organization are responsible for multiple products and systems,
where individual products can include up to 50+ teams. The impact is not just on
others in the system; architecture decisions also impact other decisions and other
architects. The findings suggest recommendations for organizations to improve
how they make and manage architecture decisions. In particular, this paper notes
the relevance of group decision-making, decision scope, and social factors such
as trust in effective architecture decision-making.
Keywords: Architecture, Architecture Decisions, Decision Making, Decision
Makers, Decision Impact, Trust, Roles, Documentation, Agile
1 Introduction
Architecture decisions can significantly affect architects and other roles. Realizing this,
a vital component of any architectural approach is having a process that promotes fol-
low through and feedback on architecture decisions. This paper presents a case study
of a large technology organization with multiple business units and product lines. This
study examines approaches to architecture decision-making and seeks to understand in
more depth the reasons for the decision-making approaches employed by architects, as
well as the challenges and context that architects must deal with. In addition, this study
attempts to understand the impact of architecture decisions. The remainder of this paper
describes the study.
2
Section 2 places this study in context through a review of relevant literature from
software architecture decision-making. This study employs a survey and three focus
groups as part of a larger case study into architecture decision-making. Section 3 de-
scribes the research approach used in this study and includes the research questions that
this study sets out to answer. Section 4 presents the findings from the survey and focus
groups. Section 5 is a discussion of the findings. Finally, section 6 presents the sum-
mary and conclusions from this study, including a set of recommendations based on the
findings, and a discussion of future research that builds on this study.
2 Literature Review
This section presents a review of the relevant architecture and decision-making litera-
ture that informs this study and helps to shape its research objectives. Bass, Clements
and Kazman [1] define architecture as, “the structure or structures of the system, which
comprise software elements, the externally visible properties of those elements, and the
relationships among them.” Traditionally, an architecture may be described using one
or more relevant views [2]. Recognizing that some architecture decisions have a broad
impact Kruchten, Capilla and Dueñas [3] propose that a “decision view” be added to
existing architecture views, superimposing the design rationale which underlies and
motivates the selection of design options realized in the architecture. Kruchten, Lago
and Van Vliet [4] suggest that “Architecture Knowledge = Design Decisions + De-
sign”. Jansen and Bosch [5] go even further to assert that a system’s architecture should
be viewed as the composition of a set of architectural design decisions. This paper takes
the position that it is a distraction to argue whether decisions should drive the selection
of relevant views as do Tyree and Akerman [6], or whether selecting relevant views
should drive important decisions [7]. Both the architecture and the relevant views that
represent it embody design decisions. Both are important to communicate. Important
to understanding any architecture are the cumulative decisions that influenced it as well
as an appreciation for how those decisions were made.
Although several formal architecture decision-making approaches have been pub-
lished, software engineering researchers find few to be used in practice. This may be
because many published decision-making approaches describe processes for making
reasoned tradeoffs between several competing options, while decision making re-
searchers observe that many complex real-world decisions are not about making
tradeoffs, but instead about finding a reasonable decision that satisfices the current sit-
uation and allows for action [8]. Rekha V. and Muccini [9] also found that none of the
published methods account for differences in expertise required to make informed de-
cisions nor do they have provisions for resolving conflicts or differences of opinion.
Surveys of architects have found that over 85% of decisions made are group decisions
[9, 10]. During group discussions, both shared and unshared (e.g. unknown to all mem-
bers of the group) information is brought out and examined.
Tyree and Akerman [6] proposed elevating architecture decisions to first-class ar-
chitectural artifacts suggesting that documented decisions can provide concrete direc-
tion for implementation and serve as an effective tool for communicating to customers
3
and management. It is unclear whether practicing architects often take their advice. In
a survey by Dasanayake, Markkula, Aaramaa and Oivo [11], architects reported that
90% of their decisions were made and communicated informally. And while these ar-
chitects were mostly satisfied with their informal decision-making processes, they also
recognized some challenges with revisiting design rationale, communicating decisions
to customers, and in knowledge gaps between engineers and architects. In our research
we also found that decisions were communicated informally through a number of me-
dia, including slide decks, wikis, and meeting recordings.
Kruchten [12] proposes the following taxonomy for architecture decisions: Existence
decisions state that some element or artifact will show up in the system’s design or
implementation. Bans or non-existence decisions are statements of things to not do.
Kruchten [12] suggests that it is especially important to precisely document bans be-
cause there isn’t a place for them in conventional architecture documentation. Property
decisions affect the overall qualities of the system and may be represented by design
rules or constraints. Executive decisions are driven by the business environment and
may affect the development process or choices of technology and tools. Although they
may place constraints on the architecture, Kruchten [12] asserts that executive-level
decisions are often not captured or appear in documents usually not associated with the
architecture. Kruchten’s model, while one of the few decision categorization models
that we can find in software architecture, is insufficient for handling decisions in the
type of environment we are studying and for classifying the types of decisions we are
uncovering.
While Miesbauer and Weinreich [13] found that architecture decisions could be
mapped to Kruchten’s taxonomy [12], they noted that architects themselves talk about
decisions they make according to level. They proposed classifying architecture deci-
sions according to these levels: Management typically makes organization-level deci-
sions with advice from software architects. Once made these decisions are rarely revis-
ited. Project managers, architects, and customers tend to make project-level decisions
at the beginning of a project. Software architects or team leads typically make archi-
tecture-level decisions after discussion in a group. Finally, implementation-level deci-
sions are made independently by developers and typically not documented. Miesbauer
and Weinreich [13] suggest further investigation into whether the levels they identify
are adequate to partition the decision space. A weakness in the framework of Miesbauer
and Weinreich [13] is it suggests that impact is contained to a specific level. In our
research, we found it fruitful to characterize decisions along multiple dimensions, in-
cluding impact and scope. We also found examples where implementation-level deci-
sions had system-wide impacts on architectural qualities.
3 Research Approach
This section presents the research questions addressed through this paper. This section
then states the epistemological stance employed by the study, and how that influences
the study and the choice of research methods. This study uses a case study of a large,
global technology organization. A survey and three focus groups are used to answer the
4
research questions. This section describes these methods, and how they contribute to
the study. Finally, this section describes the data collection and analysis methods.
3.1 Research Questions
With regard to architecture decision-making, the topics of interest to this study relate
to approaches, challenges, context, and impact. This paper aims to contribute to the
body of knowledge on architecture decision-making by answering the following re-
search questions:
• RQ1: Approaches: What are some examples of decision-making approaches used
by architects, and how those decisions are made?
• RQ2: Challenges: What challenges do architects encounter associated with the cur-
rent architecture decision-making approaches?
• RQ3: Context: How does the context influence decisions made by architects?
• RQ4: Impact: How do decisions made by architects impact other people?
Findings related to these questions are presented in section 4.
3.2 Research Method
Any study is shaped by the social and theoretical perspectives adopted by the research-
ers [14]. This study adopts an interpretivist, constructivist philosophical stance. Inter-
pretivist research acknowledges that people have potentially widely-varying percep-
tions of the same phenomenon, and that knowledge is a social product [15].
This study uses a case study to “understand complex social phenomena” related to
how architects make decisions. Case studies are well suited to research in software de-
velopment because they study contemporary phenomena in their natural setting [16,
17]. This study is concerned with how and why architects make the decisions they do,
and how those decisions impact others. Case studies can “uncover subtle distinctions
and provide a richness of understanding and multiple perspectives” [18]. This research
includes perspectives from multiple stakeholders, not just architects. Yin [19] notes that
case studies are suitable when “the boundaries between phenomenon and context may
not be clearly evident.” The primary unit of analysis is a Business Group, BG1, one of
the largest business groups in the company consisting of approximately 5,000 people
in different sites around the world. BG1 has multiple business units, each of which is
responsible for multiple product lines in a particular domain. The initial survey targeted
architects across BG1. The researchers decided to conduct a focus group with partici-
pants from each of three product lines. We targeted one product line per business unit
in order to get a representative sample of perspectives on the topic of architecture deci-
sion-making. The case study consists of one survey from BG1 and three focus groups,
FG1, FG2, and FG3.
These four units of analysis combine to provide a comprehensive picture of archi-
tecture decision-making. Initially, the researchers conducted a survey of 62 architects
located across a business group with sites in North America, Europe, Israel, and India.
5
We wanted to broadly understand how architects perceived their role and interactions
with others and more specifically what they found to be challenging and rewarding
aspects of their work. In addition to demographic information, the survey asked about
their role and interactions with other architects, engineers, product owners and product
management. The survey participants were highly experienced, as shown in Table 1.
The majority (90%) had architect, technical lead, or principal or distinguished engineer
in their job title. The remaining respondents were engineers or managers. Different
kinds of architects were represented including customer, solutions, systems, and Scrum
(team lead) architects. Following on from the survey, we conducted three focus groups
specifically on the topic of architecture decision-making.
Table 1. Survey respondents’ years of experience as architects
# years experience as an architect
% of respondents
10+ years
47%
6-10 years
23%
4-5 years
15%
1-3 years
13%
< 1 year
2%
No response
10%
Morgan [20] defines focus groups as “a research technique that collects data through
group interaction on a topic determined by the researcher”. Yin [21], on the other hand,
says the groups are focused because they share some common experience or views.
These two perspectives are complimentary for the purpose of this study. The unit of
analysis is the group itself, not the individuals within the group, so each focus group is
one data collection unit [21].
Table 2. Summary of focus groups conducted as part of this study
Focus
Group:
FG1
FG2
FG3
Focus
Architecture Decision
Making
Architecture Deci-
sion Making
Architecture Deci-
sion Making
# Partici-
pants
10
11
12
Location
Israel
USA
India
Domain
Security Products
Video Technology
Networking Tech-
nology
Roles
Architects, Program
Managers
Architects, Engi-
neers
Architects, Engi-
neers, Program Man-
agers, Engineering
Managers
The survey data was collected using a Web-based survey tool. The focus groups were
recorded and then trascribed. The authors analyzed the survey data and focus group
data independently and reviewed the analyses together through multiple iterations.
6
3.3 Threats to Validity
This section discusses potential threats to the validity of this research study.
• External Validity. The researchers do not claim that these findings are universally
applicable. They are representative of a number of business units in a specific, large
global technology organization. They serve as illustrative examples that others may
learn from.
• Construct Validity. To mitigate this threat, data was collected from multiple
sources. The researchers used triangulation between the survey data and the three
focus groups, thereby converging evidence from four distinct data sources. The re-
searchers compared results across multiple groups, where the data was collected at
different points in time and in different geographic locations.
• Reliability. Relating to the repeatability of the study, the survey instrument and fo-
cus group questions were designed over several iterations and involved other subject
matter experts and architects to review these and provide feedback. Using respond-
ent validation [21] the researchers reviewed the data with a small group of architects
from BG1 to help ensure validity of the data and the findings
• Internal validity. This study does not attempt to establish any causal relationships,
so no internal validity threats are described [17].
4 Research Findings
4.1 Survey Results
The top 6 activities surveyed architects reported in order of frequency mentioned were
architecture design, 98%; collaborating with development teams, 87%; product and so-
lution requirements specification, 84%; knowledge sharing, 82%; document review,
65%; and program and product planning, 50%. Only 18% reported coding and 15%
testing. Most, 82%, interacted daily with engineering and development teams; 13% did
so weekly. Frequency of interactions with product owners and management was less
with 44% reporting interacting daily, 37% weekly, and 10% monthly.
When asked what impacted their effectiveness, architects identified the dissipation
of knowledge, the large number of people involved in making decisions, finding relia-
ble or up to date documentation and information, misalignment with the development
team and other architects, organizational changes, time zone differences, time pres-
sures, and overlapping/unclear roles and responsibilities.
For the most part, architects perceived that they were effective in their various roles.
77% rated their effectiveness positively. 79% rated their communications with engi-
neering as successful. Slightly less, 60%, rated their communications with product man-
agement as successful. However, some architects, were unsure whether others in the
organization considered their work valuable or necessary. Architects wanted to be
heard, understood, and recognized as making valued contributions.
7
Surveyed architects found working with engineers to be extremely rewarding. They
were gratified to receive pragmatic, concrete feedback on the architecture and the de-
cisions they made; to see the final solution as implemented and its evolving design; to
answer questions, mentor engineers, and bring information about customers and
broader issues to the team; to feel part of a team working on a common goal; and to be
engaged in collaborative decision-making and mutual learning. They wanted engineers
to be more involved earlier in the definition of the product as well as to be more in-
volved themselves during implementation. Learning was perceived as bi-directional:
while engineers have broader view of technology and trends in the industry, architects
know about customer needs, product requirements, and more broadly about the existing
architecture.
Architects expressed frustrations when engineers misunderstood their designs or
when engineers lacked product knowledge. Other architects expressed frustration with
engineers’ seemingly short-term focus or focus on functionality to the exclusion of sys-
tem qualities. It was also frustrating when engineers didn’t contribute to or value archi-
tecture documentation, or when the code was viewed as “the only source of truth” about
the design.
When asked one thing they would like change to make their practice of architecture
more effective when working with engineering, several themes emerged: better
knowledge sharing, improved documentation including documentation of decisions,
and improved feedback and review of the architecture and its implementation.
Surveyed architects who regularly engaged with product owners or product manage-
ment (not all did, and some architects were also in the role of product owner) were
mixed in their perceptions. Some architects found it rewarding that they could influence
product management’s understanding of significant architecture requirements, clarify
or remove unnecessary requirements, and influence product features and their delivery
roadmap. They found it rewarding to get deeper insights into the customer and the prod-
uct provided by product management.
Architects were frustrated by conflicting requirements or when product owners
changed requirements or priorities too rapidly. Another frustration was inflexible prod-
uct managers who didn’t listen. Other architects expressed frustration with product
owners’ lack of current product or customer knowledge or when they made architecture
decisions without asking their advice. They were also frustrated when those decisions
seemed shortsighted or overly focused on satisfying a single customer.
When asked one thing they would like change to make the practice of architecture
more effective working with product management, architects on the whole wanted to
improve communications, increase collaboration, increase their visibility into and in-
fluence on the overall product strategy and backlog, and be involved in joint decision-
making on architecture.
4.2 Examples of Decisions
Focus group participants were asked to share their experiences with recent architec-
ture decisions. Examples of decisions are sown in Table 3.
8
Table 3. Examples of decisions identified in focus groups, and their context
Category
Example from this
study
Scope
Impact
Source
Technology
Investing in micro-
services frameworks
System
Business
Unit
FG2
Technology
Moving to containers
and microservices
System
Business
Unit
FG2
Technology
Moving to open stand-
ards
System
Business
Group
FG2
Design guide-
line
Defining a standard for
defining and publishing
APIs
System
Business
Group
FG2
Product
implementa-
tion
Using incompatible
technology
Product
Business
Unit
FG2
Product
implementa-
tion
Deciding to use plat-
form native encryption
components
Product
Business
Unit
FG2
Product
implementa-
tion
Taking a short-sighted,
simple decision due to
client pressure
Compo-
nent
Business
Group
FG1
Product
implementa-
tion
Taking a decision
quickly instead of ana-
lyzing new information
brought up in a meeting
Compo-
nent
Business
Unit
FG1
Product
implementa-
tion
Designing a backwards
incompatible end-to-
end solution
Compo-
nent
Business
Unit
FG1
Product
implementa-
tion
Extending an existing
solution, trying to fit a
design to an incompati-
ble platform
Product
Product
Line
Team
FG3
Product
implementa-
tion
Repeatedly bringing up
a design problem due to
lack of understanding of
an existing solution
Compo-
nent
Product
Line
Team
FG3
Infrastructure
Investing in a separate
infrastructure group
System
Business
Unit
FG2
The researchers found it difficult to classify these decisions according Miesbauer
and Weinreich [13] levels or to line them up with Kruchten’s taxonomy [13]. Conse-
quently, we characterized decisions as being technology, design guidelines, infrastruc-
9
ture, or product implementation level decisions. Analysis revealed that technology, de-
sign guidelines, and infrastructure decisions were viewed positively, while product im-
plementation decisions were not. The data in Table 3 relates the decision examples to
the case study. The scope of decisions relates to a component, a product (composed of
many components), or a system (composed of many products). The decision impact is
expressed in terms of whether the decision impacts the product team, the business unit,
or the business group. Table 3 also notes the source focus group for the example.
4.3 Approaches to Decision Making
Architects described their decision making as mostly informal, e.g. the right people get
in a room or on a phone call and discuss. While decision-making can be informal, it can
also be political. One respondent in FG1 recounted a situation where those who dis-
sented were removed from further discussion. Another observed that conversations, and
persuasion and interpersonal relationships often drive decisions, and that this was not
always positive. Depending on the decision, certain people had veto power, and for
some decisions, it was agreed that product management rightfully should make them,
although architects would like to be consulted.
The survey raised several questions about who made decisions about the architec-
ture. One respondent noted that in some cases project managers and product managers
make technical decisions without consulting architects. A different respondent ex-
pressed a desire for “more formal tracking of decisions” to help collaboration with
product managers and product owners.
4.4 Challenges with Current Approaches to Decision Making
Architects were not always involved in architecture-related decisions. Survey respond-
ents indicated that architects did not always have the influence that they thought they
should have: “significant decisions are completely centralized within the senior lead-
ership team and architects/POs/PMs have less influence than they should.” This was
echoed by another respondent who noted, “Not all the information is shared with ar-
chitects which could affect some architecture decisions in the initial phase of the pro-
ject”. There was no indication that the lack of information sharing or centralizing of
decision-making was designed to deliberately to exclude architects, but several archi-
tects certainly felt the impact.
Focus group participants cited several challenges that they associated with current
decision-making approaches. Decisions are often made without considering the tech-
nical feasibility of implementing the solution, and the long-term consequences associ-
ated with that. An example cited by one architect related to a build vs buy decision. In
the system he worked on, there were several instances where the team decided to build
the required functionality, rather than buy a commercial solution. This had the effect of
adding to the system’s technical debt.
Participants in FG2 noted that it is difficult to reverse poor decisions, and when de-
cisions are reversed or changed, it does not happen quickly enough. Participants in FG1
noted an unwillingness to change decisions: “the first decision is accepted as the final
10
one and the project leader doesn’t adjust or change direction when problems or new
information is found”.
The survey highlighted several examples where architects felt decisions were not
followed through. In one case, an architect felt that people do not follow through on
implementing architecture decisions because they are unaware of them, or they do not
trust the decision. One architect noted that architects are often out of the loop during
development and that decisions and feedback would be improved if the team were to
“…involve architects on the problems raised during the implementation”.
4.5 Context in Which Architects Make Decisions
Architects didn’t directly share many thoughts on why decisions were made the way
they were. But some decisions seemed to be made under time pressure. In those cases,
decision makers had to make tradeoffs between short-term project considerations and
longer-term architecture sustainability. Decisions sometimes were made in the narrow
context of delivering the next feature; in this case expediency drove the decision-mak-
ing process. People got together, discussed, and made a decision. One participant in
FG1 noted that it was difficult to make longer-term decisions, so he didn’t: “It’s too
hard to get enough convergence or consensus or agreement around a long term deci-
sion, so I often find that I’m making a small, local decision that serves a particular
local need and that is locally optimized, and I’m not able to take any long term wide-
ranging considerations into account”. At other times when it was important to gain
consensus, it took time to make decisions. One architect in FG2 noted that, “Because
we're focused on getting consensus over multiple engineering teams and architecture
teams all over the place, the process has just gotten more complicated.”
A large number of distributed teams is also an important part of the decision-making
context. The participants in FG2 are part of a group of 50+ teams working on a single
product. Geographic distribution between teams and multiple time zones adds to the
difficulty of their context. As one architect from FG2 stated, “we’re not compartmental-
ized enough that we can make these decisions in one timezone, or even a couple of
timezones”.
The distribution among multiple countries, time zones, and teams results in it taking
a long time to make “big” decisions. Participants cited situations where consensus-
based decisions were necessary and referred to “big” and “consensus” as attributes of
decisions that take a particularly long time. As one architect noted about working with
teams spread over 5 countries, it takes a long time “… to sell the idea, right? You have
to build consensus around that, and that does take a lot of time”.
Trust is also a factor in decision-making contexts. Focus group participants agreed
that decision-making is “more productive” when there is a higher level of trust between
the people directly involved. Participants cited situations where trust is not present. One
example is where there are “pockets of … technical feudalism” where an architect is
making decisions in isolation.
The agile development process is also a factor. One surveyed architect noted, “Agile
development as currently practiced … does not have a place for Architecture. So this
needs to be fixed before we can have a meaningful discussion about how developers
11
and architects interact.” Another expressed frustration about the way decisions are
changed, “The thing that frustrates me more is when the implementation doesn't match
the design because there is a misunderstanding and the developers make their own
decisions without checking with the architect. I'm all for letting the development teams
as much freedom as possible but the architect needs to be consulted.”
4.6 Effect of Architecture Decisions on Architects and Other People
There are examples where architecture decisions are not followed through. In one sys-
tem discussed by participants in FG2, architects defined a high-level architecture
(HLA) for several subsystems. The perception of architects is that the teams responsible
for implementing that HLA exhibited “passive aggressive non-compliance”. They did
not challenge the HLA decisions directly. Instead, they disregarded the decisions in the
HLA. The perception of the architects was that “people on various scrum teams …
decide they know better”. Participants related this problem to the context of operating
within a “giant” multi-country, multi-time zone project with 50+ teams. This context
added to a lack of visibility by architects into what teams are actually building.
While verbal communication related to follow-through on architecture decisions
happens, it is unpredictable. One architect expressed in the survey their desire for engi-
neers to contribute more to design documentation, noting “I wish engineers would have
contributed more to the knowledge sharing via documentation (and not only verbally,
which they do very happily)”.
Lack of information related to past decisions have a significant effect on architects
currently working on the product. The need for a trail of decisions and their context
came through as architects noted challenges associated with joining a product team
where the architecture already has a long history, e.g., “the very long history of the
project and the decisions that were taken before I joined the project”.
Engineers don’t always have enough context about the architecture. Sometimes en-
gineers encounter cases where the architecture does not seem to support what they need
to do. One architect noted in some cases “they don't care too much about the design; if
it seems it doesn't work, they may "adapt" the implementation, not in line with the de-
sign (that may well be wrong or incomplete) but without necessarily telling the archi-
tects or updating the documentation”. A further risk here is that if architects don’t get
this feedback, then any architecture decisions recorded earlier become out of date with-
out an appropriate feedback loop.
5 Discussion
The geographic and time zone challenges reflected by FG2 indicates that the organiza-
tion in question did not give enough consideration to the impact of these factors on
architecture decision making.
12
5.1 Perceptions of Agile Development on Architecture Decisions
Agile development emerged as a particularly strong theme from survey results and
the focus group data. The organizations that these groups were part of made decisions
and assumptions about what it meant for agile and architecture to co-exist. Architects
tend to bring up longer-term perspectives on the architecture. Some feel that short-
sighted decisions are made when the decision makers focus only on feature delivery at
the expense of architecture integrity or increased maintenance costs. An architect in
FG2 noted, “We do have this type of organization problem. I think it’s because a com-
bination of agile and I'm not sure the journey to Agile Architecture is ... It’s still a
journey … We haven’t quite figured it out.”
Several survey respondents and focus group participants echoed a common theme
around agile methods contributing to a loss of a paper trail. One survey respondent
noted, in connection with the adoption of agile methods, “we seem to have forgotten
that paper-trail is important for adequate product maintenance”.
5.2 Feedback
Architects desire more feedback on their decisions and want more follow through with
engineering. One architect in the survey stated, “In my book software architecture im-
plies that you have to work with the engineers and also be part of the development
teams…[It ] helps validate the design ideas you have as an architect.”
Some architects want more defined processes: “I believe the lack of formal develop-
ment lifecycle processes leaves the architect with a design that no one has to comply
with”. “The ‘old way’ of architecture process is gone and we don’t really have a new
one yet”. Another architect offers that one way to improve feedback “…is for the ar-
chitect to be a "virtual" member of the development team, going to some of the stand-
ups, user story reviews, retrospectives, etc.”
5.3 Group Decisions
The findings revealed that groups often make architecture decisions informally. Often
decisions are not recorded and so the nuances behind the decisions are lost or become
tacit knowledge held only by those initially involved with the decisions. Decisions com-
ing from several sources impact the architecture. Product management, with or without
architects’ involvement, sometimes makes architecture decisions. Still other architec-
ture decisions are “strategic” and made by senior management. Architects wanted more
involvement in all these decisions.
5.4 The Roles of Architects and their Decision-Making Scope
The roles of architects and their scope of decision-making are not always clearly de-
fined. There are different types of architects, ranging from those embedded in develop-
ment teams (Scrum Architect) to customer and end-to-end solutions architects. Some
architects are also Product Owners, which means they define features as well as product
13
architecture. Architects are communicators, and often are the bridge between customer
needs and engineering. However, decision-making responsibilities are not always clear.
Responsibilities of various architect roles sometimes overlap. Architects would like
more clearly defined roles that were better understood and agreed to.
5.5 Scope and Impact of Decisions
The data shown in section 4.2 indicates that the impact of decisions is generally quite
high. Relating the decisions examples to the organization structure, 16.7% impact the
product line team. 58.3% impact the business unit responsible for the product line. 25%
impact the business group responsible for the business units. The majority of examples
that impact the business group are architecture decisions that have a system scope,
while one decision example has a component scope. This illustrates that, from an ar-
chitecture perspective, a decision made at the component level can have a wide-reach-
ing impact well beyond the team. Of the example decisions shown in section 4.2, the
architecture decisions at the product scope impact either the product line team or the
business unit. The architecture decisions at the system scope impact either the business
unit or the business group.
5.6 Characterizing Decisions
During the data analysis we realized that none of the decision frameworks discussed in
the literature review adequately captured certain dimensions of a decision. During the
coding process we identified decision categories, which we noted as “Technology”,
“Product”, etc. It also proved useful to identify the level of abstraction that a decision
related to or impacted, i.e., systems, sub-system, component, etc.
We find it more useful to characterize decisions along multiple dimensions rather
than try to fit them in to a single taxonomy. Our approach was to start with the data,
and identify suitable characterizations for the data, rather than start with a framework
and force-fit the data to the framework. This approach helped us, and the participants
in our study, to gain a deeper understanding of the decisions and their context.
6 Summary and Conclusions
This section presents a summary of the research findings. Section 6.1 shows how the
research questions have been addressed. Section 6.2 provides a set of recommendations
for architects and organizations based on the research findings. Section 6.3 presents
conclusions from the research. Finally, section 6.4 outlines directions for future re-
search by the authors that builds on the topics and findings in this paper.
6.1 Answering the Research Questions
This paper sought to address five specific research questions, as outlined in section 3.1.
This section summarizes how each research question has been addressed.
14
• RQ1: Approaches. Sections 4.2 and 4.3 presented findings that show examples of
current decision-making approaches used by architects. The findings identified a
range of approaches, notably the prevalence of informal, group-based decisions.
There are also examples of scenarios where architects are not involved in architec-
ture decisions which were made by product management or management, or where
they had insufficient information to make an informed decision.
• RQ2: Challenges. Section 4.4 articulated challenges encountered by architects re-
lated to decision-making approaches. The findings identified some architecture de-
cisions were made without considering the technical feasibility or longer-term con-
sequences and other decisions were made without adequate information.
• RQ3: Context. Section 4.5 described some conditions within which architects make
decisions. The findings identified that architects worked with large, distributed
teams to make decisions which often required consensus building and gaining trust.
Architects did not directly offer explanations for their decision-making approaches.
Some decisions are made more quickly under time pressures while “bigger” deci-
sions take more time and involve gaining group consensus.
• RQ4: Impact. Section 4.6 showed how decisions made by architects, and their de-
cision-making approach, impact other people. The findings suggest that decision-
making was viewed as more effective when architects followed through with engi-
neering or decisions were made collaboratively.
6.2 Recommendations
The following recommendations are drawn from the survey and focus group findings:
1. Consider the space-time separation of teams, and how that impacts architecture de-
cisions. When dealing with teams who are separated in space (through multiple ge-
ographies) and time (through multiple time zones), make an effort to compartmental-
ize the scope of responsibility of teams such that coherent architecture decisions can
be made in each location.
2. Establish clear decision-making boundaries. Articulate who is responsible for which
type of decisions. This can be based on scope of decision (product, system, compo-
nent, etc.), nature of decision (product, technology, etc.), or something else.
3. If your organization is using an agile development approach, then take the time to
articulate how architecture fits.
4. Understand who is impacted by decisions made by architects. Establish a feedback
loop so that architects understand that impact in a timely manner.
5. Start with why. Architects in this study expressed a much higher degree of success
in decision adoption when other people understood why a decision is being taken.
This is an important part of the context of architecture decisions.
6. Take the time to foster trust among architects and those impacted by decisions.
7. Consider how architecture decisions are retained and communicated. We see a need
for retaining and communicating architecture decisions and their rationale, espe-
cially when decisions have broad impact. Documenting decisions, to be effective,
should fit into existing processes.
15
8. Some decisions are necessarily made for short-term expediency, e.g. to address an
immediate customer need. Perhaps there needs to be some mechanism to flag these
types of decisions and manage them, perhaps in a product debt backlog (especially
those that will incur architecture debt) for periodic review.
6.3 Conclusions
Having multiple dimensions that help characterize different decisions, as shown in sec-
tion 4.2, provides deeper insights into the types of decisions architects are dealing with.
Architects are generally experienced decision makers operating in an environment char-
acterized by time pressure, insufficient information, poorly defined or non-existent pro-
cedures, and a need for coordination across hundreds of people in multiple global
teams. Their perception is they are most effective in making decisions where they have
formal and direct collaboration with engineering and product management. The find-
ings showed several examples that help the researchers understand how architects ap-
proach decision-making, and the challenges, context and impact of those decisions. The
findings did not reveal sufficient data about the reasons why architects choose the de-
cision-making approaches they employ.
6.4 Future Research
Future research based on this study will focus on the following:
• Understanding how architecture decisions constrain other decisions. It is hard for
developers who get involved later, long after a decision is made, to understand the
initial design context. This research points to the potential need for a cumulative
history of decisions.
• Understanding the trade-offs and benefits between documenting decisions, and other
aspects of the architecture. In particular, is it more important to document decisions
than it is other aspects of the architecture?
The authors also intend to reproduce this study with additional organizations to un-
derstand how approaches to architecture decision making vary in different contexts.
References
1. Bass, L., Clements, P., Kazman, R.: Software Architecture in Practice. Addison-Wesley,
Upper Saddle River, NJ (2013)
2. Rozanski, N., Woods, E.: Software Systems Architecture: Working with Stakeholders Using
Viewpoints and Perspectives. Addison-Wesley, Upper Saddle River, NJ (2012)
3. Kruchten, P., Capilla, R., Dueñas, J.C.: The decision view's role in software architecture
practice. IEEE software 26, 36-42 (2009)
4. Kruchten, P., Lago, P., Van Vliet, H.: Building up and reasoning about architectural
knowledge. International Conference on the Quality of Software Architectures, pp. 43-58.
Springer (2006)
16
5. Jansen, A., Bosch, J.: Software architecture as a set of architectural design decisions.
Software Architecture, 2005. WICSA 2005. 5th Working IEEE/IFIP Conference on, pp.
109-120. IEEE (2005)
6. Tyree, J., Akerman, A.: Architecture Decisions: Demystifying Architecture. IEEE Software
(2005)
7. Clements, P., Ivers, J., Little, R., Nord, R., Stafford, J.: Documenting Software Architectures
in an Agile World. Software Engineering Institute (2003)
8. Klein, G.A.: Sources of power : how people make decisions. MIT Press, Cambridge, MA
(2017)
9. Rekha V., S., Muccini, H.: Suitability of Software Architecture Decision Making Methods
for Group Decisions. In: Avgeriou, P., Zdun, U. (eds.) Software Architecture: 8th European
Conference, ECSA 2014, Vienna, Austria, August 25-29, 2014. Proceedings, pp. 17-32.
Springer International Publishing, Cham (2014)
10. Tofan, D., Galster, M., Avgeriou, P.: Difficulty of Architectural Decisions – A Survey with
Professional Architects. In: Drira, K. (ed.) Software Architecture: 7th European Conference,
ECSA 2013 Proceedings, vol. LNCS 7957, pp. 192-199. Springer, Montpellier, France
(2013)
11. Dasanayake, S., Markkula, J., Aaramaa, S., Oivo, M.: Software architecture decision-
making practices and challenges: an industrial case study. Software Engineering Conference
(ASWEC), 2015 24th Australasian, pp. 88-97. IEEE (2015)
12. Kruchten, P.: Documentation of software architecture from a knowledge management
perspective–design representation. In: al., M.A.B.e. (ed.) Software Architecture Knowledge
Management. Springer-Verlag, Heidelberg (2009)
13. Miesbauer, C., Weinreich, R.: Classification of Design Decisions – An Expert Survey in
Practice. In: Drira, K. (ed.) Software Architecture: 7th European Conference, ECSA 2013,
vol. LNCS 7957, pp. 130-145. Springer, Montpellier, France (2013)
14. Maxwell, J.A.: Qualitative Research Design: An interactive approach. SAGE Publications,
Inc., Thousand Oaks, CA (2013)
15. Miles, M.B., Huberman, A.M., Saldana, J.: Qualitative data analysis: A methods
sourcebook. Sage, Thousand Oaks (2014)
16. Runeson, P., Höst, M., Rainer, A., Regnell, B.: Case Study Research in Software
Engineering: Guidelines and Examples. John Wiley & Sons, Inc., Hoboken, New Jersey
(2012)
17. Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in
software engineering. Empirical Software Engineering 14, 131-164 (2008)
18. Kohn, L.T.: Methods in Case Study Analysis. Technical Publication, Center for Studying
Health System Change (1997)
19. Yin, R.K.: Case study research : design and methods, 5th Edition. SAGE, London (2014)
20. Morgan, D.L.: Focus Groups as Qualitative Reearch. Sage Publications, Thousand Oaks,
California (1997)
21. Yin, R.K.: Qualitative Research from Start to Finish. The Guildford Press, New York, NY
(2016)