ChapterPDF Available

Role of Architects in Agile Organizations

Authors:

Abstract

Agile software development is broadly adopted in industry and works well for small-scale projects. In the context of large-scale development, however, there is a need for additional structure in the form of roles and practices, especially in the area of software architecture. In this chapter, we introduce the CAFFEA framework that defines a model for architecture governance. The framework defines three roles, i.e., chief architect, governance architect, and team architect, as well as a set of practices and responsibilities assigned to these roles. The CAFFEA framework has been developed and validated in close collaboration with several companies. © 2014 Springer International Publishing Switzerland. All rights reserved.
Chapter 4
Role of Architects in Agile Organizations
Antonio Martini, Lars Pareto, and Jan Bosch
Abstract Agile software development is broadly adopted in industry and works
well for small-scale projects. In the context of large-scale development, however,
there is a need for additional structure in the form of roles and practices, especially
in the area of software architecture. In this chapter, we introduce the CAFFEA
framework that defines a model for architecture governance. The framework
defines three roles, i.e., chief architect, governance architect, and team architect,
as well as a set of practices and responsibilities assigned to these roles. The
CAFFEA framework has been developed and validated in close collaboration
with several companies.
4.1 Background
Large software industries strive to make their development processes fast and more
responsive, minimizing the time between the identification of a customer need and
the delivery of a solution. Support for such goals is given by Agile software
development (ASD), which has been proven successful in small software projects
in the last decade [1].
Some frameworks including sets of practices have been proposed for
implementing Agile principles of which few are suitable for large projects (e.g.,
Scrum). However, a major gap in such frameworks is the lack of activities to
enhance agility in the task of developing and maintaining a reference architecture,
necessary for the development of portfolios of products sharing large amount of
software (e.g., platforms) [1,2]. This lack might lead to an underdeveloped or
quickly eroded architecture, which is needed to coordinate large software projects,
to enhance future development (future features), and to assure cost reduction
(through software reuse). The key need, for the companies, is to reach the ability
of Agile architecting and therefore to redefine the role of the software architects in
an Agile organization, which means understanding which activities related to
A. Martini (*) • L. Pareto • J. Bosch
Chalmers University of Technology, Gothenburg University, Gothenburg, Sweden
e-mail: antonio.martini@chalmers.se;Jan@JanBosch.com
©Springer International Publishing Switzerland 2014
J. Bosch (ed.), Continuous Software Engineering,
DOI 10.1007/978-3-319-11283-1_4
39
software architecture are necessary to be carried out and by whom. We have
therefore redefined the key roles for architects, with the responsibilities of archi-
tecture management that were missed in large Agile organizations: chief architect,
governance architect, and team architect.
Chief architects, responsible for the whole overall portfolio architecture, which
might include more products and more than one system.
Governance architects, responsible for areas of the architecture, related to single
products or systems or sub-systems, but not related to only one team.
Team architects, the usually most experienced developer in a team who have the
most knowledge about the architecture and support/lead the team on such area.
Different levels in the hierarchy are usually connected with the level of abstrac-
tion of the software architecture and design for which the architects are
accountable.
Such generic architecture roles might change from company to company, but
important questions are: What are they actually doing? What should they do? How
should they spend their time?
There are many activities suggested by the research community that need to be
done to develop and maintain a sound software architecture [312]. However, little
is said about who is responsible for which activities in a large Agile software
development organization. This gap is especially important where ASD is
employed, and the organization tends to avoid big time investments in upfront
design in favor of a shorter time to market.
In our previous studies, we have shown that the Agile teams need to communi-
cate and coordinate, and the need for strategic input is of utmost importance [13,
14]. Also, in [15] the authors introduce Agile in an SPL setting but report an open
issue as:
the management of architecture evolution and refactoring without sacrificing the principles
of agility and self-managed teams.
Leffingwell [16] introduced the concept of architecture runway as an
existing or planned infrastructure sufficient to allow incorporation of current and anticipate
requirements without excessive refactoring.
We will use such concept when we will explain the need for a Runway Team,
which in [16] has been identified as a Prototyping Team. Leffingwell also mentions
the possibility, for large systems, to have architects outside of the teams. However,
no specific roles and activities have been mentioned for the architects: on the
contrary, Leffingwell advocates the current impossibility of placing traditional
architect roles (e.g., system architects) present in many organizations (especially
in products with embedded software) into Agile frameworks. We have therefore
collected roles with specific responsibilities in a framework called CAFFEA (con-
tinuous architecture framework for embedded software and Agile).
40 A. Martini et al.
4.2 Research Design
We planned a multiple-case embedded case study involving seven sites in five large
software development companies. For confidentiality reasons, we will call the
companies A, B, C, D, and E. The main rationale for selecting the cases was that
they needed to be developing software product lines and they had adopted or at least
been in transition to ASD.
Companies A–D had extensive in-house embedded software development, while
company E was developing general-purpose software. The choice for including
company E was to compare the results with non-embedded software development.
We involved three different units within the same company, C, and we will refer to
them as C
1
,C
2
, and C
3
. We used this approach in order to assess the variance within
the same company. The companies studied were to have some years of experience
of ASD. The companies chosen were situated in the same geographical area
(Scandinavia) but were active on different international markets.
4.2.1 Cases Description
Company A is involved in the automotive industry. Part of the development is
carried out by suppliers and some by in-house teams following Scrum. The sur-
rounding organization follows a stage-gate release model for product development.
Business is driven by products for mass customization. The specific unit studied
provides a software platform for different products.
Company B is a manufacturer of recording devices. Teams work in parallel in
projects: some of the projects are more hardware oriented, while others are related
to the implementation of features developed on top of a specific Linux distribution.
The software involves in-house development with the integration of a substantial
amount of open-source components. Despite the Agile setup of the organization,
the iterations are quite long compared to the other companies involved in the study.
Company C is a manufacturer of telecommunication system product lines. Their
customers receive a platform and pay to unlock new features. The organization is
split into different units and then into cross-functional teams, most of which have
feature development roles. Most of the teams use their preferred variant of ASD
(often Scrum). Features were developed on top of a reference architecture, and the
main process consisted of a pre-study followed by few (ca. 3) sprint iterations. The
embedded cases studied slightly differed: C3 involved globally distributed teams
(Europe) while the other unit (C1 and C2) teams were co-located in the same city.
Company D is a manufacturer of a product line of devices for the control of
urban infrastructure. The organization is divided into teams working in parallel.
The organization has also adopted principles of software product line engineering,
such as the employment of a reference architecture.
4 Role of Architects in Agile Organizations 41
Company E is a company developing software for calculating optimized solu-
tions. The software is not deployed in embedded systems. The company has
employed ASD with teams working in parallel. The product is structured in a
platform entirely developed by E and a layer of customizable assets for the
customers to configure. E supports also a set of APIs for allowing development
on top of their software.
All the companies have one or more product lines and have adopted a
component-based software architecture, where they reuse a substantial part of the
system, such as several components or a shared platform. The language that is
mainly used is C and C++, with some parts of the system developed in Java and
Python. Company A uses a domain-specific language (DSL) to generate C code,
while company E uses a DSL for specifying rules to be converted into libraries. The
development at C
3
involves extensive XML.
4.2.2 Data Collection and Analysis
First we conducted a literature review to find the main state-of-the-art architecture
activities that need to be done to develop and maintain a sound software architec-
ture [312]. During the interactive workshops, we mapped such activities into roles
with the help of the informants. With such process we found which activities were
already covered by the Agile process employed at the companies and which ones
were not. The workshops were recorded and transcribed. The analysis was done
following an approach based on grounded theory [17], mixing inductive and
deductive techniques and using a tool for qualitative analysis, to keep track of the
links between the codes and the quotations they were grounded to. The overall
process is shown in Fig. 4.1.
Architecture ac vi es
Roles
Mapped to
Risk Management
Decisions and changes
Architectural knowledge
Monitoring the current
system
TA
GA
CA
Literature
Review
Dis lled
Providing
Documenta on
Fig. 4.1 CAFFEA framework and its components. The arrows show the analysis steps leading to
the results
42 A. Martini et al.
4.3 Roles and Activities
4.3.1 Chief Architect
The main role of the Chief Architect (CA) is to take high-level decisions and to
drive and support governance architects and teams in order to reach strategic goals
such as the development and maintenance of a sound architecture that would
support the business goals of the organization. In many cases such goals include
the support for multiple products or product lines, which assures multiple incoming
to the company. All of the cases that we have analyzed have such portfolio of
multiple customers. The main activities for the CA are described in the following,
grouped by the main areas.
4.3.1.1 Risk Management
The CA should drive the activities connected to risk management related to product
and architecture evolution. Such an activity is done usually on a 2–3-year scope.
The CA is usually not directly involved in the detailed development; however, in
order to take feasibility decisions, the CA needs to elicit the information about the
current status of the system from the governance architects and team architects by
the institution of a virtual architecture team. The presence of CAs in estimation is
usual, but the involvement of the GA and TA is not often applied, although many
informants reported that such roles are important for understanding the feasibility
of deliveries that require commitment. The virtual team should also involve Product
Owners since risks in terms of cost related to the architecture need to be matched
with the business model and goals of the company.
4.3.1.2 Managing Decisions and Changes
The CA takes decisions that affect more products, for example, tools and frame-
works (e.g., libraries) that need to be used cross-company, or high-level design
decisions (e.g., architectural styles, archetypes). Such decisions cannot be taken by
development teams alone but need to be taken by architects with high experience
and overall picture of the system.
Decisions on the adoption of frameworks, tools, and high-level design need to be
followed by decisions to support them by educating the development teams. Such
decisions involve the budget, so they need to be taken by (or in combination with)
product management.
4 Role of Architects in Agile Organizations 43
4.3.1.3 Pattern Distillation
The chief architect should participate in the learning activity with the governance
architects (described later) on distilling patterns. Such retrospective activity is not
done per feature, but it involves the architectural analysis of the system for a whole
product or product line. Its not a frequent activity, but its usually performed before
high-level patterns are chosen and product-wise (and therefore cost-impacting)
decisions need to be taken, for example, the starting of a complete new product
or its inclusion in the organization portfolio.
4.3.1.4 Providing Architecture Documentation (Communication
Output)
One of the main tasks for the CA should be to communicate the overall architecture
and the purpose of it. The best way, according to the informants, is to broadcast
such information in a visual way and in person. The mentioned means for this
activity are videos, “road shows,” and plenary sessions with all the consumers of the
architecture.
The CA needs to “order” education for the consumers after a decision is made to
change infrastructures, high-level design patterns, or processes. Its not likely that
the CA alone is able to take care of such education. For what concerns the
architecture, the CA should delegate most of the documentation creation and
education to the GAs.
4.3.1.5 Receiving Input About the Current Status of the System
(Communication Input)
According to the informants, the current communication practices lack good mech-
anisms for providing input to the CAs. In fact, CAs cannot go around the organi-
zation and talk to every consumer of the architecture. Input is fundamental for the
CA in order to take informed decision and to be aware of bad decisions taken
previously that need to be addressed (e.g., about tool chains not working as
expected). Some of the means for such communication input are:
Plenary sessions
• Questionnaires
• Forums
44 A. Martini et al.
4.3.2 Governance Architect
The Governance Architect (GA) is an intermediate role in the organization and the
key for scalability of the architecture development and maintenance in a large Agile
setting. Such role should function as a link among several teams that need to be
coordinated and supported when developing features within a (evolving) system
architecture. The main activities for the CA are described in the following, grouped
by the main areas.
4.3.2.1 Inter-features Architecting (Architecture Decision)
The GA should be responsible for decisions about inter-feature architecting. The
development of certain features might require adding design patterns or making
sure that the architecture is optimal not just for one feature but for more, for
example, to support features planned for the near future. This is something that
requires a wider view of the system architecture than the view focused on a single
feature typical of the Agile team. These decisions are usually taken considering also
high-level architecture. An example of such architecture is the production of a
meta-model for the understanding of the system components and their connectors
(relationships).
Besides the technical integration of the features, the GA needs to monitor and
drive an architecture that would minimize or make clear the inter-team effects or
the development of each team on the other. For this purpose, there is also the need
to define the boundaries for the teams, i.e., the critical points in the architecture in
which the risk is high for the team to interact with the others (see also documen-
tation [*]). GAs need to take architecture decisions involving at least 2 or more
teams, although the more teams are involved, the more coordination effort needs to
be spent on such activities.
The GA needs to take product-oriented decisions, but also to monitor and
support the team in following such decisions. With the focus of the team on the
features, someone needs to be able to “protect” the whole product, for example, in
terms of quality or pattern distillation (see CA [*]). In order to have such a mind-set
and information, there is the need to maintain a strong and iterative communication
link with the CA.
4.3.2.2 Architecting for Testability
An important point made by the informants is the need to architect to achieve
testability. The continuous integration team, responsible of the integration at (sub-)
system level, is a stakeholder of the team. Decisions about patterns and measure-
ment mechanisms to be embedded in the code for testing and quality management
need to be taken by the GAs.
4 Role of Architects in Agile Organizations 45
4.3.2.3 Risk Management
One of the main needs in a large organization is to balance the prioritization of
short-term and long-term goals. Such risk management activity is done on different
levels, and the GA should be a connector between the levels by participating in the
relative risk management activities. The GAs need to plan long-term product
development and architecture evolution with CAs and Product management
(High-level Product Owners), while they also need to be involved in the prioriti-
zation of features and architecture improvements at lower levels (closer to design
and implementation) with the TA and, if present, a lower-level Product Owner. In
two of the cases studied, belonging to the same large organization but being quite
disconnected sites, there was the presence of a so-called Operational Product
Owner, who was responsible for the feature prioritization directly with the team.
4.3.2.4 Controlling Erosion
The GAs are the main drivers for monitoring and reacting to architecting erosion.
Such activity is quite complex, and the shift to Agile has also led to a shift from a
document-centric architecture monitor to the more iterative and in-person partici-
pation of the architects. Controlling erosion needs also the support within the team
by the TAs, but the information provided by the team lead to decisions that have to
be taken with large perspective (see [*] decisions). On the other hand, the CA
cannot monitor the whole system to understand what is happening (see [*] CA
controlling erosion).
Consequently, the GAs are needed to lead architecture erosion control by
coordinating sub-activities, such as:
Retrospective sessions: the topic of the retrospective sessions (e.g., postmortem
analysis) suggested by Agile practices is team performance. From such activity,
the team learns how to be more efficient. However, specific sessions or
sub-sessions need to be organized and led by the GAs, in order to focus also
the retrospective towards architecture. Such sessions might be organized at the
end of:
• Sprint
• Project
Code reviews: a usual practice is to peer-review the code. However, such
practice seems to be quite a costly one and is not always possible because of
the time constraints due to the low number of GAs present in the organization
compared to the amount of code and the time invested in the other activities
listed also in this section. For this reason, the GA should be able to order
automation for such code-reviews.
Order automation: to facilitate the monitor of erosion, the inconsistencies with
respect to the desired architecture or architecture principles and requirements
46 A. Martini et al.
need to be visualized by analyzing the system. The GAs should be able to
“order” (allocate the budget and resources) automation to an infrastructure
team (or third party vendor) that could put in place a tool for visualizing and
monitoring some erosion.
4.3.2.5 Architecture Education for the Teams (Communication
Output)
Although the chief architect is responsible for the production and communication of
high-level architecture guidelines and decisions, the governance architect should be
responsible for the capillary spreading of knowledge to all the teams. In such role,
the governance architect, in the organization, needs to have enough overview and
architecture knowledge to express the important concepts.
The main means for communicating architecture and educating the teams is
maintaining architecture documentation (inter-team, inter-feature): although the
transition to ASD has questioned the need for the maintenance of superfluous
architecture documentation, part of it is critical and needs to be very well created
and kept updated. Such documentation is the one concerning architectural patterns
and requirements that are shared by more than one team. As an example, there
might be patterns that involve temporal requirements (e.g., involving the access to a
centralized database) that need to be expressed (and verified) in order to keep the
teams from hindering each other. An important requirement for the architecture
documentation is that it should be well navigable by the team members, meaning
that the link between the documents and the actual source code should be well
maintained.
4.3.2.6 Knowledge of the Status of the System (Communication Input)
Another important point in architecture communication is that the architect should
be able to have knowledge of the status of (part of) the system. The main problem
that would arise without having such knowledge would be to have architects who
define and evolve a system architecture that is completely different from what is
actually implemented. Such situation obviously hinders the decisions when risk
analysis is done for the augmentation of the products, e.g., the implementation of
new features. It is of utmost importance, therefore, that the architects continuously
and iteratively check the actual status of the system. For doing so, there might be
tools, but, according also to the Agile view of the importance of face-to-face
communication, architects should organize and participate in events including
experienced members of the teams. Such events are in line with the continuous
learning principle and with the practice, suggested in some Agile frameworks, of
conducting retrospective. In order for the architecture to be correctly communicated
to the governance architect, architecture retrospectives (i.e., retrospective sessions
4 Role of Architects in Agile Organizations 47
focused on reflecting on the current status of the architecture) have been found to be
a good means.
4.3.3 Team Architect
The Team Architect (TA) is the actual executor of the architecture in the FT. The
role is important for the capillary spreading of architecture knowledge and for
collecting input from the FT. However, this role is not a dedicated role in the team
but rather a set of responsibilities that need to be mapped to a member of the team,
who might change from time to time.
4.3.3.1 Risk Management
As mentioned in the previous sections, the TA should participate in risk manage-
ment activities with the CA, GA, and P in order to represent the interest of the teams
in feasibility discussions.
4.3.3.2 Managing Decisions and Changes
The TA leads detailed architecture decisions on a detailed design level.
4.3.3.3 Providing Architecture Documentation (Communication
Output)
The TA takes input from the GA and should be able to communicate the architec-
ture to support the FT education packages created by the GAs.
4.3.3.4 Monitoring the Current Status of the System (Communication
Input)
The TA is responsible for collecting FTs input, to be communicated to the
GA. This way, the TA lifts decisions for architecture evolution and decisions that
might affect other FTs.
4.3.4 Gap in the Current Practices
The main groups of practices that have been recognized as currently missing are:
48 A. Martini et al.
Architecture risk management (prioritization of short-term and long-term tasks)
Architecture decision and changes
Communication of architecture, composed by two-way directions:
Providing architecture directions
Monitoring the current status of the system
The combination of the previous components leads to the identification of a
major gap in the current organizations, the lack of architecture technical debt
management. Such phenomenon is recently being studied from different angles
[18,19] and is concerned with the organizations taking risk-informed architecture
decisions about which architecture changes, such as refactoring or evolution (con-
sidered the actual debt if not done) need to be conducted for having an acceptable
ratio of investment/effort. The lack of architecture technical debt management
might quickly lead the companies to crisis points where adding new business
value to the software product lines (new features or new products) incur in major
efforts, paralyzing the long-term responsiveness [20].
Conclusion
Agile software development is broadly adopted in industry and works well for
small-scale projects. In the context of large-scale development, however,
there is a need for additional structure in the form of roles and practices,
especially in the area of software architecture.
In this chapter, we introduced the CAFFEA framework that defines a
model for architecture governance. The framework defines three roles, i.e.,
chief architect, governance architect, and team architect, as well as a set of
practices and responsibilities assigned to these roles. For the chief architect,
these practices include risk management, managing decisions and changes,
pattern distillation, providing architecture documentation, and receiving
input about the current status of the system. The governance architect is
concerned with inter-features architecting (architecture decisions),
architecting for testability, risk management, controlling erosion, architecture
education for the teams, and maintaining knowledge of the status of the
system. Finally, the team architect focuses on risk management, managing
decisions and changes, providing architecture documentation, and monitoring
the current status of the system.
The CAFFEA framework has been developed and validated in close
collaboration with several companies. Currently, most of the companies in
the Software Center have adopted or are in the process of adopting the
CAFFEA framework.
Although the framework has brought significant benefit for the companies,
there still are some open items, as presented earlier in the chapter, and in
future work, we aim to address those as well as validate the framework with
other companies in other industries.
4 Role of Architects in Agile Organizations 49
References
1. Dingsøyr, T., Nerur, S., Balijepally, V., Moe, N.B.: A decade of agile methodologies: towards
explaining agile software development. J. Syst. Softw. 85(6), 1213–1221 (2012)
2. Daneva, M., van der Veen, E., Amrit, C., Ghaisas, S., Sikkel, K., Kumar, R., Ajmeri, N.,
Ramteerthkar, U., Wieringa, R.: Agile requirements prioritization in large-scale outsourced
system projects: an empirical study. J. Syst. Softw. 86(5), 1333–1353 (2013)
3. Kruchten, P.: What do software architects really do? J. Syst. Softw. 81(12), 2413–2416 (2008)
4. Tang, A., Avgeriou, P., Jansen, A., Capilla, R., Ali Babar, M.: A comparative study of
architecture knowledge management tools. J. Syst. Softw. 83(3), 352–370 (2010)
5. Pareto, L., Eriksson, P., Ehnebom, S.: Architectural descriptions as boundary objects in system
and design work. Model Driven Eng. Lang. Syst. 6395, 406–419 (2010)
6. Williams, B.J., Carver, J.C.: Characterizing software architecture changes: a systematic
review. Inf. Softw. Technol. 52(1), 31–51 (2010)
7. de Silva, L., Balasubramaniam, D.: Controlling software architecture erosion: a survey. J. Syst.
Softw. 85(1), 132–151 (2012)
8. Qumer, A.: Defining an integrated agile governance for large agile software development
environments. In: Concas, G., Damiani, E., Scotto, M., Succi, G. (eds.) Agile Processes in
Software Engineering and Extreme Programming, pp. 157–160. Springer, Berlin, Heidelberg
(2007)
9. Drury, M., Conboy, K., Power, K.: Obstacles to decision making in Agile software develop-
ment teams. J. Syst. Softw. 85(6), 1239–1254 (2012)
10. Zimmermann, O., Miksovic, C., Ku
¨ster, J.M.: Reference architecture, metamodel, and model-
ing principles for architectural knowledge management in information technology services.
J. Syst. Softw. 85(9), 2014–2033 (2012)
11. Unphon, H., Dittrich, Y.: Software architecture awareness in long-term software product
evolution. J. Syst. Softw. 83(11), 2211–2226 (2010)
12. McAvoy, J., Butler, T.: The impact of the Abilene Paradox on double-loop learning in an agile
team. Inf. Softw. Technol. 49(6), 552–563 (2007)
13. Martini, A., Pareto, L., Bosch, J.: Enablers and inhibitors for speed with reuse. In: Proceedings
of the 16th International Software Product Line Conference, vol. 1, pp. 116–125. New York,
USA (2012)
14. Martini, A., Pareto, L., Bosch, J.: Communication factors for speed and reuse in large-scale
agile software development. In: Proceedings of the 17th International Software Product Line
Conference, pp. 42–51. New York, USA, (2013)
15. Bosch, J., Bosch-Sijtsema, P.M.: Introducing agile customer-centered development in a legacy
software product line. Softw. Pract. Exp. 41(8), 871–882 (2011)
16. Leffingwell, D.: Scaling Software Agility: Best Practices for Large Enterprises. Pearson
Education (2007)
17. Strauss A., Corbin, J.M.: Grounded Theory in Practice. SAGE (1997)
18. Tom, E., Aurum, A., Vidgen, R.: An exploration of technical debt. J. Syst. Softw. 86(6), 1498–
1516 (2013)
19. Nord, R.L., Ozkaya, I., Kruchten, P., Gonzalez-Rojas, M.: In search of a metric for managing
architectural technical debt. In: 2012 Joint Working IEEE/IFIP Conference on Software
Architecture (WICSA) and European Conference on Software Architecture (ECSA), pp. 91–
100 (2012)
20. Martini, A., Bosch, J., Chaudron, M.: Architecture technical debt: understanding causes and a
qualitative model. Presented at 40th Euromicro Conference on Software Engineering and
Advanced Applications, Verona, pp. 85–92 (2014). doi:10.1109/SEAA.2014.65
50 A. Martini et al.
... Time wise, the TD metaphor is considered useful for estimating if a technical solution is actually sub-optimal or might be optimal from the business point of view. Such risk management practice is also very important in the everyday work of software architects, as mentioned in Kruchten (Kruchten, 2008) and Martini et al. (Martini et al., 2014). Although research has been done on how to take decision on architecture development (such as ATAM, ALMA, etc. (Babar and Gorton, 2004)), there is no empirical research about how sub-optimal architectural solutions (ATD) are accumulated over time and how they can be continuously managed. ...
Chapter
This chapter presents an example that provides an object lesson in understanding the real problem. Data scientists simply must engage with “customers” in their languages and talk through the apparent problems to discover the real ones. They found that many people make this more difficult than it needs to be. The authors consider three points to think through. First is data scientists' choice of the word “customer.” They found that viewing decision‐makers as customers humanizes them. A customer can come from a different part of the organization, be way up in the hierarchy, or be based in another part of the world. Second, “in their languages.” Third is “discovering the real problems.” It is a rare decision‐maker indeed who can articulate the real problem in the first try. Articulating problems is hard work, and there are so many special cases, external factors, and political considerations that may cloud one's head.
... This role might exist, but was not found to be interacting with Scrum teams in particular. However, a role similar to the "governance architect" who would be responsible for scalability of agile architecting in a large setting as an intermediate between chief architect and team [29] could be considered the "external" architect in this scenario.  The "internal and external architects" scenario is similar to the setup discussed by Fowler [17] and Abrahamsson et al. who define two types of architects in agile projects: architects who focus on the big decision, who are facing requirements and act as external coordinators ("architectus reloadus"), and architects who focus on internal team communications, mentoring, troubleshooting and code ("architectus oryzus") [2]. ...
Conference Paper
Context: Previous research investigated how to approach architecting in agile projects (e.g., in terms of processes and practices), but the role that architects play in Scrum is still not well understood. Objective: We aim at capturing scenarios of how architects (or those taking on architecture-related tasks) are involved in Scrum. Furthermore, we aim at identifying how those taking on the role of the architect interact with other roles in Scrum. Method: We conducted a multiple case study and interviews with practitioners from six Dutch software organizations. Results: We identified three generic scenarios of architects in Scrum (“internal architect”, “external architect”, “internal and external architects”). We found that how architects interact with other roles in Scrum heavily depends on the Product Owner role. Conclusions: Some of our results are not in line with recommended practices in the Scrum Guide. Our findings support those who take on architecture-related tasks in preparing for Scrum-like projects.
... Time wise, the TD metaphor is considered useful for estimating if a technical solution is actually sub-optimal or might be optimal from the business point of view. Such risk management practice is also very important in the everyday work of software architects, as mentioned in Kruchten (Kruchten, 2008) and Martini et al. (Martini et al., 2014). Although research has been done on how to take decision on architecture development (such as ATAM, ALMA, etc. (Babar and Gorton, 2004)), there is no empirical research about how sub-optimal architectural solutions (ATD) are accumulated over time and how they can be continuously managed. ...
Article
The introduction of Agile Software Development in large companies developing embedded systems aims at shortening the time-to-market for new business value for the customers. However, evidences show how system engineering needs to be integrated in the process in order to avoid the degradation of architectural system qualities. Such phenomenon has recently been identified with a financial metaphor, Architectural Technical Debt, which maps sub-optimal architectural solutions to taking a debt, and its costly impact into an interest to be paid in the long term. Through a multiple case study in 5 companies, we show what are the current practical challenges related to the long-term accumulation of Architectural Technical Debt, and how system engineers need to track and manage such phenomenon in order to avoid costly impact such as development crises and costly re-design.
... This role might exist, but was not found to be interacting with Scrum teams in particular. However, a role similar to the "governance architect" who would be responsible for scalability of agile architecting in a large setting as an intermediate between chief architect and team [29] could be considered the "external" architect in this scenario.  The "internal and external architects" scenario is similar to the setup discussed by Fowler [17] and Abrahamsson et al. who define two types of architects in agile projects: architects who focus on the big decision, who are facing requirements and act as external coordinators ("architectus reloadus"), and architects who focus on internal team communications, mentoring, troubleshooting and code ("architectus oryzus") [2]. ...
Conference Paper
Full-text available
Context: Previous research investigated how to approach architecting in agile projects (e.g., in terms of processes and practices), but the role that architects play in Scrum is still not well understood. Objective: We aim at capturing scenarios of how architects (or those taking on architecture-related tasks) are involved in Scrum. Furthermore, we aim at identifying how those taking on the role of the architect interact with other roles in Scrum. Method: We conducted a multiple case study and interviews with practitioners from six Dutch software organizations. Results: We identified three generic scenarios of architects in Scrum (" internal architect " , " external architect " , " internal and external architects "). We found that how architects interact with other roles in Scrum heavily depends on the Product Owner role. Conclusions: Some of our results are not in line with recommended practices in the Scrum Guide. Our findings support those who take on architecture-related tasks in preparing for Scrum-like projects.
... Such risk management practice is also very important in the everyday work of software architects, as mentioned in Kructhen [15] and Martini et al. [16]. Although research has been done on how to take decision on architecture development (such as ATAM, ALMA, etc. [17]), there is no empirical research about how sub-optimal architectural solutions (ATD) can be continuously managed. ...
Conference Paper
Full-text available
Architectural Technical Debt is regarded as sub-optimal architectural solutions that need to be refactored in order to avoid the payment of a costly interest in the future. However, decisions on if and when to refactor architecture are extremely important and difficult to take, since changing software at the architectural level is quite expensive. Therefore it is important, for software organizations, to have methods and tools that aid architects and managers to understand if Architecture Technical Debt will generate a costly and growing interest to be paid or not. Current knowledge, especially empirically developed and evaluated, is quite scarce. In this paper we developed and evaluated a method, AnaConDebt, by analyzing, together with several practitioners, 12 existing cases of Architecture Debt in 6 companies. The method has been refined several times in order to be useful and effective in practice. We also report the evaluation of the method with a final case, for which we present anonymized results and subsequent refactoring decisions. The method consists of several components that need to be analyzed, combining the theoretical Technical Debt framework and the practical experience of the practitioners, in order to identify the key factors involved in the growth of interest. The output of the method shows summarized indicators that visualizes the factors in a useful way for the stakeholders. This analysis aids the practitioners in deciding on if and when to refactor Architectural Technical Debt items. The method has been evaluated and has been proven useful to support the architects into systematically analyze and decide upon a case.
... Time wise, the TD metaphor is considered useful for estimating if a technical solution is actually sub-optimal or might be optimal from the business point of view. Such risk management practice is also very important in the everyday work of software architects, as mentioned in Kructhen [17] and Martini et al. [18]. Although research has been done on how to take decisions on architecture development (such as ATAM and ALMA [19]), there is no empirical research about how sub-optimal architectural solutions (ATD) are accumulated over time and how they can be continuously managed. ...
Article
Journal version of "Architecture Technical Debt: Understanding Causes and a Qualitative Model". Context - A known problem in large software companies is to balance the prioritization of short-term with long-term feature delivery speed. Specifically, Architecture Technical Debt is regarded as sub-optimal architectural solutions taken to deliver fast that might hinder future feature development, which, in turn, would hinder agility. Objective – This paper aims at improving software management by shedding light on the current factors responsible for the accumulation of Architectural Technical Debt and to understand how it evolves over time. Method - We conducted an exploratory multiple-case embedded case study in 7 sites at 5 large companies. We evaluated the results with additional cross-company interviews and an in-depth, company-specific case study in which we initially evaluate factors and models. Results - We compiled a taxonomy of the factors and their influence in the accumulation of Architectural Technical Debt, and we provide two qualitative models of how the debt is accumulated and refactored over time in the studied companies. We also list a set of exploratory propositions on possible refactoring strategies that can be useful as insights for practitioners and as hypothesis for further research. Conclusion – Several factors cause constant and unavoidable accumulation of Architecture Technical Debt, which leads to development crises. Refactorings are often overlooked in prioritization and they are often triggered by development crises, in a reactive fashion. Some of the factors are manageable, while others are external to the companies. ATD needs to be made visible, in order to postpone the crises according to the strategic goals of the companies. There is a need for practices and automated tools to proactively manage ATD.
... Fig. 1 describes our conceptual model. Such conceptual model is based on previous data collected of the same authors of this paper at the same companies involved in the study ( [7]). ...
Conference Paper
Full-text available
Architectural Technical Debt is a metaphor for representing sub-optimal architectural solutions that might cause an interest, in terms of effort or quality, to be paid by the organization in the long run. Such metaphor has been regarded as useful for communicating risks of suboptimal solutions between technical and non-technical stakeholders. However, it's fundamental to understand the information needs of the involved stakeholders in order to produce technical debt measurements that would allow proper communication and informed prioritization. We have investigated, through a combination of interviews, observations and a survey, what key information is needed by agile product owners and software architects in order to prioritize the refactoring of risky architectural technical debt items with respect to feature development.
Chapter
Agile software development is already widespread in the industry and has worked well in small project organizations. However, large-scale development in the form of roles and workflows is needed for large-scale development, especially in the area of software architecture. Software architecture, and more broadly Enterprise architecture, is critical to achieving the expected success of software projects and products. However, during agile software development, for example, Scrum, no positions are created in the team for Software architects. In this paper, we present the results of the analysis of frameworks that consider the position of Software architect, such as the SAFe method, where several positions of architects are created such as Enterprise architect, Chief architect, Administrative architect, Team architect, as well as frameworks in which the position of Software architect, for example, Scrum, is not officially created. In the second case, we examined the intersection of the activities of the Software architect with the Product owner. The results of the research show that Software architects play an important role during agile software development, despite the fact that in some agile approaches they are hidden in other position.
Chapter
Software architecture design, when performed in context of agile software development (ASD), sometimes referred to as “agile architecting,” promotes the emerging and incremental design of the architectural artifact in a sense of avoiding “big design upfront” (BDUF). This chapter presents the Agile Modeling Process for Logical Architectures (AMPLA) method, an approach for supporting the emergence of a candidate (logical) architecture, rather than BDUF, the architecture in an early phase. The architecture then emerges throughout agile iterations, where AMPLA plays a key contribution for providing traceability between models, from the business need to service specifications, ranging from design stages to deployment, hence covering a software development life cycle (SDLC).
Conference Paper
Full-text available
NB You can find the journal version of this article published as "investigating technical debt accumulation and refactoring over time: a multiple case study" also available in this website from the same authors. Abstract A known problem in large software companies is to balance the prioritization of short-term with long-term responsiveness. Specifically, architecture violations (Architecture Technical Debt) taken to deliver fast might hinder future feature development, which would hinder agility. We conducted a multiple-case embedded case study in 7 sites at 5 large companies in order to shed light on the current causes for the accumulation of Architectural Technical Debt that causes effort. We provide a taxonomy of the factors and their influence in the accumulation of debt, and we provide a qualitative model of how the debt is accumulated and recovered over time.
Conference Paper
Full-text available
An open issue in industry is the combination of software reuse in the context of large scale Agile Software Development. The speed offered by Agile Software Development is needed for short time to market, while reuse strategies such as Software Product Line Engineering are needed for long-term productivity, efficiency, and profit. The paper investigates, through a survey, communication factors affecting both speed and reuse in 3 large companies developing embedded systems and employing Agile Software Development and Software Product Line Engineering. Our results include a prioritized list of communication related factors obtained by statistical analysis and the recognition and spread of the factors in the companies. We have recognized 5 interfaces with the Agile development team that need to be improved: system engineers (architects), product management, distributed teams, inter-project teams and sales unit. Few factors (involving inter-project communication) depend on the business drivers for the company. We also reveal that Agile teams need strategic and architectural inputs in order to be implanted in a large company employing Software Product Line Engineering. Academic and industrial training as well as different tactics for co-location would improve the communication skills of engineers. There is also a need for solutions, in the reference architecture, for fostering Agile Software Development: the goal is the combination of the focus on customer value of the teams, reusability, system requirements and avoidance of organizational dependencies.
Article
Full-text available
An open issue in industry is software reuse in the context of large scale Agile product development. The speed offered by agile practices is needed to hit the market, while reuse is needed for long-term productivity, efficiency, and profit. The paper presents an empirical investigation of factors influencing speed and reuse in three large product developing organizations seeking to implement Agile practices. The paper identifies, through a multiple case study with 3 organizations, 114 business-, process-, organizational-, architecture-, knowledge- and communication factors with positive or negative influences on reuse, speed or both. Contributions are a categorized inventory of influencing factors, a display for organizing factors for the purpose of process improvement work, and a list of key improvement areas to address when implementing reuse in organizations striving to become more Agile. Categories identified include good factors with positive influences on reuse or speed, harmful factors with negative influences, and complex factors involving inverse or ambiguous relationships. Key improvement areas in the studied organizations are intra-organizational communication practices, reuse awareness and practices, architectural integration and variability management. Results are intended to support process improvement work in the direction of Agile product development. Feedback on results from the studied organizations has been that the inventory captures current situations, and is useful for software process improvement work.
Article
To be successful, a software architect-or a software architecture team, collectively-must strike a delicate balance between an external focus-both outwards: Listening to customers, users, watching technology, developing a long-term vision, and inwards: driving the development teams-and an internal, reflective focus: spending time to make the right design choices, validating them, and documenting them. Teams that stray too far away from this metastable equilibrium fall into some traps that we describe as antipatterns of software architecture teams.
Book
Grounded theory methodology and procedure have become one of the most influential modes of carrying out qualitative research when generating theory is a principle aim of the researcher. This volume presents a series of readings that emphasize different aspects of grounded theory methodology and methods. The selections are written by former students of the late Anselm Strauss and have been chosen for their accessibility and range.
Conference Paper
Practices designed to expedite the delivery of stakeholder value can paradoxically lead to unexpected rework costs that ultimately degrade the flow of value over time. This is especially observable when features are developed based on immediate value, while dependencies that may slow down future development efforts are neglected. The technical debt metaphor conceptualizes this tradeoff between short-term and long-term value: taking shortcuts to optimize the delivery of features in the short term incurs debt, analogous to financial debt, that must be paid off later to optimize long-term success. In this paper, we describe taking an architecture-focused and measurement-based approach to develop a metric that assists in strategically managing technical debt. Such an approach can be used to optimize the cost of development over time while continuing to deliver value to the customer. We demonstrate our approach by describing its application to an ongoing system development effort.
Article
ContextWhilst technical debt is considered to be detrimental to the long term success of software development, it appears to be poorly understood in academic literature. The absence of a clear definition and model for technical debt exacerbates the challenge of its identification and adequate management, thus preventing the realisation of technical debt's utility as a conceptual and technical communication device.Objective To make a critical examination of technical debt and consolidate understanding of the nature of technical debt and its implications for software development.Method An exploratory case study technique that involves multivocal literature review, supplemented by interviews with software practitioners and academics to establish the boundaries of the technical debt phenomenon.ResultA key outcome of this research is the creation of a theoretical framework that provides a holistic view of technical debt comprising a set of technical debts dimensions, attributes, precedents and outcomes, as well as the phenomenon itself and a taxonomy that describes and encompasses different forms of the technical debt phenomenon.Conclusion The proposed framework provides a useful approach to understanding the overall phenomenon of technical debt for practical purposes. Future research should incorporate empirical studies to validate heuristics and techniques that will assist practitioners in their management of technical debt.
Chapter
Capturing and sharing design knowledge such as architectural decisions is becoming increasingly important in firms providing professional Information Technology (IT) services such as enterprise application development and strategic outsourcing. Methods, models, and tools supporting explicit knowledge management strategies have been proposed in recent years; however, several challenges remain unaddressed. In this paper, we extend our previous work to overcome these challenges and to satisfy the requirements of an additional user group, presales architects that are responsible for IT service solution proposals. In strategic outsourcing, such solution proposals require complex, contractually relevant design decisions concerning many different resources such as IT infrastructures, people, and real estate. To support both presales and project architects, we define a common reference architecture and a decision process-oriented metamodel. We also present a tool implementation of these concepts and discuss their application to outsourcing proposals and application development projects. Finally, we establish twelve decision modeling principles and practices that capture the practical experience gained and lessons learned during the application of our decision modeling concepts to both proposal development and architecture design work on projects.
Article
The obstacles facing decision making in Agile development are critical yet poorly understood. This research examines decisions made across four stages of the iteration cycle: Iteration Planning, Iteration Execution, Iteration Review and Iteration Retrospective. A mixed method approach was employed, whereby a focus group was initially conducted with 43 Agile developers and managers to determine decisions made at different points of the iteration cycle. Subsequently, six illustrative mini cases were purposefully conducted as examples of the six obstacles identified in these focus groups. This included interviews with 18 individuals in Agile projects from five different organizations: a global consulting organization, a multinational communications company, two multinational software development companies, and a large museum organization. This research contributes to Agile software development literature by analyzing decisions made during the iteration cycle and identifying six key obstacles to these decisions. Results indicate the six decision obstacles are unwillingness to commit to decisions; conflicting priorities; unstable resource availability; and lack of: implementation; ownership; empowerment. These six decision obstacles are mapped to descriptive decision making principles to demonstrate where the obstacles affect the decision process. The effects of these obstacles include a lack of longer-term, strategic focus for decisions, an ever-growing backlog of delayed work from previous iterations, and a lack of team engagement.