No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Ambient Awareness of Build Status in Collocated Software Teams
Abstract
We describe the evaluation of a build awareness system that assists agile software development teams to understand current build status and who is responsible for any build breakages. The system uses ambient awareness technologies, providing a separate, easily perceived communication channel distinct from standard team workflow. Multiple system configurations and behaviours were evaluated. An evaluation of the system showed that, while there was no significant change in the proportion of build breakages, the overall number of builds increased substantially, and the duration of broken builds decreased. Team members also reported an increased sense of awareness of, and responsibility for, broken builds and some noted the system dramatically changed their perception of the build process making them more cognisant of broken builds.
... This is complemented by running Automated Static Analysis Tools (ASATs), e.g., FindBugs, Checkstyle, or JSHint as part of the CI can augment the dynamic testing phase [6]. In addition to these checks of code and system quality, CI is said to improve release frequency and predictability [7], increase developer productivity [8] and improve communication [9], hence reducing the time-to-market and allowing users to benefit from continuous updates of their software. Continuous Delivery (CD) is the development practice that enables frequent releases by help of a CI process [10]. ...
... S 138 Q1. 7 Which Scrum metrics do you usually collect? M 128 Q1. 8 Which is the main reason why a "done" user story comes back to "in-progress"? S 130 Q1. 9 Do you consider non-functional requirements as definition of "done" of a user story? S 130 Q1.10 Which kind of non-functional requirements do you consider as definition of "done" of a user story? ...
Continuous Delivery is an agile software development practice in which developers frequently integrate changes into the main development line and produce releases of their software. An automated Continuous Integration infrastructure builds and tests these changes. Claimed advantages of CD include early discovery of (integration) errors, reduced cycle time, and better adoption of coding standards and guidelines. This paper reports on a study in which we surveyed 152 developers of a large financial organization (ING Nederland), and investigated how they adopt a Continuous Integration and delivery pipeline during their development activities. In our study, we focus on topics related to managing technical debt, as well as test automation practices. The survey results shed light on the adoption of some agile methods in practice, and sometimes confirm, while in other cases, confute common wisdom and results obtained in other studies. For example, we found that refactoring tends to be performed together with other development activities, technical debt is almost always "self-admitted", developers timely document source code, and assure the quality of their product through extensive automated testing, with a third of respondents dedicating more than 50% of their time to do testing activities.
... In distributed software development, awareness has been discussed widely. Many organizations have started to implement agile software development methodology as their software process in distributed environments as alternative to traditional software methodology approach[15],[16]. The awareness of agile practitioners is required for distributed environments to achieve the software project goal[17]. ...
... Spatial distance and temporal distance does not prevent agile methodology implemented properly in distributed development as long as software practitioners aware of project, practices, task, artifacts and development teams. In Malaysia context, there is lack of awareness amongst agile practitioners reported in implementing agile practices[16],[18]. Lack of awareness among agile practitioners also has been investigated in developing countries such as Vietnam[19],[20], Indonesia[21], and Sri Lanka[22]. ...
Agile software development methodology has been implemented by software industries over a decade ago and well accepted in the practitioner community. However, there is limited understanding on how agile practitioners aware towards implementation of agile practices in software development. Lack of awareness will lead to misunderstandings among agile practitioners and misuse the agile practices. In order to understand the awareness of agile practices, this paper aims to investigate the factors that affect awareness of agile practitioners in implementing agile practices. A systematic literature review (SLR)was conducted in order to classify and define the factors of awareness in agile software development methodology. The review was based on papers between 2002 and December 2014 from seven electronic databases. The relevant papers were included 20 journal articles, 24 conference papers,16 book chapters, 9 workshop papers. Consequently, 69 papers were identified that closely related with awareness in agile software development methodology. From the thematic analysis, 13 factors were classified from 42 elements. Based on the review result, understanding the influential factors on the awareness of agile practices will provide benefit to researchers and agile practitioners.
... This opens the gates towards more empirical work in this direction. In [19] an ambient system was designed and implemented for determining build status awareness among agile teams. An industrial case study was conducted to test the approach prototype. ...
... Moreover, it is also an interesting aspect to investigate that why only communication is the preferred aspect to be studied for agile teams. [19] Lesser time between broken builds Increase in the number of builds Conference Agile service nodes (ASN) to increase awareness among globally distributed agile teams [9] Decreases delay time ...
The dynamic nature of agile methods and volatility of requirements claim frequent team collaboration.
This collaboration encompasses various socio-technical aspects like knowledge sharing, communication etc. In this paper our aim is twofold. First, we identified the most relevant socio-technical aspects of requirements-driven collaboration; and second, we reviewed those aspects in literature for agile teams.
We conducted a questionnaire based survey with agile practitioners for identification of the most relevant socio-technical aspects of requirements-driven collaboration. The survey results revealed that practitioners consider communication and awareness the most relevant socio-technical aspects of requirements-driven collaboration among many. Furthermore, the review results showed that less research has been done on social aspects (e.g. awareness) involved in agile requirements engineering activities. Hence this study provides implications in terms of more empirical investigations to further the knowledge on the social aspects of requirements-driven collaboration among agile teams.
... Kwan et al. [7] studied developers from IBM to find out whether team composition and coordination may have an impact on software build success. Dawns et al. [4] studied the operation of a build team to evaluate an automatic build management tool that enforces the build failure handling process. Philips et al. [13] studied software building teams at Microsoft and found that most challenges are on the social aspects of the team. ...
Open-source software (OSS) often needs to be built by roles who are not contributors. Despite the prevalence of build issues experienced by non-contributors, there is a lack of studies on this topic. This paper presents a study aimed at understanding the symptoms and causes of build issues experienced by non-contributors. The findings highlight certain build issues that are challenging to resolve and underscore the importance of understanding non-contributors' behavior. This work lays the foundation for further research aimed at enhancing the non-contributors' experience in dealing with build issues.
... Moreover, head-mounted AR displays (AR-HMDs), in particular, are well-suited to provide information in the visual periphery of the user and thus create an ambient awareness of the augmented information (Schmalstieg and Hollerer 2016). By using the peripheral instead of focal attention of the user (Cadiz et al. 2002), less attention is required (Downs et al. 2012), and fewer cognitive resources are used, so that the user has access to more cognitive capacities for the execution of the actual team tasks. Altogether, we assume that AR technology allows for superimposing information about the teamwork process state in an ambient manner, which generates ambient awareness of the process state and in turn improves the task state awareness of spatially dispersed teams with the result of an optimized temporal coordination (Fig. 3). ...
Augmented reality (AR) has been used in a large variety of scenarios from assembly support over gaming to collaborative work. Teamwork is a specific type of collaborative work and emerges in numerous everyday contexts. It is characterized (in addition to other aspects) by specific needs in terms of time-related synchronization of team members to enable them to coordinate the team task. This is specifically relevant in situations where team members are spatially dispersed, such that direct communication (orally and visually) is not possible. These restrictions reduce task-related communication and coordination, which are key for successful teamwork. AR has the potential to enable coordination in spatially dispersed teamwork by enhancing each team members’ view with information of the current status of the team task and additional feedback if needed, all adapted to the specific needs of the team members. As a key element of AR, this digital information is registered to the team member’s individual perspective on the work object in question. This chapter will first give an introduction to collaborative work, teams, teamwork, and taskwork followed by a broad overview of existing AR-based team-supportive methods. We will first present a taxonomy for the development and implementation of AR-based support systems and give a short example on its usage in the context of a user-centered design followed by a review of the current research conducted by the authors of the chapter and a discussion of future work.
... This can be achieved by running testing and analysis tools, reducing the cost and risk of delivering defective changes [25]. Other collateral positive effects introduced by CI in industrial environments are the improvement in developer communication [23] and the increase of their productivity [75]. Consequently, CI has become increasingly popular in software development of both, industrial and OSS projects [46,112]. ...
Continuous Integration (CI) is a software development practice that enables developers to build software more reliably and quickly. Most organizations have started adopting CI, however, only a few of them achieve the expected benefits. The reason is that living up to the recommended practices (called principles) of CI is not easy and developers tend to follow anti-patterns, which are ineffective solutions to recurrent problems. Anti-patterns violate principles and lower the effectiveness of CI. In this dissertation, we characterize the problem of anti-patterns to implement solutions that help developers follow principles. We start with classifying the anti-patterns encountered by developers in practice and identifying their four root causes, which are (i) the poor knowledge of the prerequisites for adopting CI, (ii) the difficulty of inspecting build failure logs, (iii) the presence of bad configurations, and (iv) the wrong usage of a CI process. While only better coaching in CI can efficiently remove the former, we implement several approaches to address the other causes. To improve the understandability of build failure logs, we develop Bart, a tool that produces summaries for the most common build failure types. To identify anti-patterns caused by configuration smells that developer should remove, we propose CD-Linter, a semantic linter for CI/CD configuration files. We implement CI-Odor, an automated reporting tool that leverages information from repository and build history, to monitor the wrong adoption of CI over time. The results of multiple empirical studies conducted with professional developers show that the proposed approaches are effective at identifying and removing the aforementioned causes of anti-patterns and, consequently, at enforcing a principle-driven continuous integration practice.
... Thereby the information content is implicitly presented in form of auditory, haptic, olfactory, or visual cues (Streitz, Rö cker, Prante, Stenzel, & van Alphen, 2003), like for instance a light signal that indicates by color whether a room is free or occupied. Providing information in such a way requires low attentional demands and minor cognitive load, since the information can be perceived 'ata-glance' (Cadiz et al., 2002;Downs, Plimmer, & Hosking, 2012). Additionally, the user's awareness of the information is maintained without overwhelming or distracting them (Cadiz et al., 2002) from the actual focus of work. ...
One key predictor of team performance and productivity is the teams’ ability to coordinate its subtasks as precisely as possible over time. Thereby, the quality of the temporal coordination in teams is highly dependent on several cognitive team skills, like for instance the ability to build a precise and stable mental model of the situation (shared situation awareness) and of the team task process (task state awareness/ TSA). To support these cognitive skills in teams, various methods and trainings already exist. However, considering teams that work de-located, the prerequisites and therefore the requirements for team supportive methods change. This work presents the results of an empirical usability and user experience (UX) study of an interface for an Augmented Reality-based assistance system for spatially dispersed teams, called Ambient Awareness Tool (AAT). The AAT interface consists of graphical information about the teamwork process and aims at enhancing the TSA of the team, thereby supporting its temporal coordination. Within the framework of a participatory design process we conducted a two-part expert-user-study, comprising a laboratory experiment for the evaluation of the interface usability and a follow-up online survey to additionally investigate the UX. By means of the usability scores we first inferred three interface configuration clusters. A subsequent UX evaluation then allowed us to designate the configuration cluster with the highest usability and UX scores. As a result, we defined one interface configuration that will be investigated concerning its actual impact on the temporal coordination of spatially dispersed teams in a further study.
Get temporal free access via share link: https://authors.elsevier.com/a/1cbRq4xrDwKHOM
... Furthermore, we suggest that this information should be presented in such a way, that it induces minor cognitive load, requires low attentional demands and is perceivable at a glance. Hence, we decided to develop an assistance system with an ambient (or peripheral) awareness user interface [10,11], called Ambient Awareness Tool (AAT), since such interfaces are able to maintain the user's awareness of the This work was supported by the DFG (Deutsche Forschungsgemeinschaft/ German Research Foundation). Grant number KL2207/7-1 and WE5408/3-1. ...
One major determinant for the performance and productivity of teams is the temporal coordination of interdependent sub-tasks within the teamwork process. The accuracy of the temporal coordination significantly depends on whether the team member share a precise mental model of the teamwork process, which encompasses the current process state and the desired outcome. The more the assumptions of the individual team members about the current process state deviate, the less precise the temporal coordination of the team is. Looking at spatially dispersed teams, this challenge becomes even more complicated, since such teams have no naturally existing common cues, from which they can derive corresponding process relevant information. Therefore, this work presents a user-centered design approach to develop an Augmented Reality-based assistance system for the Microsoft HoloLens 1. Its goal is to support the temporal coordination of spatially dispersed teams by utilizing a method that can present information in an at-a-glance manner and helps the user to remain aware of the information without being overwhelmed or distracted by it, denoted Ambient Awareness. In order to develop a system with the highest possible team-and task-technology fit, we aggregated usability and user experience scores in a two-part user study with n = 22 experts in the sense of a user-centered design process. To this end, we first determined the usability in a laboratory experimental setting. On the basis of the aggregated data, we identified three essential interface configuration clusters in the first step, which were evaluated subsequently according to their user experience in an online assessment. This two-step user-centered evaluation design allowed us to derive a user interface configuration that provides the highest team-and task-technology-fit under the given application-specific conditions.
... Entsprechend diesen Anforderungen, haben wir ein Assistenzsystem mit AR-basiertem Ambient Awareness-Interface (Cadiz et al. 2002;Downs et al. 2012) entwickelt (Ambient Awareness Tool/ AAT). So können benötigte Zusatzinformationen in der visuellen Peripherie eingeblendet werden, ohne dass die/der NutzerIn von der eigentlichen Aufgabe abgelenkt wird (Cadiz et al. 2002). ...
Die zeitliche Koordination räumlich verteilter Teams gestaltet sich durch die räumliche Trennung häufig suboptimal. Zur Unterstützung solcher Teams wird ein AR-basiertes Assistenzsystem vorgeschlagen, dass durch die Erhöhung der Task State Awareness die zeitliche Koordination verbessert. Der Beitrag zeigt die Ergebnisse der nutzerzentrierten Entwicklung in Form einer zweiteiligen Usability und User Experience Evaluation auf.
... Step Monitoring teams that have to work in different locations (Streitz et al., 2003). To support this kind of team task state awareness, ambient presentations, which have low attentional requirements, can be used to present dynamic information in an at-a-glance manner (Downs et al., 2012). Such awareness system can be broadly defined as systems intended to help people construct and maintain an awareness of each other's activities, context or status, even when these individuals are not co-located (Markopoulos et al., 2009, p. V). ...
This work presents a prototypic system that uses augmented reality technology to support temporal coordination of spatial dispersed production team members. This system is used to investigate the potential benefit of augmented reality for temporal coordination. We will present a study design as well as various research questions to be considered in future work.
... For communicating the current status of the build, Downs et al. [5] proposed the use of ambient awareness technologies. They have observed by providing a separate, easily perceived communication channel distinct from standard team worklow for communicating build status information, the total number of builds increased substantially, and the duration of broken builds decreased. ...
Automated builds, which may pass or fail, provide feedback to a development team about changes to the codebase. A passing build indicates that the change compiles cleanly and tests (continue to) pass. A failing (a.k.a., broken) build indicates that there are issues that require attention. Without a closer analysis of the nature of build outcome data, practitioners and researchers are likely to make two critical assumptions: (1) build results are not noisy; however, passing builds may contain failing or skipped jobs that are actively or passively ignored; and (2) builds are equal; however, builds vary in terms of the number of jobs and configurations.
To investigate the degree to which these assumptions about build breakage hold, we perform an empirical study of 3.7 million build jobs spanning 1,276 open source projects. We find that: (1) 12% of passing builds have an actively ignored failure; (2) 9% of builds have a misleading or incorrect outcome on average; and (3) at least 44% of the broken builds contain passing jobs, i.e., the breakage is local to a subset of build variants. Like other software archives, build data is noisy and complex. Analysis of build data requires nuance.
... McIntosh et al. [26] carried out an empirical study on the efforts developers spend on the building configurations of projects. Downs et al. [10] proposed an approach to remind developers in a development team about the building status of the project. On the study of building errors, Seo et al. [36] and Hassan et al. [15,17] carried out empirical studies to categorize build errors. ...
Advancements in software build tools such as Maven reduce build management effort, but developers still need specialized knowledge and long time to maintain build scripts and resolve build failures. More recent build tools such as Gradle give developers greater extent of customization flexibility, but can be even more difficult to maintain. According to the TravisTorrent dataset of open-source software continuous integration, 22% of code commits include changes in build script files to maintain build scripts or to resolve build failures. Automated program repair techniques have great potential to reduce cost of resolving software failures, but the existing techniques mostly focus on repairing source code so that they cannot directly help resolving software build failures. To address this limitation, we propose HireBuild: Hi story-Driven Rep air of Build Scripts, the first approach to automatic patch generation for build scripts, using fix patterns automatically generated from existing build script fixes and recommending fix patterns based on build log similarity. From TravisTorrent dataset, we extracted 175 build failures and their corresponding fixes which revise Gradle build scripts. Among these 175 build failures, we used the 135 earlier build fixes for automatic fix-pattern generation and the more recent 40 build failures (fixes) for evaluation of our approach. Our experiment shows that our approach can fix 11 of 24 reproducible build failures, or 45% of the reproducible build failures, within comparable time of manual fixes.
... This can be achieved by running testing and analysis tools, reducing the cost and risk of delivering defective changes [4]. Other collateral positive effects introduced by CI in industrial environments are the improvement in developer communication [7] and the increase of their productivity [8]. Consequently, CI has become increasingly popular in software development of both, industrial and OSS projects [6], [9]. ...
Continuous Integration (CI) and Continuous Delivery (CD) are widespread in both industrial and open-source software (OSS) projects. Recent research characterized build failures in CI and identified factors potentially correlated to them. However, most observations and findings of previous work are exclusively based on OSS projects or data from a single industrial organization. This paper provides a first attempt to compare the CI processes and occurrences of build failures in 349 Java OSS projects and 418 projects from a financial organization, ING Nederland. Through the analysis of 34,182 failing builds (26% of the total number of observed builds), we derived a taxonomy of failures that affect the observed CI processes. Using cluster analysis, we observed that in some cases OSS and ING projects share similar build failure patterns (e.g., few compilation failures as compared to frequent testing failures), while in other cases completely different patterns emerge. In short, we explain how OSS and ING CI processes exhibit commonalities, yet are substantially different in their design and in the failures they report.
... The difference between the practices is discussed in [1]. Downs et al. [7] recognized the importance of communication when practicing continuous integration. With increased communication, they discovered quantitative and qualitative improvements in a case study. ...
Continuous delivery is a software development discipline in which software can be released to production at any time. The proposed dissertation aims to understand the problems that emerge when adopting continuous delivery and find solutions to those problems. The goal is reached by performing a systematic literature review followed by case studies. Root cause analysis is utilized as a data collection method in some of the case studies. The contribution of the dissertation will be a theory on how continuous delivery can be adopted.
... One source extensively discusses and evaluates differences in notification methods, and concludes that a combination of multiple communication channels can have a great impact on awareness of and responsiveness to broken builds (Downs et al., 2012). ...
Continuous Integration (CI) enables developers to detect defects early and thus reduce lead time. However, the high frequency and long duration of executing CI have a detrimental effect on this practice. Existing studies have focused on using CI outcome predictors to reduce frequency. Since there is no reported project using predictive CI, it is difficult to evaluate its economic impact. This research aims to investigate predictive CI from a process perspective, including why and when to adopt predictors, what predictors to be used, and how to practice predictive CI in real projects. We innovatively employ Software Process Simulation to simulate a predictive CI process with a Discrete-Event Simulation (DES) model and conduct simulation-based experiments. We develop the Rollback-based Identification of Defective Commits (RIDEC) method to account for the negative effects of false predictions in simulations. Experimental results show that: 1) using predictive CI generally improves the effectiveness of CI, reducing time costs by up to 36.8% and the average waiting time before executing CI by 90.5%; 2) the time-saving varies across projects, with higher commit frequency projects benefiting more; and 3) predictor performance does not strongly correlate with time savings, but the precision of both failed and passed predictions should be paid more attention. Simulation-based evaluation helps identify overlooked aspects in existing research. Predictive CI saves time and resources, but improved prediction performance has limited cost-saving benefits. The primary value of predictive CI lies in providing accurate and quick feedback to developers, aligning with the goal of CI.
Context: Exploratory testing plays an important role in the continuous integration and delivery pipelines of large-scale software systems, but a holistic and structured approach is needed to realize efficient and effective exploratory testing.
Objective: This paper seeks to address the need for a structured and reliable approach by providing a tangible model, supporting practitioners in the industry to optimize exploratory testing in each individual case.
Method: The reported study includes interviews, group interviews and workshops with representatives from six companies, all multi-national organizations with more than 2,000 employees.
Results: The ExET model (Excellence in Exploratory Testing) is presented. It is shown that the ExET model allows companies to identify and visualize strengths and improvement areas. The model is based on a set of key factors that have been shown to enable efficient and effective exploratory testing of large-scale software systems, grouped into four themes: “The testers’ knowledge, experience and personality”, “Purpose and scope”, “Ways of working” and “Recording and reporting”.
Conclusions: The validation of the ExET model showed that the model is novel, actionable and useful in practice, showing companies what they should prioritize in order to enable efficient and effective exploratory testing in their organization.
Continuous experimentation (CE) refers to a group of practices used by software companies to rapidly assess the usage, value and performance of deployed software using data collected from customers and the deployed system. Despite its increasing popularity in the development of web-facing applications, CE has not been discussed in the development process of business-to-business (B2B) mission-critical systems. We investigated in a case study the use of CE practices within several products, teams and areas inside Ericsson. By observing the CE practices of different teams, we were able to identify the key activities in four main areas and inductively derive an experimentation process, the HURRIER process, that addresses the deployment of experiments with customers in the B2B and with mission-critical systems. We illustrate this process with a case study in the development of a large mission-critical functionality in the Long Term Evolution (4G) product. In this case study, the HURRIER process is not only used to validate the value delivered by the solution but to increase the quality and the confidence from both the customers and the R&D organization in the deployed solution. Additionally, we discuss the challenges, opportunities and lessons learned from applying CE and the HURRIER process in B2B mission-critical systems.
Measuring properties of software systems, organizations, and processes has much more to it than meets the eye. Numbers and quantities are at the center of it, but that is far from everything. Software measures (or metrics, as some call them) exist in a context of a measurement program, which involves the technology used to measure, store, process, and visualize data, as well as people who make decisions based on the data and software engineers who ensure that the data can be trusted. z
Software developers in big and medium-size companies are working with millions of lines of code in their codebases. Assuring the quality of this code has shifted from simple defect management to proactive assurance of internal code quality. Although static code analysis and code reviews have been at the forefront of research and practice in this area, code reviews are still an effort-intensive and interpretation-prone activity. The aim of this research is to support code reviews by automatically recognizing company-specific code guidelines violations in large-scale, industrial source code. In our action research project, we constructed a machine-learning-based tool for code analysis where software developers and architects in big and medium-sized companies can use a few examples of source code lines violating code/design guidelines (up to 700 lines of code) to train decision-tree classifiers to find similar violations in their codebases (up to 3 million lines of code). Our action research project consisted of (i) understanding the challenges of two large software development companies, (ii) applying the machine-learning-based tool to detect violations of Sun’s and Google’s coding conventions in the code of three large open source projects implemented in Java, (iii) evaluating the tool on evolving industrial codebase, and (iv) finding the best learning strategies to reduce the cost of training the classifiers. We were able to achieve the average accuracy of over 99% and the average F-score of 0.80 for open source projects when using ca. 40K lines for training the tool. We obtained a similar average F-score of 0.78 for the industrial code but this time using only up to 700 lines of code as a training dataset. Finally, we observed the tool performed visibly better for the rules requiring to understand a single line of code or the context of a few lines (often allowing to reach the F-score of 0.90 or higher). Based on these results, we could observe that this approach can provide modern software development companies with the ability to use examples to teach an algorithm to recognize violations of code/design guidelines and thus increase the number of reviews conducted before the product release. This, in turn, leads to the increased quality of the final software.
Continuous Integration is a software practice where developers integrate frequently, at least daily. While this is an ostensibly simple concept, it does leave ample room for interpretation: what is it the developers integrate with, what happens when they do, and what happens before they do? These are all open questions with regards to the details of how one implements the practice of continuous integration, and it is conceivable that not all such implementations in the industry are alike. In this paper we show through a literature review that there are differences in how the practice of continuous integration is interpreted and implemented from case to case. Based on these findings we propose a descriptive model for documenting and thereby better understanding implementations of the continuous integration practice and their differences. The application of the model to an industry software development project is then described in an illustrative case study.
Measurement programs in large software development organizations contain a large number of indicators, base and derived measures to monitor products, processes and projects. The diversity and the number of these measures causes the measurement programs to become large, combining multiple needs, measurement tools and organizational goals. For the measurement program to effectively support organization’s goals, it should be scalable, automated, standardized and flexible – i.e. robust. In this paper we present a method for assessing the robustness of measurement programs. The method is based on the robustness model which has been developed in collaboration between seven companies and a university. The purpose of the method is to support the companies to optimize the value obtained from the measurement programs and their cost. We evaluated the method at the seven companies and the results from applying the method to each company quantified the robustness of their programs, reflecting the real-world status of the programs and pinpointed strengths and improvements of the programs.
In many ways, digitalization has confirmed that the success of new technologies and innovations is fully realized only when these are effectively adopted and integrated into the daily practices of a company. During the last decade, we have seen how the speed of technology developments only accelerates, and there are numerous examples of innovations that have fundamentally changed businesses as well as everyday life for the customers they serve. In the manufacturing industry, automation is key for improving efficiency as well as for increasing safety. In the automotive domain, electrification of cars and autonomous drive technologies are replacing mechanical power and human intervention. In the telecom domain, seamless connectivity and digital infrastructures allow systems to adapt and respond within the blink of an eye. In the security and surveillance domain, intelligent technologies provide organizations with the ability to detect, respond, and mitigate potential risks and threats with an accuracy and preciseness we could only dream about a few decades ago. While these are only a few examples, they reflect how digital technologies, and the ever-increasing access to data, are transforming businesses to an extent that we have only seen the beginnings of.
Context: Agile methods have become mainstream even in large-scale systems engineering companies that need to accommodate different development cycles of hardware and software. For such companies, requirements engineering is an essential activity that involves upfront and detailed analysis which can be at odds with agile development methods.
Objective: This paper presents a multiple case study with seven large-scale systems companies, reporting their challenges, together with best practices from industry. We also analyse literature about two popular large-scale agile frameworks, SAFe® and LeSS, to derive potential solutions for the challenges.
Method: Our results are based on 20 qualitative interviews, five focus groups, and eight cross-company workshops which we used to both collect and validate our results.
Results: We found 24 challenges which we grouped in six themes, then mapped to solutions from SAFe®, LeSS, and our companies, when available.
Conclusion: In this way, we contribute a comprehensive overview of RE challenges in relation to large-scale agile system development, evaluate the degree to which they have been addressed, and outline research gaps. We expect these results to be useful for practitioners who are responsible for designing processes, methods, or tools for large scale agile development as well as guidance for researchers.
Large software companies need to support continuous and fast delivery of customer value both in the short and long term. However, this can be hindered if both the evolution and maintenance of existing systems are hampered by Technical Debt. Although a lot of theoretical work on Technical Debt has been produced recently, its practical management lacks empirical studies. In this paper, we investigate the state of practice in several companies to understand what the cost of managing TD is, what tools are used to track TD, and how a tracking process is introduced in practice. We combined two phases: a survey involving 226 respondents from 15 organizations and an in-depth multiple case study in three organizations including 13 interviews and 79 Technical Debt issues. We selected the organizations where Technical Debt was better tracked in order to distill best practices. We found that the development time dedicated to managing Technical Debt is substantial (an average of 25% of the overall development), but mostly not systematic: only a few participants (26%) use a tool, and only 7.2% methodically track Technical Debt. We found that the most used and effective tools are currently backlogs and static analyzers. By studying the approaches in the companies participating in the case study, we report how companies start tracking Technical Debt and what the initial benefits and challenges are. Finally, we propose a Strategic Adoption Model for the introduction of tracking Technical Debt in software organizations.
Even though there were many forerunners, the most widespread reference to Continuous Integration as a method was put forward by Grady Booch in 1990 [44] (page 209). In the 1995 book Microsoft Secrets, Cusumano and Selby interviewed 38 Microsoft employees to document how the world’s leading software provider was managing its own software development [80]. One of the key practices found was the Daily Build concept. In the popular literature, this was described as everyone needed to check in their code and build the product at the end of the workday. In tight connection to the build, some smoke tests were run to ensure that the individual contributions could be integrated. If the build was broken, the person who broke it had to fix his/her code before going home. There was also modern folklore mentioning that the breaker of the build had to wear a funny hat for the remainder of the day.
The term artificial intelligence (AI) triggers many things in terms of its inherent meaning and potential. The notion of a machine with the same level of intellect as a human or even far exceeding it is enthralling and scary at the same time. Several science fiction movies build on the HAL 9000 or Terminator theme of artificial intelligence bent on controlling or even exterminating humankind.
Development of high-quality complex software, in particular in modern embedded and cyber-physical systems, requires careful attention to the software architecture and design in order to achieve the desired quality attributes. Generally speaking, the evolution in software development methods during the last decade, towards more agile practices with short iterations and early feedback, has focused more on implementation and validation activities than architectural design. It is sometimes argued, even, that the concept of architecture is obsolete in modern software development. However, architectural decisions still have a significant impact on software quality, including crucial aspects like performance, safety, and security. Moreover, although architecture can, and should, evolve over time, it does so at a slow pace compared to implementation changes, meaning that the architecture impacts how quickly new functionality can be implemented in response to changed market needs. Thus, for any long-lived systems, but in particular for systems where for example safety assurance is critical, there is definitely a need to document and reason about architecture. Architectural documentation no longer plays the role of a static, a priori, specification for developers to follow but should rather be viewed as an artifact that continuously evolves together with the implementation.
Agile software development is increasingly adopted by companies evolving and maintaining software products to support better planning and tracking the realization of user stories and features. While convincing success stories help to further spread the adoption of Agile, mechatronics-driven companies need guidance to implement Agile for non-software teams. In this comparative case study of three companies from the Nordic region, we systematically investigate expectations and challenges from scaling Agile in organizations dealing with mechatronics development by conducting on-site workshops and surveys. Our findings show that all companies have already successfully implemented Agile in their software teams. The expected main benefit of successfully scaling agile development is a faster time-to-market product development; however, the two main challenges are: (a) An inflexible test environment that inhibits fast feedback to changed or added features, and (b) the existing organizational structure including the company’s mind-set that needs to be opened-up for agile principles.
Artificial intelligence (AI) and machine learning (ML) are increasingly broadly adopted in industry, However, based on well over a dozen case studies, we have learned that deploying industry-strength, production quality ML models in systems proves to be challenging. Companies experience challenges related to data quality, design methods and processes, performance of models as well as deployment and compliance. We learned that a new, structured engineering approach is required to construct and evolve systems that contain ML/DL components. In this chapter, we provide a conceptualization of the typical evolution patterns that companies experience when employing ML as well as an overview of the key problems experienced by the companies that we have studied. The main contribution of the chapter is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions and an overview of open items that need to be addressed by the research community at large.
Background: Profiling software development projects, in order to compare them, find similar sub-projects or sets of activities, helps to monitor changes in software processes. Since we lack objective measures for profiling or hashing, researchers often fall back on manual assessments.
Objective: The goal of our study is to define an objective and intuitive measure of similarity between software development projects based on software defect-inflow profiles.
Method: We defined a measure of project similarity called SimSAX which is based on segmentation of defect-inflow profiles, coding them into strings (sequences of symbols) and comparing these strings to find so-called motifs. We use simulations to find and calibrate the parameters of the measure. The objects in the simulations are two different large industry projects for which we know the similarity a priori, based on the input from industry experts. Finally, we apply the measure to find similarities between five industrial and six open source projects.
Results: Our results show that the measure provides the most accurate simulated results when the compared motifs are long (32 or more weeks) and we use an alphabet of 5 or more symbols. The measure provides the possibility to calibrate for each industrial case, thus allowing to optimize the method for finding specific patterns in project similarity.
Conclusions: We conclude that our proposed measure provides a good approximation for project similarity. The industrial evaluation showed that it can provide a good starting point for finding similar periods in software development projects.
In model-based development projects, models at different abstraction levels capture different aspects of a software system, e.g., specification or design. Inconsistencies between these models can cause inefficient and incorrect development. A tool-based framework to assist developers creating and maintaining models conforming to different languages (i.e. heterogeneous models) and consistency between them is not only important but also much needed in practice. In this work, we focus on assisting developers bringing about multi-view consistency in the context of agile model-based development, through frequent, lightweight consistency checks across views and between heterogeneous models. The checks are lightweight in the sense that they are easy to create, edit, use and maintain, and since they find inconsistencies but do not attempt to automatically resolve them. With respect to ease of use, we explicitly separate the two main concerns in defining consistency checks, being (i) which modelling elements across heterogeneous models should be consistent with each other and (ii) what constitutes consistency between them. We assess the feasibility and illustrate the potential usefulness of our consistency checking approach, from an industrial agile model-based development point-of-view, through a proof-of-concept implementation on a sample project leveraging models expressed in SysML and Simulink. A continuous integration pipeline hosts the initial definition and subsequent execution of consistency checks, it is also the place where the user can view results of consistency checks and reconfigure them.
Agile software development is well-known for its focus on close customer collaboration and customer feedback. In emphasizing flexibility, efficiency and speed, agile practices have led to a paradigm shift in how software is developed. However, while agile practices have succeeded in involving the customer in the development cycle, there is an urgent need to learn from customer usage of software also after delivering and deployment of the software product. The concept of continuous deployment, i.e. the ability to deliver software functionality frequently to customers and subsequently, the ability to continuously learn from real-time customer usage of software, has become attractive to companies realizing the potential in having even shorter feedback loops. However, the transition towards continuous deployment involves a number of barriers. This paper presents a multiple-case study in which we explore barriers associated with the transition towards continuous deployment. Based on interviews at four different software development companies we present key barriers in this transition as well as actions that need to be taken to address these.
Software development companies are increasingly aiming to become data-driven by trying to continuously experiment with the products used by their customers. Although familiar with the competitive edge that the A/B testing technology delivers, they seldom succeed in evolving and adopting the methodology. In this paper, and based on an exhaustive and collaborative case study research in a large software-intense company with highly developed experimentation culture, we present the evolution process of moving from ad-hoc customer data analysis towards continuous controlled experimentation at scale. Our main contribution is the “Experimentation Evolution Model” in which we detail three phases of evolution: technical, organizational and business evolution. With our contribution, we aim to provide guidance to practitioners on how to develop and scale continuous experimentation in software organizations with the purpose of becoming data-driven at scale.
A systematic review of 459 Ambient Displays, reported in 410 publications between 1996 and 2016 was used as the basis for an analysis of the high-level design features associated with the technology. An analysis of these displays considers three main aspects: the modalities used to display information, the physical form of the displays and the level of implemented interaction. The paper provides a longitudinal overview of the various forms of Ambient Displays over a twenty-year period. This allows for the provision of a comprehensive timeline of past work in the field of Ambient Display and establishes a sound basis for further reviews or studies related to this technology.
Continuous integration (CI) systems automate the compilation, building, and testing of software. Despite CI rising as a big success story in automated software engineering, it has received almost no attention from the research community. For example, how widely is CI used in practice, and what are some costs and benefits associated with CI? Without answering such questions, developers, tool builders, and researchers make decisions based on folklore instead of data. In this paper, we use three complementary methods to study the usage of CI in open-source projects. To understand which CI systems developers use, we analyzed 34,544 open-source projects from GitHub. To understand how developers use CI, we analyzed 1,529,291 builds from the most commonly used CI system. To understand why projects use or do not use CI, we surveyed 442 developers. With this data, we answered several key questions related to the usage, costs, and benefits of CI. Among our results, we show evidence that supports the claim that CI helps projects release more often, that CI is widely adopted by the most popular projects, as well as finding that the overall percentage of projects using CI continues to grow, making it important and timely to focus more research on CI.
This paper contributes long-term findings on a custom ambient display solution (Ambient Surfaces) in the informative workspace of co-located agile (Scrum) software development teams. The study is based on the premise that displaying relevant information in a team's informative workspace is beneficial, as it readily provides information of common interest. However, existing research shows a lack of experience with software visualization tools in commercial settings. To our current knowledge, there is no study which examines ambient displays in Scrum teams with a longitudinal approach in order to address challenges in awareness and informal communication. A mixed-methods field study, which began in February 2014, is currently being conducted in cooperation with a medium-sized software company near Hamburg, Germany. Our current findings, which the following addresses, support the value of the Ambient Surfaces as a beneficial tool for Scrum teams.
Microenterprises (companies with less than 10 employees) are the dominating form of organizations in Europe. Unfortunately, many approaches to improve software development processes based on measurement are not tailored for such small companies. This poster proposes a measurement infrastructure that is developed with the goal to support microenterprises in measuring their process, product, and usage of the developed software to provide feedback to the entire development team. What the here presented tool wants to propose is to automate not only the data collection, but to be consequent in the rest of the feedback loop: to setup the interpretation and visualization of the data that, once this is done, no more intervention is needed.
While efficient testing arrangements are the key for software companies that are striving for continuous integration, most companies struggle with arranging these highly complex and interconnected testing activities. There is often a lack of an adequate overview of companies’ end-to-end testing activities, which tend to lead to problems such as double work, slow feedback loops, too many issues found during post-development, disconnected organizations, and unpredictable release schedules. We report from a multiple-case study in which we explore current testing arrangements at five different software development sites. The outcome of the study is a visualization technique of the testing activities involved from unit and component level to product and release level that support the identification of improvement areas. This model for visualizing the end-to-end testing activities for a system has been used to visualize these five cases and has been validated empirically.
This book illustrates how goal-oriented, automated measurement can be used to create Lean organizations and to facilitate the development of Lean software, while also demonstrating the practical implementation of Lean software development by combining tried and trusted tools. In order to be successful, a Lean orientation of software development has to go hand in hand with a company's overall business strategy. To achieve this, two interrelated aspects require special attention: measurement and experience management. In this book, Janes and Succi provide the necessary knowledge to establish "Lean software company thinking," while also exploiting the latest approaches to software measurement. A comprehensive, company-wide measurement approach is exactly what companies need in order to align their activities to the demands of their stakeholders, to their business strategy, etc. With the automatic, non-invasive measurement approach proposed in this book, even small and medium-sized enterprises that do not have the resources to introduce heavyweight processes will be able to make their software development processes considerably more Lean. The book is divided into three parts. Part I, "Motivation for Lean Software Development," explains just what "Lean Production" means, why it can be advantageous to apply Lean concepts to software engineering, and which existing approaches are best suited to achieving this. Part II, "The Pillars of Lean Software Development," presents the tools needed to achieve Lean software development: Non-invasive Measurement, the Goal Question Metric approach, and the Experience Factory. Finally, Part III, "Lean Software Development in Action," shows how different tools can be combined to enable Lean Thinking in software development. The book primarily addresses the needs of all those working in the field of software engineering who want to understand how to establish an efficient and effective software development process. This group includes developers, managers, and students pursuing an M.Sc. degree in software engineering. © Springer-Verlag Berlin Heidelberg 2014. All rights are reserved.
Agile methods [1] are becoming popular in the software industry. In agile software development projects, it is imperative
that all software written by each developer integrates properly into the entire project. To this end, most agile teams adopt
Continuous Integration (CI). CI is the practice of automatically compiling, deploying and testing the entire codebase against a suite of prewritten
tests. This occurs after any change to the codebase, usually multiple times per day.
When integration is finished, it is important for the developers to become aware of the result so that any problems can be
immediately fixed. Undetected bugs can cause further problems as other developers may synchronize with a broken version of
the codebase, and this may result in increased effort required to fix the problem and delays in integrating their changes
to the latest build. Thus, awareness of the build status is essential, especially immediately after submitting new code to
the codebase.
Current configuration management systems promote workspaces that isolate developers from each other. This isolation is both good and bad It is good, because developers make their changes without any interference from changes made concurrently by other developers. It is bad, because not knowing which artifacts are changing in parallel regularly leads to problems when changes are promoted from workspaces into a central configuration management repository. Overcoming the bad isolation, while retaining the good isolation, is a matter of raising awareness among developers, an issue traditionally ignored by the discipline of configuration management. To fill this void, we have developed Palantir, a novel workspace awareness tool that complements existing configuration management systems by providing developers with insight into other workspaces. In particular, the tool informs a developer of which other developers change which other artifacts, calculates a simple measure of severity of those changes, and graphically displays the information in a configurable and generally non-obtrusive manner. To illustrate the use of Palantir, we demonstrate how it integrates with two representative configuration management systems.
A decentralized variant of continuous integration can be defined in terms of two fundamental rules: (1) Developers’ access to add contributions to the development version at any time, and (2) developers’ obligation to integrate their own contributions properly. Decentralized, continuous integration may adapt well to organizations where developers work relatively independently, as in many open source projects. The approach raises the issue of how these organizations can exercise central control, as attaining the benefits of continuous integration requires that contributions are useful and satisfy the project’s definition of successful integration. We have investigated the use of continuous integration in FreeBSD and Mozilla. Our account of quality assurance activities in the two open source projects distinguishes between Mintzberg’s three complementary forms of central control: Standardization and control of work output, work processes, and worker skills. Our study indicates that two major challenges face projects using decentralized, continuous integration: (1) To balance the access to add contributions against the need to stabilize and mature the software prior to a release, and (2) to consider the developers’ limited time and resources when interpreting their obligation to integrate their changes properly.
Global software teams face challenges when collaborating over long distances, such as communicating changes in the project. During a four-month case study at IBM Ottawa Software Lab we observed the collaboration patterns of a multi-site development project team. In this period, we inspected project documentation, interviewed team leaders, attended project meetings, and spoke with developers to identify problems originated by the lack of awareness of changes related to the implementation of work items. Our observations show (1) that organizational culture has an effect on how developers are made aware; (2) that communication-based social networks revolving around particular work items are dynamic throughout development, and therefore awareness needs to be maintained in infrastructures of work; and (3) that information overload and communication breakdowns contributed to the generation of a broken integration build. We discuss these breakdowns in communication and implications for the design of collaboration tools that could mitigate these problems.
Success in software development dictates that the right information reaches the right people. In the development process, information flows among developers, managers, and customers. The information is represented in a multitude of sources: customer requirements, application domains, product specifications, development processes, schedules, budgets, project status reports, and task priorities. Unfortunately, in the transmission of information, vital tidbits are filtered away, made inaccessible, or withheld from the stakeholders. Software visualization systems have traditionally focused on the complexities of the relations and detailed structure of the software artifacts. Recently, attention has been given to visualizing software artifacts from the perspective of supporting teams in coordinating efforts. In this paper, we describe the nature of this information source and provide design guidelines for developing ambient software visualizations in the workplace. In particular, we describe how developers can better understand more about project management, recall and perform tasks in their personal work flow, and coordinate project state that is continually changing.
Developers ought to maintain awareness of the status of a software project. However, there are very few recorded best practices for defining what constitutes relevant status information and the appropriate modalities for communicating this information. In this industry case study, we conducted in-depth interviews with members of an agile development team. We found that their daily work practices, while well-defined and regular, were heavily influenced by the status information they integrated from a number of sources. In particular, continuous integration builds had a substantial effect on the team's workflow. Based on our findings, we provide a set of guidelines for build monitoring systems which encourage collective and individual responsibility while working within the established team environment.
Open-source software development projects are almost always collaborative and distributed. Despite the difficulties imposed by distance, these projects have managed to produce large, complex, and successful systems. However, there is still little known about how open-source teams manage their collaboration. In this paper we look at one aspect of this issue: how distributed developers maintain group awareness. We interviewed developers, read project communication, and looked at project artifacts from three successful open source projects. We found that distributed developers do need to maintain awareness of one another, and that they maintain both a general awareness of the entire team and more detailed knowledge of people that they plan to work with. Although there are several sources of information, this awareness is maintained primarily through text-based communication (mailing lists and chat systems). These textual channels have several characteristics that help to support the maintenance of awareness, as long as developers are committed to reading the lists and to making their project communication public.
to continuously run regression tests in the background, providing rapid feedback about test failures as source code is edited. It is intended to reduce the time and energy required to keep code well-tested and prevent regression errors from persisting uncaught for long periods of time. This paper reports on a controlled human experiment to evaluate whether students using continuous testing are more successful in completing programming assignments. We also summarize users' subjective impressions and discuss why the results may generalize.
Automated continuous integration and the Ambient Orb
- M Swanson
M. Swanson, " Automated continuous integration and the Ambient Orb, " 2004. [Online]. Available: http://blogs.msdn.com/mswanson/articles/169058.aspx. [Accessed: 30-Jan-2012].
Continuous integration
- M Fowler
M. Fowler, " Continuous integration, " 2006. [Online]. Available: http://www.martinfowler.com/articles/continuousIntegration.html. [Accessed: 30-Jan-2012].
Brian the Build Bunny
- M Woodward
M. Woodward, " Brian the Build Bunny, " 2008. [Online]. Available: http://www.woodwardweb.com/gadgets/000434.html. [Accessed: 30-Jan-2012].
Hawthorne and the Western Electric Company: The Social Problems of an Industrial Civilisation. Routledge
- E Mayo
E. Mayo, Hawthorne and the Western Electric Company: The Social Problems of an Industrial Civilisation. Routledge, 1949.
Automated Continuous Integration and the BetaBrite LED Sign
- J Atwood
J. Atwood, " Automated Continuous Integration and the BetaBrite LED Sign, " 2005. [Online]. Available: http://www.codinghorror.com/blog/archives/000238.html. [Accessed: 30-Jan-2012].
Research on instructional media
- R E Clark
- B M Sugrue
R. E. Clark and B. M. Sugrue, " Research on instructional media, 1978-1988, " Instructional Technology: Past, Present and Future, pp. 336-346, 1996.
Pragmatic Project Automation
- M Clark
M. Clark, Pragmatic Project Automation. Raleigh, NC: The Pragmatic Programmers, LLC, 2004.
Team Build 2008 API Example: The Build Wallboard
- M Woodward
M. Woodward, " Team Build 2008 API Example: The Build Wallboard, " 2007. [Online]. Available: http://www.woodwardweb.com/vsts/000395.html. [Accessed: 30-Jan-2012].
Nabaztag Scala Library
- J.-L De
J.-L. de Morlhon, " Nabaztag Scala Library, " 2009. [Online]. Available: http://morlhon.net/blog/2009/11/09/nabaztag-scala-library/. [Accessed: 30-Jan-2012].
Putting the SnowMan to work: Cruise Control .Net Build Status Monitor Available: http://blog.analysisuk.com/post/33-Putting-the-SnowMan-to-work-Cruise-Control-Net-Build-Status-Monitor.aspx
- S Harrison
S. Harrison, " Putting the SnowMan to work: Cruise Control.Net Build Status Monitor, " 2007. [Online]. Available: http://blog.analysisuk.com/post/33-Putting-the-SnowMan-to-work-Cruise-Control-Net-Build-Status-Monitor.aspx. [Accessed: 30-Jan-2012].
Bamboo Remote API Available: http://confluence.atlassian.com/display
- Ltd Atlassian Pty