Fig 3 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
The necessity of software as stand-alone products, and as central parts of non-traditional software products have changed how software products are developed. It started with the introduction of the agile manifesto and has resulted in a change of how software process improvements (SPI) are conducted. Although there are agile SPI methods and several...
Contexts in source publication
Context 1
... decision of which backlog to assess, selection of a representative sample of items from the backlog, team members to involve -as well as the details of the actual assessment are detailed in Fig. 3. The following sub-sections describes each step of BAM in more ...
Context 2
... organization may have several products, thus several product backlogs, or different type of backlogs for the same product, as seen in Step 1, Fig. 3. Having multiple backlogs at several different levels is not an uncommon way to organize the work and requirements in industry. In particular for large organizations where the work requires decisions on multiple levels to select what to realize in the next phase/sprint. The case company Telenor has, in essence, three types of backlogs ...
Context 3
... in the product are included in the backlog, and not all items are written as user stories. Instead, it is common to write items using natural lan-guage [24,25]. Telenor Sweden uses several different styles of writing/specifying requirements/backlog items, including use-case diagrams, use-case scenarios, and natural language, as seen in Step 2, Fig. 3. This is not surprising as different specification styles are appropriate at different levels in the decision chain as the abstraction levels differ, and the items are used for different purposes ...
Context 4
... in the backlog for, e.g. decision-making, planning future products/sprints, estimations, coordination with other teams/products, or prioritization. The identification of stakeholders could be a simple one-to-one mapping, one backlog and one team working with the backlog, or there could be multiple teams working with the same backlog, (Step 3, Fig. 3). At Telenor there is usually one team with various roles from different departments/areas working with the backlog in the feasibility phase, while other teams work with the product and sprint backlogs. That is, it is not the same team/roles that work with the items from potential ideas to on-queue for development. This is an important ...
Context 5
... the core, BAM consists of four BAM perspectives, as seen in Step 4, Fig. 3. These are used to grade individual items in the backlog. The four BAM perspectives are described in ...
Context 6
... the fifth and final step of BAM, the identified stakeholders perform the assessment. Each stakeholder individually estimates items by grading them using each of the four perspectives from Step 4. The assessment is performed using predefined templates as seen in Step 4, Fig. 3. The template can be used in two ways, (1) using one template for each item, or (2) adding several items to the same template. The case company used one template for several items (see Fig. 4, Items A -D) by placing the item's ID on the scale (to preserve confidentiality, the two examples of detail for Items A and C are anonymized in ...
Context 7
... all stakeholders have completed their assessment, an overview of the results is constructed (Step 5, Fig. 3). The constructed overview comes in two parts. ...
Context 8
... constructed (Step 5, Fig. 3). The constructed overview comes in two parts. First, the overview shows the total estimated span of items in the backlog for each of the four perspectives combined with the overlap of all stakeholders. At Telenor Sweden, the four stakeholders had different opinions about the span of items in their backlog (see Step 5, Fig. 3). Each of the colored lines represents one stakeholder. For example, one stakeholder believed that the Scope of the items span from an isolated change to the platform (blue line), while another stakeholder believed that the Scope spans from an isolated change to the application (green line). The grey boxes represent the shared view ...
Context 9
... (blue line), while another stakeholder believed that the Scope spans from an isolated change to the application (green line). The grey boxes represent the shared view among the stakeholders for each scale. Second, the assessment results can also be used to create "typical profiles" of items that exist in the same backlog, as seen in Step 5, Fig. 3. Four typical item profiles were identified in one backlog at Telenor. For example, one typical profile (profile Item A, Step 5 in Fig. 3) was items with a narrow Scope, wide Abstraction/Level, with constantly changing information (Maturity) that was specified with little Detail. In the same backlog, other items (profile Item D) had a ...
Context 10
... boxes represent the shared view among the stakeholders for each scale. Second, the assessment results can also be used to create "typical profiles" of items that exist in the same backlog, as seen in Step 5, Fig. 3. Four typical item profiles were identified in one backlog at Telenor. For example, one typical profile (profile Item A, Step 5 in Fig. 3) was items with a narrow Scope, wide Abstraction/Level, with constantly changing information (Maturity) that was specified with little Detail. In the same backlog, other items (profile Item D) had a wider Scope, more Mature information that was specified in greater ...
Context 11
... case company. Seven practitioners representing different roles were selected. The roles chosen were: 1 Process developer, 2 Business analysts, 2 Product managers, 1 Project manager, and 1 Operational manager. The selected practitioners identified 100 backlog items that were used as a representative sample of items from the backlog (Step 2 in BAM, Fig. 3). Then the practitioners used the template for BAM perspectives (Step 4 in BAM, Fig. 3) to perform the assessment of the backlog items (Step 5 in BAM, Fig. 3). After the practitioners had used BAM, data -in terms of BAM's usability and usefulness -was collected. Semi-structured interviews [20] were used to ask questions about how BAM ...
Context 12
... chosen were: 1 Process developer, 2 Business analysts, 2 Product managers, 1 Project manager, and 1 Operational manager. The selected practitioners identified 100 backlog items that were used as a representative sample of items from the backlog (Step 2 in BAM, Fig. 3). Then the practitioners used the template for BAM perspectives (Step 4 in BAM, Fig. 3) to perform the assessment of the backlog items (Step 5 in BAM, Fig. 3). After the practitioners had used BAM, data -in terms of BAM's usability and usefulness -was collected. Semi-structured interviews [20] were used to ask questions about how BAM was perceived by the practitioners. The interviews were carried out by one interviewer ...
Context 13
... 1 Project manager, and 1 Operational manager. The selected practitioners identified 100 backlog items that were used as a representative sample of items from the backlog (Step 2 in BAM, Fig. 3). Then the practitioners used the template for BAM perspectives (Step 4 in BAM, Fig. 3) to perform the assessment of the backlog items (Step 5 in BAM, Fig. 3). After the practitioners had used BAM, data -in terms of BAM's usability and usefulness -was collected. Semi-structured interviews [20] were used to ask questions about how BAM was perceived by the practitioners. The interviews were carried out by one interviewer and one interviewee. For all interviews, we took records in form of ...
Context 14
... of Use. In general, the practitioners at Telenor found that BAM is easy to understand, and that the five steps of BAM make sense for assessing a backlog. The practitioners explained that the steps of BAM are helpful and easy to follow, and in particular the provided examples for each step (Fig. 3). In addition, BAM does not seem to take too much time to use in practice. An assessment of about 100 items for one backlog with 4-5 practitioners did not take more than a day, and if the number of items is decreased to 30-40 items, then the time spent on the assessment is only 3-4 h, which could be done on a regular basis by individual ...
Similar publications
The specialized literature defines Software Process Improvement (SPI) as the fundamental approach to improving software products in software development organizations. In this context, studies report several problems and difficulties that organizations face during the implementation of improvements. Although there are studies that address the probl...
While many organizations successfully embrace and experience software process improvement (SPI) benefits, others abandon the effort before realizing the total potential result of an SPI initiative. Therefore, researchers' interest in understanding the reasons why software organizations that have a successful start in adopting SPI abandon improvemen...
Citations
... However, in practice, SP are often initially estimated based on incomplete information. Such information may be discovered late or changed (Hoda and Murugesan 2016;Svensson et al. 2019), which may lead to the SP changes. To minimize the impact, there are needs to better understand the SP changes and an automated approach to predict the SP changes. ...
Story Points (SP) are an effort unit that is used to represent the relative effort of a work item. In Agile software development, SP allows a devel- opment team to estimate their delivery capacity and facilitate the sprint plan- ning activities. Although Agile embraces changes, SP changes after the sprint planning may negatively impact the sprint plan. To minimize the impact, there is a need to better understand the SP changes and an automated approach to predict the SP changes. Hence, to better understand the SP changes, we examine the prevalence, accuracy, and impact of information changes on SP changes. Through the analyses based on 19,349 work items spread across seven open-source projects, we find that on average, 10% of the work items have SP changes. These work items typically have SP value increased by 58%-100% rel- ative to the initial SP value when they were assigned to a sprint. We also find that the unchanged SP reflect the development time better than the changed SP. Our qualitative analysis shows that the work items with changed SP of- ten have the information changes relating to updating the scope of work. Our empirical results suggest that SP and the scope of work should be reviewed prior or during sprint planning to achieve a reliable sprint plan. Yet, it could be a tedious task to review all work items in the product (or sprint) backlog. Therefore, we develop a classifier to predict whether a work item will have SP changes after being assigned to a sprint. Our classifier achieves an AUC of 0.69-0.8, which is significantly better than the baselines. Our results suggest that to better manage and prepare for the unreliability in SP estimation, the team can leverage our insights and the classifier during the sprint planning. To facilitate future studies, we provide the replication package and the datasets, which are available online.
... However, in practice, SP are often initially estimated based on incomplete information. Such information may be discovered late or changed (Hoda and Murugesan, 2016;Svensson et al., 2019), which may lead to the SP changes. To minimize the impact, there are needs to better understand the SP changes and an automated approach to predict the SP changes. ...
Story Points (SP) are an effort unit that is used to represent the relative effort of a work item. In Agile software development, SP allows a development team to estimate their delivery capacity and facilitate the sprint planning activities. Although Agile embraces changes, SP changes after the sprint planning may negatively impact the sprint plan. To minimize the impact, there is a need to better understand the SP changes and an automated approach to predict the SP changes. Hence, to better understand the SP changes, we examine the prevalence, accuracy, and impact of information changes on SP changes. Through the analyses based on 13,902 work items spread across seven open-source projects, we find that on average, 10% of the work items have SP changes. These work items typically have SP value increased by 58%-100% relative to the initial SP value when they were assigned to a sprint. We also find that the unchanged SP reflect the development time better than the changed SP. Our qualitative analysis shows that the work items with changed SP of- ten have the information changes relating to updating the scope of work. Our empirical results suggest that SP and the scope of work should be reviewed prior or during sprint planning to achieve a reliable sprint plan. Yet, it could be a tedious task to review all work items in the product (or sprint) backlog. Therefore, we develop a classifier to predict whether a work item will have SP changes after being assigned to a sprint. Our classifier achieves an AUC of 0.69-0.8, which is significantly better than the baselines. Our results suggest that to better manage and prepare for the unreliability in SP estimation, the team can leverage our insights and the classifier during the sprint planning. To facilitate future studies, we provide the replication package and the datasets, which are available online.
... Based on the Agile manifesto that comprehensive documentation has lower priority than working software [14], it is possible that the team may not foster adequate or up-to-date information in documentation when it is needed for effort estimation and sprint planning [19,25,38,51]. Indeed, a recent survey reported that most of the participated agile developers considered that documented information is important when estimating effort and reestimation would be performed when the documented information was changed [38]. ...
... Hence, the team needs to analyze the documented information to understand the work to be done for the work item. Prior studies reported that the quality of documented information could impact the effort estimation accuracy [24,51,56,57]. Furthermore, the documented information may also be changed after the sprint plan is finalized, requiring the team to spend more effort in re-estimating and re-planning the sprint [19,33,38]. ...
In agile iterative development, an agile team needs to analyze documented information for effort estimation and sprint planning. While documentation can be changed, the documentation changes after sprint planning may invalidate the estimated effort and sprint plan. Hence, to help the team be aware of the potential documentation changes, we developed DocWarn to estimate the probability that a work item will have documentation changes. We developed three variations of DocWarn, which are based on the characteristics extracted from the work items (DocWarn-C), the natural language text (DocWarn-T), and both inputs (DocWarn-H). Based on nine open-source projects that work in sprints and actively maintain documentation, DocWarn can predict the documentation changes with an average AUC of 0.75 and an average F1-Score of 0.36, which are significantly higher than the baselines. We also found that the most influential characteristics of a work item for determining the future documentation changes are the past tendency of the developers and the length of description text. Based on the qualitative assessment, we found that 40%-68% of the correctly predicted documentation changes were related to scope modification. With the prediction of DocWarn, the team will be better aware of the potential documentation changes during sprint planning, allowing the team to manage the uncertainty and reduce the risk of unreliable effort estimation and sprint planning.
... Motivation: Accurate effort estimation requires an adequate understanding of a task [28,44,47,48]. While documentation can convey the information to understand the task [3,19,24], it has a lower priority than delivering working software in the Agile concept [16]. ...
... Motivation: Comprehensive documentation can be considered as an overhead in Agile [16,24]. On the other hand, inadequate documentation may lead to a misunderstanding of a task [29,44]. Hence, just-enough and just-in-time documentation would be suitable for Agile practices [21]. ...
... [...] the change of user stories/functional requirements changes the original estimation often." Since the effort estimates are used in iteration planning, such uncertainty may cause the plan to become unreliable [8,44]. Hence, to facilitate effort estimation and improve its accuracy, these useful types of information should be documented and reviewed. ...
... Motivation: Accurate effort estimation requires an adequate understanding of a task [28,44,47,48]. While documentation can convey the information to understand the task [3,19,24], it has a lower priority than delivering working software in the Agile concept [16]. ...
... Motivation: Comprehensive documentation can be considered as an overhead in Agile [16,24]. On the other hand, inadequate documentation may lead to a misunderstanding of a task [29,44]. Hence, just-enough and just-in-time documentation would be suitable for Agile practices [21]. ...
... [...] the change of user stories/functional requirements changes the original estimation often." Since the effort estimates are used in iteration planning, such uncertainty may cause the plan to become unreliable [8,44]. Hence, to facilitate effort estimation and improve its accuracy, these useful types of information should be documented and reviewed. ...
Effort estimation is an integral part of activities planning in Agile iterative development. An Agile team estimates the effort of a task based on the available information which is usually conveyed through documentation. However, as documentation has a lower priority in Agile, little is known about how documentation effort can be optimized while achieving accurate estimation. Hence, to help practitioners achieve just-enough documentation for effort estimation, we investigated the different types of documented information that practitioners considered useful for effort estimation. We conducted a survey study with 121 Agile practitioners across 25 countries. Our survey results showed that (1) despite the lower priority of documentation in Agile practices, 98% of the respondents considered documented information moderately to extremely important when estimating effort, (2) 73% of them reported that they would re-estimate a task when the documented information was changed, and (3) functional requirements, user stories, definition of done, UI wireframes, acceptance criteria, and task dependencies were ranked as the most useful types of documented information for effort estimation. Nevertheless, many respondents reported that these useful types of documented information were occasionally changing or missing. Based on our study results, we provide recommendations for agile practitioners on how effort estimation can be improved by focusing on just-enough documentation.
