No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Combining Estimates with Planning Poker--An Empirical Study
Abstract
Combination of expert opinion is frequently used to produce estimates in software projects. However, if, when and how to combine expert estimates, is poorly understood. In order to study the effects of a combination technique called planning poker, the technique was introduced in a software project for half of the tasks. The tasks estimated with planning poker provided: 1) group consensus estimates that were less optimistic than the mechanical combination of individual estimates for the same tasks, and 2) group consensus estimates that were more accurate than the mechanical combination of individual estimates for the same tasks. The set of control tasks in the same project, estimated by individual experts, achieved similar estimation accuracy as the planning poker tasks. However, for both planning poker and the control group, measures of the median estimation bias indicated that both groups had unbiased estimates, as the typical estimated task was perfectly on target.
... Recently, a new estimation method appeared, named Team Estimation Game (TEG), for which its proponents claim that it enables faster evaluation and more accurate estimates [12]. While the Planning Poker method was already the subject of some scientific studies [19][20][21], we were not able to find a systematic study about Team Estimation Game except some recommendations of agile practitioners who recommend TEG as a better assessment method [12,22]. In order to better understand TEG, we wanted to further investigate it and compare it to the widely used PP method. ...
... It is a robust and sensible measure. Another reason for using BRE was compatibility with several studies that have been conducted in the past and use BRE as their key measure [42,43], including related studies on PP [19][20][21]. Since BRE values failed to comply with normal distribution according to skewness (larger than AE1) and kurtosis (larger than AE2), we analyzed the BRE results with nonparametric tests instead of parametric tests. ...
Effort estimation is a crucial part of software development projects. Despite the availability of several assessment techniques, accurate assessment still remains an extremely difficult task. Team Estimation Game is a relatively new estimation technique for agile software development methods that has not received significant attention from the scientific community despite its growing popularity between practitioners. In this paper, we attempt to bridge this gap by presenting the results of an empirical study with undergraduate students in which we compare Team Estimation Game with the more established Planning Poker technique. We mainly focus our analysis of the two techniques on the time needed for user story estimation and estimation accuracy. The results of the empirical study reveal that Team Estimation Game produces more accurate story estimates than Planning Poker. Additionally, we found that for the Team Estimation Game, estimation and planning skills of the development teams improve from Sprint to Sprint. Team Estimation Game proved to be a useful estimation method for agile projects within the capstone course. Furthermore, we have shown that the study can be successfully incorporated into a software engineering capstone course without hindering the teaching goals while retaining the validity of research goals.
... It has been shown that letting the developers who will be implementing the features in question estimate efforts leads to higher estimation accuracy [19], likely because they know the project better and have a higher motivation to understand it than a non-developing estimator [12] . Studies have also shown that the accuracy of individual estimates can be improved by averaging several independent estimates, rather than relying on estimates of a single estimator [20]. Group estimates meanwhile rely on the discussion among experts, which can be structured or unstructured [13]. ...
... To answer RQ2 regarding the teams' estimation accuracy, we used the balanced measure of relative error (BRE). Following [20], we choose the BRE because of criticism of the magnitude of relative error (MRE) regarding its lack of balance [38]. The BRE is calculated as: ...
... Planning Poker is a technique for sizing Product Backlog Items (PBIs) that was first described by James Grenning in 2002 and then popularized by Mike Cohn in 2006, is the most used estimation technique in Scrum projects. It's a teambased exercise that is commonly used for assigning relative estimate values to user stories/requirements to express the effort required to deliver specific features or functionality [11]. ...
... If the estimations differ too much the estimators discuss their proposals. The high and low estimators should especially share their reasons, the process is repeated until consensus is achieved [23] [9] [11]. ...
Planning Poker is a consensus-based technique mostly used for estimating effort or relative size of software development goals. This is applicable to estimate the size of user stories, developing releases and iteration plans. It is used generally with Scrum. Planning Poker has a lot of benefits, however, this method is not entirely efficient because the result is always based on the observation of an expert. This paper proposes the identification and validation of the most important factors taken into account by Scrum teams to assign complexity and importance to tasks, which are two important factors to consider for task scheduling. We define a knowledge structure that represents different aspects and the influence of each factor in the final decision. We use a Bayesian Network to co-relate these factors in order to have accurate in the estimation. The Bayesian Network give us the complexity of a task according to Fibonacci scale. Using our technique, software teams can focus in the most important tasks to obtain a quality software.
... Results indicated planning poker produced more accurate estimates than unstructured group estimates when the team had related experience with similar projects. Two other studies [11,12], conducted in a Scrum environment, compared group consensus estimates calculated by planning poker and statistical combination of individual expert estimates. Results showed that group consensus estimates were less optimistic and more accurate than statistical combination of individual estimates. ...
... Results showed that group consensus estimates were less optimistic and more accurate than statistical combination of individual estimates. Results of another study [13] also conducted in a Scrum environment do not fully support the results reported in [11,12]. However, it was observed in [13] that optimism bias and estimation accuracy can be improved when planning poker is used by experienced practitioners. ...
Context: Ever since the emergence of agile methodologies in 2001, many software companies have shifted to Agile Software Development (ASD), and since then many studies have been conducted to investigate effort estimation within such context; however to date there is no single study that presents a detailed overview of the state of the art in effort estimation for ASD. Objectives: The aim of this study is to provide a detailed overview of the state of the art in the area of effort estimation in ASD. Method: To report the state of the art, we conducted a systematic literature review in accordance with the guidelines proposed in the evidence-based software engineering literature. Results: A total of 25 primary studies were selected; the main findings are: i) Subjective estimation techniques (e.g. expert judgment, planning poker, use case points estimation method) are the most frequently applied in an agile context; ii) Use case points and story points are the most frequently used size metrics respectively; iii) MMRE and MRE are the most frequently used accuracy metrics; iv) team skills, prior experience and task size are cited as the three important cost drivers for effort estimation in ASD; and v) Extreme Programming (XP) and SCRUM are the only two agile methods that are identified in the primary studies. Conclusion: Subjective estimation techniques, e.g. expert judgment-based techniques, planning poker or the use case points method, are the one used the most in agile effort estimation studies. As for the size metrics, the ones that were used the most in the primary studies were story points and use case points. Several research gaps were identified, relating to the agile methods, size metrics and cost drivers, thus suggesting numerous possible avenues for future work.
... The second and third studies (Moløkken-Østvold and Haugen, 2007;Moløkken-Østvold et al., 2008) concentrated on comparison between group consensus estimates obtained through planning poker and the statistical combination (the mean) of individual expert estimates. It was found that: (1) group consensus estimates were less optimistic than the statistical combination of individual estimates of the same tasks and (2) group consensus estimates were more accurate than the statistical combination of individual estimates of the same tasks. ...
... Another reason for using BRE was compatibility with previous related research. Many recent studies on software effort estimation use BRE as the key measure (e.g., Moløkken-Østvold and Jørgensen, 2005;Moløkken-Østvold and Furulund, 2007), including related studies of planning poker by Moløkken-Østvold and Haugen (2007) and Moløkken-Østvold et al. (2008). The decision to use BRE made it possible to directly compare our results to the results of previous research of the planning poker technique. ...
While most studies in psychology and forecasting stress the possible hazards of group processes when predicting effort and schedule, agile software development methods recommend the use of a group estimation technique called planning poker for estimating the size of user stories and developing release and iteration plans. It is assumed that the group discussion through planning poker helps in identifying activities that individual estimators could overlook, thus providing more accurate estimates and reducing the over-optimism that is typical for expert judgment-based methods. In spite of the widespread use of agile methods, there is little empirical evidence regarding the accuracy of planning poker estimates. In order to fill this gap a study was conducted requiring 13 student teams to develop a Web-based student records information system. All teams were given the same set of user stories which had to be implemented in three Sprints. Each team estimated the stories using planning poker and the estimates provided by each team member during the first round were averaged to obtain the statistical combination for further comparison. In the same way the stories were estimated by a group of experts. The study revealed that students’ estimates were over-optimistic and that planning poker additionally increased the over-optimism. On the other hand, the experts’ estimates obtained through planning poker were much closer to actual effort spent and tended to be more accurate than the statistical combination of their individual estimates. The results indicate that the optimism bias caused by group discussion diminishes or even disappears as the expertise of the people involved in the group estimation process increases.
... Other practices include communities of practice [6] and job rotation. Within the agile development community a practice called Planning Poker has been established as a light-weight, work estimation practice for teams that can improve accuracy of estimates and reduce occurrences of optimistic estimates [16,26]. Contributing to these effects is that participants must justify estimates which again promote discussions and identification of lack of information [5]. ...
... A limitation of the investigation is lack of data to enable analysis of the team estimation accuracy. However, these effects of Planning Poker have been reported elsewhere [16,26]. During my observations in the post-action phase I did notice significantly better Scrum burndowns. ...
Specialist organizational environments and lack of redundant knowledge reduce flexibility and therefore inhibits transition
to agile development. This action research reports from the adoption of team estimation as a vehicle to increase redundant
knowledge within a group of specialists. The group suffered from low levels of group learning legitimized by high work pressure
and a specialist organizational environment. This resulted in poor planning and optimistic task estimates which contributed
to increase the work pressure even higher. I framed the research as double-loop learning; I illustrate how different barriers
to team estimation arose from conflicts with existing efficiency norms and then how benefits from team estimation created
sufficient momentum to change practice. The results are obtained from qualitative analysis of empirical data gathered during
one year of collaboration with the group. The article contributes to understanding of barriers to group learning and agile
adoption in software organizations.
... A study about using planning poker for user story estimation found that planning poker produced more accurate estimates than unstructured group estimates [38]. Other studies found that planning poker in scrum projects yielded less optimistic and more accurate estimates than a statistical combination of individual expert estimates [39,40]. ...
This study compares the reliability of estimation, productivity, and defect rate metrics for sprints driven by a specific instance of the agile approach (i.e., scrum) and an agile model-Bbased software engineering (MBSE) approach called the integrated Scrum Model-Based System Architecture Process (sMBSAP) when developing a software system. The quasi-experimental study conducted ten sprints using each approach. The approaches were then evaluated based on their effectiveness in helping the product development team estimate the backlog items that they could build during a time-boxed sprint and deliver more product backlog items (PBI) with fewer defects. The commitment reliability (CR) was calculated to compare the reliability of estimation with a measured average scrum-driven value of 0.81 versus a statistically different average sMBSAP-driven value of 0.94. Similarly, the average sprint velocity (SV) for the scrum-driven sprints was 26.8 versus 31.8 for the MBSAP-driven sprints. The average defect density (DD) for the scrum-driven sprints was 0.91, while that of the sMBSAP-driven sprints was 0.63. The average defect leakage (DL) for the scrum-driven sprints was 0.20, while that of the sMBSAP-driven sprints was 0.15. The t-test analysis concluded that the sMBSAP-driven sprints were associated with a statistically significant larger mean CR, SV, DD, and DL than that of the scrum-driven sprints. The overall results demonstrate formal quantitative benefits of an agile MBSE approach compared to an agile alone, thereby strengthening the case for considering agile MBSE methods within the software development community. Future work might include comparing agile and agile MBSE methods using alternative research designs and further software development objectives, techniques, and metrics.
... Planning poker estimates effort from expert judgment that was first defined by James Grenning in 2002. By combining the opinions of various experts, this technique generates an estimate of effort [18]. The origins of neural networks (NN) go back to the work of Warren McCulloch and Walter Pitts [19]. ...
... Haugen (2007) [PS16] used the planning poker technique and concluded that group consensus estimates were less optimistic than the mechanical combination of individual estimates for the same tasks. Coyle et al. (2013) [PS43] assessed the existence of group process losses during decision-making in an agile software development team, such as groupthink and inappropriate Influences and group member domination. ...
Background: In software project management, the decision-making process is a complex set of tasks mainly based on human relations, individual knowledge, and cultural background. The factors that affect the decisions of Software Project Managers (SPMs), as well as their potential consequences, require attention because project delays and failures might be related to a series of poor decisions. Aims: To understand how SPMs make decisions based on how they interpret their experiences in the workplace. Further, to identify antecedents, moderators and consequences of those decisions to increase the effectiveness of project management. Method: Firstly, an exploratory study based on semi-structured interviews was conducted with SPMs from a large Brazilian governmental organization and from a small Portuguese private organization to shed light on the causal factors of SPMs’ cognitive biases and how they deal with them, including techniques and tools they used to minimize the cognitive biases’ adverse effects. The initial findings suggested that we needed a more grounded understanding of the mechanisms of decision-making. Thus, a broader research protocol based on semi-structured interviews was carried out with SPMs within a large Brazilian governmental organization and a large Brazilian private organization. We also conducted interviews with software engineers and PMO managers to triangulate the data, which was analyzed using techniques from grounded theory. Data from observations, document analysis and selected studies from a systematic literature review were also used. Results: We found that decision-making in software project management is based on knowledge sharing in which the SPM acts as a facilitator. This phenomenon is influenced by individual factors, such as experience, knowledge, leadership style, and skills, and by situational factors such as the autonomy of the SPM, task complexity and team members' technical competence. Conclusions: Due to the uncertainty and dynamism inherent to software projects, the SPMs focus on making, monitoring and adjusting decisions in an argument-driven way. Also, the involvement of the team members in decision-making aims to minimize the SPM's decision regret and cognitive biases as well as to maximize the team member's commitment.
... The big issue is that most often Planning Poker practice is subject to various limitations [27,33]. Because of these limitations, companies and teams have no choice rather than using average of the suggested values for software cost estimation. ...
... He also took an unstructured group estimation process and then compared the results. Kjetil Moløkken-Østvold[6]et al. combined their estimates with planning poker. They discussed different techniques and prepared research questions. ...
Agile Software Development techniques are worldwide accepted, regardless of the definition of agile we all must agree with the fact that agile is maturing day by day, suppliers of software systems are moving away from traditional waterfall techniques and other development practices in favor of agile methods. There are numerous types of methodologies, domains/ methods in agile for which are to be selected according to the current situation and demand of the current project. As a case scenario in the following research will discuss scrum as a development technique in which we will focus on the effort estimation(s) and their effects by discussing distinct metrics. Mainly estimation refers directly to cost, time and complexity during the life cycle of project. Metrics will help the teams to better understand the development progress and building releasing (releases) of software easier in a fluent and robust way. The following paper thus identifies aspects mainly ignored by the development team(s) during estimation.
... The planning poker game [27], [28] provides a collaborative method for estimating efforts for software engineering. The players take turns to estimate the efforts of a task in the first round, discuss the reasoning for their estimations and estimate again in a second round. ...
Social engineering is the acquisition of information about computer systems by methods that deeply include non- technical means. While technical security of most critical systems is high, the systems remain vulnerable to attacks from social engineers. Social engineering is a technique that: (i) does not require any (advanced) technical tools, (ii) can be used by anyone, (iii) is cheap. Traditional security requirements elicitation approaches often focus on vulnerabilities in network or software systems. Few approaches even consider the exploitation of humans via social engineering and none of them elicits personal behaviours of indi- vidual employees. While the amount of social engineering attacks and the damage they cause rise every year, the security awareness of these attacks and their consideration during requirements elicitation remains negligible. We propose to use a card game to elicit these requirements, which all employees of a company can play to understand the threat and document security requirements. The game considers the individual context of a company and presents underlying principles of human behaviour that social engineers exploit, as well as concrete attack patterns. We evaluated our approach with several groups of researchers, IT administrators, and professionals from industry.
... The planning poker game [26], [27] provides a collaborative method for estimating efforts for software engineering. The players take turns to estimate the efforts of a task in the first round, discuss the reasoning for their estimations and estimate again in a second round. ...
Social engineering is the acquisition of information about computer systems by methods that deeply include non- technical means. While technical security of most critical systems is high, the systems remain vulnerable to attacks from social engineers. Social engineering is a technique that: (i) does not require any (advanced) technical tools, (ii) can be used by anyone, (iii) is cheap.
Traditional security requirements elicitation approaches often focus on vulnerabilities in network or software systems. Few approaches even consider the exploitation of humans via social engineering and none of them elicits personal behaviours of indi- vidual employees. While the amount of social engineering attacks and the damage they cause rise every year, the security awareness of these attacks and their consideration during requirements elicitation remains negligible.
We propose to use a card game to elicit these requirements, which all employees of a company can play to understand the threat and document security requirements. The game considers the individual context of a company and presents underlying principles of human behaviour that social engineers exploit, as well as concrete attack patterns. We evaluated our approach with several groups of researchers, IT administrators, and professionals from industry.
... Diese Rollen werden nachfolgend einzeln vorgestellt und ihre Verantwortungen und Pflichten innerhalb des Prozesses beleuchtet. [15], [16], [24]). ...
Die agile Softwareentwicklung gewinnt immer mehr an Bedeutung. Im Gegensatz zu klassischem Projektmanagement werden Anforderungen nicht in einem Lasten- und Pflichtenheft aufgeführt, stattdessen wird ein sogenannter Product Backlog aufgestellt. In diesem werden die einzelnen Anforderungen als User Stories - Anwendungsfälle aus der Sicht eines Akteurs - beschrieben. Die Schätzung des Aufwands zur Umsetzung der Stories liegt nicht mehr in der Hand des Projektmanagers, sondern wird vom Entwicklerteam selbst vorgenommen. Dieser Schätzprozess stellt einen elementaren Bestandteil der Projektplanung dar und versucht eine realistische Abschätzung der Größe beziehungsweise des Ausmaßes eines Projektes frühzeitig zu ermitteln. Dazu wurden verschiedene Methoden entwickelt.
Im Rahmen dieser Arbeit sollen nun zunächst ausgewählte Ansätze, Vorgehensweisen und existierende Softwarewerkzeuge zur Durchführung des Schätzprozesses untersucht und bewertet werden.
Ferner soll anhand aufgestellter Anforderungen eine Multi-Touch-Anwendung, basierend auf der bewährten Schätzmethode „Planning Poker“, auf dem Microsoft Surface entwickelt werden. Zunächst soll ein Konzept entwickelt werden, wie das Spielprinzip angepasst und als Multi-User-Anwendung umgesetzt werden kann. Nachfolgend soll dieses Konzept realisiert werden. Es soll untersucht werden, inwiefern sich der Schätzprozess mittels gegebener Möglichkeiten der modernen Multi-Touch-Technologie, wie zum Beispiel Objektinteraktion und Berührungs- bzw. Gestenerkennung, weiter optimieren lässt, um Verwaltungs- und damit auch Zeitaufwand einzusparen. Zusätzlich soll die Durchführung der Schätzung als ein motivationsförderndes Erlebnis gestaltet werden.
Um eine Integration in ein bestehendes Prozessverwaltungswerkzeug zu ermöglichen, soll eine Schnittstelle implementiert werden, über welche die Anwendung auf vorhandene Datensätze zugreifen kann. Diese Daten sollen entsprechend visualisiert und gegebenenfalls mit den ermittelten Schätzwerten aktualisiert werden.
... In order to compare actual effort and estimated effort, and to measure any differences in project overruns dependent on the studied properties, we used the BREbias measure, previously used in related research, e.g., [21,22]. It is calculated as The BREbias measures both the magnitude and direction of effect when comparing the actual effort to estimated effort. ...
... One Norwegian industrial team " immediately took a liking to the new estimation process " [15] of Planning Poker, and the technique was " very well received. " Another Norwegian [25] " decided to implement it for all tasks in the project's future " because they felt it was an effective means of collaboratively producing unbiased estimates. Planning Poker is " played " by the team as a part of the Sprint Planning meeting. ...
The Scrum methodology is an agile software development process that works as a project management wrapper around existing engineering practices to iteratively and incrementally develop software. With Scrum, for a developer to receive credit for his or her work, he or she must demonstrate the new functionality provided by a feature at the end of each short iteration during an iteration review session. Such a short-term focus without the checks and balances of sound engineering practices may lead a team to neglect quality. In this paper we present the experiences of three teams at Microsoft using Scrum with an additional nine sound engineering practices. Our results indicate that these teams were able to improve quality, productivity, and estimation accuracy through the combination of Scrum and nine engineering practices.
This paper presents the impact of team size and dynamics on agile estimation accuracy and strategies for improving estimation in diversified teams. We employed a mixed-method approach with online surveys, interviews, and case studies. The data received and analyzed in this research came from 150 agile teams representing different industries. Our results show that strong interdependency exists between team size and the dispersion of estimations. The best estimation accuracy was when the team had 5-9 members. Team dynamics, particularly cohesion and psychological safety, emerged as important in the estimation outcome. Based on these insights, we propose a framework for improving estimation practice within agile teams, including tailoring and continuous improvement.
Im dritten Band der Reihe "Elche fangen" finden Sie Methoden und Instrumente aus der Ökonomie, der Organisationsentwicklung, der Arbeit mit Teams und einige Werkzeuge für das Selbstmanagement. Dies reicht von der Balanced Scorecard über Retreat, Kreativtechniken, Großgruppenereignisse bis hin zum Umgang mit den digitalen Medien und sozialen Netzwerken. Diese Toolbox soll Sie anregen: Bauen Sie sich Ihre eigene Werkzeugkiste.
Vieses cognitivos são erros sistemáticos de racionalidade na cognição humana, podendo afetar decisões em diversas áreas, com consequências de variadas dimensões. Uma das áreas na qual se identifica amplo impacto de vieses cognitivos é a engenharia de software, área em constante crescimento no mundo todo. Sendo assim, se faz fundamental ampliar o corpo de conhecimento acerca dos fatores que da cognição humana que podem influenciar a engenharia de software. O presente trabalho constitui em um mapeamento sistemática acerca das técnicas de desenviesamento em engenharia de software. Foram registradas 31 técnicas de desenviesamento, divididas em 10 categorias. Do total de técnicas, apenas 13 foram testadas empiricamente, das quais 11 obtiveram resultados positivos na mitigação de ao menos um viés cognitivo.
SDCE (Software Development Cost Estimation) has always been an interesting and budding field in Software Engineering. This study supports the SDCE by exploring its techniques and models and collecting them in one place. This contribution in the literature will assist future researchers to get maximum knowledge about SDCE techniques and models from one paper and to save their time. In this paper, we review numerous software development effort and cost estimation models and techniques, which are divided into different categories. These categories are parametric models, expertise-based techniques, learning-oriented techniques, dynamicsbased models, regression-based techniques, fuzzy logic-based methods, size-based estimation models, and composite techniques. Some other techniques which directly do not lie in any specific category are also briefly explained. We have concluded that no single technique is best for all situations; rather they are applicable in different nature of projects. All techniques have their own pros and cons and they are challenged by the rapidly changing software industry. Since no single technique gives a hundred percent accuracy, that is why one technique and model should not be preferred over all others. We recommend a hybrid approach for SDCE because in this way the limitations of one model and technique are complemented by the merits of the other model/technique. We also recommend a model calibration to obtain accurate results because if a model was developed in a different environment, we cannot expect reliable estimates from it in a completely new environment.
The error messages displayed after compilation of MySQL query shows the MySQL error code, SQLSTATE value
and text string in command prompt. The displayed error messages will lead an inconvenience to users who are willing
to assess the full details of the errors. The proposed framework presents the design of pre-processor with attractive
interactive user interfaces to reduce the ambiguity of the error messages generated after the compilation of MySQL query.
The output of the proposed pre-processor gives the packed error details when an error occurs during the compilation
of query and it shows clear error messages to the users instead of giving existing warning message in the command
prompt. This will facilitate the users who are not an expert in MySQL and they easily tackle the type error message and
where they occurred. The proposed pre-processor also provides a lot of interactive interfaces for handling databases
and tables stored within the MySQL server as supporting elements for creation of queries generates Interactive text
editor with intellisence facilities for users to minimize their careless or typing errors. Finally, the prototype of this pre-processor supports different types of queries with an expected error details and reduces the ambiguity of the warning
messages displayed after the compilation of MySQL query.
One source of software project challenges and failures is the systematic errors introduced by human cognitive biases. Although extensively explored in cognitive psychology, investigations concerning cognitive biases have only recently gained popularity in software engineering research. This paper therefore systematically maps, aggregates and synthesizes the literature on cognitive biases in software engineering to generate a comprehensive body of knowledge, understand state of the art research and provide guidelines for future research and practice. Focusing on bias antecedents, effects and mitigation techniques, we identified 65 articles (published between 1990 and 2016), which investigate 37 cognitive biases. Despite strong and increasing interest, the results reveal a scarcity of research on mitigation techniques and poor theoretical foundations in understanding and interpreting cognitive biases. Although bias-related research has generated many new insights in the software engineering community, specific bias mitigation techniques are still needed for software professionals to overcome the deleterious effects of cognitive biases on their work.
In theory, software analytics shouldn’t work because software project behavior shouldn’t be predictable. However, it does. Why?
One source of software project challenges and failures is the systematic errors introduced by human cognitive biases. Although extensively explored in cognitive psychology, investigations concerning cognitive biases have only recently gained popularity in software engineering research. This paper therefore systematically maps, aggregates and synthesizes the literature on cognitive biases in software engineering to generate a comprehensive body of knowledge, understand state of the art research and provide guidelines for future research and practise. Focusing on bias antecedents, effects and mitigation techniques, we identified 65 articles (published between 1990 and 2016), which investigate 37 cognitive biases. Despite strong and increasing interest, the results reveal a scarcity of research on mitigation techniques and poor theoretical foundations in understanding and interpreting cognitive biases. Although bias-related research has generated many new insights in the software engineering community, specific bias mitigation techniques are still needed for software professionals to overcome the deleterious effects of cognitive biases on their work.
This study aims to explore the project management practices of Danish organizations.
The inductive study compares the approaches taken by Danish organization in the IT
and Engineering industry, to the socio-technical system approach to organizational development. This has been done by conducting a case-study of the approaches that
organizations and project managers have implemented. The explorative study also aims
to identify the differences in how organizations approach the concept of project management and how certifications and project management methodologies are utilized by
the organizations. Additionally this study explores how project managers perceive and
utilize the different paradigms that make up the socio-technical system.
Projects are becoming increasingly complex and the emerging globalisation and pressure to innovate necessitates increased efficiency. The existing literature argues that a
socio-technical approach towards project management will provide additional benefits
to the projects, causing increased success rates. This report reveals the state of project
management in comparison to the approaches that have been identified as significant in
the socio-technical system.
It is indicated by the literature that project management can be perceived from either
a social or a technical perspective. It was indicated by the interviews that Danish organizations primarily utilized the technical approach while many of the project managers
acknowledge the importance of the social perspective. Similarly, the approaches towards
the use of methodologies were found to be primarily customized, as was also indicated by
the literature to be effective. However many organizations did not employ the social approaches necessary to become a more mature project organization and while there were
distinct differences in approaches, the findings did not indicate that any organization
fully followed a socio-technical approach to project management
Like in every process model, agile processes (e.g. Scrum, eXtreme Programming) depend on accurate estimations to enable meaningful prioritization, iteration- and release planning. The emphasis of this paper is on "Planning Poker", a widely used estimation technique in agile context. The goal is to identify inaccurate effort estimates to enable more precise project scheduling and release planning. In a first step, basic terms (e.g. development effort, functional effort) of traditional effort estimation were declared in the agile context. Afterwards-within the main part of the paper-a set of metrics is proposed, which can be used to evaluate accuracy of estimates. These metrics are based on the estimated efforts and interim results of the estimation process. Since the usage of these metrics needs to be seamlessly integrated into the Planning Poker process, we have conceptualized a computer-aided tool to collect and evaluate necessary data. To get a first proof of concept and preliminary feedback, we developed a corresponding prototype and applied it to a students' project. The prototype is based on a stationary multi-touch device and offers an intuitive user interface to perform the Planning Poker process. While estimation takes place, it collects required data and identifies inaccurate estimates with the help of the proposed metrics. The second part of the paper summarizes assets and drawbacks concerning usability and the user interface of the prototype.
Software systems of today are often complex, making development costs difficult to estimate. This paper uses data from 50
projects performed at one of the largest banks in Sweden to identify factors that have an impact on software development cost.
Correlation analysis of the relationship between factor states and project costs was assessed using ANOVA and regression analysis.
Ten out of the original 31 factors turned out to have an impact on software development project cost at the Swedish bank including
the: number of function points, involved risk, number of budget revisions, primary platform, project priority, commissioning
body’s unit, commissioning body, number of project participants, project duration, and number of consultants. In order to
be able to compare projects of different size and complexity, this study also considers the software development productivity
defined as the amount of function points per working hour in a project. The study at the bank indicates that the productivity
is affected by factors such as performance of estimation and prognosis efforts, project type, number of budget revisions,
existence of testing conductor, presentation interface, and number of project participants. A discussion addressing how the
productivity factors relate to cost estimation models and their factors is presented. Some of the factors found to have an
impact on cost are already included in estimation models such as COCOMO II, TEAMATe, and SEER-SEM, for instance function points
and software platform. Thus, this paper validates these well-known factors for cost estimation. However, several of the factors
found in this study are not included in established models for software development cost estimation. Thus, this paper also
provides indications for possible extensions of these models.
Software systems of today are often complex, making development costs difficult to estimate. This paper uses data from 50 projects performed at one of the largest banks in Sweden to identify factors that have an impact on software development cost. Correlation analysis of the relationship between factor states and project costs were assessed using ANOVA and regression analysis. Ten out of the original 32 factors turned out to have an impact on software development project cost at the Swedish bank, including the number of function points and involved risk. Some of the factors found to have an impact on cost are already included in estimation models such as COCOMO II and SEER-SEM, for instance function points and software platform. Thus, this paper validates these well-known factors for cost estimation. However, several of the factors found in this study are not included in established models for software development cost estimation. Thus, this paper also provides indications for possible extensions of these models.
Tracking organizations such as the US CERT show a continuing rise in security vulnerabilities in software. But not all discovered vulnerabilities are equalsome could cause much more damage to organizations and individuals than others. In the inevitable absence of infinite resources, software development teams must prioritize security fortification efforts to prevent the most damaging attacks. Protection Poker is a collaborative means of guiding this prioritization. A case study of a Red Hat IT software maintenance team demonstrates Protection Poker's potential for improving software security practices and team software security knowledge.
Discovery of security vulnerabilities is on the rise. As a result, software development teams must place a higher priority on preventing the injection of vulnerabilities in software as it is developed. Because the focus on software security has increased only recently, software development teams often do not have expertise in techniques for identifying security risk, understanding the impact of a vulnerability, or knowing the best mitigation strategy. We propose the Protection Poker activity as a collaborative and informal form of misuse case development and threat modeling that plays off the diversity of knowledge and perspective of the participants. An excellent outcome of Protection Poker is that security knowledge passed around the team. Students in an advanced undergraduate software engineering course at North Carolina State University participated in a Protection Poker session conducted as a laboratory exercise. Students actively shared misuse cases, threat models, and their limited software security expertise as they discussed vulnerabilities in their course project. We observed students relating vulnerabilities to the business impacts of the system. Protection Poker lead to a more effective software security learning experience than in prior semesters. A pilot of the use of Protection Poker with an industrial partner will begin in October 2008.
Distributed software development teams are common-place today. One good reason for distribution is the need to combine special
skills or competencies from different locations. However, integrating skills flexibly is both a technical and a communication
challenge. Lean and agile projects depend on direct communication. In this contribution, we investigate how agile teams can
be distributed by adding a “remote partner” - and still maintain agile advantages. We analyze communication using the goal-question-metric
paradigm (GQM) and apply it to a programming project, part of which was distributed. We discuss our insights on the minimal
set of additions (technical and organizational) that are required to turn distributed while staying agile.
Most agile projects rely heavily on good collaboration with the customer in order to achieve project goals and avoid overruns. However, the role of the customer in software projects is not fully understood. Often, successful projects are attributed to developer competence, while unsuccessful projects are attributed to customer incompetence. A study was conducted on eighteen of the latest projects of a software contractor. Quantitative project data was collected, and project managers interviewed, on several issues related to estimates, key project properties, and project outcome. It was found that in projects where collaboration was facilitated by daily communication between the contractor and the customer, they experienced a lesser magnitude of effort overruns. In addition, employing a contract that facilitates risk-sharing may also have a positive impact.
This work describes the optimum design of a permanent magnet assisted reluctance rotor of a 110 kW reluctance synchronous machine for traction applications. Previous studies show that the performance of the reluctance synchronous machine, as compared to the induction machine, vastly deteriorates as the flux-weakening operating region is widened. To address this problem, use of thin bonded permanent magnet sheet material inside the flux barriers of the reluctance rotor is made to improve the performance of the reluctance synchronous machine especially in the flux-weakening region. A design optimisation algorithm is implemented to minimise the volume and hence the cost of the permanent magnet material subject to voltage and torque constraints of the machine. It is shown clearly that with optimum amount of permanent magnet material included the performance of the reluctance synchronous machine, in terms of torque and supply voltage, is significantly improved and compares favorably with that of induction machine at both rated and maximum speeds.
The customer reads a story to the team. Two guys are involved in discussing the impact of the story on the system. Reluctantly, an estimate is tossed out on the table. They go back and forth for quite a while. Everyone else in the room is drifting off, definitely not engaged. The discussion oscillates from one potential solution to another, avoiding putting a number on the card. When the discussion finally ends, you discover that the estimate did not really change over all that discussion. You just wasted 20 minutes of valuable time. You have 25 more stories to estimate, you don't want to make a career of release planning. Extreme programming employs two levels of planning, iteration planning and release planning. Iteration planning is short term and very detailed. Iteration planning is a good time to get into the details of how to implement the story. Release planning is a higher-level plan with a long-range time horizon. During release planning, the team is taking a less precise and longer-term view of the project. There is usually a long backlog of stories. The release-planning objective is to get a ballpark estimate of the effort to build the product, and to split the product into interesting release. Precision of individual estimates is not the goal. Determining the project scope is. Story estimates are just what their name implies. They are estimates. Some estimates are over-estimates. Some are under-estimates. Every now and then, the estimates turn out to be just right. The team is looking out far, at many stories, including ones that will never be implemented. It is OK to be less precise. Why invest in precision before it is needed? A deck of fairly accurate estimates are better than a pair of very precise estimates (and these probably are not that precise anyway). The team should be trying to get good at using the intuition, their gut. Don't be cavalier about the estimates, but speed is important if the team wants to finish the meeting and get back to building the product.
Expert opinion is often necessary in forecasting tasks because of a lack of appropriate or available information for using statistical procedures. But how does one get the best forecast from experts? One solution is to use a structured group technique, such as Delphi, for eliciting and combining expert judgments. In using the Delphi technique, one controls the exchange of information between anonymous panelists over a number of rounds (iterations), taking the average of the estimates on the final round as the group judgment. A number of principles are developed here to indicate how to conduct structured groups to obtain good expert judgments. These principles, applied to the conduct of Delphi groups, indicate how many and what type of experts to use (five to 20 experts with disparate domain knowledge); how many rounds to use (generally two or three); what type of feedback to employ (average estimates plus justifications from each expert); how to summarize the final forecast (weight all experts’ estimates equally); how to word questions (in a balanced way with succinct definitions free of emotive terms and irrelevant information); and what response modes to use (frequencies rather than probabilities or odds, with coherence checks when feasible). Delphi groups are substantially more accurate than individual experts and traditional groups and somewhat more accurate than statistical groups (which are made up of noninteracting individuals whose judgments are aggregated). Studies support the advantage of Delphi groups over traditional groups by five to one with one tie, and their advantage over statistical groups by 12 to two with two ties. We anticipate that by following these principles, forecasters may be able to use structured groups to harness effectively expert opinion.
Managers and 231 members of 41 work groups representing 4 diverse organizations participated in an experiment involving disciplinary decisions. Managers and group members responded individually to scenarios describing a group member's poor performance, followed by group members meeting to reach consensus on the disciplinary decisions. As hypothesized, manager disciplinary decisions were more severe than decisions made by individual group members. Contrary to predictions, the severity of manager and group disciplinary decisions did not differ. A test of choice shifts revealed that when the prevailing view among individual group members was for a relatively lenient disciplinary action, the group consensus decision was more severe than the average of the individual decisions. Attributions and outcome seriousness were found to influence the severity of manager, group member, and group decisions. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Two Choice Dilemma Questionnaire items were used to investigate the influence of persuasive arguments and social decision schemes on group decisions. Furthermore, the predictions of the persuasive arguments theory on the polarization of individual preferences were tested. Ss were given lists of persuasive arguments and a 2nd individual decision was requested before the group discussion. The lists of persuasive arguments were compiled through a stepwise process of rating data gathered from the content analysis of former group discussions. In Condition 1, the Ss received the lists of arguments before the 1st individual decision; in Condition 2, between the 1st and 2nd individual measure; and in Condition 3 (control), they received no list at all. In all 3 conditions the reduced paired comparison median model showed the best fit and the highest hit rate in predicting the group decisions. The resulting choice shift could not be explained by the influence of arguments, whereas the best-fitting aggregation rule was able to clarify the choice shift. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
When financial columnist James Surowiecki wrote The Wisdom of Crowds, he wished to explain the successes and failures of markets (an example of a "crowd") and to understand why the average opinion of a crowd is frequently more accurate than the opinions of most of its individual members. In this expanded review of the book, Scott Armstrong asks a question of immediate relevance to forecasters: Are the traditional face-to-face meetings an effective way to elicit forecasts from forecast crowds (i.e. teams)? Armstrong doesn't believe so. Quite the contrary, he explains why he considers face-to-face meetings a detriment to good forecasting practice, and he proposes several alternatives that have been tried successfully.
Intuition and previous research indicate that individuals commonly display an optimistic bias in time prediction. The present studies extend research on task completion forecasts to examine tasks performed collaboratively by groups, and predictions generated through group discussion. Participants predicted—individually and collaboratively—when they would complete upcoming group projects ranging from brief laboratory tasks to extensive real-world projects, and their actual completion times were measured. Results supported the three guiding hypotheses. First, there was an optimistic bias for both individual (Studies 1 and 2) and group predictions (Studies 1–3). Second, predictions generated through group discussion were more optimistic than those generated individually. Third, this “group accentuation” effect was mediated by the informational focus at the time of prediction. Group discussion heightened participants’ tendency to focus primarily on factors promoting successful task completion, and this selective focus on “planning for success” enhanced their optimistic outlook. Discussion centers on theoretical contributions to the individual and group decision making literatures, as well as applied implications for planning and forecasting within organizational contexts.
This paper summarizes the current state ofthe art and recent trends in software engineering economics. It provides an overview of economic analysis techniques and their applicability to software engineering and management. It surveys the field of software cost estimation, including the major estimation techniques available and the state of the art in algorithmic cost models.
A summary is presented of the current state of the art and recent trends in software engineering economics. It provides an overview of economic analysis techniques and their applicability to software engineering and management. It surveys the field of software cost estimation, including the major estimation techniques available, the state of the art in algorithmic cost models, and the outstanding research issues in software cost estimation.
When the values of dependent variables, such as software size, change greatly relative error should be used to evaluate the fidelity of the prediction model. Relative error is usually defined by the equation: RY1=(Z1−Y1)/Y1 where Z, is the estimated value of a dependent variable and Yi is the actual value of the variable for the ith sample data, respectively. According to this definition, RYi has a lower bound of — 1 and an unlimited upper bound. Therefore, lenient evaluation is obtained only in the case of underestimation. To complement this defect, the paper defines the relative error R1 as: •• If Zi − Yi ⩾ 0 then Ri = (Zi − Yi)/Yi•• If Zi − Yi < 0 then Ri = (Zi − Yi)/ZiRi has no bound for both underestimation and overestimation. Hence a balanced evaluation is possible for both cases.To estimate the parameter values in a software prediction model, the paper proposes using the least squares method, which minimizes the sum of squares of Ri, instead of the ordinary least squares method, which minimizes the sum of squares of errors. With the sizing data from 15 completed systems, the paper shows that the method actually makes it possible to obtain better parameter values compared with some other methods, including the ordinary least squares method. The result also shows that there is a distinct possibility that a practical model, which enables sizing estimation in early phases of software development, can be developed for business application systems by the method proposed.
When a panel of experts is assembled to make predictions about some aspect of the future, they invariably disagree. The possible strategies for dealing with such disagreement include (1) taking the statistical average of the individual forecasts, (2) face-to-face discussion until consensus is achieved, (3) the Delphi procedure, and (4) the Estimate-Talk-Estimate procedure proposed by D. H. Gustafson, R. K. Shukla, A. Delbecq, and G. W. Walster (Organizational Behavior and Performance, 1973). This paper does two things. First, it very briefly reviews the literature relevant to opinion aggregation when forecasts are expressed as subjective probability distributions. Second, it describes an experimental comparison of the four procedures listed above using a subjective probability forecasting task. Together, the review and the experiment lead to two conclusions. First, subjective probability forecasts can be substantially improved by aggregating the opinions of a group of experts rather than relying on a single expert. Second, from a practical standpoint, there is no evidence to suggest that the method used to aggregate opinions will have a substantial impact on the quality of the resulting forecast.
From the Publisher:Software Metrics explains how software measurement can be used to support software process improvement by providing objective methods of characterizing process capability and evaluating the effect of process changes. The author explains what is meant by software measurement and how to decide what to measure; how to use measurement to support different aspects of a process improvement programme; how to set quantitative goals using a pragmatic approach to the Goal-Question-Metric paradigm; how to set up a metrication programme and design a data collection system; and how to analyse the software data collected. Further, the author provides insight into how software measurement can act as a process improvement agent for quality control and software estimating. This book sets out an approach to the formal validation of measures and the theory of statistical data analysis for students. It also contains, for practitioners, many examples of the use of real data in real projects.
The item you have requested is not an article but an entire textbook. You will have to purchase it at a bookstore, probably a textbook store at a University or college.
Success is found relatively rare in the world of software projects. One possible reason may be the difference in the perception of the meaning of ‘success’ in the minds of people who evaluate the project performance. Usually, stakeholders external to the project organization use target cost and time for inferring ‘project success’ while those internal to project agree that the attainment of ‘scope’ of development decides the ‘project success’. Hence, project success criteria, as believed by different groups of stakeholders, do not match. In this study, we examine the views of one such internal set of stakeholders: Programmer/Developers, Project Managers and Customer Account Managers. We conducted an exploratory survey to determine their view of a successful project. We found surprising uniformity in different constituents of this particular group of stakeholders and all of them overwhelmingly considered meeting the ‘scope’ of software projects, which comprises the functionality and quality of the project outcome, as the highest determinant of success. We believe that this view of software project success in the eyes of software professionals should be studied further to build a comprehensive project evaluation framework and should be utilized effectively to achieve success in terms of external objectives like customer satisfaction and customer happiness.
This paper summarizes the current state of the art and recent trends in software engineering economics. It provides an overview of economic analysis techniques and their applicability to software engineering and management. It surveys the field of software cost estimation, including the major estimation techniques available, the state of the art in algorithmic cost models, and the outstanding research issues in software cost estimation.
The results from the study described in this paper suggest that customer expectations of a project's total cost can have a very large impact on human judgment-based estimates (expert estimates) of the most likely use of software development effort. The information that the customer expectations did not constitute valid input data for making estimates did not remove the impact. Surprisingly, the estimators did not notice this impact or assessed it to be low. An implication of the results is that the provision of a realistic project estimate of most likely use of effort may require that the estimators do not know the customer's expectations of the total cost of the project.
This paper reviews some past research that may be relevant to the way that ‘expert judgement’ is used to produce estimates of software size and implementation effort. It then describes a survey of software estimators in one organization conducted via written questionnaires and some follow-up interviews. The survey paid particular attention to the information requirements of the estimators. A large proportion of the system development was found to involve the modification of existing systems and as a consequence a large part of the estimating process comprised the assessment of the impact of changes on existing code. The paper concludes by suggesting that some work could profitably be applied to attempting to support expert judgement rather than displacing it.
ABSTRACT Many software companies,track and analyze project performance by measuring cost estimation accuracy. A high estimation error is frequently interpreted as poor ,estimation ,skills. This is not necessarily a correct interpretation. High estimation error can also bea result of other factors, such as high estimation complexity and insufficient cost control of the ,project. Through a real-life example,we illustrate ,how ,the lack of proper ,estimation error analysis technique can bias analyses of cost estimation accuracy and lead to wrong conclusions. Further, we examine a selection of cost estimation studies, and show that they frequently do not take the necessary ,actions to ensure ,meaningful ,interpretations of estimation error data. Motivated by these results, we propose a general framework that, we believe, will improve analyses of software cost estimation error. Keywords
Group estimation of user stories is an important part of extreme programming (XP), used for both planning releases and iterations. Research has shown that although group estimation in many cases is superior to individual estimation, there is still room for improvement. For instance, group estimation performance can be reduced by dominant personalities and anchoring effects. Through the analysis of 101 user story estimates, made by an XP team for release planning, we investigate whether the introduction of the planning poker estimation process improved the estimation ability of the team. The results show that planning poker improved the team's estimation performance in most cases, but that it increased estimation error in the extreme cases
The effort required to complete software projects is often estimated, completely or partially, using the judgment of experts, whose assessment may be biased. In general, such bias as there is seems to be towards estimates that are overly optimistic. The degree of bias varies from expert to expert, and seems to depend on both conscious and unconscious processes. One possible approach to reduce this bias towards over-optimism is to combine the judgments of several experts. This paper describes an experiment in which experts with different backgrounds combined their estimates in group discussion. First, 20 software professionals were asked to provide individual estimates of the effort required for a software development project. Subsequently, they formed five estimation groups, each consisting of four experts. Each of these groups agreed on a project effort estimate via the pooling of knowledge in discussion. We found that the groups submitted less optimistic estimates than the individuals. Interestingly, the group discussion-based estimates were closer to the effort expended on the actual project than the average of the individual expert estimates were, i.e., the group discussions led to better estimates than a mechanical averaging of the individual estimates. The groups’ ability to identify a greater number of the activities required by the project is among the possible explanations for this reduction of bias.
To outline the key concepts and principles of the Delphi technique.
Reference is made to a selection of studies that illustrate a variety of methodological interpretations. Drawing on Heshusius's concept of 'goodness criteria', particular emphasis is given to the question of scientific merit and means of evaluation.
Although the technique should be used with caution, it appears to be an established method of harnessing the opinions of an often diverse group of experts on practice-related problems.
This paper systematically reviews empirical studies looking at the effectiveness of the Delphi technique, and provides a critique of this research. Findings suggest that Delphi groups outperform statistical groups (by 12 studies to two with two ‘ties’) and standard interacting groups (by five studies to one with two ‘ties’), although there is no consistent evidence that the technique outperforms other structured group procedures. However, important differences exist between the typical laboratory version of the technique and the original concept of Delphi, which make generalisations about ‘Delphi’ per se difficult. These differences derive from a lack of control of important group, task, and technique characteristics (such as the relative level of panellist expertise and the nature of feedback used). Indeed, there are theoretical and empirical reasons to believe that a Delphi conducted according to ‘ideal’ specifications might perform better than the standard laboratory interpretations. It is concluded that a different focus of research is required to answer questions on Delphi effectiveness, focusing on an analysis of the process of judgment change within nominal groups.
The required effort of a task can be estimated subjectively in interviews with experts in an organization in different ways. Interview techniques dealing with which type of questions to ask are evaluated and techniques for combining estimates from individuals into one estimate are compared in an experiment. The result shows that the interview technique is not as important as the combination technique. The estimate which is best with respect to mean value and standard deviation of the effort is based on an equal weighting of all individual estimates. The experiment is performed within the Personal Software Process (PSP)
Whenever I'm asked to recommend estimation methods and techniques, I always point to 12 issues you must consider in making accurate estimates, independent of the method, tool, or technique used. Satisfy these conditions and, as the article shows, your chances of making accurate estimates using your method or tool of choice will improve significantly. Violate one or more of them, however, and you'll run a significant risk of making inaccurate estimates regardless of the method or technique you've used.
The Wisdom of Crowds Doubleday
- J Surowiecki