Content uploaded by Burak Turhan
Author content
All content in this area was uploaded by Burak Turhan on Feb 20, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Implementation of a Software Quality Improvement Project in an SME: A Before and After Comparison
Abstract and Figures
Software process management is a crucial task in small and medium size enterprises (SME) due to scarce resources, small development teams and heavy workload.We have worked with an SME to institutionalize process improvement practices, aligned with CMMI maturity levels2 and 3. Main goals of this project were to achieve effective resource management and increase the product quality with fewer defects. The project had two phases: Phase I conducts an ¿as-is analysis¿ in order to set well-defined processes.Phase II measures the software reliability in terms of time/effort allocation and defect rates. Our before and after analysis showed that: a) time allocation for requirements,coding and testing steps improved from (4%, 31%, 65%) to(47%, 20%, 33%), b) defect rates decreased from 11% to6.5%, c) estimated testing effort also decreased from 47%to 30%.
Figures - uploaded by Burak Turhan
Author content
All figure content in this area was uploaded by Burak Turhan
Content may be subject to copyright.
... As noted in Table 2, in all studies (Wang et al., 2020b;Lin et al., 2020;Kumar and Mishra, 2016;Puri-Jobi, 2015;Hilton et al., 2017;Lee and Hwang, 2012;Collins and de Lucena, 2012;Tosun et al., 2009;Williams et al., 2009;Berner et al., 2005) that have observed how test automation maturity affects product quality, the findings verified the positive impact. These studies view product quality as to which extent a software product free from defects during the quality assurance process or in the production environment. ...
... It is different from scholars who defined product quality as conformance to a standard or to which extent a software product bears on its ability to satisfy given needs (Int'l Standards Organization, 2001;García-Mireles et al., 2015). Studies from Wang et al. (2020b), Hilton et al. (2017), Lee and Hwang (2012), Berner et al. (2005), Puri-Jobi (2015) and Tosun et al. (2009) reported that better product quality can be achieved after improving test automation maturity against certain standard best practices. ...
... An experience study (Ramler et al., 2014) reported that the direct effect of test automation maturity on test automation effort is negative. However, a different school of viewpoint reported by studies from Kumar andMishra (2016), Puri-Jobi (2015), and Tosun et al. (2009) views that, test automation effort will be reduced after improving the level of test automation maturity and product quality after a certain period of time -even though the considerable effort is required to develop and execute rigorous automated tests in attaining high product quality, the later effort savings may offset the invested effort. ...
The popularity of continuous integration (CI) is increasing as a result of market pressure to release product features or updates frequently. The ability of CI to deliver quality at speed depends on reliable test automation. In this paper, we present an empirical study to observe the effect of test automation maturity (assessed by standard best practices in the literature) on product quality, test automation effort, and release cycle in the CI context of open source projects. We run our test automation maturity survey and got responses from 37 open source java projects. We also mined software repositories of the same projects. The main results of regression analysis reveal that, higher levels of test automation maturity are positively associated with higher product quality (p-value=0.000624) and shorter release cycle (p-value=0.01891); There is no statistically significant evidence of increased test automation effort due to higher levels of test automation maturity and product quality. Thus, we conclude that, a potential benefit of improving test automation maturity (using standard best practices) is product quality improvement and release cycle acceleration in the CI context of open source projects. We encourage future research to extend our findings by adding more datasets with different programming languages and CI tools, closed source projects, and large-scale industrial projects. Our recommendation to practitioners (in the similar CI context) is to utilize standard best practices to improve test automation maturity.
... In the case of Mozilla Firefox, there is no significant association between release cycle frequency and product quality. [39] 2009 A software quality improvement project within a health care company in Turkey Experience report Test automation maturity; product quality; test automation effort ...
... [30] 2009 Test automation maturity achieved by using several standard best practices (define a good test automation strategy, carefully design and reuse automated tests, and build a testable SUT) can lead to shorter release cycles and better product quality. Table 2, in all studies [27,33,34,35,28,37,38,39,30,32] that have observed how test automation maturity affects product quality, the findings verified the positive impact. These studies view product quality as to which extent a software product free from defects during the quality assurance process or in the production environment. ...
... It is different from scholars who defined product quality as conformance to a standard or to which extent a software product bears on its ability to satisfy given needs [40,41]. Studies [27,28,37,32,35,39] reported that better product quality can be achieved after improving test automation maturity against certain standard best practices. ...
The popularity of continuous integration (CI) is increasing as a result of market pressure to release product features or updates frequently. The ability of CI to deliver quality at speed depends on reliable test automation. In this paper, we present an empirical study to observe the effect of test automation maturity (assessed by standard best practices in the literature) on product quality, test automation effort, and release cycle in the CI context of open source projects. We run our test automation maturity survey and got responses from 37 open source java projects. We also mined software repositories of the same projects. The main results of regression analysis reveal that, higher levels of test automation maturity are positively associated with higher product quality (p-value=0.000624) and shorter release cycle (p-value=0.01891); There is no statistically significant evidence of increased test automation effort due to higher levels of test automation maturity and product quality. Thus, we conclude that, a potential benefit of improving test automation maturity (using standard best practices) is product quality improvement and release cycle acceleration in the CI context of open source projects. We encourage future research to extend our findings by adding more datasets with different programming languages and CI tools, closed source projects, and large-scale industrial projects. Our recommendation to practitioners (in the similar CI context) is to utilize standard best practices to improve test automation maturity.
... [16] 2. Working with tight budgets and resources, which makes the deliverables, depends on the heroic efforts of a few people. [17] 3. Late delivery and Low Productivity, cause a lot of customer complaints, and too much rework. ...
... However, their response time for a change request is long and sometimes the level of quality of services is poor. [17] 5. The employee may play multiple roles in the company, for example a programmer might play the role of a technical architect, developer and tester simultaneously. ...
... [16] 6. Heavy reliance on engineers rather than processes. [17] 7. ...
... Tosun et al. [41] present a software quality improvement project (SQIP) by selecting specific practices from the CMMI maturity levels 2 and 3, then they conduct an internal, non-formal appraisal on two projects -one had no quality processes and another had SQIP and the differences were studied. The selected developer for the case study is a small enterprise, with challenges in several issues such as defect rate and performance measurement, project management, and employee training. ...
Software process improvement (SPI), and by extension Software Quality Assurance (SQA), is the approach to understand the software development process lifecycle and implement necessary changes to the processes to achieve a high-quality, maintainable product. Small software enterprises face enormous challenges to gain a competitive advantage in the software industry, especially with the presence of large conglomerates. Much of these small-to-medium enterprises (SMEs) adopt agile models such as Scrum to quickly react to clients' demands. However, agile methodologies lack direct addressing of maturity in process, project and product that larger enterprises are capable of. It is important for the software engineering community to aid in enabling SMEs to have process maturity without compromising agility. In this paper, we use the Capability Maturity Model Integration (CMMI)-a world-renowned software quality assurance methodology, to address some shortcomings in the Scrum model. Specific practices are selected out of eighteen process areas based on both literature research and field study from the second and third levels of maturity to address missing elements. The proposed model prototype keeps Scrum intact while allowing small enterprises to produce high-quality software without compromising agility or going over budget, thus reducing the 'low-quality' stigma attached to small software developers around the world.
... 3) Widespread adoption of the agile model: The agile model focuses on quickly developing usable software, making the incorporation of security practices in all stages of the agile SDLC challenging [46], [47]. 4) Reliance on developers for security: Software SMEs rely more heavily on developers than processes [48]. Security in the SDLC is thus being left to developers, who usually do not have the security expertise to adequately manage and implement secure software practices [13], [49]. ...
Software is widely deployed and used for managing critical daily domestic, social, and economic activities. Due to software’s economic value, software is a high-value target of malicious actors and a primary source of many information security vulnerabilities. Software must be engineered to be secure because of its value. Traditional approaches to software security treat software as an addon and have been proven inadequate at producing secure software. Practicing the secure software development lifecycle (SSDLC) is recommended in academic literature. Software SMEs must adopt and practice the SSDLC for increased security of published software. This paper explores the SSDLC and makes a case for its adoption with the goal of informing security decision-makers of Software SMEs.
... SSCs 17 [23], [24], [55], [58], [59], [61], [65], [67], [75], [83], [96], [97], [101], [106], [113], [122], [125] 4 SME 16 [56], [57], [63], [66], [76], [85], [86], [93]- [95], [104], [105], [108], [110], [118], [123], [124] (c) Framework/standards 1 None 36 [19], [23], [32], [35], [36], [63], [66], [68], [70], [78], [81], [83]- [87], [93], [94], [96]- [102], [104], [109], [118]- ...
Small software companies face numerous challenges of complexity, unstructured software development processes and scarce resources. This notwithstanding, the companies have dominated the software market by 80 percent. The practice and products of these companies are still persistently marred by quality issues arising from the processes, with evidence indicating that process tools do not fit the unique contexts in which they operate. Significant strides have been made in transforming software development practice; however, the challenges are still evidently apparent. Hence the need to establish how knowledge areas are applied in process practice, understand the context of software development and its implication in practice, how process tools are utilised in practice and evaluate quality of research in software literature The researchers undertook a systematic mapping study to determine the state of practice in empirical literature on software engineering of SSCs by examining and classifying 1096 publications. Other than the finding that research quality was low and affecting generalisation and transferability, the results also revealed interesting findings which we finally consolidated and integrated to develop two contributions (i) a software development process adoption theoretical framework which provides new insights of understanding software development and (ii) a 3-point guideline for research quality.
... In addition, it is necessary to have good documentation from each employee. So, if there are additions, developments, changes to a feature requested by the customer, then the work does not have to be done by the same person [37]. ...
A software house, that established in 2005 based in Indonesia, got 31 projects in 2019. By the end of year, Project Management Officer released documents to inform company’s project health. There are 14 projects confirmed late, 6 projects on time and 11 projects scheduled complete on the next year. That late projects cause serious problem like loses revenue and gets disrupted of company’s cash flow. Based on the root cause analysis, it found that no standardization of software development process in the company. Before designing the standardization to improve process, we need to analyze the obstacles that might be happened. Therefore, this study aims to identify the obstacles on software process improvement in software house. We performed a systematic literature review to determine the obstacles, then we do empirical research to 58 employees on company’s development department to sort the priority of obstacles in the company. From the systematic literature review, we found studies that relevant and there are 13 obstacles of software process improvement, then from the empirical research we got top three obstracles. We also proposed recommendations to solve that obstacles.
... For example, the study by Díaz-Ley et al. [9] reported the experiences of a Spanish SME in implementing an MP and reported that the practitioners could now objectively evaluate the trade-off between on-time releases and software product reliability. Tosun et al. [50] collaborated with a Turkish healthcare SME to institutionalize process improvement practices, and reported improvements in the organization's time allocation for requirements, coding, and testing steps. Furthermore, the authors found that defect rates as well as testing effort estimation decreased. ...
Context: Agile software development has become commonplace in software development companies due to the numerous benefits it provides. However, conducting Agile projects is demanding in Small and Medium Enterprises (SMEs), because projects start and end quickly, but still have to fulfil customers’ quality requirements. Objective: This paper aims at reporting a practical experience on the use of metrics related to the software development process as a means supporting SMEs in the development of software following an Agile methodology. Method: We followed Action-Research principles in a Polish small-size software development company. We developed and executed a study protocol suited to the needs of the company, using a pilot case. Results: A catalogue of Agile development process metrics practically validated in the context of a small-size software development company, adopted by the company in their Agile projects. Conclusions: Practitioners may adopt these metrics in their Agile projects, especially if working in an SME, and customise them to their own needs and tools. Academics may use the findings as a baseline for new research work, including new empirical studies.
... Exploratory Testing is an approach that helps to identify unknown bugs which are missed during traditional testing activities [13][14][15][16][17][18]. In agile development process, exploratory testing plays an important role which can also be combined with other analytical risk-based testing, model-based testing and regression-averse testing [19]. ...
... Tosun et al. [24] present their work, named software quality improvement project (SQIP) by selecting specific practices from the CMMI maturity levels 2 and 3. They use two projects to compare the results of the one with no quality processes (default) with the results of the one using SQIP. ...
Software process improvement (SPI) is the approach to understand the software development process lifecycle and implement necessary changes to achieve a high-quality, maintainable product. The software industry is comprised of small and medium-sized enterprises that adopt agile models as their preferred model of development. The Capability Maturity Model Integration (CMMI) model for software process improvement and quality is used to address aspects of software quality and maturity that Scrum was not initially designed to consider. CMMI is used for its reliable practices and consideration of many aspects of software development in process, project and product. After consideration of literature, three process areas are selected to address necessary elements in Scrum used by software developers in offshoring destinations. The practices are selected based on their practicality in small, agile settings and the ability to be incorporated into Scrum activities without disrupting the models sprint cycles. This opens the way for smaller enterprises to create produce quality.
In this paper, we present a defect prediction model based on ensemble of classifiers, which has not been fully explored so far in this type of research. We have conducted several experiments on public datasets. Our results reveal that ensemble of classifiers considerably improve the defect detection capability compared to Naive Bayes algorithm. We also conduct a cost-benefit analysis for our ensemble, where it turns out that it is enough to inspect 32% of the code on the average, for detecting 76% of the defects.
Small software development companies need it to ensure they stay on course and are able to respond to the market's ebbs and flows.
The United States Air Force sponsored research within the Department of Defense (DoD) so~are development community to determine the applicability of the Sojiware Engineering Institute’s (SEI) Capability Maturity Model (CW) for Sojlware to small businesses and small softiare organizations. The research found that small businesses are faced not only with a lack of resources and funds required to implement many of the practices stated in the CM, but also with the task of basing their process improvement initiatives on practices that do not apply to a small business and small sojlware organization. This paper discusses, from industryh perspective, why small businesses and organizations are experiencing dif]culties implementing CMM-based process improvement programs and how they are tailoring their approach to the GUM to meet their quality goals.
This paper reports on the construction and validation of fault- proneness prediction models in the context of an object-oriented, evolving, legacy system. The goal is to help QA engineers focus their limited verification resources on parts of the system likely to contain faults. A number of measures including code quality, class structure, changes in class structure, and the history of class- level changes and faults are included as candidate predictors of class fault-proneness. A cross-validated classification analysis shows that the obtained model has less than 20% of false positives and false negatives, respectively. However, as shown in this paper, statistics regarding the classification accuracy tend to inflate the potential usefulness of the fault-proneness prediction models. We thus propose a simple and pragmatic methodology for assessing the cost-effectiveness of the predictions to focus verification effort. On the basis of the cost-effectiveness analysis we show that change and fault data from previous releases is paramount to developing a practically useful prediction model. When our model is applied to predict faults in a new release, the estimated potential savings in verification effort is about 29%. In contrast, the estimated savings in verification effort drops to 0% when history data is not included.