ArticlePDF Available

From Pragmatic to Systematic Software Process Improvement: An Evaluated Approach

Authors:

Abstract

Software processes improvement (SPI) is a challenging task, as many different stakeholders, project settings, and contexts and goals need to be considered. SPI projects are often operated in a complex and volatile environment and, thus, require a sound management that is resource-intensive requiring many stakeholders to con- tribute to the process assessment, analysis, design, realisation, and deployment. Although there exist many valuable SPI approaches, none address the needs of both process engineers and project managers. This article presents an Artefact-based Software Process Improvement & Management approach (ArSPI) that closes this gap. ArSPI was developed and tested across several SPI projects in large organisations in Germany and Eastern Europe. The approach further encompasses a template for initiating, performing, and managing SPI projects by defining a set of 5 key artefacts and 24 support artefacts. We present ArSPI and discus results of its validation indicating ArSPI to be a helpful instrument to set up and steer SPI projects.
From%Pragmatic%to%Systematic%Software%
Process%Improvement:%An%Evaluated%
Approach
!
Marco!Kuhrmann1,2,!Daniel'Méndez'Fernández2!
1University!of!Southern!Denmark,!Mærsk!Mc;Kinney!Møller!Institute,!
Campusvej!55,!DK;5230!Odense!M,!Denmark!
2Technische!Universität!München,!Faculty!of!Informatics!–!Software!&!
Systems!Engineering,!Boltzmannstr.!3!85748!Garching,!Germany
!
Corresponding!Contact:!!
Marco!Kuhrmann!
!! E;Mail:!kuhrmann@mmmi.sdu.dk!!
!! Phone:!+45!24!60!14!22!
!
©!IET!2015.!Preprint.!This!is!the!author's!version!of!the!work.!The!definite!version!
was!accepted!in!IET!Software,!Issue!assignment!pending,
The!final!version!is!available!in!the!IET!Digital!Library!
http://digital;library.theiet.org/content/journals/iet;sen!
!
!
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 1
From Pragmatic to Systematic Software Process
Improvement: An Evaluated Approach
Marco Kuhrmann1,2, Daniel Méndez Fernández2
1University of Southern Denmark, The Mærsk Mc-Kinney Møller Institute, 5230 Odense, Denmark
E-Mail: kuhrmann@mmmi.sdu.dk
2Technische Universität München, Faculty of Informatics, 85748 Garching, Germany
E-Mail: mendezfe@in.tum.de
Correspondence to: Marco Kuhrmann, kuhrmann@mmmi.sdu.dk
Abstract
Software processes improvement (SPI) is a challenging task, as many different stakeholders, project settings,
and contexts and goals need to be considered. SPI projects are often operated in a complex and volatile envi-
ronment and, thus, require a sound management that is resource-intensive requiring many stakeholders to con-
tribute to the process assessment, analysis, design, realisation, and deployment. Although there exist many valu-
able SPI approaches, none address the needs of both process engineers and project managers. This article pre-
sents an Artefact-based Software Process Improvement & Management approach (ArSPI) that closes this gap.
ArSPI was developed and tested across several SPI projects in large organisations in Germany and Eastern Eu-
rope. The approach further encompasses a template for initiating, performing, and managing SPI projects by
defining a set of 5 key artefacts and 24 support artefacts. We present ArSPI and discus results of its validation
indicating ArSPI to be a helpful instrument to set up and steer SPI projects.
Keywords
Software Process Improvement, Software Process Management, Artefact Orientation
1 Introduction
Software processes comprise many process assets, which need to be designed, implemented, quality assured,
and managed in the context of an organisation-wide software process management (SPM; [27]). Software pro-
cess improvement (SPI; [12]) aims at the systematic analysis, re-/design, and evaluation of a particular process.
As part of SPM, it forms an important step for organisations of all sizes to succeed in the market [11]. However,
SPI is costly and improved processes need time to be disseminated, making the impact of SPI hard to measure
and justify [4], [6], [8]. Therefore, and because of the associated costs, many software managers are reluctant to
conduct SPI [6], or companies give up SPI at all [10]. Niazi et al. [28] mention the importance of an effective
strategy to successfully implement SPI, what is especially true for the management of a process’s evolution
after its initial deployment, i.e. changes must be tracked [29] and the evolution of external standards must be
considered in the process maintenance [23]. Furthermore, a sound project organisation, i.e. allocating resources,
defining deliverables, or tracking progress, is crucial to set up a systematic SPI [3].
From the perspective of a process engineer, we still miss a guidance to conduct a flexible but systematic SPI
project going beyond purely assessment-driven approaches, like CMMI [5] or ISO/IEC 15504 [14]. That is, we
need support for process engineers to systematically organise and perform SPI projects in the context of an SPM
strategy, while leaving open the way of conducting particular improvements.
In this article, we contribute a model for an Artefact-based Software Process Improvement & Management
(ArSPI). The model emerges from initial pragmatically conducted improvement activities [18]. Based on our
experiences across several SPI projects in large organisations in Germany, we inferred and systematised our
approach, which we applied and validated in practice in Germany and Eastern Europe. Our approach relies on
the principle of artefact orientation [25]. That is, by concentrating on the key artefacts to be created in SPI pro-
jects, we abstract from actual SPI activities. We thus give process engineers the freedom to apply methods ap-
propriate for their particular situation while being able to clearly define the interfaces to supporting activities,
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 2
e.g., quality management. ArSPI defines a template that process engineers can use to set up and manage SPI
projects.
The remainder of this article is organised as follows. In Section 2, we discuss related work, which gaps are left
open, and how we intent to close those gaps. In Section 3, we introduce our approach in detail, before giving a
concluding summary in Section 4.
2 Related Work
In this article, we present an SPI method that, compared to existing models, follows an alternative approach.
Instead of focussing on assessments or specific improvement procedures, our model is based on the principle of
artefact orientation [25]. According to Frailey [9], relying on artefacts is advantageous as artefacts ease, inter
alia, the creation of a common terminology. In a study on the perception of artefact-based software processes
[20] and in an experiment on the perception of process modelling [21], we further found indicators supporting
Frailey’s conclusion. Reviewing available and well-disseminated SPI models, e.g., CMMI [5], ISO/IEC 15504
[14], and ISO/IEC 12207 [13], we find, however, that the focus in current approaches lies on providing compre-
hensive descriptions of principles and procedures rather than on providing precise artefact models. Furthermore,
these models are often criticised to be too voluminous, too complex, or to result in processes that might lead to
an improvement alien to the organisation [2], [26], [32]. In response to this shortcoming, tailored variants of
these models aim at better addressing small and very small companies, e.g., ISO/IEC 29110 [15]. Those ap-
proaches, however, remain normative and they usually focus on process assessments only. Even the recently
published standard ISO/IEC 33014 [16] focuses on activities without precisely defining the required artefacts.
Apart from the standards, several method proposals were made that rely on best practices and standards empha-
sising needs of small companies. PROCESSUS [11], BG-SPI [1], LAPPI [31], and BOOTSTRAP [24], are
representatives of such methods. These proposals comprise activity-based guidelines providing detailed proce-
dures process engineers should follow (e.g., [11], [1], and [31]), or they aim to simplify process assessments
(e.g., [24]). Artefacts are mentioned (e.g., PROCESSUS and BG-SPI), but detailed models of artefact structures
and relationships are not provided. Activity-based approaches therefore encounter problems in practice: What if
the described order of activities does not meet the needs of the actual context? A missing description of the
expected outcomes hampers learning curves of process engineers, limits the comparability of SPI projects, and
thereby limits the opportunities to create reusable assets for enhancing improvement processes, which are all
aspects investigated in the context of SPI success factors (e.g., Melzer and Stellis [33]) and human aspects in
SPI (e.g., Viana et al. [34]).
Our proposed approach precisely defines the artefacts allowing for creating a model of the expected results that
can be tested, e.g., for completeness and consistency. Furthermore, an artefact-based approach allows for bridg-
ing the gap between single SPI project instances and organisation-wide SPI programs to provide SPI projects
with a stable environment as, for instance, recommended by Rainer and Hall [30]. Artefact models remain sta-
ble and only the respective methods for the artefact creation need to be adapted for the respective context, which
allows for, e.g., a flexible tailoring of improvement endeavours as considered crucial by Melzer and Stellis [33].
The subsequently presented ArSPI1 model provides scalable and adaptable SPI project- and artefact templates
[17] supporting process engineers to set up and organise SPI projects. ArSPI defines a method-agnostic, but
general structure (the embodiment with particular methods is out of scope) that process engineers can use in
combination with their preferred methods as, e.g., previously contributed for the RE improvement domain [26].
This article is based on previously published material: We first presented ArSPI as method proposal [19], gave a
brief overview of the key concepts, and provided two practical examples. The full ArSPI model, i.e. all UML
models, tailoring profiles, and so forth, are documented in our complementing technical report [17], and the
overall construction procedure of ArSPI can be depicted from [18]. In this article, we extend the presentation of
ArSPI by providing more details and background, and we provide information on the validation of ArSPI in
academia and practice.
1 The ArSPI website: http://www4.in.tum.de/~kuhrmann/arspi.shtml
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 4
artefacts, accompany these key artefacts. Table 2 lists selected support artefacts and provides a brief description.
The complete list of support artefacts can the depicted from [17].
Table 1 Overview of the ArSPI key artefacts.
ID
Artefact Description
SPI Project Life
Cycle Phase
PRQ
The Process Requirements artefact contains all requirements regarding the process. To
collect all relevant requirements, the PRQ defines the following top-level structure:
Goals
Stakeholders and Roles
Requirements
Overall Process Draft
Technical Infrastructure
Basic Conditions
Analysis
CPD
The Conceptual Process Design contains all designs of a process without paying attention
to any technical realisation. It refines all process-related requirements and transfers them
into concrete processes and process parts. It defines the following top-level structure:
Goals (shared with PRQ)
Principles
Planned Adaptations: Organisation and Roles, Artefacts, and Processes
Additional Requirements: Tailoring, Process Documentation, and Supporting Material
Requirements Tracing (shared with PRQ and TPD)
Conceptualisation
TPD
The Technical Process Design refines the CPD regarding a concrete technical realisation
and tool/tool infrastructure to be used for its realisation:
(Refinement of the CPD structure, cf. Figure 2)
Logical and Physical Model Organisation
Realisation
PLC
The Process Life Cycle Support comprehends all information, agreements, and definition
regarding complementing processes supporting the deployment, training, and further
development of a concrete process as well as its evaluation and measurement:
Training
Deployment and Further Development
Measurement and Evaluation
Change Management
Created early in
the Analysis phase,
at latest in the
Deployment phase
PR
A Process Release is a concrete process package that is shipped and deployed. The
results produced in the SPI project dynamically define the PR’s structure.
Deployment
Table 2 Selection of the ArSPI support artefacts.
Artefact
Description
User Evaluation Plan
While the measurement plan of a process aims at measuring the process performance in gen-
eral, a User Evaluation Plan aims to evaluate the actual use of a process. In contrast to “classic
KPI-based measuring, the user evaluation is more of a qualitative nature.
Training Material
Training Material consists of material to train the process consumers. The Training Material is
specific for certain user groups/stakeholders and for particular Process Releases. Usually, Train-
ing Material is explicitly defined for stakeholder groups and, therefore, provides different per-
spectives and information at different levels of abstraction.
SPL-Delta Report
If the considered process is based on a software process line (SPL), a delta report is helpful to
analyse deviations from the SPL base process to support long-term development (having the
SPL’s evolution in mind), and to support compliance assessments.
ArSPI’s artefact model defines the structure of the particular artefacts. A comprehensive set of associations
connects the artefacts with each other to allow for refinements and tracing, e.g., which requirements are how
designed and realised. Figure 2 shows an example of the UML model in which the Conceptual Process Design
and Technical Process Design, and their relationships are illustrated.
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 9
ments, (2) to analyse the model’s consistency and completeness, and to (3) develop/refine the instruments to be
used in the external validation. Furthermore, the internal validation paves the way for independent replication
studies. The external validation aims at providing insights into practical settings regarding benefits and short-
comings to prepare dissemination and further investigations. As our approach was developed based on our ex-
periences to systematise the pragmatic approaches in the past [18], the conducted case studies thereby aim at
investigating whether and to which extent ArSPI generally supports process engineers in conducting a systemat-
ic SPI.
3.3.1 Overview of conducted Studies and Outcomes
In Table 3, we give an overview of the overall strategy implemented so far (studies that were recently added to
the evaluation from [18] are marked with “new”). We summarise the instruments, context, a brief study descrip-
tion, and outcomes.
Table 3 Overview of the elements of the validation strategy.
No.
Validation
Inst
I/E
Ctx
Description
Process
Templates
Tool
Training
1
Exp
I
U
This quasi-experiment was conducted in a course on soft-
ware process modelling [22], and aimed at investigating the
feasibility of an artefact-based SPI approach in general. The
whole experiment is described in [21].
X
2
CS
E
I
In this industry-hosted case study, a new process should be
developed, which aimed to define management and devel-
opment procedures. The new process was based on the V-
Modell XT. As no case existed in advance, the case study
could not be conducted in a comparative manner. However,
in order to set up a continuous improvement and manage-
ment, the process was evaluated using interviews to create
reference values for further evaluation. A detailed descrip-
tion can be depicted from [18] and Section 3.2.2.
X
X
X
3new
CS
I
U
In this investigation, the release 0.9 of ArSPI was analysed
for completeness. The overall goal was to define require-
ments for ArSPI’s further improvement based on the experi-
ences gathered so far and using CMMI as external refer-
ence.
X
X
4new
CS
E
I
In this case study, ArSPI was evaluated from the perspective
of the process owner who is responsible for his company’s
SPM. We combined elements from experimentation and
case study research to evaluate ArSPI in a real-world set-
ting, and to provide a controlled environment to gather de-
tailed insights into the execution of the project. For this, two
industry partners defined the requirements and acted as
clients in the project. A student performed the SPI activities.
The objective was to tailor the Scrum process respecting the
predefined requirements. We as developers of ArSPI moni-
tored the project and performed a continuous evaluation
regarding, e.g., product and process quality.
X
X
5
CS
E
I
This study is a long-term industry case study in which the
organisation-wide software process is subject to continuous
improvement. In this particular setting, the ArSPI model is
applied to SPI projects as well as to the organisation-wide
quality- and process management. A description of this
setting can be depicted from [18] and Section 3.2.1.
X
X
X
X
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 10
No.
Validation
Inst
I/E
Ctx
Description
Process
Templates
Tool
Training
6
Int
E
I
During the construction of the ArSPI model, we conducted
interviews with external partners from industry and academ-
ia, who are experienced in SPI. The interviews aimed at
investigating the completeness of the constructed model,
and to figure out improvement potential. A description of the
interview and its outcomes can be depicted from [18].
X
Legend:
Inst Instrument: Exp = experiment, CS = Case Study, Int = interview; I/E internal/external validation: I = internal, E = external;
CtxContext: U = university/academic, I = industry
3.3.2 Summary of Conclusions
So far, we developed ArSPI in an inductive manner complemented with continuous validation and evaluation
activities serving its improvement. From the initially conducted studies, we could extract the following findings:
Process consumers, e.g., process owners or tool developers, benefit from an artefact-based SPI approach as
the artefact-based approach allows for, e.g., a precise definition of process entities for tool support or pro-
cess enactment [21]. A major finding was that we can rate the success of an SPI project by rating the out-
come, i.e. we imply a notion of SPI quality in relation to the quality of the outcome while abstracting from
the way of producing the outcome which is the fundamental principle of artefact orientation.
Process engineers benefit from an artefact-based SPI approach by being provided with a clearly structured
model serving as reference to design/improve processes [18], [21]. For example, in study 2 (Table 3), the
evaluation of release 0.9 of the developed process indicated to gaps, which could be directly aligned to
change requests; process owners mentioned missing artefacts and 5 missing artefacts could be identified. As
figured out in [21], we can rate the quality of SPI projects by rating the outcomes.
Experts consider ArSPI useful, as, for instance, it helps to structure SPI projects, and to reflect on SPI activ-
ities [18]. A major finding was the flexibility of the ArSPI model that allows for tailoring and applying
ArSPI in different contexts, e.g., large and small, and short- and long-term SPI projects/programs.
However, the number and character of the conducted case studies limit our initial findings. For instance, so far,
completed case studies mainly address stakeholders related to process management, and, thus, project managers
and software developers were not in scope as primary study subjects. However, in a complementing study [20],
we could find indication to benefits for these stakeholder groups as well.
3.3.3 Exemplary Results
In the studies 3 and 4 (Table 3), we aimed at conducting a comparative in-depth analysis of ArSPI compared to
previously used approaches. In the following, we provide insight into the industry-hosted study 4 in which we
conducted a completely monitored case study. Two industry partners were personally invited to participate in
the case study and were asked to rate the ArSPI approach in relation to their experiences.
Figure 6 illustrates the final rating of the experts as an exemplarily evaluation of ArSPI (the ratings are based
questionnaires and interviews, values are on an 8-point Likert scale). Expert 1 has experienced 6 medium- to
large-sized SPI projects, mainly in the context of public administration. Expert 2 has conducted about 50 SPI
and SPI-related projects in different industry contexts.
Figure 6 shows that especially expert 2 rates the approach significantly better than the previously experienced
ones. He stated that although there were some limitations by the study’s setting, the ArSPI approach worked
“better that everything else compared to what happens in practice right now.” The evaluation of expert 1 shows
a different picture. Expert 1 also rates structuredness, knowledge transfer, and explicit analysis and design pro-
cedures “good” (5 to 6). However, based on his experiences, he gave a low rating for the other criteria resulting
from “the way the process engineer applied the model in this case study.”
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 12
4.1 Impact and Implications
The ArSPI model provides a blueprint for setting up and organising SPI projects. The model, its documentation,
and templates are available online. ArSPI is focused on the artefacts needed by process engineers to analyse
process requirements, to design and implement processes, and to ship processes and establish a continuous im-
provement. Since ArSPI is focused on artefacts, process engineers can directly apply the model to structure SPI
activities.
Researchers and practitioners as well get with our contribution already insights into benefits and shortcomings
in SPI in general and in artefact-based SPI in particular. As we created an experimental setting in which SPI
activities can be analysed, compared, and evaluated, we actively contributed to the dissemination into academia
and practice and support to the replications of our studies and to further expand our knowledge on the broad
spectrum of SPI knowledge.
4.2 Future Work
ArSPI still needs a continuous validation to foster its improvement. Beyond an initial Eclipse Process Frame-
work-based implementation of ArSPI, we are also working on implementations using other frameworks and on
the development of further supporting material, e.g. checklists, evaluation questionnaires, etc. Findings from the
conducted studies become part of the next iteration of ArSPI, e.g., findings from recent studies 3 and 4 (Sec-
tion 3.3.3) define improvement requirements. Furthermore, ArSPI is under analysis for integration opportunities
with existing standards, e.g., the ISO/IEC 33014 [16]. In addition to the practical dissemination, we also plan to
extend the process-engineering lab [21] to systematically analyse and understand findings from practical stud-
ies. Those different steps serve the dissemination of our approach and, especially, the continuous, joint evalua-
tion of ArSPI to which we cordially invite researchers and practitioners.
Acknowledgements
We owe special thanks to Sarah Beecham and Ita Richardson for the fruitful discussion on SPI methods. Fur-
thermore, we thank Jens Calamé for reviewing the article manuscript.
5 References
[1] Aysolmaz, B., Demirörs, O.: ‘A Detailed Software Process Improvement Methodology: BG-SPI’, Proc.
European Conf. System & Software Process Improvement and Innovation, Roskilde, Denmark, June
2011, pp. 97108
[2] Beecham, S.: A Requirements-based Software Process Maturity Model’. PhD Thesis, Department of
Computer Science, University of Hertfordshire, UK, 2003
[3] Birk, A., Pfahl, D.: A Systems Perspective on Software Process Improvement’, Proc. Int. Conf. Prod-
uct Focused Software Process Improvement, Rovaniemi, Finnland, December 2002, pp. 418
[4] Boria, J. L.: A framework for understanding software process improvement’s return on investment’,
Proc. Portland Int. Conf. on Management and Technology Innovation in Technology Management,
Portland, USA, July 1997, pp. 847851
[5] CMMI Product Team,CMMI for Development, Version 1.3 (Software Engineering Institute, Carne-
gie Mellon University, 2010)
[6] Coleman, G., O’Connor, R.: Investigating software process in practice: A grounded theory perspec-
tive’, Journal of Systems and Software, 2008, 81, (5), pp. 772784
[7] Deming, W. E. Out of the Crisis. MIT Press, 2000.
[8] Eberlein, A., Jiang, L.: Description of a process development methodology’, Software Process: Im-
provement and Practice, 2007, 12, (1), pp. 101118
[9] Frailey, D. J.: Defining a corporate-wide software process’, Proc. Int. Conf. on the Software Process,
Redondo Beach, USA, October 1991, pp. 113121
[10] Hardgrave, B. C., Armstrong, D. J.: Software process improvement: It’s a journey not a destination’,
Communications of the ACM, 2005, 48, (11), 9396
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 13
[11] Horvat, R. V., Rozman, I., Györkös, J.: Managing the Complexity of SPI in Small Companies’, Soft-
ware Process: Improvement and Practice, 2000, 5, (1), pp. 4554
[12] Humphrey, W. S.: ‘Managing the Software Process’ (Addison Wesley, 1989)
[13] ISO, ISO/IEC 12207:2008: Systems and software engineering Software life cycle processes(Inter-
national Organization for Standardization, 2008)
[14] ISO, ISO/IEC 15504-4:2004: Software Process Assessment - Part 4: Guidance on use for process im-
provement and process capability determination’ (International Organization for Standardization, 2004)
[15] ISO, ISO/IEC 29110:2011: Systems and Software Life Cycle Profiles and Guidelines for Very Small
Entities (VSEs)’ (International Organization for Standardization, 2011)
[16] ISO, ISO/IEC 33014:2013: Information Technology Process Assessment Guide for Process Im-
provement’ (International Organization for Standardization, 2013)
[17] Kuhrmann, M.: ArSPI: An Artifact Model for Software Process Improvement and Management
(Technische Universität München, 2013)
[18] Kuhrmann, M.: Crafting a Software Process Improvement Approach A Retrospective Systematiza-
tion’, Journal of Software: Evolution and Process, 2015, 27, (2), pp. 114145
[19] Kuhrmann, M., Beecham, S.: Artifact-based Software Process Improvement and Management: A
Method Proposal’, Proc. Int. Conf. on Software and Systems Process, Nanjing, China, May 2014, pp.
119123
[20] Kuhrmann, M., Lange, C., Schnackenburg, A.: A Survey on the Application of the V-Modell XT in
German Government Agencies’, Proc. European Conf. System & Software Process Improvement and
Innovation, Roskilde, Denmark, June 2011, pp. 4960
[21] Kuhrmann, M., Méndez Fernández, D., Knapp, A.: ‘A First Investigation About the Perceived Value of
Process Engineering and Process Consumption’, Proc. Int. Conf. Product Focused Software Process
Improvement, Paphos, Cypros, June 2013, pp. 138152
[22] Kuhrmann, M., Méndez Fernández, D., Münch, J.: Teaching Software Process Modeling’, Proc. Int.
Conf. on Software Engineering, San Francisco, USA, May 2013, pp. 11381147
[23] Kuhrmann, M., Méndez Fernández, D., Ternité, T.: Realizing Software Process Lines: Insights and
Experiences’, Proc. Int. Conf. on Software and Systems Process, Nanjing, China, May 2014, pp. 110
119
[24] Kuvaja, P.: Bootstrap 3.0 A Spice Conformant Software Process Assessment Methodology’, Soft-
ware Quality Journal, 1999, 8, (1), pp. 7–19
[25] Mendéz Fernández, D., Penzenstadler, B., Kuhrmann, M., Broy, M.: A Meta Model for Artefact-
Orientation: Fundamentals and Lessons Learned in Requirements Engineering’, Proc. Int. Conf. on
Model Driven Engineering Languages and Systems, Oslo, Norway, October 2010, pp. 183197
[26] Mendéz Fernández, D., Wieringa, R.: ‘Improving Requirements Engineering by Artefact Orientation’,
Proc. Int. Conf. Product Focused Software Process Improvement, Paphos, Cypros, June 2013, pp. 108
122
[27] Münch, J., Armbrust, O., Sotó, M., Kowalczyk, M.: Software Process Definition and Management
(Springer, 2012)
[28] Niazi, M., Wilson, D., Zowghi, D.: A maturity model for the implementation of SPI’, Journal of Sys-
tems and Software, 2003, 74, (2), pp. 155172
[29] Ocampo, A., Münch, J.: Rationale Modeling for Software Process Evolution’, Journal on Software
Process: Improvement and Practice, 2009, 14, (2), pp. 85105
[30] Rainer, A., Hall, T.: An analysis of some core studies’ of software process improvement’, Software
Process: Improvement and Practice, 2002, 6, (4), pp. 169187
[31] Raninen, A., Ahonen, J. J., Sihvonen, H.-M., Savolainen, P., Beecham, S.: LAPPI: A light-weight
Technique to Practical Process Modeling and Improvement Target Identification’, Journal of Software:
Evolution and Process, 2012, 25, (9), pp. 915933
[32] Staples, M., Niazi, M., Jeffery, R., Abrahams, A., Byatt, P., Murphy, R.: An Exploratory Study of
Why Organizations do not Adopt CMMI’, Journal of Systems and Software, 2007, 80, (6), pp. 883895
[33] Stelzer, D., Mellis, W.: ‘Success Factors of Organizational Change in Software Process Improvement’,
Software Process: Improvement and Practice, 1998, 4, (4), pp. 227250
© IET. PREPRINT. This is the author's version of the work. It is posted here by permission of IET for your personal use. Not for
redistribution. The definitive version was published in the IET Software, 2015.
ArSPI-IET Page 14
[34] Viana, D., Conte, T., Vilela, D., De Souza, C. R. B., Santos, G., Prikladnicki, R.: The influence of
human aspects on software process improvement: Qualitative research findings and comparison to pre-
vious studies’, Proc. Int. Conf. on Evaluation & Assessment in Software Engineering, Ciudad Real,
Spain, May 2012, pp. 121125
... It further defines general control flow dependencies within requirements engineering processes. Further fields of applications can be found in the broader field of software engineering process modelling, e.g. for synchronising various software process models via artefacts as interfaces [10] or for modelling artefact-focused method-neutral software process improvement models [9]. Another view on the term 'artefact' is provided by Silva et al. [17] who structure artefacts by defining a metamodel that explicitly lists specifying the basic elements (fragments) to create generic software artefacts. ...
... As available approaches (e.g. SPEMrelated approaches) further focus on methods and description techniques as well as the complex dependencies between the methods rather than on clear result structures, contents, and dependencies among the artefacts, project participants remain unaware of how to create consistent artefacts in their projects independently of underlying restrictive processes [9]. ...
Article
Full-text available
Artefacts play a vital role in software and systems development processes. Other terms like documents, deliverables, or work products are widely used in software development communities instead of the term artefact. In the following, we use the term ‘artefact’ including all these other terms. Despite its relevance, the exact denotation of the term ‘artefact’ is still not clear due to a variety of different understandings of the term and to a careless negligent usage. This often leads to approaches being grounded in a fuzzy, unclear understanding of the essential concepts involved. In fact, there does not exist a common terminology. Therefore, it is our goal that the term artefact be standardised so that researchers and practitioners have a common understanding for discussions and contributions. In this position paper, we provide a positioning and critical reflection upon the notion of artefacts in software engineering at different levels of perception and how these relate to each other. We further contribute a metamodel that provides a description of an artefact that is independent from any underlying process model. This metamodel defines artefacts at three levels. Abstraction and refinement relations between these levels allow correlating artefacts to each other and defining the notion of related, refined, and equivalent artefacts. Our contribution shall foster the long overdue and too often underestimated terminological discussion on what artefacts are to provide a common ground with clearer concepts and principles for future software engineering contributions, such as the design of artefact-oriented development processes and tools.
... It further defines general control flow dependencies within requirements engineering processes. Further fields of applications can be found in the broader field of software engineering process modelling, e.g. for synchronising various software process models via artefacts as interfaces [10] or for modelling artefact-focused method-neutral software process improvement models [9]. Another view on the term "artefact" is provided by Silva et al. [17] who structure artefacts by defining a meta model that explicitly lists specifying the basic elements (fragments) to create generic software artefacts. ...
... Moreover, there is still no common agreement on the structure and semantics of artefact-based methodologies [12]. As available approaches (e.g., SPEM-related approaches) further focus on methods and description techniques as well as the complex dependencies between the methods rather than on clear result structures, contents, and dependencies among the artefacts, project participants remain unaware of how to create consistent artefacts in their projects independently of underlying restrictive processes [9]. ...
Preprint
Full-text available
Artefacts play a vital role in today's software and systems development processes. The notion of artefacts is widely used in software development communities. Despite the relevance and frequent use of the term `artefact', the exact denotation of this term is still not clear, due to a variety of different understandings of the term and to a careless negligent usage. This often leads to conceptual and formal results being grounded in a fuzzy, unclear understanding of the essential concepts involved. In fact, there does not exist a common terminology. Other terms like documents, deliverables, or work products are used instead of the term artefact. Therefore, it is our goal that the term artefact be standardised so that researchers and practitioners have a common understanding for discussions and contributions. In this position paper, we provide a critical reflection upon the notion of artefact in software engineering at different levels of perception and how these relate to each other. We further contribute a meta model that provides a description of an artefact that is independent from any underlying process. The meta model defines an artefact at three levels. Abstraction and refinement relations between these levels allow correlating artefacts to each other and defining the notion of equivalent artefacts. Our contribution shall foster the long overdue and too often underestimated terminological discussion on what artefacts are to provide a common ground with clearer concepts and principles for future software engineering contributions, such as the design of artefact-oriented development processes and tools.
... Adopting any of the SPI approaches do not come without upfront investment. SPI initiatives are usually "costly and improved processes need time to be disseminated, making the impact of SPI hard to measure and justify" [30], [31], at least on the short term. Accordingly, interested practitioners are reluctant to conduct SPI [31]. ...
Article
There is a big consensus that the quality of the software process has a great influence on the quality of the developed software. Hence, improving process quality helps in developing better software products with fewer defects on time and within budget. Accordingly, many software companies worldwide are conducting software process improvement (SPI) initiatives to enhance their maturity in developing software. The success in pursuing such initiatives is affected by several factors. The literature documents various experience studies in conducting SPI initiatives. Very few of these studies have been conducted in the Middle East region in general and particularly in Saudi Arabia. In this paper, we report the results of a survey-based empirical study to identify factors that supported the SPI initiatives in Saudi Arabia. A survey to identify reasons for such success has been dispatched to practitioners in various Saudi IT companies. Responses have been collected and analyzed. We have also reviewed various factors reported in the literature related to Saudi SPI initiatives. Results from the literature review and our conducted empirical study have been synthesized and analyzed in this paper. The presented work in this paper complements a previously published work that studied the failure factors that hamper SPI initiatives. This would complete the whole picture of what should and what should not be adopted by companies aiming to conduct successful SPI initiatives.
... Adopting any of the SPI approaches do not come without upfront investment. SPI initiatives are usually "costly and improved processes need time to be disseminated, making the impact of SPI hard to measure and justify" ( Coleman and O'Connor, 2008;Méndez Fernández and Kuhrmann, 2015), at least on the short term. Accordingly, interested practitioners are reluctant to conduct SPI (Coleman and O' Connor, 2008). ...
Article
Full-text available
Software product quality is affected by the quality of the process used to develop it. Improving process quality helps software organizations in developing better software products on time and within budget. To achieve these benefits, software organizations are becoming more interested to pursue Software Process Improvement (SPI) initiatives. The failure in pursuing such initiatives is affected by several factors. The literature reports many studies that document different factors impacting the success of SPI initiatives. Very few of these studies have been conducted in the Middle East and particularly in Saudi Arabia. In this study, we report the results of a survey-based empirical study that identify factors that hamper the SPI initiatives conducted by 26 organizations located in Saudi Arabia. A survey has been sent to various organizations. Responses have been collected and analyzed. Results from the literature review and from our survey have been synthesized and presented in this study along with a comparative analysis of similar factors reported in similar studies worldwide. Knowing, prioritizing and understanding these factors can help both researchers and software development organizations avoid them to successfully plan and execute SPI initiatives.
... This standard describes the identification of the most important process improvement support elements to enhance and includes an evaluation [38]. However, it focuses on activities without precisely defining the required artefacts [39]. ISO/ IEC 33014 is based on the ISO/IEC 15504part 4 and 7and addresses two dimensions: i) three control levels (strategic, tactical and operational) and ii) three improvement perspectives (Organizational improvability, Project improvability and Process improvement) [40]. ...
Article
Full-text available
Small software companies have to work hard in order to survive. They usually find it challenging to spend time and effort on improving their operations and processes. Therefore, it is important to address such needs by the introduction of a proposed framework that specifies ways of getting things done while consciously encourage them to enhance their ability to improve. Although there are many software process improvement approaches, none of them address the human factors of small companies in a comprehensive and holistic way. Samay is a proposed framework to integrate human factors in the daily work as a way to deal with that challenge. This study suggests managing human factors but pointing out the software process life cycle. The purpose is to converge toward a continuous improvement by means of alternative mechanisms that impact on people. This framework was developed based upon reviews of relevant standards (such as ISO/IEC 29110, ISO 10018, OMG Essence and ISO/IEC 33014) and previously published studies in this field. Moreover, an expert review and validation findings supported the view that Samay could support practitioners when small software companies want to start improving their ways of work.
... and there is a Public Site of the ISO Working Group Mandated to Develop ISO/IEC 29110 Standards and Guides for Very Small Entities involved in the Development or Maintenance of Systems and/or Software References. Recommended literature for further information about MoProSoft: http://www.nyce.org.mx/moprosoft and recommended further reading regarding ArSPI is available from [50,51]. ...
Chapter
Full-text available
The software development industry is dominated by a myriad of small- and medium-sized enterprises (SMEs). The main goal of this chapter is to provide a characterization of SMEs based on previous studies. It also includes an overview of a number of software process models and software process improvement (SPI) models, which are aimed at assisting SMEs in improving the way they develop software. Furthermore, this chapter discusses the extent of SPI approaches published in the literature as a way to understand the particular context and some of the major challenges faced. From there, we propose an approach to integrate software process practices. This proposal is based on the results of our study on this topic carried out in small software companies. It is focused on what small organizations could actually do, more than on what they are currently practicing.
... A first case study was conducted in 2012/2013 (cf. [34,24]). As case, we selected an SPI project in Eastern Europe (Table VII, project P7), the research design and the evaluation material was taken from the student lab that we described in Section 4.4.1. ...
Article
Full-text available
Structured approaches are beneficial for successful software process improvement (SPI). However, process engineers often struggle with standardized SPI methods, such as capability maturity model integration (CMMI) or International Organization for Standardization (ISO) 15504, and complain about too generic or voluminous approaches or methods that are alien to the organizations in which SPI is conducted. Therefore, process engineers need to customize existing SPI models or develop new approaches for company-specific SPI programs. While conducting SPI in the context of the German V-Modell XT, we faced the need to develop a new method for artifact-based SPI. In the process, we found that the construction procedures of SPI models are barely documented, and thus, their successful adaptation solely depends on the process engineers' expertise. With this article, we aim to address this lack of support and provide a structured reflection on our experiences from creating and adopting the Artifact-based Software Process Improvement & Management (ArSPI) model. We present the steps of the construction procedure, the validation, and the dissemination of the model. Furthermore, we detail on the applied methods, the design decisions, and the challenges encountered. By providing a reference procedure and tested methods, we support process engineers with the creation and adoption of SPI approaches.
Thesis
Companies that want to improve the quality of software products or speed up software development, e.g., to improve the time-to-market, put emphasis on the software process used to organize and steer software development projects. A software process describes the basic development approach, artifacts being produced, activities to be performed, and roles taking responsibility or contributing to development artifacts or activities. In order to address the specific circumstances, companies look for software process improvement (SPI) programs. The thesis at hand aims to support organizations and process engineers to better conduct SPI in a systematic and organized manner by instrumenting the paradigm of artifact-orientation. Artifact-orientation is a design paradigm that puts emphasis on the artifacts being produced in a project - the “what”. By describing the artifacts as detailed as possible while giving process engineers the freedom to apply their preferred methods or methods that best fit the actual context, artifact-orientation allows for an increased flexibility. In this thesis, we apply artifact-orientation to the domain of software process improvement. In 10 selected papers, we describe the context for software process improvement and put special emphasis on modeling of software processes, variability of software processes, software process life cycle models and software process infrastructures. We then apply artifact-orientation to software process improvement by defining an artifact-based approach to systematically conduct SPI, and extend our discussion by presenting an integrated Artifact-based Software Improvement & Management model. This model goes beyond isolated SPI endeavors and allows for implementing SPI in the context of a company-wide software process management (SPM). Finally, we discuss the feasibility of the artifact-based approach in the context software process enactment, and we also describe an approach to educate process engineers to apply the presented approach in their respective contexts.
Article
Full-text available
Structured approaches are beneficial for successful software process improvement (SPI). However, process engineers often struggle with standardized SPI methods, such as capability maturity model integration (CMMI) or International Organization for Standardization (ISO) 15504, and complain about too generic or voluminous approaches or methods that are alien to the organizations in which SPI is conducted. Therefore, process engineers need to customize existing SPI models or develop new approaches for company-specific SPI programs. While conducting SPI in the context of the German V-Modell XT, we faced the need to develop a new method for artifact-based SPI. In the process, we found that the construction procedures of SPI models are barely documented, and thus, their successful adaptation solely depends on the process engineers' expertise. With this article, we aim to address this lack of support and provide a structured reflection on our experiences from creating and adopting the Artifact-based Software Process Improvement & Management (ArSPI) model. We present the steps of the construction procedure, the validation, and the dissemination of the model. Furthermore, we detail on the applied methods, the design decisions, and the challenges encountered. By providing a reference procedure and tested methods, we support process engineers with the creation and adoption of SPI approaches.
Data
Full-text available
Software process adaptation and improvement (SPI) addresses the need of companies to adapt new and/or improve their software processes in order to meet, e.g. business optimization goals or regulative requirements. Such an initiative comprises manifold activities, e.g. analyzing, designing, realizing, evaluating, and deploying new software processes or, respectively, new versions/variants of a maintained software process. Therefore, such initiatives are often considered to be (self-contained) projects. Although reference models such as CMMI and ISO 15504 contain practices and assessment methods they, however, lack in defining supporting artifacts for SPI projects, which help process engineers to structure the outcomes, to guide process engineers during the set-up and the operation of SPI projects. Therefore, setting-up and operating SPI projects highly depends on the individual expertise of process engineers. In this report, we present a compact artifact model and a set of complementing processes to support SPI projects and software process management (SPM), which we inferred from six years of experience. The presented model serves as template for creating analysis, design, and supporting artifacts in SPI projects. Furthermore, the artifact model is embedded into an organizational context in which SPI projects are initiated and executed, and the results are deployed to the process consumers. The report at hands serves as “data sink” and contains all detailed artifact model descriptions and further information aiding the definition of a software process management approach and, furthermore, provides guidance to process engineers to organize and manage a particular SPI project.
Conference Paper
Full-text available
Software process lines provide a systematic approach to construct and manage software processes. A process line defines a reference process containing general process assets, whereas a well-defined customization approach allows process engineers to create new process variants by, e.g., extending or altering process assets. Variability operations are a powerful instrument to realize a process line. However, little is known about which variability operations are suitable in practice. In this paper, we present a study on the feasibility of variability operations to support process lines in the context of the German V-Modell XT. We analyze which variability operations were defined and used to which extent, and we provide a catalog of variability operations as an improvement proposal for other process models. Our findings show 69 variability operations defined across several meta-model versions of which 25 remain unused. Furthermore, we also find that variability operations can help process engineers to compensate process metamodel evolution.
Conference Paper
Full-text available
When it comes to software process improvement (SPI), process engineers look for SPI methods to support process analysis, design, realization, deployment, and management. Although a number of different SPI methods and models exist, process engineers tend to view these as too generic, too large, or a poor fit for the organization in which SPI is conducted. A strategy to overcome these shortcomings is to concentrate on the artifacts, which precisely define the de-sired outcomes, rather than on specific methods. In this paper, we present the Artifact-based Software Process Improvement & Management (ArSPI) model that provides a unified perspective on SPI and company-wide software process management (SPM), the required key artifacts, and the life cycle models. ArSPI is shown to be of practical sup-port to industry who called for a practical way to define the interfaces between SPI projects. This paper concludes with an example of how ArSPI paved the way for several organizations through applying the model in real-world SPI-projects.
Conference Paper
Full-text available
When it comes to designing a software process, we have experienced two major strategies. Process engineers can either opt for the strategy in which they focus on designing a process using an artefact model as backbone or, on the other hand, they can design it around activities and methods. So far, we have first studies that directly analyse benefits and shortcomings of both approaches in direct comparison to each other, without addressing the questions relevant to pro-cess engineers and which implications the selection of a particular design strategy has on the process consumption. We contribute a first controlled investigation on the perceived value of both strategies from the perspectives of process engineers and process consumers. While our results underpin the artefact-oriented design strategy to be an advantageous instrument for process engineers, process con-sumers do not evidently care about the selected design strategy. Furthermore, our first investigation performed in an academic environment provides a suitable em-pirical basis, which we can use to steer further replications and investigations in practical environments.
Article
Realizing the benefits of continuous software process improvement.
Conference Paper
Software process improvement (SPI) has become a well-established discipline in the last part of the 1990s. Some interesting numbers have been reported in the literature as to what is the return on investment from SPI. However, very little attention has been devoted to individual actions and where their payoff lies. This paper reports on a systematic approach developed for an international company to understand, and potentially measure, the return on investment of SPI