Conference PaperPDF Available

A Case Study on Artefact-based RE Improvement in Practice


Abstract and Figures

Background: Most requirements engineering (RE) process improvement approaches are solution-driven and activity-based. They focus on the assessment of the RE of a company against an external norm of best practices. A consequence is that practitioners often have to rely on an improvement approach that skips a profound problem analysis and that results in an RE approach that might be alien to the organisational needs. Objective: In recent years, we have developed an RE improvement approach (called ArtREPI) that guides a holistic RE improvement against individual goals of a company putting primary attention to the quality of the artefacts. In this paper, we aim at exploring ArtREPI's benefits and limitations. Method: We contribute an industrial evaluation of ArtREPI by relying on a case study research. Results: Our results suggest that ArtREPI is well-suited for the establishment of an RE that reflects a specific organisational culture but to some extent at the cost of eciency resulting from intensive discussions on a terminology that suits all involved stakeholders. Conclusions: Our results reveal first benefits and limitations, but we can also conclude the need of longitudinal and independent investigations for which we herewith lay the foundation.
Content may be subject to copyright.
A Case Study on
Artefact-based RE Improvement in Practice
Daniel endez Fern´andez1and Stefan Wagner2
1Technische Universit¨at unchen, Germany
2University of Stuttgart, Germany
Abstract. Background: Most requirements engineering (RE) process
improvement approaches are solution-driven and activity-based. They fo-
cus on the assessment of the RE of a company against an external norm
of best practices. A consequence is that practitioners often have to rely
on an improvement approach that skips a profound problem analysis and
that results in an RE approach that might be alien to the organisational
needs. Objective: In recent years, we have developed an RE improve-
ment approach (called ArtREPI ) that guides a holistic RE improvement
against individual goals of a company putting primary attention to the
quality of the artefacts. In this paper, we aim at exploring ArtREPI’s
benefits and limitations. Method: We contribute an industrial evaluation
of ArtREPI by relying on a case study research. Resu lts: Our results
suggest that ArtREPI is well-suited for the establishment of an RE that
reflects a specific organisational culture but to some extent at the cost of
eciency resulting from intensive discussions on a terminology that suits
all involved stakeholders. Conclusions: Our results reveal first benefits
and limitations, but we can also conclude the need of longitudinal and
independent investigations for which we herewith lay the foundation.
Keywords: Requirements Engineering, Artefact Orientation, Software
Process Improvement, Case Study Research
1 Introduction
Requirements engineering (RE) constitutes an important success factor for soft-
ware development projects since stakeholder-appropriate requirements are im-
portant determinants of quality. Its interdisciplinary nature, the uncertainty,
and the complexity in the process, however, make the discipline dicult to in-
vestigate and to improve [1]. For an RE improvement, process engineers have to
decide whether to opt for problem orientation or for solution orientation [2,3].
In a solution-driven improvement, the engineers assess and adapt their RE ref-
erence model, which provides a company-specific blueprint of RE practices and
artefacts, against an external norm of best practices. The latter is meant to lead
to a high quality RE based on universal, external goals (see, e.g. CMMI for
© Springer. This is the author's version of the work. It is posted here by permission of Springer for your personal use. Not for
redistribution. The definitive version was published in the conference/workshop proceeding
RE [4]). Solution-driven improvement approaches might thus serve the purpose
of achieving externally predefined goals by implementing a set of best practices
adhered by many organisations [2] (e.g. as part of a certification). They do not
necessarily consider company-specific goals, however, that dictate the notion of
RE quality within a particular socio-economic context (e.g. a company) and,
thus, may result in an RE reference model that is alien to the organisational cul-
ture. In consequence, those RE improvement approaches encounter problems and
are often rejected by practitioners [5,1]. A notion of RE quality where company-
specific goals dictate the improvement is the core of problem-driven approaches.
Besides the improvement principles, the paradigm in which the targeted RE
reference model is structured (and, thus, improved) plays an important role. A
reference model can either be activity-based or artefact-based [6]. In short, an
activity-based improvement approach focuses on improving the quality of the
RE practices while an artefact-based one puts its focus on improving the quality
of the RE artefacts.
Most available RE improvement approaches today are solution-driven [7].
Yet, RE is complex by nature, and we postulate that RE quality depends on the
contribution of an RE reference model to context-specific goals. Therefore, im-
provements cannot be meaningfully implemented without a qualitative problem
investigation that reveals which goals must be achieved [3,2] and which artefacts
should be created in which way. In response to the lack of problem-driven and
artefact-based RE improvement approaches, we elaborated such an approach [8]
which we call ArtREPI. We further realised our approach using the EPF Com-
poser3and made first experiences using ArtREPI in practice (see also Sect. 2).
Problem. Although we made first conceptual and empirical contributions
to support a problem-driven and artefact-based RE improvement, we still have
little evidence on its practical benefits and limitations.
Contribution. In this paper, we report on the first industrial case study
to evaluate ArtREPI in comparison to solution-driven and activity-based REPI
approaches previously used in the same contexts. The purpose, however, is not
to evaluate only the particularities of our approach itself, but also
1. to reveal first qualitative insights into the benefits and limitations of a
problem-driven and artefact-based RE improvement in general, and
2. to lay the foundation for future independent empirical investigations.
Based on our contribution and the disclosed material [9], practitioners can
therefore already apply our RE improvement approach, and researchers can build
their conceptual and empirical work on our results to further explore the full
spectrum of an RE improvement.
Outline. In Sect. 2, we discuss fundamentals and related work. In Sect. 3,
we introduce the artefact-based RE improvement approach. We provide our case
study design in Sect. 4, the results in Sect. 5, and a critical reflection in Sect. 6,
before concluding our paper in Sect. 7.
2 Fundamentals and Related Work
Requirements engineering process improvement, as software process improve-
ment in general, is a cyclic approach to continuously analyse problems/the cur-
rent situation in RE as part of an appraisal, plann an improvement, realise the
improvement and evaluate the improvement before initiating the next iteration.
We can distinguish solution-driven approaches and problem-driven approaches
as well as artefact and activity orientation.
In literature, there exist mostly solution-driven contributions [3]. R-CMM,
proposed by Beecham et al. [4], is a prominent representative of these approaches.
It is based on CMMI and an empirical investigation in twelve companies [10].
The investigation revealed patterns and best practices based on problems expe-
rienced by practitioners. Therefore, it aimed at a generalised, external notion of
RE quality. A technical validation using an expert panel [11] further illustrates
selected success criteria, such as understandability. Approaches of this category
focus on a solution-driven benchmarking of the maturity of RE according to a
specific norm of best practices and may thus lead to the problems described in
the introduction (see also [5,1] for richer investigations).
In response to their shortcoming, Pettersson et al. contributed an approach
to problem-driven RE improvement [2] called the iFLAP approach. Same as
in ArtREPI, they make use of qualitative methods for the problem analysis
and postulate the importance of strong stakeholder involvement. Although their
concepts are promising to conduct a problem-driven REPI, the consequential
next steps, i.e. the actual improvement realisation by crafting a new RE reference
model, was not in scope of their contribution.
To the best of our knowledge, there exists no holistic approach to a REPI
covering all improvement phases in a seamless manner, let alone considering
an improvement specifically directed at the RE artefacts. Recent work in this
direction is made by Kuhrmann et al. [12], but taking more a perspective on
the management of software process models. The focus is thereby set on how to
manage an artefact-based improvement rather than on how to conduct it. They
do not look at how to analyse, design, and evaluate a process in a problem-driven
manner focusing on the quality of the artefacts which is in scope of ArtREPI.
Available validation and evaluation research, which would be directly related
to the contributions of this paper, focuses on the evaluation of methods or metrics
used in isolated REPI phases, such as the analysis, on experience reports, or
on the analysis of general success factors and lessons learnt. In the case study
we present here, we therefore do not discuss the relation to existing evidence
taking into account particular approaches, but exclusively take a qualitative
view and rely on the evaluation of ArtREPI against the general perceptions and
the experiences of the participants with solution-driven, activity-based REPI in
the same context. Further details on the publication landscape on REPI can be
taken from our previously published mapping study [7].
Previously Published Material. In [8], we first introduced the basic con-
cepts of ArtREPI and its design science principles. Since then, we realised our
approach using the EPF Composer as a means of a technical validation and
made all material (models, process documentation, document templates, and
evaluation instruments) publicly available [9] to support the dissemination. In a
previous short paper [13], we then briefly reported on initial experiences from an
ongoing case study. In the paper at hands, we report on the by now completed
case study in detail including the case study design, the results containing a
second case, and the implications the results have.
3 ArtREPI: Artefact-based RE Process Improvement
Figure 1 gives an overview of the basic structure of ArtREPI, which we use for
our evaluation presented in this paper. As shown in the figure, we distinguish two
contexts important to the notion of RE quality: an external context that con-
tains norms of best practices and the socio-economic context where the notion
of quality is dictated by individual demands. External norms of best practices
are not only key to solution-driven RE improvement approaches but also impor-
tant to a problem-driven improvement as one principle is to support technology
transfer according to context-specific goals.
The socio-economic context is further characterised by a set of disciplines that
aim at managing software processes. Of particular interest is that we have process
owners who usually have the sovereignty over an RE reference model (although
often underrepresented in practice [1]) and a set of project participants who work
according to an (explicitly established or implicitly lived) RE reference model.
The latter is subject to an improvement in case an organisational change is
triggered. For reasons of complexity, we omit the discussion of further disciplines
or roles, such as an improvement sponsor. A change is performed as part of an RE
process improvement project which is in scope of ArtREPI. ArtREPI consists of
four phases and yields a set of mandatory and optional results. The mandatory
ones eventually serve a seamless improvement based on a unified underlying data
model of the improvement results. The full model can be taken from our online
sources [9] while in the following, we provide a brief overview of the single phases.
3.1 Improvement Preparation
In the preparation phase, we lay the groundwork for the improvement and aim
at getting an understanding about the application domain, such as the chemical
sector, typical constructs and rules followed therein, and the terminology used.
We discuss contemporary problems and the primary improvement goals which,
in collaboration with the process owners, we refine to concrete metrics and mea-
surements to evaluate the success of the improvement after completion. Where
possible and reasonable, we reuse metrics from previously conducted ArtREPI.
The outcome of the preparation phase is an improvement plan that defines the
concrete procedures and time schedules as well as resource allocations (cases and
subjects) for subsequent steps including interviews or workshops.
Management) Longitudinal)
Role)) Discipline)
1* 2* 3* 4*
Fig. 1. ArtREPI: Structuring and context.
3.2 Problem Analysis
The second stage aims at discovering problems in the use of the current RE
reference model independent of whether it is explicitly defined as a company
standard or not. In case a company standard exists, we analyse it first to get
an overview of the basic artefacts, roles, and activities. We then conduct a doc-
ument analysis in selected exemplary development projects and abstract from
the contents to build an as-is model of the RE artefacts, milestones, and roles.
We complement this syntactic analysis with a semantic analysis where we anal-
yse, for example, linguistic defects in document templates. We use the defined
as-is model to conduct a gap analysis where we compare the current state of
practice with an external standard wich serves as a knowledge base. More pre-
cisely, we rely on AMDiRE (artefact model for domain-independent RE) that
defines a best practice norm for artefact-based RE (see also [14]). We use poten-
tial gaps from our analysis as candidates for further validation which we do via
semi-structured interviews. There, besides asking about experiences and expec-
tations, we ask the participants why certain contents identified as incomplete in
comparison to AMDiRE have or have not been specified in their projects. We
take their reasoning as our primary input to establish later on a tailoring profile
for the artefact-based RE reference model. The aim of the whole analysis is not
to benchmark the process against an external standard. Instead and according to
the problem-driven nature of ArtREPI, it aims at getting a better understanding
on potential improvements which then can only be validated by project partici-
pants. To this end, we trigger a self-reflection via interviews by indicating only
possibilities for changes that eventually only they can judge upon.
3.3 Improvement Design
In the third stage, we conduct the actual realisation of the improvement based
on the candidates previously identified during the gap analysis. To this end, we
conduct a series of action research workshops where we build a new model can-
didate by deciding which content items to consider in the candidate (e.g. “use
case model”) before we then subsequently define the actual content model that
abstracts from concrete modelling concepts used to specify the content items
in a project. The latter serves to build a prototypical (modelling) tool or docu-
ment templates, and to establish content-related dependencies to the artefacts of
further development phases, such as of testing, serving the purpose of a process
integration into the overall software process model. We enrich the artefact model
with process elements (roles, milestones) and a tailoring profile that emerges from
the interviews and that defines under which project circumstances to document
certain content items or not. Depending on the improvement project complexity
or criticality, we may perform a validation of the model candidate before enter-
ing the last stage. In this validation step, we validate the model via feedback
gathered by domain experts not involved in the improvement workshops.
3.4 Improvement Evaluation and Transfer Preparation
The success of a problem-driven improvement can eventually only be determined
by the degree to which the improvement outcome satisfies the improvement goals.
In the last step, we therefore conduct technical action research workshops with
project participants in pilot projects where we apply the resulting RE reference
model under realistic conditions. We specify a set of RE artefacts following the
new RE reference model. Afterwards, we compare the outcomes and the followed
process with the artefacts previously created in the same project environment by
an interview. The participants rate the new model following the criteria agreed
on in the preparation phase. In case of a positive rating, we release the new
model complemented with supporting material.
4 Case Study Design
In [8], we introduced in detail how we inferred our ArtREPI approach presented
in Sect. 3 from fundamental and applied research projects where we conducted
an artefact-based and problem-oriented RE improvement. We argued so far that
our approach is successful, because it emerged from successful RE improvements
leading to new RE company standards. We were aware, however, that we need
to better understand the benefits and limitations we can expect from ArtREPI
and whether our approach can be used by others if we are not involved. This
motivates the design for our empirical investigations:
1. We rely on case study research with action research components in an in-
dustrial environment as we are particularly interested in elaborating qualita-
tive insights into benefits and shortcomings in applying an RE improvement
approach that also depends on subjective (and social) factors. The design
follows the guideline of Runeson and ost [15].
2. We included two cases where the second one was conducted with little direct
involvement of the ArtREPI authors. This should allow us to get an indica-
tion whether the success of an improvement depends on our influence, and
it should provide a first step in scaling up to practice.
4.1 Objectives and Research Questions
The study has the general objective to understand the benefits and limitations of
applying ArtREPI in practice. We formulate two research questions to steer the
study design structured into the evaluation of the improvement tasks (ArtREPI
stage 1-3) and of the resulting RE reference model (ArtREPI stage 4):
RQ 1 How well are process engineers supported in their RE improvement tasks?
RQ 2 How well are project participants supported by the resulting RE reference
4.2 Cases and Subjects
We applied ArtREPI as part of two improvement projects in two dierent con-
texts. For each project, we give a summary of the most important context in-
formation we are able to provide within the limits of existing non-disclosure
agreements in Tab. 1. We chose the contexts because of their suitability to our
research questions. We looked at two dierent settings (large versus small and
agile). Furthermore, REPI project 2 has, in contrast to the first case, only little
involvement of those researchers developing ArtREPI (see also the next section).
REPI 1: Large scale RE process. The first improvement project was
conducted as part of a research cooperation between Technische Universit¨at
unchen and Wacker Chemie and was described, in parts, as the intermedi-
ate evaluation in [8].4Wacker Chemie is a German company that works in the
4In [8], we reported on first results from the first evaluation steps where the context-
specific evaluation (ArtREPI stage 4) was still ongoing.
Tabl e 1 . Overview of cases (REPI 1 and REPI 2).
No. Aspects Descriptions
1 Improvement goal Integration of RE into quality management.
REPI context 2 process engineers, 4 domain experts, 4 external improvement consultants.
Eort: 8 PM including 13 workshops.
Pilot projects 3 pilot projects, 8 participants.
2 Improvement goal Re-design of RE to support agility.
REPI context 2 main process engineers (previously coached by improvement consultants).
Eort: approx. 6 PM.
Pilot projects 3 project participants to rate the reference model w/o pilot projects.
chemical business and develops custom software for their operation processes
and their production sites. The improvement project aimed at defining a de-
tailed artefact-based RE with a seamless integration into quality management.
REPI 2: Agile RE process. The second improvement project was con-
ducted as part of a Master’s Thesis at the company SupplyOn AG which is a
software as a service provider. The improvement project aimed at the re-design of
the RE of the Rational Unified Process to an artefact-based agile RE approach.
In contrast to project 1, where we evaluated the resulting RE reference model
explicitly via pilot projects, we relied on informal reviews and expert opinion.
4.3 Data Collection and Analysis Procedures
We answer our research questions by collecting and analysing data from the
application of ArtREPI and the assessments via questionnaires in both cases
by improvement consultants who also authored ArtREPI. To this end, the im-
provement consultants coach the process engineers on ArtREPI, its underlying
principles, and corresponding tools in a 3 hour workshop. These process engi-
neers then apply ArtREPI as described in Sect. 3 and conduct the last phase of
ArtREPI (i.e. the evaluation) by applying the resulting RE reference model in
pilot projects of the company. In REPI 1, the improvement consultants formed
part of the improvement project team after the coaching. In REPI 2, they where
not directly involved anymore after the coaching sessions.
We collect the data for both research questions using questionnaires with
open and closed questions. Each closed question is formulated as a statement
where the participants should state their agreement on a Likert scale defined on
an ordinal scale from 0 (“I strongly disagree”) to a maximum of 7 (“I strongly
agree”) to avoid that they choose the middle. They have, however, always the
possibility to refuse an answer. The open questions give them the possibility to
provide a rationale for their decision. The original questionnaires can also be
found in our online material [9].
To answer RQ 1, we conduct an assessment by letting the involved process en-
gineers rate ArtREPI on the basis of a questionnaire where they directly compare
ArtREPI to previously followed solution-driven, activity-based improvement ap-
proaches. We summarise the questions in Tab. 2. At the end, we finally asked
the process engineers three concluding questions about the overall suitability of
the approach to cover the particularities of their context.
Tabl e 2 . Condensed RQ 1 instrument: Support of process engineers in improvement.
Criteria Statements (closed question)
Structuredness The approach was systematic.
Simplicity The approach was easy to use.
Goal orientation The approach considered problems and needs of all involved stakeholders.
Experience orientation The approach considered company culture and stakeholder experiences.
Transparency The decisions during the workshops were reproducible.
Eectivity The improvement has led to the desired results.
Eciency I perceived the eciency of the undertaking as high.
Knowledge transfer The approach actively supported knowledge transfer.
Overall suitability (1) ArtREPI was better suited than solution-driven approaches.
Overall suitability (2) I would apply ArtREPI again.
Overall suitability (3) I want to add following positive/negative aspects (open)
To answer RQ 2, we conduct an assessment by letting the involved project
participants rate the outcome of ArtREPI using again a questionnaire. This
evaluates the resulting RE reference model after application in the pilot projects
in direct comparison with the one previously used in same context to evaluate
whether the improvement goals have eventually been achieved. Table 3 sum-
marises the questions for this RQ. In the questions, we use “model” as a reference
to the RE reference model (upper part in the table) and “artefacts” as a short
reference for the artefacts created using the model (lower part in the table).
Tabl e 3 . Condensed RQ 2 instrument: Support of pro ject participants by RE ref-
erence model (resulting from the improvement tasks).
Criteria Statements (closed question)
Flexibility The model allows for flexibility in the RE process.
Ease of use The model is easy to understand.
Eectivity The application of the model has led to the desired results.
Eciency When applying the model, I perceived the eciency as high.
Customisation / Tailoring The model can be tailored to pro ject-specific situation of the company.
Process integration The model is integrated into further development activities and within
the line organisation.
Structuredness of artefacts The artefacts are well structured and can be understo od by people
not involved in their creation.
Syntactic artefact quality The model supports a high syntactic artefact quality w.r.t. complete-
ness and consistency.
Traceability of artefact The model supports traceability within RE and between RE and fur-
ther disciplines.
Semantic artefact quality The mo del supports semantically consistent and complete artefacts.
Testability of artefacts The model supports the creation of testable artefacts.
For the analysis of our results, we rely on descriptive analysis and qualita-
tive interpretation of our data. We intentionally refrain from summarising visual
accumulations of the ratings (e.g. via radar charts) because of the heterogene-
ity in the data as providing answers to each criterion was not obligatory. We
therefore report on every rating given by each subject. We additionally indicate
to whether the participants experienced an improvement or deterioration of ap-
plying ArtREPI in direct comparison to previously used improvement approach
which we assume to be present if the mean values dier by at least one point.
5 Case Study Results
In the following, we summarise the results from the rating given in the assess-
ments, structured according to the research questions. For reasons of confiden-
tiality, we cannot provide details about workshop contents, the company-specific
RE reference model and the pilot projects, but we will report the rating results.
5.1 RQ 1: Support in RE Improvement Tasks
Table 4 summarises the ratings of the ArtREPI approach by the process engi-
neers in direct comparison to previously used approaches which were based on
CMMI. For each criterion, we show each subject’s ratings. We had 6 subjects
in context 1 and 2 subjects in context 2 (see Tab.1). Furthermore, we give the
mean and median as central tendencies and further show whether ArtREPI is
clearly considered better (+) or worse (). No clear comparison is indicated by
a 0. At the bottom of the table, we show the results of the rating of whether
ArtREPI was considered better suited for the respective context than previously
used REPI approaches.
Tabl e 4. Results for RQ 1: Rating of improvement procedure by subjects Sxon a
scale of 0 (“I strongly disagree”)to7(“I strongly agree”) from a process engineering
perspective. See also Tab. 2 for details on the used instrument.
S1 S2 S3 S4 S5 S6 Mean Median Comp. S1 S2 Mean Median Comp.
Structuredness Old --56534.75 5 +64 5 5 0
New 656775 6 6 65 5.5 5.5
Simplicity Old --5555 5 5 065 5.5 5.5 0
New 5565534.83 5 65 5.5 5.5
Goal Orientation Old --4644 4.5 4 044 4 4 +
New 335664 4.5 4.5 66 6 6
Experience Orientation Old --4-364.33 4
66 6 6
New 225- 52 3.2 2 43 3.5 3.5
Sustainability Old --54444.25 4 +45 4.5 4.5 0
New 6564745.33 5 45 4.5 4.5
Eectivity Old --55634.75 5 034 3.5 3.5 +
New 3365654.66 5 76 6.5 6.5
Eciency Old --5654 5 5
34 3.5 3.5 +
New 126654 4 4.5 75 6 6
Knowledge Transfer Old --4644 4.5 4 034 3.5 3.5 +
New 555663 5 5 76 6.5 6.5
Overall Suitability New --5--5 5 5 76 6.5 6.5
Overall, ArtREPI was rated as a structured improvement approach that
tends to better support knowledge transfer than previously used approaches.
Surprising to us, however, is the result of REPI 2 conducted by people not
involved at all in the development of ArtREPI. Our assumption was that the
rating would be worse than in REPI 1, but it was better regarding the goal ori-
entation, the eectivity and eciency, and the support for knowledge transfer.
Qualitative statements from the open questions provide some explanations. The
subjects in REPI 1 rated that the initial preparation phase was performed in
a “too academic” fashion with “too many discussions to clarify the terminol-
ogy”, especially by those subjects with no REPI experience made before (S1
and S2). This might also be the reason for the negative comparison regarding
the experience orientation. In contrast, REPI 2 was conducted solely by process
engineers employed by the respective company and familiar with the culture and
the domain.
Finally, the overall rating whether ArtREPI was better suited to achieve
the company-specific improvement goals in comparison to previously used ap-
proaches were answered positively in both cases. The process engineers stated for
both cases that they would apply ArtREPI in follow-up improvement cycles. The
answers to the open questions showed for REPI 1, however, that the engineers
expect an integration into the organisation as a prerequisite for a repetition.
Another suggestion was to check in a follow-up study whether the eciency of
ArtREPI could be improved. Further informal statements included that:
1. the action research workshops, where the new artefact-based RE reference
model was crafted by the process engineers, together with the group discus-
sions, fostered discussions the engineers would otherwise not have, e.g. about
roles and responsibilities in the RE that just seemed clear to everybody.
2. independent of the results from the pilot projects (described next), they
would need additional longitudinal studies before declaring the new RE ref-
erence model as a new standard as too many changes have been made.
5.2 RQ 2: Support by resulting RE Reference Model
The success of an improvement eventually depends on whether the resulting
RE reference model achieves the improvement goals. RQ 2 therefore focuses on
evaluating the REPI outcome in pilot projects. Table 5 shows the rating of the
artefact-based RE reference model resulting from the improvement cycle from
the perspective of project participants. We distinguish criteria to rate the appli-
cation of the reference model itself and criteria to evaluate the RE artefacts pro-
duced following the reference model. In REPI 1, we applied the model in three
pilot projects (two with custom software development and one with standard
software). In REPI 2, it was not possible to fully implement the new model im-
mediately, but they included the artefacts created during the coaching sessions.
Therefore, their rating is does not reflect the full experience with ArtREPI.
The results indicate that ArtREPI supported the participants in achieving
their improvement goals (see Sect. 4.2). For REPI 1, we could improve, e.g.,
the traceability, the testability, and the process integration of the RE reference
model, thus, supporting the better integration into quality management which
formed the improvement goal. The negative results in the comparison to the
previously used RE reference model regarding the process quality might be ex-
plained from the complexity of the new (richer) model and the learning curve
associated with all new methods in general. In contrast, the positive rating in
REPI 2 might be explained by the new RE reference model to support agility
due to its new (light weight) simplicity.
Tabl e 5. Results for RQ 2: Rating of resulting RE reference model by subjects Sxon a
scale of 0 (“I strongly disagree”)to7(“I strongly agree”) from a project perspective.
S1 S2 S3 S4 S5 S6 S7 S8 Mean Med. Comp. S1 S2 S3 Mean Med. Comp.
Process Quality
Flexibility Old 770425664.63 5.5 0561 4 5 +
New 446652654.75 5 6676.33 6
Ease of Use Old 5-5464454.71 5
525 4 5 +
New 3-5352443.71 4 555 5 5
Eectivity Old 4-56-564 5 5 05-1 3 3 +
New 6-56-2665.16 6 6-7 6.5 6.5
Eciency Old 3-45-6654.83 5
6-0 3 3 +
New 1-56-2633.83 4 5-7 6 6
Customisation Old 7-131664 4 4 05745.33 5 +
New 6-6641654.86 6 6776.67 7
Process Old 3-2336533.75 3 +5354.33 5 +
Integration New 6-4466555.14 5 5655.33 5
Artefact Quality
Structuredness Old 253532523.38 3 0342 3 3 +
New 473642534.25 4 366 5 6
Syntactic Old 05440263 3 3.5 +-22 2 2 +
Quality New 775525665.38 5.5 - 76 6.5 6.5
Traceability Old 441322633.13 3 +1301.33 2 +
New 766745645.63 6 666 6 6
Semantic Old 4423- 3733.71 3 +435 4 4 +
Quality New 6654- 5765.57 6 6655.67 6
Testability Old 123452633.25 3 +5333.67 3 +
New 545675655.38 5 6766.33 6
Overall, we observe that ArtREPI can show its strength especially in the
artefact quality as expected, because it is oriented towards artefacts. In the
process quality, the picture is, however, more mixed. Qualitative feedback to the
open questions provides explanations:
The success of a pilot study strongly depends, beyond political factors, on
the quality of the prototypical implementation. Hence, project participants
should be involved in the technical validation, too.
Coaching sessions before the pilot studies should not only focus on the new
RE reference models, but on the underlying principles, e.g. new levels of
abstraction in the requirements or new roles and responsibilities.
Although artefact-based RE reference models are inherently process agnos-
tic, they should include suggestions for methods and modelling techniques
(e.g. UML-based ones) project participants are familiar with. This should
increase the organisational willingness to change.
5.3 Threats to Validity
There exist many threats to validity inherent to case study research [15]. Threats
to the internal validity mainly arise from potential bias during the data collec-
tion, let alone because of our action research components, and because of the
general subjective nature in the ratings. We applied selected techniques to reduce
those threats, e.g. researcher and method triangulation, but to a certain extent
we were particularly interested in gathering subjective opinions by the intervie-
wees. Another mitigation strategy followed in advance was to apply ArtREPI in
the second case with little influence of the approach authors. It provides a first
indication towards scaling up to practice [16], thus, it provides a first step in
strengthening the external validity. This generalisation, however, needs further
attention in future work.
Finally, a more general problem is that many things remain (objectively)
unmeasurable. As a matter of fact, we still have a limited understanding on how
to reliably measure long-term improvement eects going beyond RE, because
of the complexity of confounding variables in a software project ecosystem. In
our studies, we therefore refrained from such measurements in advance. Mea-
surements are still important to determine the success of an improvement given
that (1) RE forms part of a larger context that needs to be taken into account,
while (2) we consider problem orientation where we cannot rely on an external
reference to determine a notion of software process quality (in the sense of an
Another facet important to an improvement is finally the question how much
the notion of process quality is determinable by the quality of the artefacts
and also how much project participants eventually rely on the created artefacts.
We can observe first empirical investigations in that direction (e.g. [17]). Still,
even if we can measure certain phenomena on basis of the artefacts, we still
do not fully understand to which extend the notion of RE quality eventually
manifests itself in the created artefacts. That is, the investigations are based
on the critical assumption that the success of a project and, in particular, of
an artefact-based improvement depends on the documented results on which
project participants rely. To elaborate the extent to which the application of
ArtREPI eventually leads to an improvement, and how to measure the success
of an improvement (including subjective and cognitive facets), we first need a
better understanding on the measurability of such an improvement. In [18], we
provide a richer discussion on the limitations of measurements in RE.
6 Discussion
Our results indicate that ArtREPI is well suited to cover the needs of a struc-
tured improvement where problem and artefact orientation are important, while
supporting knowledge transfer by continuous stakeholder involvement. The di-
rect comparison of the two cases further indicates that the eectivity, eciency,
the knowledge transfer, and even the goal orientation are strengthened when
the improvement is conducted by company members with no intervention from
outside. The results from pilot projects further suggest that the improvement
eventually achieved the local improvement goals. Our overall results therefore
strengthen our confidence in the benefits of ArtREPI as a self-contained and
holistic approach to a problem-driven RE improvement as long as the improve-
ment goals are in tune with the (known) benefits of artefact orientation.
6.1 Limitations of ArtREPI
One benefit in case study research is that it gives us the possibility to get qual-
itative feedback, i.e. explanations for the particular ratings. We are particularly
interested in revealing limitations of ArtREPI as this supports us steering the
development and the evaluation of the discussed improvement principles, and it
helps us fostering the discussions on RE improvement in general.
Limitations we could reveal by our second case concern more social aspects
of a process improvement. For example, we, as researchers involved in the de-
velopment of ArtREPI, seemed to lower the eciency and eectivity of an im-
provement due to long preparation phases to increase our understanding of the
domain and the terminology used. Our initial assumption that the success of
ArtREPI depends on that we ourselves should conduct the improvement thus
is wrong. However, the exact consequences of applying ArtREPI without any
involvement of the authors at all (e.g. without coaching) remain still unknown.
Also, there exists a plethora of organisational factors, such as the support by
the management, the support at project level (a champion), and general social
as well as empirical skills that constitute success factor for an improvement. It
might be possible to include some factors in the approach. For instance, one
suggestion was to involve project participants earlier during technical validation
stages to mitigate the threats arising from a missing organisational willingness
to change. Other factors, however, might not be covered at all, because models
abstracts, by nature, from desires, beliefs, experiences, and expectations, which
all are important. To fully reveal those factors, and to fully explore to which
extent they can eventually be influenced on a methodological layer, we need
more (especially longitudinal) investigations in that direction.
6.2 Success Factors for ArtREPI
Besides the general focus of problem orientation, which is to emphasise context-
specific goals over the possibility for an external certification of an RE, we found
several factors that influence the success of an RE improvement:
Improvement goals need to be in tune with the expected benefits of the
chosen paradigms (in our case artefact orientation) known in advance.5
Support by higher management, especially when communicating new roles
and responsibilities.
Backup by project environments, e.g. via the early involvement of project
participants in the validation stages of a new RE reference model (before
initiating the evaluation stages).
Domain knowledge.
Reflection of the organisational culture.
Social and empirical skills.
Those factors might not be surprising in themselves but together in their ex-
tent. The success of a problem-driven improvement is determined by the reflec-
tion of the organisational culture of a company in every facet of the improvement
approach. That is, all relevant stakeholders need to be involved in early stages to
5This implies that we need sucient evidence on potential benefits of artefact ori-
entation to the chosen context beforehand to justify the decision to conduct an
organisational change.
cover their needs, a backup by representative projects needs to be ensured (in the
sense of champions), and project participants need to identify themselves with
the resulting reference model. All those facets aim at supporting the willingness
for a change in the way of working to reshape an existing organisation [19].
In our current understanding, this can only be achieved by applying ex-
haustive, qualitative empirical method that foster continuous stakeholder in-
volvement but which also come themselves with limitations, e.g. regarding the
measurability of improvement eects (see also our Sect. 5.3).
7 Conclusion
In this paper, we reported on our first steps to empirically evaluate the princi-
ples of an artefact-based and problem-driven RE improvement (ArtREPI). Our
investigation should give a first qualitative impression of the general benefits
and shortcomings in direct comparison to existing principles of solution-driven
RE improvements that currently dominate the publication landscape [7]. The
purpose was also to lay the foundation for independent empirical investigations
to eventually explore the full spectrum of RE improvement.
To this end, we reported on two industrial case studies and analysed (1)
how ArtREPI would support process engineers in achieving a problem-driven
and artefact-based RE improvement, and (2) how well project participants were
supported in their project environments by the resulting artefact-based RE ref-
erence model. We further conducted the second case with very little influence of
the approach authors to investigate, in a first step, to what extent the results
might be influenced by us researchers being involved in the RE improvement.
Our results strengthen our confidence that ArtREPI is suitable for a self-
contained, problem-driven RE improvement where the improvement goals are
in tune with the scope of artefact orientation. More important, we could also
reveal first limitations and factors important to future work in RE improvement
research. We need to further explore the measurability of an improvement and
its long-term eects, and to further support scaling up to practice. To support
the generalisation of our observations that, so far, are only valid for chosen local
improvement contexts [16], we need to (1) foster independent replications of our
case studies which we support by our contribution and our publicly accessible
online material [9], and (2) we need to take new perspectives.
Rather than further investigating the technicalities of applying ArtREPI our-
selves, we need to concentrate on exploring further social and cognitive facets
and the long-term eects of an RE improvement for which our contribution has
provided one foundation and sensitisation. We thereby encourage researchers and
practitioners to join us in this endeavour to fully understand the broad spectrum
of possibilities and limitations in an artefact-based and problem-driven RE im-
Acknowledgements. We want to thank R. Wieringa and M. Broy for the
collaboration during the development of ArtREPI. We further want to thank
R. Bossek and M. Kuhrmann for their support during the systematisation of
ArtREPI, and S. Wiesi, J. Mund, J. Eckhardt, and H. Femmer for their support
in the REPI projects. Finally, we are grateful to all subjects involved during the
case studies and to B. Penzenstadler, M. Daneva, and A. Vetr`o for their valuable
feedback on previous versions of this manuscript.
1. endez Fern´andez, D. and Wagner, S.: Naming the Pain in Requirments Enginer-
ing: A Design for a global Family of Surveys and First Results from Germany. IST
2. Pettersson, F., Ivarsson, M., Gorschek, T., ¨
Ohman, P.: A practitioner’s Guide to
light weight Software Process Assessment and Improvement Planning. JSS (2008)
3. Napier, N., Mathiassen, L., Johnson, R.: Combining Perceptions and Prescriptions
in Requirements Engineering Process Assessment: An Industrial Case Study. TSE
4. Beecham, S., Hall, T., Rainer, A.: Defining a requirements process improvement
model. Software Quality Control (2005)
5. Staples, M., Niazi, M., Jeery, R., Abrahams, A., Byatt, P., Murphy, R.: An
Exploratory Study of why Organizations do not adopt CMMI. JSS (2007)
6. endez Fern´andez, D. and Penzenstadler, B. and Kuhrmann, M. and Broy, M.:
A Meta Model for Artefact-Orientation: Fundamentals and Lessons Learned in
Requirements Engineering. In: MoDELS. (2010)
7. endez Fern´andez, D., Ognawala, S., Wagner, S., Daneva, M.: Where Do We Stand
in Requirements Engineering Improvement Today? First Results from a Mapping
Study. In: ESEM. (2014)
8. endez Fern´andez, D. and Wieringa, R.: Improving Requirements Engineering by
Artefact Orientation. In: PROFES. (2013)
9. endez Fern´andez, D.: ArtREPI Online Resources.,
DOI: 10.13140/RG.2.1.2165.5521
10. Beecham, S., Hall, T., Rainer, A.: Software Process Improvement Problems in
Twelve Software Companies: An Empirical Analysis. EMSE (2003)
11. Beecham, S., Hall, T., Britton, C., Cottee, M., Austen, R.: Using an Expert Panel
to Validate A Requirements Process Improvement Model. JSS (2005)
12. Kuhrmann, M. and Beecham, S.: Artifact-Based Software Process Improvement
and Management: A Method Proposal. In: ICSSP. (2014)
13. endez Fern´andez, D.: Artefact-based Requirements Engineering Improvements:
Learning to Walk in Practice. In: PROFES. (2014)
14. endez Fern´andez, D. and Penzenstadler, B.: Artefact-based requirements engi-
neering: The AMDiRE pproach. RE Journal (2014)
15. Runeson, P., ost, M.: Guidelines for Conducting and Reporting Case Study
Research in Software Engineering. EMSE (2009)
16. Wieringa, R.: Empirical Research Methods for Technology Validation: Scaling up
to Practice. JSS (2013)
17. Liskin, O.: How Artifacts Support and Impede Requirements Communication. In:
REFSQ. (2015)
18. endez Fern´andez, D. and Mund, J and Femmer, H. and Vetr`o, A.: In Quest for
Requirements Engineering Oracles: Dependent Variables and Measurements for
(good) RE. In: EASE. (2014)
19. Pfleeger, S.: Understanding and Improving Technology Transfer in Software Engi-
neering. JSS (1999)
... Yeniden mühendislik (reengineering), yalnızca iş süreçlerindeki değişiklik değil; organizasyonun misyonu doğrultusunda, vizyonunu (Nowakowski, 1994), hedeflerini, yapılarını, yönetim sistemlerini (Hammer, 1990), performans sonuçlarını ve müşterilerini özenli bir şekilde yeniden değerlendirmesi (reassesment) ve değişimi anlamına gelmektedir (Brock ve diğerleri, 1997: 8). Yeniden mühendislik (reengineering), organizasyonların süreçlerinin maliyet, kalite, hız gibi performanslarını ve yeni uygulamalarını değerlendirip (Fernández ve Wagner, 2015) yeniden iyileştirmesidir (reimprovement) (Hammer ve Champy 2009: 35). Krizler ve gerileme dönemleri gibi olumsuz durumların yanı sıra teknolojik değişimleri takip etmek için iş süreçlerini yeniden mühendislik (reengineering) yaklaşımıyla dönüştüren organizasyonlar, sorumlulukları kuruluşa, çalışanlara ve tedarikçilere, yetki ve takdiri müşteriye (Radhakrishnan ve Balasubramanian, 2008: 18) ve kaynakları da tasarruf sağlamak için yeni süreçlere Verimlilik Dergisi / Journal of Productivity yeniden tahsis ederler (reallocation) (Myronenko, 2009). ...
Full-text available
Amaç: Bu makalenin amacı, organizasyonlara değişim için bir ‘Reorganizasyon Modeli’ önermek, modelin uygulanmasına yardımcı olacak bir yol haritası sunmak ve modelin yerel yönetimlerde uygulandığında verimliliği artırdığını, kurumsal performans ve çalışan memnuniyeti sonuçları üzerinden göstermektir. Yöntem: Reorganizasyon modelindeki her bir kavramın literatür taraması yapılmış, taslak bir model ve yol haritası hazırlanmıştır. Modeli kısmen uygulamış ve EFQM ödülünü kazanarak başarılı olmuş bir kuruluştaki üst düzey yöneticilerle yapılandırılmamış görüşme tekniği yapılarak taslaklara son hali verilmiştir. Modelin kurumsal verimliliği artırdığını göstermek için, uygulama sonuçları, aynı kuruluşta belge analizleri metoduyla incelenmiştir. Bulgular: Önerilen modeli uygulayan organizasyonun kurumsal performans ve çalışan memnuniyeti göstergelerinde yıllara sâri bir artış gözlemlenmiş ve modelin kurumsal verimliliği artırdığı ispatlanmıştır. Özgünlük: Bu makaleyi diğerlerinden farklılaştıran nokta, reorganizasyon modelinin literatüre dayalı bütünsel ve kapsayıcı bir şekilde ele alınması, ortak bir dil oluşturacak ve kuruluşlara değişim süreçlerinde yardımcı olacak ilk ve tek model olması, modelin tüm aşamalarının sistemsel bakış açısına sahip olması ve uygulanabilmesi için bir yol haritası önermesidir.
... In this paper, we have reported on a technique including tooling for a lightweight self-assessment of security compliance in the development process. One particular focus was set on covering both the artifact view and the process view alike as we consider both imperative in context of software process improvement [18,20]. The transformation from a regular development process to a secure development process can be further tracked with metrics based on artifacts and tasks. ...
Conference Paper
Full-text available
Companies are often challenged to modify and improve their software development processes in order to make them compliant with security standards. The complexity of these processes renders it difficult for practitioners to validate and foresee the effort required for compliance assessments. Further, performing gap analyses when processes are not yet mature enough is costly and involving auditors in early stages is, in our experience, often inefficient. An easier and more productive approach is conducting a self-assessment. However, practitioners, in particular developers, quality engineers, and product owners face difficulties to identify security-relevant process artifacts as required by standards. They would benefit from a proper and light-weight tool to perform early compliance assessments of their processes w.r.t. security standards before entering an in-depth audit. In this paper, we report on our current effort at Siemens Corporate Technology to develop such a light-weight assessment tool to assess the security compliance of software development processes with the IEC 62443-4-1 standard, and we discuss first results from an interview-based evaluation.
The effective communication of the requirements influences the success of software development projects. Achieving effective communication of the requirements is difficult due to the involvement of several persons with different roles, skills, knowledge and responsibilities. Although many studies analyze the communication between clients and system analysts, they do not focus on the communication within the development team. In this research, we propose the creation of a set of artifacts and models to support the communication of requirements. We will base our proposal on the different perspectives of the team members according to their experience in the artifacts and models adopted in the development process within their organization. We will follow a methodology based on Design Science Research guidelines, which will guide us through the creation and evaluation of the artifacts and models to solve problems with the communication of requirements. Our goal is to improve the communication of requirements between the members of a development team, reducing the loss of requirements information during the execution of the software project
Conference Paper
Full-text available
Requirements engineering process improvement (REPI) has gained much attention in research and practice. Most REPI approaches are of solution-driven and activity-based nature. They focus on the assessment of company-specific RE reference models against an external norm of best practices, and they propagate an improvement by forecasting the adaptation of the processes and methods in the RE reference model towards that norm. In recent years, we could develop a first problem-driven RE improvement approach that supports an improvement against individual goals and problems of a company putting primary attention to the quality of the RE artefacts (named ArtREPI). In this short paper, we briefly illustrate our resulting approach and report on our initial experiences from ongoing empirical evaluations in practice. We conclude with a summary of planned next steps.
Full-text available
Context: Requirements engineering process improvement (REPI) approaches have gained much attention in research and practice. Goal: So far, there is no comprehensive view on the research in REPI in terms of solutions and current state of reported evidence. We aims to provide an overview on the existing solutions, their underlying principles and their research type facets, i.e. their state of empirical evidence. Method: To this end, we conducted a systematic mapping study of the REPI publication space. Results: This paper reports on the first findings regarding research type facets of the contributions as well as selected methodological principles. We found a strong focus in the existing research on solution proposals for REPI approaches that concentrate on normative assessments and benchmarks of the RE activities rather than on holistic RE improvements according to individual goals of companies. Conclusions: We conclude, so far, that there is a need to broaden the work and to investigate more problem-driven REPI which also targets the improvement of the quality of the underlying RE artefacts, which currently seem out of scope.
Conference Paper
Full-text available
Context: For many years, researchers and practitioners have been proposing various methods and approaches to Requirements Engineering (RE). Those contributions remain, however, too often on the level of apodictic discussions without having proper knowledge about the practical problems they propagate to address, or how to measure the success of the contributions when applying them in practical contexts. While the scientific impact of research might not be threatened, the practical impact of the contributions is. Aim: We aim at better understanding practically relevant variables in RE, how those variables relate to each other, and to what extent we can measure those variables. This allows for the establishment of generalisable improvement goals, and the measurement of success of solution proposals. Method: We establish a first empirical basis of dependent variables in RE and means for their measurement. We classify the variables according to their dimension (e.g. RE, company, SW project), their measurability, and their actionability. Results: We reveal 93 variables with 167 dependencies of which a large subset is measurable directly in RE while further variables remain unmeasurable or have too complex dependencies for reliable measurements. We critically reflect on the results and show direct implications for research in the field of RE. Conclusion: We discuss a variety of conclusions we can draw from our results. For example, we show a set of first improvement goals directly usable for evidence-based RE research such as "increase flexibility in the RE process", we discuss suitable study types, and, finally, we can underpin the importance of replication studies to obtain generalisability.
Full-text available
The various influences in the processes and application domains make Requirements Engineering (RE) inherently complex and di�cult to im- plement. In general, we have two options for establishing an RE approach: we can either establish an activity-based RE approach or we can establish an artefact-based one where project participants concentrate on the RE artefacts rather than on the way of creating them. While a number of activity-based RE approaches have been proposed in recent years, we have gained much em- pirical evidence and experiences about the advantages of the artefact-based paradigm for RE. However, artefact orientation is still a young paradigm with various interpretations and practical manifestations whereby we need a clear understanding of its basic concepts and a consolidated and evaluated view on the paradigm. In this article, we contribute an artefact-based approach to RE (AMDiRE) that emerges from six years of experiences in fundamental and evidence-based research. To this end, we first discuss the basic notion of artefact orientation and its evolution in recent years. We briefly introduce a set of artefact-based RE models we developed in industrial research cooperations for di↵erent ap- plication domains, show their empirical evaluations, and their dissemination into academia and practice, eventually leading to the AMDiRE approach. We conclude with a discussion of experiences we made during the development and di↵erent industrial evaluations, and lessons learnt.
Conference Paper
Full-text available
Context: For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, those studies either concentrate on validating specific methods or on single companies or countries. Therefore, they allow only for limited empirical generalisations. Objective: To lay an empirical and generalisable foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method: We designed a globally distributed family of surveys in joint collaborations with different researchers from different countries. The instrument is based on an initial theory inferred from available studies. As a long-term goal, the survey will be regularly replicated to manifest a clear understanding on the status quo and practical needs in RE. In this paper, we present the design of the family of surveys and first results of its start in Germany. Results: Our first results contain responses from 30 German companies. The results are not yet generalisable, but already indicate several trends and problems. For instance, a commonly stated problem respondents see in their company standards are artefacts being underrepresented, and important problems they experience in their projects are incomplete and inconsistent requirements. Conclusion: The results suggest that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.
Conference Paper
Full-text available
When it comes to software process improvement (SPI), process engineers look for SPI methods to support process analysis, design, realization, deployment, and management. Although a number of different SPI methods and models exist, process engineers tend to view these as too generic, too large, or a poor fit for the organization in which SPI is conducted. A strategy to overcome these shortcomings is to concentrate on the artifacts, which precisely define the de-sired outcomes, rather than on specific methods. In this paper, we present the Artifact-based Software Process Improvement & Management (ArSPI) model that provides a unified perspective on SPI and company-wide software process management (SPM), the required key artifacts, and the life cycle models. ArSPI is shown to be of practical sup-port to industry who called for a practical way to define the interfaces between SPI projects. This paper concludes with an example of how ArSPI paved the way for several organizations through applying the model in real-world SPI-projects.
Full-text available
Requirements engineering (RE) is a key discipline in software development and several methods are available to help assess and improve RE processes. However, these methods rely on prescriptive models of RE; they do not, like other disciplines within software engineering, draw directly on stakeholder perceptions and subjective judgments. Given this backdrop, we present an empirical study in RE process assessment. Our aim was to investigate how stakeholder perceptions and process prescriptions can be combined during assessments to effectively inform RE process improvement. We first describe existing methods for RE process assessment and the role played by stakeholder perceptions and subjective judgments in the software engineering and management literature. We then present a method that combines perceptions and prescriptions in RE assessments together with an industrial case study in which the method was applied and evaluated over a three-year period at TelSoft. The data suggest that the combined method led to a comprehensive and rich assessment and it helped TelSoft consider RE as an important and integral part of the broader engineering context. This, in turn, led to improvements that combined plan-driven and adaptive principles for RE. Overall, the combined method helped TelSoft move from Level 1 to Level 2 in RE maturity, and the employees perceived the resulting engineering practices to be improved. Based on these results, we suggest that software managers and researchers combine stakeholder perceptions and process prescriptions as one way to effectively balance the specificity, comparability, and accuracy of software process assessments.
Conference Paper
[Context & motivation] Requirements artifacts, like specifications, diagrams, or user stories, are often used to support various activities related to requirements. How well an artifact can support a specific activity depends on the artifact’s nature. For example, a plain text document can be adequate to provide contextual information, but is not well suited in terms of documenting changes. [Questions / problem] We wanted to understand how practitioners in various roles use requirements artifacts, how they manage to work with multiple artifacts at a time, and whether they use current practices for linking related artifacts. [Principal ideas / results] We have conducted an interview study with 21 practitioners from 6 companies. The interviews indicate that often a variety of artifact types is needed to successfully conduct a project. At the same time, using multiple artifacts causes problems like manual translation effort and inconsistencies. Mapping mechanisms that explicitly relate different artifacts are needed. However, existing methods are often not used. We investigate why these methods challenge developers in practice. [Contribution] We show challenges and chances of requirements artifacts. Our findings are grounded on true experiences from the industry. These experiences can support software developers in planning and improving their processes with regard to better requirements communication and researchers in making mapping methods more applicable in industry.
Context For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, they still do not allow for empirical generalisations. Objective To lay an empirical and externally valid foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method We designed a globally distributed family of surveys in joint collaborations with different researchers and completed the first run in Germany. The instrument is based on a theory in the form of a set of hypotheses inferred from our experiences and available studies. We test each hypothesis in our theory and identify further candidates to extend the theory by correlation and Grounded Theory analysis. Results In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from Germany with participants from 58 companies. The results reveal, for example, a tendency to improve RE via internally defined qualitative methods rather than relying on normative approaches like CMMI. We also discovered various RE problems that are statistically significant in practice. For instance, we could corroborate communication flaws or moving targets as problems in practice. Our results are not yet fully representative but already give first insights into current practices and problems in RE, and they allow us to draw lessons learnt for future replications. Conclusion Our results obtained from this first run in Germany make us confident that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.