Content uploaded by Birgit Penzenstadler
Author content
All content in this area was uploaded by Birgit Penzenstadler on Oct 07, 2019
Content may be subject to copyright.
A Case Study on the Application of an
Artefact-Based Requirements Engineering Approach
Daniel M´
endez Fern´
andez, Klaus Lochmann, Birgit Penzenstadler, Stefan Wagner
Institut f¨
ur Informatik
Technische Universit¨
at M¨
unchen
mendezfe, lochmann, penzenst, wagnerst@in.tum.de
Abstract—[Background:] Nowadays, industries are facing the
problem that the Requirements Engineering (RE) process is
highly volatile, since it depends on project influences from
the customer’s domain or from process models used. Artefact-
based approaches promise to provide guidance in the creation
of consistent artefacts in volatile project environments, because
these approaches concentrate on the artefacts and their depen-
dencies, instead of prescribing processes. Yet missing, however, is
empirical evidence on the advantages of applying artefact-based
RE approaches in real projects. [Aim:] We developed a customis-
able artefact-based RE approach for the domain of business
information systems. Our goal is to investigate the advantages
and limitations of applying this customisable approach in an
industrial context. [Method:] We conduct a case study with our
artefact-based RE approach and its customisation procedure. For
this, we apply it at a software development project at Siemens
following the steps of the customisation procedure. We assess
our approach in direct comparison with the previously used RE
approach considering possible improvements in the process and
in the quality of the produced artefacts. [Results:] We show that
our approach is flexible enough to respond to the individual needs
in the analysed project environment. Although the approach is
not rated to be more productive, we find an improvement in
the syntactic and the semantic quality of the created artefacts.
[Conclusions:] We close a gap in the RE literature by giving
empirical evidence on the advantages of artefact orientation in
RE in an industrial setting.
Keywords—Requirements Engineering, Artefact Model, Devel-
opment Process Model, Case Study
I. INTRODUCTION
Requirements Engineering (RE) lays the foundation for
successful software and system development projects regard-
ing cost and quality (Broy 2006). The activities, which are
performed as part of the RE process, aim at the discovery
and specification of requirements that unambiguously reflect
the purpose of a software system as well as the needs of
all relevant stakeholders (Nuseibeh and Easterbrook 2000).
The precise definition of requirements supports subsequent
software development activities like architectural design or
project management. As a software engineering discipline,
RE contributes with precise requirements specifications to
appropriateness and cost-effectiveness in the development of
a system (Nuseibeh and Easterbrook 2000) and, thus, RE
is an important factor for productivity and (product) qual-
ity (Damian and Chisan 2006).
An important step for companies towards RE excellence
consists in the establishment of an RE reference process
for a company-wide use among different projects. However,
companies are facing the problem that the RE process is
highly volatile, since it depends on project influences from the
customer’s domain or from process models used. Thus, volatile
project environments restrict the appropriate application of the
reference process at project level and complicate the decisions
on what artefacts to produce in RE and on how to produce
them in a syntactically consistent and complete manner. An-
other complicating aspect is that many influences arise during
the execution of the process, as many aspects are not clear
from the beginning of a project (Berenbach et al. 2009).
An observable consequence of the diversity in the project
influences is that engineers often act solution-biased. This can
lead to incomplete and inconsistent artefacts and thereby to
disruptions in the development life cycle (Mendez Fernan-
dez et al. 2010b).
Yet missing is an RE approach that provides:
1) a reference model (model blueprint) of the basic concepts
to guide the creation of precise RE artefacts.
2) guidance for the customisation (tailoring) of the reference
model at project level in response to project parameters,
which affect the precision in the artefacts.
Artefact orientation seems an appropriate philosophy to
provide such an approach. Artefact models define a blueprint
of the results to be created and thereby abstract from processes,
i.e., from the actual creation of specification documents by
the use of particular methods in a particular sequence. Hence,
as artefact models abstract from complex processes while
capturing the basic concepts of an application domain, they
should provide a means to tackle the problems stated above.
Problem Statement: Although the advantages of artefact
orientation for RE are recognised (Mendez Fernandez et al.
2010a), no empirical evidence has been given yet on the
application of artefact-based RE approaches in real projects.
Research Objective: We want to investigate the advan-
tages and limitations of applying a customisable artefact-based
RE approach at project environments. We developed such
an artefact-based RE approach for the application domain
of business information systems in a research cooperation
with Capgemini Technology Services (TS) (Mendez Fernandez
and Kuhrmann 2009). Because this approach is declared as
the standard reference model for RE at Capgemini TS, we
gained practical experience there, but we have little empirical
evidence on how the approach finally tackles the stated prob-
lems beyond that. Hence, we want to investigate whether our
Proceedings of EASE 2011
104
approach can be generally applied in real projects independent
of the organisational culture of particular companies and
whether it satisfies the need of a flexible RE process supporting
the creation of precise results.
Contribution: In this paper, we contribute a case study
on the application of a customisable artefact-based refer-
ence model for business information systems’ analysis (short:
BISA). For this, we apply our previously developed BISA
approach in an industrial case study to analyse possible process
improvements in a practical setting. We investigate the actual
RE process and the resulting artefacts in a project at Siemens
considering the development of a traffic control system. We
apply our artefact-based approach in the same context and
compare both our approach and the previously used one
with respect to the performed process and the quality in the
resulting artefacts.
II. FUNDAMENTALS AND RELATED WORK
We distinguish between an activity-based and an artefact-
based philosophy. In both areas there exists a variety of contri-
butions, which in turn can be structured into single approaches
and comprehensive development process models. The latter
abstract from the idealised project execution including a set
of methods, milestones, roles, and artefacts. We subsequently
discuss in both philosophies related work with respect to
customisable approaches and conclude with an introduction
into our previously published work.
A. Activity Orientation
Activity-based approaches rely on the philosophy of defin-
ing a concrete process by a set of methods to be performed in a
particular order by a specific set of roles. Each of the methods
provides a construction procedure to combining description
techniques (Nuseibeh and Easterbrook 2000). These tech-
niques are used to record the results into previously defined
specification documents (or data sets) (Braun et al. 2005).
Prominent examples are development process models, which
are based on the Software & Systems Process Engineering
Meta-Model (SPEM) (OMG 2008), such as the Rational
Unified Process (RUP) (Kroll and Kruchten 2003). However,
while such approaches offer the necessary elements to define a
process, the customisation of the process to particular project
environments is barely described and left to the expertise of
project participants.
As a response to this shortcoming, Situational Method
Engineering (Brinkkemper 1996) provides approaches to se-
lect and combine methods from a repository. This area can
further be complemented by (content-centric) Decision Sup-
port Systems (see e.g. Regnell et al. 2001, Ngo-The and
Ruhe 2005), which contribute approaches to select, classify,
and rate a set of alternatives in the choice of methods (and
description techniques) according to project parameters. Still,
although the importance of a well-defined artefact model is
recognised in this area (Foorthuis et al. 2008), artefacts and
their dependencies are not in scope in available approaches.
Braun et al. (2005) discover that only 50% of the analysed
approaches include an artefact description at all, while the
approaches that include an artefact description reduce the
artefacts to an optional outcome of the methods.
Activity orientation thereby does in general not guide in
the creation of precise result structures, since available ap-
proaches emphasise the selection and combination of methods
rather than their integration and application considering their
syntactic compatibility and the consistency in the resulting
artefacts (Arthur et al. 1997). The project-specific combination
of methods is additionally hampered by the diversity in the
project parameters and by the variability in the RE processes.
Hence, activity orientation does not satisfy the demand for
an RE approach that guides in the creation of syntactically
consistent artefacts while supporting the necessary flexibility
during this creation. This is also reflected by the absence of
empirical work in this area. Despite available case studies that
analyse the application of isolated methods, further studies,
which consider comprehensive development process models,
put emphasis on the resulting process rather than on its
customisation and application (Pedreira et al. 2007).
B. Artefact Orientation
A possibility to provide guidance in the creation of consis-
tent artefacts in volatile project environments is to concentrate
on the artefacts rather then on the way of creating them. This
leads to process-neutral RE approaches following an artefact-
based philosophy.
The idea of artefact-based approaches consists of defining
a reference model of all artefacts that are an intermediate
result of a development process. At project level, the actual
process is then defined by agreeing on a set of artefacts to be
created by a particular role and to be delivered when reaching
a particular milestone. The diversity in the process definitions
is thereby reduced to the dependencies between the artefacts
themselves without having to take into account the complexity
of differing processes. Artefact-based approaches are thereby
meant to guide in the creation of precise results while offering
the necessary flexibility during their creation.
However, there exist different views onto the structure and
the syntax of artefact models depending on the intended
purpose of the models. We can distinguish two major areas,
in which artefact models are applied: development process
models and model-based development approaches.
A prominent development process model, which relies on
the artefact-based philosophy is the V-Modell XT1, a Ger-
man standard for IT development projects. The V-Modell
XT defines an artefact (e.g. requirements specification) as a
deliverable, which is coupled to a milestone, a role, and a set
of activities. The included customisation approach describes
mechanisms to apply the development process model as part of
an initial project plan by selecting (according to a pre-defined
set of project parameters) those artefacts, which are necessary
in a project, similar to what is done with methods in decision
support systems. The next steps consider the assignment of
roles and milestones to the selected artefacts.
1Available at http://www.V-Modell-XT.de
Proceedings of EASE 2011 105
While the V-Modell XT and its customisation approach with
its abstract view onto an artefact model ease process integra-
tion and customisation, they neglect, however, the domain-
specific content in the artefacts.
Such content descriptions can be found, in turn, in the
area of model-based development, in which artefact models
(often referred to as “RE meta models”) capture the basic
concepts and relations found in the description techniques used
for a design methodology with respect to a chosen family
of systems (Sch¨
atz et al. 2002). Thus, available approaches
support the demanded precision in the creation of the artefacts.
At the same time, these artefact models become complex
and thereby hamper the process integration and the customi-
sation (by coupling the process elements to the artefact model).
C. Previous Work on Artefact-based RE
In (Mendez Fernandez et al. 2010a), we developed a meta
model for artefact-based RE approaches, which, if instantiated
for a particular application domain, unifies the advantages
of artefact-based development process models and model-
based development. We aimed at supporting a flexible process
definition and at the same time offering guidance for the
creation of precise results. For this purpose, we combined the
two different views onto an artefact that have been treated in
an isolated manner before. A structure view captures for each
artefact type (e.g., “requirements specification”) the content
items to be considered (e.g., “use case model”). Each artefact
type is then coupled to the elements necessary to define a
process, i.e., to roles, methods, and milestones. For each
content item, we define the content view via the modelling
concepts, e.g., the elements and relations of a use case model
and different description techniques that can be used to in-
stantiate these concepts, such as a UML activity diagram. The
structure model thus supports the flexible process definition
that we consider in the artefact-based customisation approach
in Sect. IV. The content model supports the precision of the
results during this customisation.
Figure 1 illustrates on its top both views taken by the
meta model. The middle part of the figure illustrates an
exemplary excerpt of the artefact model, which is part of the
artefact-based RE reference model presented in the following
Sect. III. The bottom part illustrates an exemplary outcome
of customising the reference model, i.e., of creating project-
specific exemplars of the reference model (see Sect. IV).
III. ARTE FACT-BASED REQUIREMENTS ENGINEERING
REFERENCE MODEL
According to our understanding about artefact orientation in
RE (see Sect. II-C), we developed an artefact-based reference
model for business information systems analysis (BISA) in
a cooperation with Capgemini TS. The development was
performed over two years while we continuously evaluated
the approach within 12 project environments. With the evalu-
ation, we ensured the validity of the approach for the chosen
application domain so that it correctly covers its basic concepts
in a sufficiently complete manner.
Meta ModelRE Reference Model
Structure Content
Project-specific
Exemplars
instance ofinstance of
PRODUKT.PROJEKTBEZEICHNUNG - PRODUKT.NAME
Zuletzt geändert: 27.10.2010 13:28 3/20
Content
1Introduction..........................................................................................................................6
1.1Overview.......................................................................................................................6
1.2Purpose..........................................................................................................................6
1.3References.....................................................................................................................7
1.4Scope.............................................................................................................................8
2System Vision......................................................................................................................8
2.1Summary of Business Specification..............................................................................8
2.2Scope of Information System under Consideration......................................................8
2.2.1System Overview...................................................................................................8
2.2.2External Systems..................................................................................................10
2.2.3Use Case Overview..............................................................................................10
2.2.4Information System Service Overview................................................................10
3Information System Requirements.....................................................................................11
3.1Actors..........................................................................................................................11
3.2Generic Scenarios........................................................................................................11
3.3Domain-specific Application Capabilities..................................................................12
3.3.1<<Business Domain>> <Name>..........................................................................12
3.4Information System Objects........................................................................................14
3.5System Quality Requirements.....................................................................................16
3.6Architectural Constraints.............................................................................................16
3.6.1Logical Restrictions..............................................................................................17
3.6.2Technical Restrictions..........................................................................................17
4Integrational Requirements................................................................................................18
4.1Deployment Requirements..........................................................................................18
4.2Migration Requirements..............................................................................................18
5Organisational Requirements.............................................................................................19
5.1Project Requirements..................................................................................................19
5.2Obligations..................................................................................................................19
5.3Glossary.......................................................................................................................19
6Abbreviations.....................................................................................................................20
7References..........................................................................................................................20
Travel Ordering System
Requirements Specification
Version: 0.1
Project Name <Name>
Project Lead <Name>
Responsible <Name>
Created on <Date>
Last changed
X In process
Submitted
State
Completed
Document File
V-Modell XT Version VMRELEASE 1.3with BISA Extension
illustrative
Fig. 1. Two Views onto an Artefact Unified by our Meta Model
The artefact-based BISA approach defines two artefact
types as a domain-specific interpretation of RE: The Business
Specification and the Requirements Specification. For each
artefact type, we define:
1) Methods and description techniques as construction pro-
cedures, which describe how to create the content of the
artefacts by the choice of a modelling language that has
to incorporate the concepts and relations defined by the
content model.
2) Roles, which define responsibilities.
3) Milestones, which define points in time on when to
complete an artefact.
We subsequently give a brief overview of the two artefact
types. Further elements of the BISA approach, such as a
status model to support progress control, are not in scope
of this paper and can be taken from (Mendez Fernandez
and Kuhrmann 2009). Figure 2 illustrates selected concepts
defined by the BISA approach with respect to their levels of
abstraction and used modelling views. A level of abstraction
is, in our context, a refinement stage over which the content
in the artefacts is refined and / or decomposed. We use
differently coloured shapes to illustrate to which artefact type
the depicted concepts relate and omit in the figure (for reasons
of complexity) the dependencies between the concepts.
A. Business Specification
The business specification contains all goals (statements of
intent), capabilities (behaviour), restrictions, and conditions
that affect the business of a company without describing
requirements towards the underlying systems.
In particular, the concepts used in the business specification
describe long-term objectives of a company and the desired
Proceedings of EASE 2011
106
Modelling Views
Environment
Levels of Abstraction
Data
Organisation's
Context
Business
Process
Hierarchy
Business
Process Logic
Information
System
Service
Hierarchy
Information
System's
Constraints
Business Vision
System Vision
Business
Process
Business
Service
Business
Tas k
StructureBehaviour
Process Step
Use CaseIS Service
Functional
Requirement
Objective
Principle
Business
Goal
Process
Owner
User Group
Process-
related Goal
IT-related
Goal
Actor
Architectural Constraint
System Quality Requirement
Business
Object
Information
Object
Information
System Object
Business Unit
Business
Domain
System
Overview
Legend
Concepts of
Requirements Specification
Concepts of
Business Specification
Fig. 2. Artefact Model of BISA in a Nutshell
business processes. We decompose goals over different levels
of abstraction. Goals motivate the decomposition of business
processes to a sequence of atomic process steps performed
by user groups. In relation to the decomposition of behaviour
models, we model the interchanged data and the organisation’s
structure. For describing structure, we distinguish between
the description of business units and business domains. The
latter groups logically related business processes for which one
process owner is responsible.
B. Requirements Specification
The requirements specification comprehends demanded
properties of information systems, organisational restrictions
on the development process, and restrictions on the integration,
that all shall be accomplished by a development project
according to the content of the business specification.
In particular, we make use of concepts for specifying
how the business process shall be realised by a particular
system. For this, we make use of use cases and services. The
first specifies interaction scenarios between actors (logically
representing user groups) and the system as a whole. In
constrast, services describe a logical representation of a use
case, not necessarily involving actors or concrete sequences
of interaction. Regarding the data modelling view, we use
information system objects as a system-side representation of
the information objects, which are interchanged in the business
processes. The system overview captures the context of a
system in interaction with surrounding systems. Finally, we
decompose behaviour models to further concepts, including ar-
chitectural constraints and system quality requirements, which
constrain a system and its environment in its properties and
conditions by means of metrics and values.
IV. ARTEFAC T-BASED CUSTOMISATION APPROACH
The customisation approach guides the creation of the
artefacts at project level in response to individual project
characteristics. Figure 3 illustrates on its left side the actual
approach and on its right side how the single steps relate to
decision support.
To enable this decision support in an artefact-based context,
we introduce a project repository. In contrast to activity-based
decision support systems, however, we characterise projects by
the impact of project parameters on the creation of artefacts,
instead of indirectly characterising projects by parameters
impacting the choice of methods for producing the artefacts.
The parameters’ impact can argue for the creation of certain
content items or hamper their creation. For instance, if user
groups are unavailable, this condition can hamper the creation
of use case models and at the same time argue for the
specification of risk status lists. An exemplary set of project
parameters can be taken from our previously performed field
study (Mendez Fernandez et al. 2010b). By the use of such a
project repository, we can support project participants in mak-
ing certain decisions. To support the transfer of made decisions
backwards from projects to the project repository, we introduce
the additional artefact BISA Diary. In this artefact, we capture
for each content item in the artefact, project parameters, which
affect the item’s completeness and corresponding decision that
has been taken. This way, we can use a project repository
in multi-project environments to re-use organisation-specific
experiences and decisions.
Regarding the actual customisation approach, we distinguish
two stages. The first stage considers the instantiation of the
BISA reference model during the Initial Project Set-Up to
initially define a process frame, similar as done in the V-
Modell XT (see Sect II-B). Within this initial set-up, we
elaborate the project background and create initially agreed
on exemplars of the artefacts. We choose for each artefact
the desired document structure and include the initial content,
which is already available. We assign the roles, define the
milestones, and set-up of the tool infrastructure. This step is
performed during initial workshops, e.g., during a kick-off
workshop. The second stage considers within this (planned)
process frame the dynamic content creation of the artefacts
during project execution, which results in a probable Project-
specific BISA Execution Strategy. This dynamic content cre-
ation is performed in response to project parameters that affect
the completeness of the artefacts. We decide on how to proceed
for the content of each business domain and document our
decisions within the BISA diary. For instance, user groups
might be unavailable for negotiating interaction scenarios in
the context of the business processes of the business domain
“Customer Administration”, for the domain “Sales” they could
be available. In that case, we could decide to document
services for the first business domain instead of use cases and
document this decision in the BISA diary.
The outcome of the project-specific BISA execution strat-
egy are artefacts in a certain quality and the documented
Proceedings of EASE 2011 107
Decision
Support
Customisation Approach
Stage 1: Initial Project Set-Up
Project
Background,
Documents,...
tt '
Stage 2: Project-specific BISA Execution Strategy
t
Artefact
Type
Assign Roles
Create Artefacts
- Structure
- Initial Content
Define Milestones
Set-Up Infrastructure Reflect at Decision Gate on Project Situation
(Impacts of potentially underspecified Artefacts on further Development Activities)
Project
Repository
BISA Diary
Project
Parameter
Analyse
Possibilities
Document
Rationale
Content
Item
Act
(Create Content)
Dynamic Content Creation
Reflect on
Project
Parameter
Content
Items
Impact
Fig. 3. Customisation Stages Considered by the BISA Approach
decisions. In contrast to activity orientation, we now are
able to objectively assess the quality of the artefacts with
respect to the artefact-based reference model. We can check
whether the artefacts are underspecified, i.e., syntactically
incomplete or inconsistent with respect to the reference model.
When reaching the milestone of the corresponding artefact,
we can reflect on potentially underspecified artefacts and their
consequences on further development activities that rely on
those artefacts.
Therefore, due to the notion of underspecified artefacts we
can objectively decide on what risks the quality in the results
implicate and, thus, achieve guidance in the creation of precise
results and of the made decisions that have lead to those
results.
V. C ASE STUDY DESIGN
We organise the case study according to Runeson and H¨
ost
(2009). After defining the goal and the research questions of
the study, we describe how we select the case and the subjects.
Finally, we describe how we collect and analyse the data,
before concluding with a discussion on the validity procedures.
A. Research Questions
The goal is to investigate the advantages and limitations
in the application of our customisable artefact-based RE
approach in an industrial environment. For this, we formulate
three research questions.
RQ 1 Does the artefact-based approach improve the usabil-
ity of the RE process?
One aim of artefact orientation consists in the flexibility of
the process, i.e., its usability in differing project environments
while leading to reproducible results. Thus, our first research
question aims at analysing whether we can achieve an im-
provement in the usability of the RE process.
RQ 2 Does the artefact-based approach improve the syn-
tactic quality of the created artefacts?
Once we analysed the actual process for creating the arte-
facts with respect to individual project influences, we want
to know whether the created artefacts are of higher syntactic
quality due to the underlying model-based philosophy of our
approach.
RQ 3 Does the artefact-based approach improve the se-
mantic quality of the created artefacts?
Finally, for implementing the requirements it is not only
important that the syntactic quality is high, but also that the
requirements are stated correctly and sufficiently detailed, i.e.,
the semantic quality is high. Thus, we want to know whether
the approach also improves the semantic quality of the created
content.
B. Case and Subjects Selection
We apply the artefact-based reference model and the corre-
sponding customisation approach to a software development
project and repeat the RE process for a part of the system
under consideration. Both the case and the subject selection are
opportunistic, because we need a real development project and
access to its participants and documentation. Nevertheless, we
choose a project that considers a similar application domain
as the one for which BISA has been developed but differs
in its industrial context. Especially, we are interested in an
evolving system, because Capgemini mostly replaces legacy
applications completely by new systems.
To select a representative part of the system, we hold a
discussion between the industry participants and researchers.
We select a set of business domains (logically clustering
potential processes and use cases) for which corresponding
stakeholders are available. This way, the approach can be
conducted entirely including the creation of the business and
the requirements specification. We define three main groups
of participants as study subjects:
1) Industry participants: Experts from industry responsible
Proceedings of EASE 2011
108
for performing the (baseline) RE phase of the system de-
velopment project under consideration. Ideally, they have
different viewpoints on the requirements specification,
e.g., product managers and developers.
2) Researchers: Researchers familiar with the BISA ap-
proach take the role of requirements analyst and support
the customisation procedure at the industry partner.
3) External reviewer: In order to achieve an unbiased assess-
ment of the produced specifications an external reviewer
(not involved in the actual process) will be called in.
C. Data Collection Procedures
The collection of the data for the case study comprises the
participation of the researchers in the RE process as well as
a concluding assessment of the performed process by internal
and external reviewers. We first perform the process according
to the customisation approach as part of a series of workshops
between the researchers and the industry participants. After-
wards, we assess the process and the produced artefacts in di-
rect (benchmark) comparison to the legacy process previously
used in the company. The industry participants do an internal
assessment. An external reviewer is called in especially for
conducting an independent external assessment.
1) Requirements Engineering Workshops: We conduct the
steps of the BISA approach in a series of workshops, which are
organised according to the customisation approach (Sect. IV).
In these workshops, the researchers and the industrial par-
ticipants are present while the researchers take the role of
requirements analysts.
Initial Project Set-Up: At the kick-off workshop, the
researchers present the BISA reference model and the cus-
tomisation approach. We customise the BISA reference model
to initially set up the project. We select the artefacts to be
created, decide on a preferable document structure, assign the
roles, define the milestones for when to complete the artefacts,
and finally agree on the used tool infrastructure. Then, we
discuss a set of specification documents, which we use as an
initial input. This input includes a description of the business
goals, the relevant stakeholders and the business domains. We
use the latter to organise the subsequent workshops.
Dynamic Content Creation: As part of the second cus-
tomisation stage, we perform a workshop for each of the
relevant business domains. In each of the workshops, we
discuss the business processes of a domain and corresponding
use cases. We use both the processes and the use cases to
initially sketch the corresponding vision documents (business
vision and the system vision). After each of the workshops,
the researchers further specify the contents on the basis of the
given information, i.e., they complete the use cases or infer
probable system quality requirements, which are presented and
approved at the beginning of the next workshop.
Approval and Acceptance: After having created the con-
tent for each of the identified business domains, i.e., when
reaching the agreed milestones, we jointly review the artefacts
and the industry participants can reject or accept them with
the possibility to demand further changes.
2) Assessment: As a preparation for the assessment, each
industry participant and the external reviewer review both the
previously documented specification documents and the BISA
artefacts (specifications and BISA diary). As a guideline for
the reviews, the reviewers get a questionnaire that will later
be used in the interviews. We perform these interviews with
the industry participants as a group interview (directed by
the researchers). The interview with the external reviewer is
performed in isolation.
The goal of both interviews is to answer the questionnaire,
which includes a set for different assessment criteria. Since the
external reviewer has no insights into the actually performed
process, the assessment criteria in his questionnaire is mostly
reduced to the ones that consider the quality of the produced
results. For each assessment criteria, we define a closed and an
open question. In a closed question we ask for the agreement
to a given statement. It can be answered on a Likert-scale from
1 = I strongly disagree to 8 = I strongly agree. We deliberately
choose an 8-point scale to avoid that the interviewees check
the middle. In the open question, we ask for their expert
opinion, in which the interviewee answers as free text. This
open question may also be used for additional remarks and
explanations regarding the selected grade on the Likert-scale.
The condensed questionnaire with the statements of the closed
questions is shown in Table I.
To answer RQ 1, we ask for information on the execution
of the process, respectively the customisation approach. Since
only the industry participants and researchers were present
while performing the process, only the industry participants
will answer this part of the questionnaire. An exception is
given by the assessment of the sustainability of the approach,
which is analysed by the external reviewer (only he can
analyse whether the process is reproducible on the basis of
given specifications).
We answer RQ2 by assessing the syntactical structure of the
specifications. This includes the structuring of the artefacts
into topics, which, e.g., provide an easy understanding of
traceability from high-level requirements to detailed ones. We
additionally investigate the content in the artefacts with respect
to cross-references between different content items, which
serve, e.g., as background information or as a rationale.
Regarding RQ 3, we assess the actual content of the pro-
duced specifications. To rate the minimality and the complete-
ness of the requirements, deep domain knowledge is necessary
whereby only the industry participants can answer these two
questions.
D. Analysis Procedures
Due to the low number of participants for the questionnaire,
statistical hypothesis testing is not applicable. Therefore, we
present the results of the closed questions as a radar chart.
We analyse the answers to the open question qualitatively to
further explain the answers of the closed questions and to
discuss the differences between the legacy process and BISA-
specific one.
Proceedings of EASE 2011 109
TABLE I
QUESTIONNAIRE FOR THE ASSESSMENT (CONDENSED)
Criteria Statement
RQ 1 Ease of use The approach is clear and understandable.
Sustainability The process and the taken decisions are reproducible.
Effectivity All information and requirements are used in design and management activities.
Flexibility The approach supports a flexible process in response to individual project characteristics.
Productivity The perceived productivity was high.
Structuredness The RE process is systematic.
RQ 2 Syntactic Consistency Elements in the specifications are used consistently.
Complexity The complexity in the cross-references is low.
Syntactic Completeness All syntactic elements needed to specify the requirements are given.
Syntactic Minimality There are no unnecessary syntactic elements in the specifications.
Modularity The specification is organised in modules, separated according to certain topics.
Traceability Each requirement has a rationale.
Ease of Perception The specifications are well-suited to be understood by people not involved into the process.
RQ 3 Unambiguity The requirements are stated unambiguously.
Testability The fulfilment of each requirement is measurable / testable.
Semantic Completeness All stakeholder needs are reflected by the specifications.
Semantic Minimality There are no needless requirements in the specifications.
Semantic Consistency There are no contradictory statements in the specification.
E. Validity Procedures
To increase the reliability of the statements of the industry
participants, and thus the internal validity, we perform a
group interview. Through the interaction between the group
members, memories and experiences of the participants are
stimulated. This way, they can produce insights that would
be less accessible without this technique. Furthermore, the
different group participants serve as quality control, because
extreme opinions are filtered out by the participants (Lindlof
and Taylor 2002).
Additionally, researcher triangulation is used to increase
internal validity: in addition to the assessment of the speci-
fications by the industrial participants (internal assessment),
the assessment is done by a researcher not participating in the
whole process (external assessment).
Moreover, methodological triangulation is used, by asking
both open and closed questions. Through the open questions
the interviewees can express their opinion more freely. On the
other side, the closed questions force them to agree on one
statement.
To mitigate the threat of a bias toward the BISA specifica-
tion by the external reviewer, he is not involved in the study
prior to the actual assessment. He is not allowed access to
further documents, like background information, to ensure his
judgment relies only on the produced artefacts.
VI. CASE STUDY RESULTS
After the description of the cases and subjects, we describe
the results and structure them according to the research ques-
tions. For reasons of confidentiality, we can not give detailed
information on the (baseline) RE approach, the system under
consideration, or illustrate the exact content of the produced
(BISA) specifications.
A. Case Description
The case study is conducted with a department of
Siemens AG. This department develops a traffic control system
(TCS). The system is a hybrid of geographically distributed
embedded controllers in traffic lights and a central information
processing and monitoring system. To stick to the application
domain for which the BISA approach has been developed, the
analysed sub-system is such a monitoring system of the TCS.
The TCS is in production for more than 30 years, while
each year a new release is developed. In each of those
releases, Siemens conducts a development project with an
RE process, which takes three months. The reference process
definition, the Siemens Reference Process House, underlies the
activity-based philosophy. The development process used in
our case is specialised according to the particular project needs
and characterised by a set of milestones and corresponding
methods, which are used to document the requirements in
natural language into previously structured, self-contained
Excel Specifications.
The BISA specifications were documented in Microsoft
Word documents, which we structured according to the (arte-
fact) structure model. To record the content of the BISA
specifications, we extended the UML tool Enterprise Architect
with a plugin, which is based on the content model, i.e., we
developed an UML profile and defined individual modelling
shapes to specify, e.g. business processes and goal graphs in
conformance to the artefact model.
For the content of the specification documents, we consider
the activities of three different business domains, which are
supported by the monitoring system: Planning,Operations,
and Maintenance. During operations, an operator is provided
with communication and controller states by the system. In
case of anomalies, these states are analysed and an initial
fault clearance is initiated. Furthermore, the operators perform
statistical analysis over a chosen period of time, and in case
of traffic accidents they provide juridical evidence about the
actual status of the corresponding traffic lights. Figure 4
illustrates an exemplary excerpt of the resulting requirements
specification (see Sect. III), which contains the use case
overview for the business domain “Operations”.
Proceedings of EASE 2011
110
Fig. 4. Exemplary (Operations) Use Cases
Regarding decision taking during the content creation, we
documented each decision and the underlying project param-
eter in the BISA diary (see Sect. IV) created with Microsoft
Word. As a whole, the diary contains 26 project parameters.
For instance, we documented the mentioned use cases in
natural language, because of their low (functional) complexity.
A further project parameter which took effect on the content
creation was, e.g., “timeboxing”, which we used as a rationale
for documenting business services instead of detailed business
processes.
B. Subject Description
As described in the subject selection, there are three groups
of participants. In the group industry participants there are two
roles, which were assigned to three employees of Siemens AG:
The Product Manager is responsible for defining the re-
quirements for the control and monitoring system from the
customer/user viewpoint. He represents potential customers.
The Project Lead is responsible for broader management activ-
ities of the development department. Regarding requirements
engineering, he is responsible for negotiating the requirements
with the product manager.
The group of researchers consists of three software engi-
neering researchers from the Technische Universit¨
at M¨
unchen
with a special focus on requirements engineering. Daniel
Mendez Fernandez is the main developer of the BISA ap-
proach. Klaus Lochmann and Birgit Penzenstadler also have
detailed knowledge on the BISA approach and participate in
the RE workshops.
As external reviewer acts Stefan Wagner of the Technische
Universit¨
at M¨
unchen, who also has knowledge about the BISA
approach. He was, however, not involved in the earlier steps
of the case study.
C. Analysis Results
Figure 5 illustrates the results of the assessment as a radar
plot, depicting the ratings of the closed questions.
In the following, we describe the results with respect to our
research questions taking into account the answers made in
the open questions.
1) Usability of the Process (RQ 1): The BISA approach
achieves an improvement of the usability in the process. The
industry participants judged the BISA approach as easier to
use due to its flexibility and its guidance given by the artefact
model. In the interview, they explained that BISA defines a
clear customised process. It needs, however, guidance during
its customisation whereby deep knowledge in the approach
(“skills”) is needed. The legacy process to produce the Excel
specification is more ad-hoc, since is offers no guidance with
respect to individual project situations.
As the customisation approach gives guidance on struc-
turing elicitation workshops according to logically related
requirements clusters (business domains), they also saw an
increase of the structuredness in the approach. The industry
participants also stated an improvement in the effectivity as
they noticed a higher acceptance ration in the requirements,
i.e., less requirements had to be re-adjusted and negotiated
with corresponding architects after acceptance.
Regarding the productivity, they concluded that the BISA
approach is more heavy-weight due to the artefact-based ref-
erence model, which demands for the specification of several
topics in a certain syntactic quality. Still, they see this artefact-
based approach to be beneficial if a previously unknown sys-
tem is specified from scratch. If all participants have already a
common understanding of the problem space, however, – like
in the department where the case study was conducted – then
a more lightweight approach is also adequate. Therefore, they
judged productivity to be equal in both approaches.
Regarding the sustainability of the approach, the external
reviewer stated that the BISA process could be reproduced.
The reason is to be seen in the BISA diary, which documents
all the made decisions leading to the created artefacts. In
contrast to the BISA approach, he could not reproduce the
legacy process, because he was only confronted with the
produced Excel specification.
2) Syntactic Quality of the Artefacts (RQ 2): Both the
internal and external reviewers assessed the syntactic qual-
ity of the specifications. The internal reviewers judged the
syntactic completeness to have slightly increased in BISA.
They explained that in the Excel specification they are able to
define columns for all information needed. The use of further
description techniques like UML, however, is not possible.
On the other hand, Excel offers filtering functions that can
be used to aggregate needed information. BISA offers far
more syntactic elements and proposes different description
techniques for representing requirements, which are better
suited for certain kinds of information.
This is also reflected by the judgment of the external
review, who stated BISA to be substantially better regarding
syntactic completeness. This is because the artefact model
and the supported description techniques in BISA are more
specific and therefore easier to understand to external people.
This assessment is supported by the statement of the internal
reviewers that the syntactical elements in Excel are inconsis-
tently used whereby the meaning of the syntactical elements
often remain unclear.
Both reviews judged that there are less unused syntactic
elements in BISA. The external reviewer found out that
for example the column state in Excel is never used. One
Proceedings of EASE 2011 111
,
-
.
/
0
1
2
3
4
()
(
)
#&( )
%&
" #&($
)
" #&("$
)
""
& &
%&(&
)
&
& &
"&
&
("
)
(#)
"&
&
&
&
%
&
'
(
)
*
+
,
-
!
!
!!
!
!!
!
!
!
!
(a) Internal Review (b) External Review
Fig. 5. Analysis Results Shown as Radar Plots
reason for the syntactic minimality of BISA in contrast to the
elements given by the Excel specification is that the BISA
artefact model has been customised.
The judgment of the traceability is different for both re-
viewers. The internal reviewers see a marginal increase in
traceability in BISA, while the external reviewer assesses
the traceability in the Excel specifications with the lowest
grade and that of BISA with the highest. The reason for
this difference can be found in the explanations the internal
reviewers gave in the interview. They acknowledge that there is
no rationale for requirements given in Excel. They know, how-
ever, that there are other documents in their company where
the rationales are implicitly given. They further acknowledge
that background information, like goals, are specified in BISA
more comprehensively and structuredly. The difference in
the judgment is further explained by the comments of the
external reviewer. He could not discover any rationales for
requirements in Excel, thus he judged the traceability with
the lowest grade. In BISA however, he sees a clear top-down
hierarchy given by the refinement notion in the artefact model
(see Fig. 2 on page 4) whereby implicitly necessary domain
knowledge has to be made explicit.
3) Semantic Quality of the Artefacts (RQ 3): Both the
internal and external reviewers judged the testability of the
requirements slightly better in BISA than in the Excel spec-
ifications. The testability, as well as the unambiguity, are
improved in BISA, because the artefact model proposes a strict
content model including several requirements attributes, such
as acceptance criteria. Furthermore, both reviewers noticed
that the free text formulations in the Excel specifications
allows for more freedom in interpreting them. Again, the
artefact model reduced this freedom and gives more guid-
ance to create precise requirements. The internal reviewers,
however, judged the resulting contents to be more perceptible
by the development department rather than for the product
management, since the content model demands for detailed
specification of, e.g., architectural constraints.
The semantic consistency could be judged only by the
internal reviewers, because of their domain knowledge. They
rated the semantic consistency for the BISA specifications
slightly higher than for the Excel specification, because the
dependencies in the artefact model are more suitable to find
inconsistencies. The semantic completeness was stated to be
higher in the BISA specification for a similar reason. The
artefact model and the customisation approach supports a
structured discussion of business needs and requirements,
which would not have been considered before.
D. Evaluation of Validity
Regarding construct validity we see the threat that the
used questionnaire might not adequately represent the research
questions. Although the questionnaire was developed jointly
by all researchers participating in the study, it cannot be
ensured that there are no topics missing. In addition, the
answers of the participants are inherently subjective. Through
the intensive discussions of the answers and different view
points of the participants we could improve their objectivity.
The internal validity could be threatened by a bias towards
the BISA specification of the reviewers, because the external
reviewer was involved in the original development of the BISA
approach. This threat is only minor for the internal reviewers,
because they are comparing their legacy specification with
the BISA specification. Another threat to the internal validity
could be that different efforts were spent in creating both
specifications. The better completeness and traceability of the
BISA specification, for example, could be explained by more
effort dedicated to it. However, this threat is seen as minor,
because both approaches got the same rating on productivity.
As explained in the validity procedures, a researcher triangula-
tion was done to mitigate reliability threats. When comparing
the results of the internal and external assessment, we can see
that the answers are differing, but have the same trend. This
fact strengthens our confidence in the collected data. Also the
Proceedings of EASE 2011
112
methodological triangulation supports the internal validity. No
major contradictions between the explanations given in the
open questions and the rating of the closed questions could be
found.
Regarding external validity, the major concern is the gener-
alisability of the results, because we conducted this particular
case study only in one company. From the viewpoint of the
industry and research participants, however, a representative
part of the system under consideration was selected.
VII. CONCLUSION
Due to the missing agreement on artefact orientation, we
developed a meta model for this philosophy (Mendez Fer-
nandez et al. 2010a). Based on this meta model, we devel-
oped a reference model for business information systems’
analysis including a customisation approach, which transfers
the principles of decision support to the new artefact-based
philosophy (Mendez Fernandez and Kuhrmann 2009). In this
paper, we applied our approach in a case study at Siemens to
investigate the advantages and limitations of artefact orienta-
tion.
Summary of Conclusions: The proposed artefact-based
approach shows to be suitable for use in practice. The study
participants rated it as flexible enough to respond to their indi-
vidual project needs, especially in comparison to their current
activity-based approach. They only did not feel to be more
productive, but acknowledged in particular an improvement
in the syntactic and also the semantic quality of the created
artefacts.
Relation to Existing Evidence: To the best of our knowl-
edge there does not exist any reproducible case study that
compares activity-based approaches and artefact-based ones,
whereby the case study at hand closes this gap in literature.
Limitations: Inherent in case study research is the lim-
itation to a small number of cases and subjects. We cannot
generalise the strength of the approach, since it is intended
for the domain of business information systems. For software
with substantially different characteristics it might perform not
as well. Achieved improvements also depend on the existing
approach on site. In other contexts, in which model-based
requirements approaches are already used, the improvements
may be smaller. Finally, one discovered limitation is that the
approach seeks rather an application in unknown problem
domains, e.g., reflected by the productivity.
Impact/Implications: In addition to considering more
unknown problem domains, the establishment of artefact
orientation is, in general, an elaborate task (see Sect. III
introducing the development of the BISA approach). However,
taking RE as a critical process that is mostly driven by
uncertainty it is important to support a flexible process design
leading to high quality results. We could show that artefact
orientation is promising to satisfy exactly those demands. We
showed, e.g., an increase of both the syntactical and semantical
completeness and consistency of artefacts while those were
flexibly and reproducibly created in response to individual
project needs.
From a practitioner’s perspective, the results show that the
efforts necessary to establish an artefact-based RE approach
should be justified by the resulting benefits. From a re-
searcher’s perspective, it seems promising to further investigate
the field of artefact orientation in order to integrate available
methods for different application domains.
REFERENCES
H.M Arthur, T. Hofstede, and T. Verhoef, Jan 1997. On the feasibility of
situational method engineering* 1. Information Systems, 42-6-7 (1997),
401–422.
B. Berenbach, D. Paulish, J. Kazmeier, and A. Rudorfer, 2009. Software &
Systems Requirements Engineering: In Practice. McGraw-Hill Osborne
Media.
C. Braun, F. Wortmann, M. Hafner, and R. Winter, 2005. Method construction
- a core approach to organizational engineering. In SAC ’05: Proceedings
of the 2005 ACM symposium on Applied computing, p. 1295–1299. ACM.
S. Brinkkemper, 1996. Method Engineering: Engineering of Information
Systems Development Methods and Tools. Information and Software
Technology, (1996), 275–280.
M. Broy, 2006. Requirements Engineering as a Key to Holistic Software
Quality. Lecture Notes in Computer Science, 4263 (2006), 24.
D. Damian and J. Chisan, 2006. An empirical study of the complex rela-
tionships between requirements engineering processes and other processes
that lead to payoffs in productivity, quality, and risk management. IEEE
Transactions on Software Engineering, 32-7 (2006), 433–453.
R. Foorthuis, S. Brinkkemper, and R. Bos, 2008. An Artifact Model for
Projects Conforming to Enterprise Architecture. The Practice of Enterprise
Modeling, (2008), 30–46.
P. Kroll and P. Kruchten, 2003. The rational unified process made easy:
a practitioner’s guide to the RUP. Boston, MA, USA, Addison-Wesley
Longman Publishing Co., Inc.
Thomas R. Lindlof and Bryan C. Taylor, 2002. Qualitative communication
research methods. Thousand Oaks, Calif, Sage Publ., 2nd ed.eedition.
D. Mendez Fernandez and M. Kuhrmann, 2009. Artefact-Based Requirements
Engineering and its Integration into a Process Framework: A Customis-
able Model-based Approach for Business Information Systems’ Analysis.
Technischer Bericht, Technische Universit¨
at M¨
unchen.
D. Mendez Fernandez, B. Penzenstadler, M. Kuhrmann, and M. Broy,
2010a. A Meta Model for Artefact-Orientation: Fundamentals and Lessons
Learned in Requirements Engineering. In Proc. 13th Model Driven
Engineering Languages ans Systems (Models), D. Petriu, N. Rouquette,
and O. Haugen, redactie, volume 6395/2010, p. 183–197. Springer-Verlag
Berlin Heidelberg.
D. Mendez Fernandez, S. Wagner, K. Lochmann, and A. Baumann, 2010b.
Field Study on Requirements Engineering Artefacts and Patterns. In
Proceedings of the 14 th International Conference on Evaluation and
Assessment in Software Engineering. BCS eWIC.
A. Ngo-The and G. Ruhe, 2005. Decision Support in Requirements Engi-
neering, chapter 12, in Engineering and Managing Software Requirements.
Springer.
B. Nuseibeh and S. Easterbrook, 2000. Requirements Engineering: A
Roadmap. In Proceedings of the Conference on the Future of Software
Engineering, p. 35–46. ACM New York, NY, USA.
OMG, 2008. Software and Systems Process Engineering Meta-Model Speci-
fication 2.0. Technical report, Object Management Group.
O. Pedreira, M. Piattini, M.R. Luaces, and N.R. Brisaboa, 2007. A Sys-
tematic Review of Software Process Tailoring. ACM SIGSOFT Software
Engineering Notes, 32-3 (2007), 1–6.
B. Regnell, B. Paech, A. Aurum, C. Wohlin, A. Dutoit, and J.N. och Dag,
2001. Requirements Mean Decisions!–Research issues for understanding
and supporting decision-making in Requirements Engineering. In Fir st
Swedish Conference on Software Engineering Research and Practise:
Proceedings.
P. Runeson and M. H ¨
ost, 2009. Guidelines for conducting and reporting Case
Study Research in Software Engineering. Empirical Software Engineering,
14-2 (2009), 131–164.
B. Sch¨
atz, A. Pretschner, F. Huber, and J. Philipps, 2002. Model-Based
Development of Embedded Systems. In Advances in Object-Oriented
Information Systems, Lecture Notes in Computer Science, volume 2426,
p. 298–311. Springer-Verlag.
Proceedings of EASE 2011 113
