Content uploaded by Henry Muccini
Author content
All content in this area was uploaded by Henry Muccini on Aug 31, 2015
Content may be subject to copyright.
Handling Non-functional Requirements in
Model-Driven Development:
An Ongoing Industrial Survey
David Ameller∗, Xavier Franch∗, Cristina G´
omez∗,
Joao Araujo†, Richard Berntsson Svensson‡, Stefan Biffl§, Jordi Cabotk, Vittorio Cortellessa∗∗, Maya Daneva††,
Daniel M´
endez Fern´
andez‡‡, Ana Moreira†, Henry Muccini∗∗, Antonio Vallecillox, Manuel Wimmer§,
Vasco Amaral†, Hugo Bruneli`
erexi , Loli Burgue˜
nox, Miguel Goul˜
ao†, Bernhard Sch¨
atzxii and Sabine Teuflxii
∗Universitat Polit`
ecnica de Catalunya, Spain. {dameller, franch, cristina}@essi.upc.edu
†NOVA LINCS, Universidade Nova de Lisboa, Portugal. {p191, amm, vasco.amaral, mgoul}@fct.unl.pt
‡Chalmers |University of Gothenburg, Sweden. richard@cse.gu.se
§Vienna University of Technology, Austria. Stefan.Biffl@tuwien.ac.at, Wimmer@big.tuwien.ac.at
kICREA - UOC, Spain. jordi.cabot@icrea.cat
∗∗University of L’Aquila, Italy. {vittorio.cortellessa, henry.muccini}@univaq.it
††University of Twente, The Netherlands. M.Daneva@utwente.nl
‡‡Technische Universit¨
at M¨
unchen, Germany. mendezfe@in.tum.de
xUniversidad de M´
alaga, Spain. {av, loli}@lcc.uma.es
xi AtlanMod Team (Inria, Mines Nantes & LINA), France. hugo.bruneliere@inria.fr
xii Fortiss GmbH, Germany. {schaetz, teufl}@fortiss.org
Abstract—Model-Driven Development (MDD) is no longer a
novel development paradigm. It has become mature from a
research perspective and recent studies show its adoption in
industry. Still, some issues remain a challenge. Among them, we
are interested in the treatment of non-functional requirements
(NFRs) in MDD processes. Very few MDD approaches have been
reported to deal with NFRs (and they do it in a limited way).
However, it is clear that NFRs need to be considered somehow
in the final product of the MDD process. To better understand
how NFRs are integrated into the existing MDD approaches, we
have initiated the NFR4MDD project, a multi-national empirical
study, based on interviews with companies working on MDD
projects. Our project aims at surveying the state of the practice
for this topic. In this paper, we summarize our research protocol
and present the current status of our study. The discussion will
focus on the peculiarities of our study’s context and organization
involving about 20 researchers from 8 European countries.
Index Terms—Non-Functional Requirements, Model-Driven
Development, Quality Requirements, NFR, MDD, Empirical
Study, Survey, Semi-Structured Interviews.
I. INTRODUCTION
Model-Driven Development (MDD) is a development
paradigm, where models play a central role [1], [2]. “Model-
driven development is simply the notion that we can construct
a model of a system that we can then transform into the real
thing” [2]. One of the benefits of using MDD is the higher
abstraction level by providing platform independence. Other
related concepts are Model-Driven Architecture (MDA) [3],
the OMG perspective of MDD; and Model-Driven Engineering
(MDE) [4], which includes other software engineering activ-
ities in addition to development. In this paper, we will focus
on MDD approaches for software production.
Non-Functional Requirements (NFRs) are one of the main
research targets in the Requirements Engineering commu-
nity [5] and their industrial impact has been documented in
many empirical studies (e.g., [6], [7], [8], [9]). There are many
definitions for the concept of NFR [5]. For example, they
have been defined as: “(...) global requirements on [the sys-
tem] development or operational cost, performance, reliability,
maintainability, portability, robustness, and the like.” [10].
However there is no common agreement on the concrete
meaning of the term NFR [11].
Dealing with NFRs is a major challenge in software devel-
opment. The lack of integration of NFRs with functional re-
quirements can result in increased time-to-market and project
cost [5], [12]. This holds also for MDD. Being supported or
not by MDD approaches, practitioners have to deal with NFRs
in one way or another.
Ameller et al. [13] presented at RE’10 a vision paper
on the impact of NFRs in MDD processes and foresaw
different ways to handle them. The authors pointed out the
lack of support of NFRs in MDD (based on a literature
review) and hypothesized that this situation is probably even
more dramatic in the industrial practice. This topic was also
subject of a keynote1, given at the MoDRE@RE’14 workshop,
where some industrial attendees and academics with industrial
experience reinforced this hypothesis through their questions
and opinions. Moreover, recent empirical studies that report
the adoption of MDD in industry [14], [15], [16], [17] do not
provide insights on how practitioners deal with NFRs.
1www.slideshare.net/gessiupc/mo-dre-2014- keynote
978-1-4673-6905-3/15 c
2015 IEEE RE 2015, Ottawa, ON, Canada
RE:Next! Track
Accepted for publication by IEEE. c
2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/
republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
52
208
TABLE I
RESEARCH QUESTIONS
RQ1 In which context is MDD adopted by companies?
RQ1.1 What factors motivate the adoption of MDD?
RQ1.2 Which types of NFRs are linked to these motivating factors?
RQ1.3 To what extent are NFRs relevant for those projects that adopt MDD?
RQ2 To what extent do MDD approaches adopted by companies support NFRs?
RQ2.1 Which types of NFRs are supported by the adopted MDD approaches?
RQ2.2 Which characteristics do these NFRs exhibit?
RQ2.3 Which notations and tools are used for the supported types of NFRs?
RQ2.4 At which stages of the adopted MDD approach are these NFRs handled?
RQ3 How do companies deal with NFRs when the adopted MDD approach does not support them?
RQ3.1 How are MDD approaches customized to take into account the previously unsupported types of NFRs?
RQ3.2 How do companies deal with an NFR which is not supported by MDD?
RQ3.3 To what extent do the drawbacks of dealing with unsupported types of NFRs compensated by the benefits of adopting MDD?
Further investigating this hypothesis is the main motivation
of our work. In this paper, we present the ongoing NFR4MDD
project, a multinational (Europe-wide) empirical study on the
industrial practices in MDD, focusing on how companies
handle NFRs when using MDD. The study involves about
20 researchers from 8 countries and is expected to involve
around 30 companies whose experiences and perceptions will
be surveyed using semi-structured interviews.
The rest of the paper is divided into the following sections:
Section II, the objective and the research questions of this
work; Section III, the context in which this research is being
held; Section IV, the working plan and the research method-
ology; and finally Section V, the conclusions.
II. OBJECTIVE OF THE ST UDY
The main goal of the NFR4MDD project is to obtain a
detailed view on the way that companies handle NFRs when
they use MDD. Based on this goal, we define the research
questions (RQs) shown in Table I.
The first RQ (RQ1) shall provide an overview of the context
in which MDD approaches are used by European companies.
In particular, we are interested in identifying the factors (e.g.,
speed up the development process, improve the reusability,
better documentation) that motivate the adoption of MDD as
the most suitable option (RQ1.1). Specifically, we want to
search for links between these factors and specific types of
NFRs (RQ1.2). Since our working hypothesis is that most
MDD approaches do not handle any type of NFRs, with
RQ1.3 we want to know if MDD is adopted in projects where
the NFRs are less important than functional requirements, or
whether relevant NFRs are already satisfied by the technolo-
gies or the infrastructure used. While RQ1.1 shall allow us to
compare our findings with other surveys on MDD in practice
(see Section I), RQ1.2 and RQ1.3 aim at characterizing the
participating companies with regard the topic of study.
The second RQ (RQ2) is focused on understanding how
NFRs are supported by the MDD approaches currently adopted
by the participating companies. First, we want to know if the
MDD approaches adopted by the companies tend to support
any specific NFR (RQ2.1). For the supported NFRs, we want
to determine: a) their characteristics, for example, how near
or far the NFRs are from the solution, the granularity of their
specification, etc. (RQ2.2); b) how are they represented in
the models, for example, the languages or extensions used
(RQ2.3). Also, we want to map the supported NFRs in the
adopted MDD approaches, in particular into the stages at
which the NFRs are handled (RQ2.4). As part of our study,
we plan to compare the results of the survey to previously
published findings.
Even if some types of NFRs might be supported by the
MDD approaches adopted by companies, in most cases we
expect to find only limited support [13]. For this reason, we
aim with RQ3 at understanding the strategies used by the
companies when they have to deal with those types of NFRs
that are unsupported by the MDD approaches adopted in their
context. Ameller et al. [13] envisaged two possible strategies:
the first one is to adapt the MDD process so that it can account
for the unsupported types of NFRs (RQ3.1) and the second one
is to manually adapt the resulting artifact of the MDD process
in order to satisfy the unsupported NFRs (RQ3.2). However,
we expect that companies may well use other strategies, for
example, postpone the NFRs for future releases, or simply
discard the NFRs (RQ3.2). Finally, since all of these situations
can affect the development, we want to better understand their
impact from the point of view of the participating companies
(RQ3.3), i.e., we want to determine if bearing the drawbacks
due to the loose integration of NFRs into the MDD process is
compensated by the benefits provided by MDD approaches.
III. CON TE XT O F TH E STU DY
The NFR4MDD project was initiated by the Universitat
Polit`
ecnica de Catalunya (UPC) team after the keynote at
MoDRE@RE’14 mentioned in Section I. The collective ob-
servation that the challenge remains open and the interest
209
Fig. 1. Participant countries.
expressed by the community were the triggers for the study
described in this paper.
The collaborative nature of the survey was also decided
from the very beginning. Not just the difficulty of getting
enough subjects by only one research group was considered,
but in fact the main driver for this decision was to aim at
the best possible quality of the project and its outcome by
involving experienced researchers of the two main areas of
the study (NFRs and MDD). A subsequent decision was that
UPC would act as a research coordinator. The criteria that
UPC used to select participating researchers were:
•European research groups. We invited only one research
group per country, thus ensuring representation of coun-
tries is well balanced (see Fig. 1). Differences emerging
from the characteristics of a particular country may
eventually be investigated.
•The researchers should have demonstrated expertise in at
least one of the main areas of the study, MDD or NFRs.
•The researchers should have profound expertise in em-
pirical research methods, ideally in qualitative research.
In particular, membership to the ISERN network2was
welcome.
The invited researchers received a formal invitation letter3
composed of the following parts: general presentation of the
project, research questions, list of responsibilities, and the
initial work plan. In general, the response was quite positive,
especially considering that NFR4MDD is not a funded project.
With regards to the responsibilities, the coordinators of the
project will lead most of the tasks, while the participating
researchers will collaborate in the: review of the protocol,
search for interviewees in companies using MDD, execution
of interviews, feedback on data analysis, and collaboration in
reporting (e.g., writing papers).
A. Management Issues
Coordinators had to deal with several management issues.
For instance, to discuss the preliminary versions of documents
2isern.iese.de
3www.essi.upc.edu/∼gessi/papers/RE15-letter.pdf
we opted to use collaborative space that allows several re-
searchers working at the same time on the same document
without conflicts. For the consolidated versions of the docu-
ments and the research material (e.g., interview transcriptions,
data analysis) we use a shared repository with easy-to-use
storage capabilities. We also created a mailing list to be used
as the main communication channel of the project.
Not surprisingly for such a collaborative endeavor, several
circumstances started to arise from the very beginning, e.g.:
•Initially, it was planned to have two researchers per coun-
try, but some countries kindly required to have more than
two researchers, therefore it was decided to exceptionally
allow to exceed the original numbers.
•Even in a short space of time, some researchers started
to change affiliation and even country. This will not
be considered a problem as far as the interviews are
conducted in the originally planned country.
•We wanted to define as soon as possible the author order
in the artifacts produced in the project (in particular,
research papers). Authorship rules (already applied in
the paper at hand) are: first, coordinators; next, national
representatives; last, supporting researchers. Alphabetical
order is proposed in every category.
•Some of the research groups have expressed concerns
about the need to record and transcribe the interviews
(e.g., national privacy regulations). Here we opted to
go case-by-case in order to find the best solution that
minimizes the impact on the quality of the study.
•The language used in the interviews was also a matter
of discussion. The final decision was to use at every
interview the most appropriate language, with the only
restriction to provide the final transcription in English.
B. Open Discussions
More interestingly, we started some discussions about the
contents of our empirical study, e.g.:
•We are currently deciding whether we use a common
NFRs taxonomy or the taxonomy that better fits the
company domain in each case and we consolidate ter-
minology during data analysis. One reason is that we
expect a varying understanding of the notion of NFRs
for the various industrial sectors and application domains
covered by our study (“there is not any formal definition
or complete list of nonfunctional requirements” [10]).
•We are currently discussing the various context- and
domain-specific interpretations of MDD and we are con-
sidering to initiate the interviews by letting the partici-
pants define the notion of MDD for their context first.
•We initially proposed that each country representative
would select their participants, but as some concerns
about the diversity arose, we decided to make the selec-
tion at a global level, once all the countries have proposed
their own candidates.
210
IV. PLA NN IN G OF T HE ST UDY
Among the variety of types of empirical studies (e.g., sur-
veys, case studies, experiments), this study will be a qualitative
survey. Surveys (e.g., interviews, online questionnaires) are
a common way for collecting qualitative and quantitative
information [18]. They provide a snapshot of the current state
of the studied topic by collecting information to describe,
compare, or explain knowledge, attitudes and behavior [18].
Surveys aim at understanding a population from which a
sample will be drawn.
This survey will include descriptive and exploratory inten-
tions. Descriptive surveys are conducted to enable assertions
about some population. They are designed to measure what
occurred, rather than why. Exploratory surveys are used as a
pre-study to a more thorough investigation.
We followed Ciolkowski et al.’s guidelines [19] to define a
protocol for this study. In these guidelines the following steps
are defined (iterations are not represented):
1) Survey definition. Objective and research questions.
2) Survey design. Planning the key elements of the study,
including population, sample, variables, data collection
and analysis, and mitigation of threats to validity.
3) Survey implementation. Make the survey executable.
4) Survey execution. Data collection and processing.
5) Survey analysis. Analysis and interpretation.
6) Survey packaging. Report the results of the study.
A. Survey Steps
Our survey definition (Step 1) is stated in Section II. In this
section we focus on the survey design (Step 2) and we outline
the remaining steps to provide a full overview of the process
and the planned work.
Following the guidelines in [19], the survey design should
consider: the target population and the survey sample; the
variables of the survey; the approach for data collection; the
questionnaire design; the approaches for data analysis; and the
threats to validity.
Population. Our population is the software companies. We
do not restrict our population with regard to the company size
or application domain, but we require experience using MDD
in software projects. We deliberately avoided a more restrictive
population in order to facilitate the selection of candidates and
to avoid a narrow view on the topic of study. However, we re-
quire that the representative of each participating company has
the adequate background in MDD (i.e., s/he has participated,
at least, in one project that applied MDD in the company).
Sampling. The sampling method used in this study is non-
probability sampling. As explained in Section III, our selection
of candidates is made up by the national representatives
using their known industrial contacts. Taking into account the
geographical distribution of the participants, we consider that
our sampling method is a good approach to obtain the sample
of the target population. Since we do not have a sufficient
theoretical basis to make estimates on the target population,
we will limit our findings to the obtained sample rather than
inferences over the population.
Variables. Next we defined the list of variables linked to
the research questions and the questions in the questionnaire.
These variables form the basis of the survey analysis (Step
5). It is worth mentioning that even if we made an effort to
determine most of the variables that will appear in the survey,
some new ones may appear during the survey execution in Step
4 (e.g., some recurrent pattern during the content analysis).
Data collection. The data collection instrument, as already
mentioned, will be face-to-face interviews, which gives us
the possibility to get the participant answer our questions
without skipping any question (providing clarifications when
necessary). Also, it is possible for the interviewer to observe
and ask questions to the participants to extend their responses
(e.g., to understand the reasoning of the participant, or to go
deeper into the details).
Questionnaire design. We followed the writing recommen-
dations given by Dillman et al. [20] to reduce the common
mistakes when designing a questionnaire (e.g., keep positive
and negative sides in the questions; questions polarized to one
side can bias the responses). We also made several internal it-
erations to improve the questionnaire, and we planned internal
and real-case pilots before proceeding to the survey execution.
Data analysis. The obtained data will be analyzed using
basic descriptive analysis and content analysis. For the basic
descriptive analysis, we will use frequency analysis and com-
parison of different variables. The use of statistical correlation
analysis may not be feasible due to the expected sample
size. For the content analysis, we will identify categories
using coding techniques, and we will use qualitative analysis
techniques (which often work well with small samples) such
as summarizing, explaining, and structuring [21].
Threats to validity. We identified the most relevant threats of
internal, external, conclusion, and construct validity as defined
in the book by Wohlin et al. [22]. The next subsection details
the most relevant threats to the planned survey.
B. Threats to Validity
1) Internal Validity: Understandability problems. Some
questions may be understood differently of what they are
intended for resulting in lower quality answers, or even not
valid answers. As mitigation the questionnaire was designed
following the indications given by Dillman et al. [20] and will
be piloted in several iterations to ensure its understandability.
Language difficulties. The language used during the inter-
views will be chosen by the participant, therefore, in some
cases, the transcription of the interview will be translated
to English. During the translation, some information may be
lost or incorrectly translated, especially for technical terms.
As mitigation, the translation will be verified by the local
researchers who had participated in the interview and know
the technical terms of the research topic.
Insufficient knowledge of the participant. Having a wrong
profile participating in the survey may result in lower quality
answers, or the inability of the participant to provide an
answer to some questions. To mitigate this problem, the
selected participants should have experience using MDD; more
211
1 Dec 1 Jan 1 Feb 1 Mar 1 Apr 1 May 1 Jun 1 Jul 1 Aug 1 Sep 1 Oct 1 Nov
- Complete research team
- First version of the protocol
- Consolidate protocol
- Selection of participants
- Execution of pilots
- Conduct the interviews
- Validate gathered data
- Internal organization
- Project planning - Prepare data for analysis
- Data analysis
- Report of the results
- Define schedule for publications
- Start writing pape rs
RE 15
starts
Survey definition, design, and implementation
in several iterations
(Steps 1, 2, 3)
Survey execution
(Step 4)
Survey analysis
(Step 5)
Survey packaging (starts)
(Step 6)
2015
Fig. 2. Timeline of the work plan.
concretely, they should have participated as software engineers
in at least one MDD project that has successfully finished in
their current company.
Untruthful answers. Participants may be reluctant to provide
sincere responses when they have to explain negative aspects
about their work and their company. In order to minimize this
situation, when the survey is executed, the interviewers will
inform the participants that the data collected will be used
anonymously and that the purpose of the interview is not to
evaluate them or their company.
2) External Validity: As it is the usual case in interview-
based surveys, the sample size and the sampling technique
used do not provide the statistical basis to generalize the results
to the target population. The results should yield, however, an
initial theory which can be used to steer future research.
3) Conclusion Validity: Interviewers’ bias. Due to the
context of this study, the interviews will be executed by
several different interviewers, therefore there is a risk that they
conduct the interview in different ways (e.g., making more
emphasis on some parts than others). As countermeasure we
are creating documentation for the execution: a guide for the
execution with the specific steps that need to be performed
before, during and after the interviews, and a questionnaire
populated with specific instructions for the interviewers. In
addition, we will hold a plenary meeting before starting the
execution phase to explain and clarify all the aspects related
to the interviews execution.
Replicability of this study. Once the study is finalized, the
protocol, and all the related material used to perform this
study will be made available under CC-BY4license. The
open character of our project will support researchers and
practitioners to replicate this study, and, in the long run, to
generalize and verify the results.
4) Construct Validity: The underlying methodology is not
robust. The protocol may be incomplete, or provide insuf-
ficient details for the success of the project. To guarantee
4creativecommons.org/licenses/by/4.0
a solid foundation, we have followed Ciolkowski et al.’s
guidelines [19]. These guidelines cover the most important
aspects to be considered when performing a survey. As result,
we defined a detailed protocol5.
The questionnaire6used to drive the semi-structured inter-
view is not well linked to the research questions. Missing im-
portant data during the interviews may lead to the impossibility
to provide an answer to some of the research questions. To
verify the suitability of the questionnaire, we have related each
question in the questionnaire with one or more variables (to
be used for the analysis), and these variables with the research
questions (one variable may provide insightful information for
more than one research question). We are currently working on
a conceptual model that clarifies the relationships between the
variables (e.g., context, input and output factors of the MDD
process), which we will use as the foundation for our research
questions. The model will be available in the next iteration of
the protocol.
C. Work Plan
Due to the large number of researchers involved in this
study, it was deemed critical to put in place a well-defined
work plan since the very beginning. The work plan and its
timeline is depicted in Fig. 2. It shows the tasks that have
been and will be performed.
The current status (at the time of submitting this paper) is a
complete survey definition and design (Steps 1 and 2), which
may still be subject of minor modifications. For the survey
implementation (Step 3), our first version of the questionnaire
is almost ready but the pilots are to be executed. Steps 4 and
5 are not initiated yet. This paper can be seen as part of the
reporting activities of the survey (which belong to Step 6). As
part of the survey packaging we envision several publications
at conferences and in journals, including RE’16.
5www.essi.upc.edu/∼gessi/papers/RE15-protocol.pdf
6www.essi.upc.edu/∼gessi/papers/RE15-questionnaire.pdf
212
V. CONCLUSION
In this paper, we have reported the planning and current
status of an empirical study about the management of NFRs
in MDD processes in industry. We have explained the objective
and the research questions, the context, the methodology used,
and the planning of the tasks remaining. We have explained the
difficulties that we have encountered during the organization of
this project together with our solutions. We also have discussed
the threats to the validity of the survey and selected mitigation
actions for these threats.
The RE Next call for papers has provided us with a great
opportunity for the study. First, given the current preliminary
stage, we expect to get relevant feedback from the reviewers
for improving our protocol in a timely manner. Should the
paper be accepted, the presentation at the conference itself
would provide an excellent opportunity to share our first
impressions on the ongoing analysis and raise awareness of
the study. The writing of the paper itself has also represented
a kind of proof of concept for the full study in terms of
collaborative work. In this respect, this paper has served as
a motivating instrument to activate the participation of all
researchers involved. We also consider that this paper provides
a valuable experience in organizing such an Europe-wide kind
of study which could help other researchers organizing similar
collaborative projects.
Once this study is completed, our intention is to extend it
to worldwide scale. We are open for future iterations with
participants from all the continents. We think that the RE
conference will be a great venue to foster such participation.
ACKNOWLEDGMENT
Several projects and grants have supported our work:
Secretaria d’Universitats i Recerca de Catalunya; TIN2013-
44641-P; TIN2011-23795; FCT/MEC NOVA LINCS PEst
UID/CEC/04516/ 2013; the Christian Doppler Forschungsge-
sellschaft and the BMWFJ, Austria.
REFERENCES
[1] C. Atkinson and T. K ¨
uhne, “Model-Driven Development: a metamodel-
ing foundation,” IEEE Software, vol. 20, no. 5, pp. 36–41, Sep. 2003.
[2] S. J. Mellor, A. N. Clark, and T. Futagami, “Guest Editors’ Introduction:
Model-Driven Development,” IEEE Software, vol. 20, pp. 14–18, 2003.
[3] J. Miller and J. Mukerji, “MDA Guide Version 1.0.1,” Object Manage-
ment Group (OMG), Tech. Rep., 2003.
[4] D. C. Schmidt, “Guest Editor’s Introduction: Model-Driven Engineer-
ing,” Computer, vol. 39, no. 2, pp. 25–31, Feb. 2006.
[5] L. Chung and J. C. S. do Prado Leite, Conceptual Modeling: Foun-
dations and Applications. Springer, 2009, ch. On Non-Functional
Requirements in Software Engineering, pp. 363–379.
[6] D. Ameller, C. Ayala, J. Cabot, and X. Franch, “How do Software Ar-
chitects Consider Non-functional Requirements: An Exploratory Study,”
in 20th IEEE International Requirements Engineering Conference (RE),
2012, pp. 41–50.
[7] D. Ameller, M. Galster, P. Avgeriou, and X. Franch, “A survey on quality
attributes in service-based systems,” Software Quality Journal, 2015.
[8] R. Berntsson Svensson, T. Gorschek, B. Regnell, R. Torkar,
A. Shahrokni, and R. Feldt, “Quality Requirements in Industrial Practice:
An Extended Interview Study at Eleven Companies,” IEEE Trans. Softw.
Eng., vol. 38, no. 4, pp. 923–935, Jul. 2012.
[9] M. Daneva, S. Marczak, and A. Herrmann, “Engineering of Quality
Requirements As Perceived by Near-shore Development Centers’ Ar-
chitects in Eastern Europe: The Hole in the Whole,” in Proceedings
of the 8th ACM/IEEE International Symposium on Empirical Software
Engineering and Measurement, 2014.
[10] J. Mylopoulos, L. Chung, and B. A. Nixon, “Representing and Us-
ing Nonfunctional Requirements: A Process-Oriented Approach,” IEEE
Trans. Softw. Eng., vol. 18, no. 6, pp. 483–497, Jun. 1992.
[11] M. Glinz, “On Non-Functional Requirements,” in 15th IEEE Interna-
tional Requirements Engineering Conference (RE’07), 2007, pp. 21–26.
[12] L. Chung, B. A. Nixon, E. Yu, and J. Mylopoulos, Non-functional
requirements in software engineering. Kluwer Academic, 2000.
[13] D. Ameller, X. Franch, and J. Cabot, “Dealing with Non-Functional Re-
quirements in Model-Driven Development,” in 18th IEEE International
Requirements Engineering Conference (RE), 2010, pp. 189–198.
[14] J. Hutchinson, J. Whittle, and M. Rouncefield, “Model-Driven Engineer-
ing Practices in Industry: Social, Organizational and Managerial Factors
that lead to Success or Failure,” Science of Computer Programming,
vol. B, no. 89, pp. 144–161, September 2014.
[15] P. Mohagheghi, W. Gilani, A. Stefanescu, and M. Fernandez, “An em-
pirical study of the state of the practice and acceptance of model-driven
engineering in four industrial cases,” Empirical Software Engineering,
vol. 18, no. 1, pp. 89–116, 2013.
[16] L. T. W. Agner, I. W. Soares, P. C. Stadzisz, and J. M. Sim˜
ao, “A
Brazilian survey on UML and model-driven practices for embedded
software development,” Journal of Systems and Software, vol. 86, no. 4,
pp. 997–1005, 2013.
[17] M. Torchiano, F. Tomassetti, F. Ricca, A. Tiso, and G. Reggio, “Rele-
vance, benefits, and problems of software modelling and model driven
techniques: A survey in the Italian industry,” Journal of Systems and
Software, vol. 86, no. 8, pp. 2110–2126, 2013.
[18] C. Wohlin, M. H ¨
ost, and K. Henningsson, “Empirical Research Methods
in Software Engineering,” in Empirical Methods and Studies in Software
Engineering, ser. Lecture Notes in Computer Science, R. Conradi and
A. Wang, Eds. Springer Berlin Heidelberg, 2003, vol. 2765, pp. 7–23.
[19] M. Ciolkowski, O. Laitenberger, S. Vegas, and S. Biffl, “Practical
Experiences in the Design and Conduct of Surveys in Empirical Software
Engineering,” in Empirical Methods and Studies in Software Engineer-
ing, ser. Lecture Notes in Computer Science, R. Conradi and A. Wang,
Eds. Springer Berlin Heidelberg, 2003, vol. 2765, pp. 104–128.
[20] D. A. Dillman, J. D. Smyth, and L. M. Christian, Internet, Phone, Mail,
and Mixed-Mode Surveys: The Tailored Design Method, 4th ed. Wiley
Publishing, 2014.
[21] P. Mayring, Qualitative Content Analysis. Theoretical Foundation,
Basic Procedures and Software Solution. Social Science Open Access
Repository, 2014. [Online]. Available: http://nbn- resolving.de/urn:nbn:
de:0168-ssoar-395173
[22] C. Wohlin, P. Runeson, M. Hst, M. C. Ohlsson, B. Regnell, and
A. Wessl´
en, Experimentation in Software Engineering. Springer, 2012.
213
