PreprintPDF Available
Preprints and early-stage research may not have been peer reviewed yet.

Abstract

Requirements engineering (RE) is one of the most natural language-intensive fields within the software engineering area. Therefore, several works have been developed across the years to automate the analysis of natural language artifacts that are relevant for RE, including requirements documents, but also app reviews, privacy policies, and social media content related to software products. Furthermore, the recent diffusion of game-changing natural language processing (NLP) techniques and platforms has also boosted the interest of RE researchers. However, a reference framework to provide a holistic understanding of the field of NLP for RE is currently missing. Based on the results of a recent systematic mapping study, and stemming from a previous ICSE tutorial by one of the authors, this technical briefing gives an overview of NLP for RE tasks, available techniques, supporting tools and NLP technologies. It is oriented to both researchers and practitioners, and will gently guide the audience towards a clearer view of how NLP can empower RE, providing pointers to representative works and specialised tools. I. TECHNICAL BRIEFING PROPOSAL a) Topic: The topic of the proposed technical briefing is natural language processing (NLP) for requirements engineering (RE) [1], [2]. This is a specialised, yet highly multi-faceted research topic that has received attention since the '90s, with seminal contributions mostly focused on generating models from requirements [3], [4]. Later, NLP has been applied to automate the typical time-consuming RE tasks, such as defect detection [5], requirements tracing [6], and categorization [7]. With the emerging of novel RE-relevant formats and sources, such as user stories [8], but also app reviews [9] and privacy policies [10], the field has expanded its scope of application to tackle issues specifically related to these types of artifacts. In parallel, NLP, and artificial intelligence techniques in general, have seen a tremendous breakthrough in the last decade after the development of game-changing technologies generally classified under the umbrella of deep learning [11]. These two trends have created further synergies between NLP and RE, and a mapping study was performed by the proponents of this technical briefing to provide a holistic understanding of the field [2]. The study analyses 404 papers on the topic, and provides statistics on multiple relevant facets (e.g., documents used, techniques, NLP tools, resources), giving not only a report on the state-of-the-art, but also a conceptual reference framework to make sense of current research in the field, and its possible evolution. This technical briefing stems from the results of the mapping study, and aims to provide a blueprint of the landscape of NLP for RE, in terms of (1) tasks that can be automated (2) NLP-based techniques that are available to support the tasks (3) tools that support the techniques, and (4) established and novel NLP technologies that are relevant for RE and can further advance the field. In particular, part of the talk will present the concept of transfer learning, and the BERT language model [12]. Transfer learning is based on reusing pre-trained models to new and unseen problems, and this can be particularly useful in RE, where datasets to train models are particularly scarce [1]. The specific merit of BERT is its versatility to be used in different NLP downstream tasks such as text classification, question-answering and named-entity recognition, which are useful to several RE tasks as reported in the mapping study [2]. For example, in the case of requirements classification, using a pre-trained model such as BERT and fine-tuning it with some labelled requirements generates promising results, as shown by Hey et al. [13]. The topic of NLP for RE is particularly relevant and timely, given the recent NLP developments not yet absorbed by the RE community [2]. It is also the right time for technology transfer, as it has been observed that only 16% of companies use some form of automation in requirements analysis [14], and only 7% of the works retrieved in the systematic mapping study concern industrial applications, while a plethora of different solution proposals is available (67% of the studies) [2]. b) Format: The technical briefing consists of 90 minutes of presentation, and is divided into three main parts. The first part is dedicated to provide an overview of the different RE tasks that can be supported by NLP techniques, such as traceability, defect detection, requirements classification, etc. This is followed by an overview of the results of the mapping study, and representative contributions are presented to illustrate available techniques and tools developed to support each RE task. Particular relevance will be given to point to enabling technologies and platforms from the NLP field. The mapping study has identified a clear lack of adoption of cutting-edge NLP technologies in RE. Therefore, to trigger interest and open to further development, the final slot will focus on the presentation of one specific NLP technique that we consider of potential interest for both RE and software engineering (SE) practitioners, namely the BERT model [12].
NLP for Requirements Engineering:
Tasks, Techniques, Tools, and Technologies
Alessio Ferrari
CNR-ISTI
Pisa, Italy
Email: alessio.ferrari@isti.cnr.it
Liping Zhao
University of Manchester
Manchester, UK
Email: liping.zhao@manchester.ac.uk
Waad Alhoshan
Al-Imam Muhammad ibn Saud Islamic University
Riyadh, Saudi Arabia
Email: wmaboud@imamu.edu.sa
Abstract—Requirements engineering (RE) is one of the most
natural language-intensive fields within the software engineering
area. Therefore, several works have been developed across the
years to automate the analysis of natural language artifacts that
are relevant for RE, including requirements documents, but also
app reviews, privacy policies, and social media content related
to software products. Furthermore, the recent diffusion of game-
changing natural language processing (NLP) techniques and plat-
forms has also boosted the interest of RE researchers. However,
a reference framework to provide a holistic understanding of the
field of NLP for RE is currently missing. Based on the results of a
recent systematic mapping study, and stemming from a previous
ICSE tutorial by one of the authors, this technical briefing
gives an overview of NLP for RE tasks, available techniques,
supporting tools and NLP technologies. It is oriented to both
researchers and practitioners, and will gently guide the audience
towards a clearer view of how NLP can empower RE, providing
pointers to representative works and specialised tools.
I. TECHNICAL BRI EFI NG PRO PO SA L
a) Topic: The topic of the proposed technical briefing is
natural language processing (NLP) for requirements engineer-
ing (RE) [1], [2]. This is a specialised, yet highly multi-faceted
research topic that has received attention since the ’90s, with
seminal contributions mostly focused on generating models
from requirements [3], [4]. Later, NLP has been applied to
automate the typical time-consuming RE tasks, such as defect
detection [5], requirements tracing [6], and categorization [7].
With the emerging of novel RE-relevant formats and sources,
such as user stories [8], but also app reviews [9] and privacy
policies [10], the field has expanded its scope of application to
tackle issues specifically related to these types of artifacts. In
parallel, NLP, and artificial intelligence techniques in general,
have seen a tremendous breakthrough in the last decade after
the development of game-changing technologies generally
classified under the umbrella of deep learning [11]. These two
trends have created further synergies between NLP and RE,
and a mapping study was performed by the proponents of this
technical briefing to provide a holistic understanding of the
field [2]. The study analyses 404 papers on the topic, and
provides statistics on multiple relevant facets (e.g., documents
used, techniques, NLP tools, resources), giving not only a
report on the state-of-the-art, but also a conceptual reference
framework to make sense of current research in the field, and
its possible evolution.
This technical briefing stems from the results of the mapping
study, and aims to provide a blueprint of the landscape of
NLP for RE, in terms of (1) tasks that can be automated
(2) NLP-based techniques that are available to support the
tasks (3) tools that support the techniques, and (4) established
and novel NLP technologies that are relevant for RE and
can further advance the field. In particular, part of the talk
will present the concept of transfer learning, and the BERT
language model [12]. Transfer learning is based on reusing
pre-trained models to new and unseen problems, and this can
be particularly useful in RE, where datasets to train models
are particularly scarce [1]. The specific merit of BERT is
its versatility to be used in different NLP downstream tasks
such as text classification, question-answering and named-
entity recognition, which are useful to several RE tasks as
reported in the mapping study [2]. For example, in the case
of requirements classification, using a pre-trained model such
as BERT and fine-tuning it with some labelled requirements
generates promising results, as shown by Hey et al. [13].
The topic of NLP for RE is particularly relevant and timely,
given the recent NLP developments not yet absorbed by the RE
community [2]. It is also the right time for technology transfer,
as it has been observed that only 16% of companies use some
form of automation in requirements analysis [14], and only
7% of the works retrieved in the systematic mapping study
concern industrial applications, while a plethora of different
solution proposals is available (67% of the studies) [2].
b) Format: The technical briefing consists of 90 minutes
of presentation, and is divided into three main parts. The
first part is dedicated to provide an overview of the different
RE tasks that can be supported by NLP techniques, such
as traceability, defect detection, requirements classification,
etc. This is followed by an overview of the results of the
mapping study, and representative contributions are presented
to illustrate available techniques and tools developed to support
each RE task. Particular relevance will be given to point
to enabling technologies and platforms from the NLP field.
The mapping study has identified a clear lack of adoption of
cutting-edge NLP technologies in RE. Therefore, to trigger
interest and open to further development, the final slot will
focus on the presentation of one specific NLP technique that
we consider of potential interest for both RE and software
engineering (SE) practitioners, namely the BERT model [12].
c) Interest for the Community: The first author provided
a technical briefing at ICSE’18 on the topic [15], and the
presentation received high interest from the audience—20
to 30 participants were present. While the previous briefing
was mostly focused on consolidated NLP technologies that
could be used in daily RE practice, this new one has the
objective of giving a holistic view of the field, also based on
empirical data from the mapping study, and considering recent
developments. This can be of interest to researchers willing to
address current gaps, but also to practitioners who want to
know where the field is now, what are available tools, and
who are the reference experts to involve in research-industry
collaborations. Furthermore, we believe that the final slot,
dedicated to the presentation of BERT, can raise the interest of
the whole SE community, as NLP has seen several applications
also in software testing (see, e.g., the recent mapping study
from Garousi et al. [16]), and maintenance [17], [18]. Overall,
our goal is also to raise awareness about current research on
NLP for RE within the SE community, suggest NLP-centered
synergies, and pave the way for more integration between
different SE perspectives.
II. BIOGRAPHIES
Alessio Ferrari is research scientist at CNR-ISTI (Con-
siglio Nazionale delle Ricerche - Istituto di Scienza e Tec-
nologia dell’Informazione was “A. Faedo”, Pisa, Italy -
http://www.isti.cnr.it), where he works since 2011. He received
his Ph. D. in Computer Engineering from the University of
Florence, Italy, in 2011. His current research interests are
applications of NLP techniques to RE, requirements elicita-
tion, and RE teaching. In particular, his main focus is natural
language ambiguity detection and mistakes identification in re-
quirements elicitation interviews and requirements documents.
Ferrari participated in several European Projects, including
Learn PAd, ASTRail, and DESIRA. He is author of more than
70 papers in relevant conferences (RE, ICSE) and journals
(REJ, EMSE, IEEE Software). He served in the PC of ICSE,
IEEE RE, REFSQ, he has been co-organiser of two editions of
the NLP4RE workshop, and local organiser of REFSQ 2020.
Liping Zhao is Associate Professor in Department of
Computer Science, the University of Manchester. Her current
research focuses on using NLP and machine learning to sup-
port RE. From 2004 to 2014 she had been in collaboration with
IBM on Pattern Language for the Design and Development E-
business Applications and received three IBM Faculty Awards
(2004, 2005, and 2008) for her contributions. From 2007 to
2012, she co-founded and led a multidisciplinary academic
network in the UK on service science (SSMEnetUK), funded
by the UK Research Council, BT, HP, and IBM. She is
an Associate Editor for Requirements Engineering (Springer)
and Expert Systems (Wiley). She has served on numerous
conferences and workshops, and has been co-organiser of
IEEE International Workshops on Requirements Patterns, co-
located with RE Conference (from 2012 to 2016) and Inter-
national Workshop on Advances and Applications of Problem
Orientation (IWAAPO), co-located with ICSE (2010).
Waad Alhoshan is an Assistant Professor in the Department
of Computer Science, IMSIU. She received her PhD degree
in Computer Science in 2020 from the University of Manch-
ester, where she studied corpus-based and language modeling
techniques to investigate approaches for detecting semantic
relationships between software requirements. During her PhD,
Waad published several papers in peer-reviewed conferences
such as LREC, RE and ESEM. Currently, she is cooperating
on multiple research projects on designing NLP-based systems
to support software in Arabic and English languages. One of
these projects is a collaboration project between IMSIU and
the Saudi Authority for Intellectual Property (SAIP) to design
AI-driven systems for processing legal documents.
REFERENCES
[1] A. Ferrari, F. Dell’Orletta, A. Esuli, V. Gervasi, and S. Gnesi, “Natural
language requirements processing: a 4D vision,” IEEE Software, vol. 34,
no. 6, pp. 28–35, 2017.
[2] L. Zhao, W. Alhoshan, A. Ferrari, K. J. Letsholo, M. A.
Ajagbe, E.-V. Chioasca, and R. T. Batista-Navarro, “Natural
Language Processing (NLP) for Requirements Engineering: A
Systematic Mapping Study,” 2020 (Under Review). [Online]. Available:
https://arxiv.org/abs/2004.01099v2
[3] C. Rolland and C. Proix, “A natural language approach for requirements
engineering,” in International Conference on Advanced Information
Systems Engineering. Springer, 1992, pp. 257–277.
[4] L. Mich, “NL-OOPS: from natural language to object oriented require-
ments using the natural language processing system LOLITA,Natural
language engineering, vol. 2, no. 2, pp. 161–187, 1996.
[5] H. Femmer, D. M. Fern´
andez, S. Wagner, and S. Eder, “Rapid quality
assurance with requirements smells,” JSS, vol. 123, pp. 190–213, 2017.
[6] J. Guo, J. Cheng, and J. Cleland-Huang, “Semantically enhanced soft-
ware traceability using deep learning techniques,” in ICSE’17. IEEE,
2017, pp. 3–14.
[7] M. Binkhonain and L. Zhao, “A review of machine learning algorithms
for identification and classification of non-functional requirements,”
Expert Systems with Applications: X, vol. 1, p. 100001, 2019.
[8] M. Robeer, G. Lucassen, J. M. E. van der Werf, F. Dalpiaz, and
S. Brinkkemper, “Automated extraction of conceptual models from user
stories via nlp,” in RE’16. IEEE, 2016, pp. 196–205.
[9] W. Martin, F. Sarro, Y. Jia, Y. Zhang, and M. Harman, “A survey of
app store analysis for software engineering,” TSE, vol. 43, no. 9, pp.
817–847, 2016.
[10] D. Torre, S. Abualhaija, M. Sabetzadeh, L. Briand, K. Baetens, P. Goes,
and S. Forastier, “An ai-assisted approach for checking the completeness
of privacy policies against gdpr,” in RE’20. IEEE, 2020, pp. 136–146.
[11] T. Young, D. Hazarika, S. Poria, and E. Cambria, “Recent trends in
deep learning based natural language processing,” IEEE Computational
intelligence magazine, vol. 13, no. 3, pp. 55–75, 2018.
[12] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training
of deep bidirectional transformers for language understanding,” arXiv
preprint arXiv:1810.04805, 2018.
[13] T. Hey, J. Keim, A. Koziolek, and W. F. Tichy, “NoRBERT: Transfer
learning for requirements classification,” in RE’20. IEEE, pp. 169–179.
[14] D. M. Fern´
andez, S. Wagner, M. Kalinowski, M. Felderer, P. Mafra,
A. Vetr`
o, T. Conte, M.-T. Christiansson, D. Greer, C. Lassenius et al.,
“Naming the pain in requirements engineering,” EMSE, vol. 22, no. 5,
pp. 2298–2338, 2017.
[15] A. Ferrari, “Natural language requirements processing: from research to
practice,” in ICSE’18. IEEE, 2018, pp. 536–537.
[16] V. Garousi, S. Bauer, and M. Felderer, “Nlp-assisted software testing:
A systematic mapping of the literature,” IST, p. 106321, 2020.
[17] X. Hu, G. Li, X. Xia, D. Lo, and Z. Jin, “Deep code comment generation
with hybrid lexical and syntactical information,” EMSE, vol. 25, no. 3,
pp. 2179–2217, 2020.
[18] S. Gupta and S. Gupta, “Natural language processing in mining un-
structured data from software repositories: a review,” S¯
adhan¯
a, vol. 44,
no. 12, p. 244, 2019.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Reference to the paper: https://arxiv.org/abs/2004.01099 Context: NLP4RE-Natural language processing (NLP) supported requirements engineering (RE)-is an area of research and development that seeks to apply NLP techniques, tools and resources to a variety of requirements documents or artifacts to support a range of linguistic analysis tasks performed at various RE phases. Such tasks include detecting language issues, identifying key domain concepts and establishing traceability links between requirements. Objective: This article surveys the landscape of NLP4RE research to understand the state of the art and identify open problems. Method: The systematic mapping study approach is used to conduct this survey, which identified 404 relevant primary studies and reviewed them according to five research questions, cutting across five aspects of NLP4RE research, concerning the state of the literature, the state of empirical research, the research focus, the state of the practice, and the NLP technologies used. Results: (i) NLP4RE is an active and thriving research area in RE that has amassed a large number of publications and attracted widespread attention from diverse communities; (ii) most NLP4RE studies are solution proposals having only been evaluated using a laboratory experiment or an example application; (iii) most NLP4RE studies have focused on the analysis phase, with detection as their central linguistic analysis task and requirements specification as their commonly processed document type; (iv) 130 new tools have been proposed by the selected studies to support a range of linguistic analysis tasks, but there is little evidence of adoption in the long term, although some industrial applications have been published; (v) 140 NLP techniques (e.g., POS tagging and tokenization), 66 NLP tools (e.g., Stanford CoreNLP and GATE) and 25 NLP resources (WordNet and British National Corpus) are extracted from the selected studies, but most of them-particularly those novel NLP techniques and specialized tools-are not in frequent use; by contrast, frequently used NLP technologies are syntactic analysis techniques, general-purpose tools and generic language lexicons. Conclusion: There is a huge discrepancy between the state of the art and the state of the practice in current NLP4RE research, indicated by insufficient industrial validation of NLP4RE research, little evidence of industrial adoption of the proposed tools, the lack of shared RE-specific language resources, and the lack of NLP expertise in NLP4RE research to advise on the choice of NLP technologies. 2 CCS Concepts: • Software and its engineering➝ Software creation and management➝ Designing software➝ Requirements analysis KEYWORDS Requirements engineering, software engineering, natural language processing, NLP, systematic mapping study, systematic review
Article
Full-text available
Context: To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been proposed in the literature. Given the large amount of approaches in this area, and since many practitioners are eager to utilize such techniques, it is important to synthesize and provide an overview of the state-of-the-art in this area. Objective: Our objective is to summarize the state-of-the-art in NLP-assisted software testing which could benefit practitioners to potentially utilize those NLP-based techniques. Moreover, this can benefit researchers in providing an overview of the research landscape. Method To address the above need, we conducted a survey in the form of a systematic literature mapping (classification). After compiling an initial pool of 95 papers, we conducted a systematic voting, and our final pool included 67 technical papers. Results This review paper provides an overview of the contribution types presented in the papers, types of NLP approaches used to assist software testing, types of required input requirements, and a review of tool support in this area. Some key results we have detected are: (1) only four of the 38 tools (11%) presented in the papers are available for download; (2) a larger ratio of the papers (30 of 67) provided a shallow exposure to the NLP aspects (almost no details). Conclusion: This paper would benefit both practitioners and researchers by serving as an “index” to the body of knowledge in this area. The results could help practitioners utilizing the existing NLP-based techniques; this in turn reduces the cost of test-case design and decreases the amount of human resources spent on test activities. After sharing this review with some of our industrial collaborators, initial insights show that this review can indeed be useful and beneficial to practitioners.
Article
Full-text available
During software maintenance, developers spend a lot of time understanding the source code. Existing studies show that code comments help developers comprehend programs and reduce additional time spent on reading and navigating source code. Unfortunately, these comments are often mismatched, missing or outdated in software projects. Developers have to infer the functionality from the source code. This paper proposes a new approach named Hybrid-DeepCom to automatically generate code comments for the functional units of Java language, namely, Java methods. The generated comments aim to help developers understand the functionality of Java methods. Hybrid-DeepCom applies Natural Language Processing (NLP) techniques to learn from a large code corpus and generates comments from learned features. It formulates the comment generation task as the machine translation problem. Hybrid-DeepCom exploits a deep neural network that combines the lexical and structure information of Java methods for better comments generation. We conduct experiments on a large-scale Java corpus built from 9,714 open source projects on GitHub. We evaluate the experimental results on both machine translation metrics and information retrieval metrics. Experimental results demonstrate that our method Hybrid-DeepCom outperforms the state-of-the-art by a substantial margin. In addition, we evaluate the influence of out-of-vocabulary tokens on comment generation. The results show that reducing the out-of-vocabulary tokens improves the accuracy effectively.
Conference Paper
Full-text available
Automated manipulation of natural language requirements, for classification, tracing, defect detection, information extraction, and other tasks, has been pursued by requirements engineering (RE) researchers for more than two decades. Recent technological advancements in natural language processing (NLP) have made it possible to apply this research more widely within industrial settings. This technical briefing targets researchers and practitioners, and aims to give an overview of what NLP can do today for RE problems, and what could do if specific research challenges, also emerging from practical experiences, are addressed. The talk will: survey current research on applications of NLP to RE problems; present representative industrially-ready techniques, with a focus on defect detection and information extraction problems; present enabling technologies in NLP that can play a role in RE research, including distributional semantics representations; discuss criteria for evaluation of NLP techniques in the RE context; outline the main challenges for a systematic application of the techniques in industry. The crosscutting topics that will permeate the talk are the need for domain adaptation, and the essential role of the human-in-the-loop.
Article
Full-text available
In most safety-critical domains the need for traceability is prescribed by certifying bodies. Trace links are generally created among requirements, design, source code, test cases and other artifacts, however, creating such links manually is time consuming and error prone. Automated solutions use information retrieval and machine learning techniques to generate trace links, however, current techniques fail to understand semantics of the software artifacts or to integrate domain knowledge into the tracing process and therefore tend to deliver imprecise and inaccurate results. In this paper, we present a solution that uses deep learning to incorporate requirements artifact semantics and domain knowledge into the tracing solution. We propose a tracing network architecture that utilizes Word Embedding and Recurrent Neural Network (RNN) models to generate trace links. Word embedding learns word vectors that represent knowledge of the domain corpus and RNN uses these word vectors to learn the sentence semantics of requirements artifacts. We trained 360 different configurations of the tracing network using existing trace links in the Positive Train Control domain and identified the Bidirectional Gated Recurrent Unit (BI-GRU) as the best model for the tracing task. BI-GRU significantly out-performed state-of-the-art tracing methods including the Vector Space Model and Latent Semantic Indexing.
Article
Full-text available
Natural language processing (NLP) and requirements engineering (RE) have had a long relationship, yet their combined use isn’t well established in industrial practice. This situation should soon change. The future evolution of the application of NLP technologies in RE can be viewed from four dimensions: discipline, dynamism, domain knowledge, and datasets.
Article
Full-text available
Deep learning methods employ multiple processing layers to learn hierarchical representations of data, and have produced state-of-the-art results in many domains. Recently, a variety of model designs and methods have blossomed in the context of natural language processing (NLP). In this paper, we review significant deep learning related models and methods that have been employed for numerous NLP tasks and provide a walk-through of their evolution. We also summarize, compare and contrast the various models and put forward a detailed understanding of the past, present and future of deep learning in NLP.
Conference Paper
Classifying requirements is crucial for automatically handling natural language requirements. The performance of existing automatic classification approaches diminishes when applied to unseen projects because requirements usually vary in wording and style. The main problem is poor generalization. We propose NoRBERT that fine-tunes BERT, a language model that has proven useful for transfer learning. We apply our approach to different tasks in the domain of requirements classification. We achieve similar or better results F 1 -scores of up to 94%) on both seen and unseen projects for classifying functional and non-functional requirements on the PROMISE NFR dataset. NoRBERT outperforms recent approaches at classifying non-functional requirements subclasses. The most frequent classes are classified with an average F 1 -score of 87%. In an unseen project setup on a relabeled PROMISE NFR dataset, our approach achieves an improvement of 15 percentage points in average F 1 score compared to recent approaches. Additionally, we propose to classify functional requirements according to the included concerns, i.e., function, data, and behavior. We labeled the functional requirements in the PROMISE NFR dataset and applied our approach. NoRBERT achieves an F 1 -score of up to 92%. Overall, NoRBERT improves requirements classification and can be applied to unseen projects with convincing results.