Content uploaded by Itzel Morales-Ramirez
Author content
All content in this area was uploaded by Itzel Morales-Ramirez on Jul 17, 2020
Content may be subject to copyright.
Providing Foundation for User Feedback Concepts by
Extending a Communication Ontology
Itzel Morales-Ramirez1,2, Anna Perini1, and Renata Guizzardi3
1Software Engineering Research Unit. Fondazione Bruno Kessler - IRST
imramirez,perini@fbk.eu
2International Doctoral School ICT- University of Trento, Italy
3Ontology and Conceptual Modeling Research Group, UFES, Brazil
rguizzardi@inf.ufes.br
Abstract. The term user feedback is becoming widely used in requirements en-
gineering (RE) research to refer to the comments and evaluations that users ex-
press upon having experienced the use of a software application or service. This
explicit feedback takes place in virtual spaces (e.g., issue tracking systems, app
stores), aiming, for instance, at reporting on discovered bugs or requesting new
features. Founding the notion of explicit user feedback with the use of an ontology
may support a deep understanding of the feedback nature, as well as contribute
to the development of tool-components for its analysis at use of requirements
analysts. In this paper, we present a user feedback ontology as an extension of
an existing communication ontology. We describe how we built it, along with a
set of competency questions, and illustrate its applicability on an example taken
from a collaborative communication related to RE for software evolution.
Keywords: User Feedback, Communication Ontology, Requirements Engineer-
ing
1 Introduction
More and more, software users make use of social media to express their comments
about a software service or to rate applications they often use. This information, which
is easily accessible via the Internet, is considered an invaluable asset for software de-
velopers and service providers, to help them understand how to improve their software
application or service, and get inspiration for new functionalities and products [1, 2].
This information deliberately provided by the users is generally called explicit user
feedback and can be used as a complement to implicit user feedback, i.e. information
collected by means of observing the user behaviour through logs.
In [3] it is stated, “Feedback is one of the primary results of introducing the im-
plemented software system into the real world. There is an immediate response to the
system from those affected by it”. This definition leads us to recognise user feedback
both as an artefact and as a process. Taking the perspective of user feedback as an arte-
fact, we revise the previous definition and propose the following one: “User feedback
is a reaction of the user upon her experience in using a software service or application.
Explicit user feedback could be based on multi-modal communication, such as natural
language text, images, emoticons, etc.”. Taking the perspective of the communication
2 Itzel Morales-Ramirez, Anna Perini, and Renata Guizzardi
process, we claim that the roles of sender and receiver are essential to make clear the
purpose of explicit feedback.
In our research, we focus on explicit user feedback, with the ultimate goal of defin-
ing methods and techniques to support software system maintenance, as well as col-
laborative RE tasks as described in a recent paper [4]. We consider issues related to
collecting and analysing user feedback. Indeed, different techniques may be needed for
collection and for analysis, depending on the feedback type, and on the volume of the
corresponding data. On one hand, structured feedback is collected according to a pre-
defined input template. Consequently, in this case the analysis is driven by the schema
underlying the template itself. On the other hand, unstructured user feedback is col-
lected freely, without the aid of any predefined structure. Thus, the analysis in this case
may require data mining and natural language processing (NLP) techniques to discover,
retrieve, and extract information and opinions from huge textual information [5].
In this paper we present an ontology of user feedback, key for a deep understanding
of the explicit user feedback and its further exploitation. To build the user feedback
ontology we adopt a goal-oriented methodology that guide us in specifying the main
stakeholders, including the users of our ontology (e.g. designers of feedback collector
tools and analysts), and a set of competency questions (CQs) that the resulting ontology
will answer [6]. For example, the following CQs emerged from the analysis of the goals
of feedback analysts: (a) what are the types of user feedback presentation formats?;
(b) how can user feedback be classified?; (c) and what are the speech acts commonly
expressed by users in their explicit feedback?
Since feedback collection is seen as a communication process, we develop our on-
tology as an extension of a well-founded existing communication ontology [7]. Besides
this, we consider users’ feedback expressed in natural language (NL), therefore we take
into account the speech act theory (SAT) of Searle [8, 9].
The rest of the paper is organised as follows. Section 2 presents the concepts we
borrow from pre-existing ontologies and SAT theory. Section 3 describes the user feed-
back ontology and the concepts that it involves. Related work is recalled in Section 4.
Finally we draw some conclusions and point out future work in Section 5.
2 Baseline
Collecting user feedback basically consists in a process of communication between the
developers and the users of existing software systems. We build on an existing Commu-
nication ontology [7] and extend it including user feedback concepts. This Communica-
tion ontology is especially attractive because it is grounded on a foundational ontology,
namely the Unified Foundational Ontology (UFO) [10], which has been successfully
applied to provide real-world semantics for ontologies in different fields. In the follow-
ing we briefly recall the concepts that we reuse from UFO and from the Communication
ontology.
UFO distinguishes between endurants and perdurants. Endurants do not have tem-
poral parts, and persist in time while keeping their identity (e.g. a person and the colour
of an apple). A Perdurant (also referred to as event), conversely, is composed of tem-
poral parts (e.g. storm, heart attack, trip). Substantials are existentially independent
endurants (e.g. a person or a car). Moments, in contrast, are endurants that are ex-
Title Suppressed Due to Excessive Length 3
istentially dependent on other endurants or events, inhering in these individuals (e.g.
someone’s headache and the cost of a trip). Moments can be intrinsic or relational. In-
trinsic Moments are those that depend on one single individual in which they inhere.
ARelator is a relational moment, i.e. a moment that inheres simultaneously in multi-
ple individuals. Agents are substantials that can perceive events, perform action con-
tributions and bear special kinds of intrinsic moments, named Intentional Moments
(examples of agents are person, student and software developer). Action contributions
are intentional participations of agents within an event (e.g. saying something to some-
one, writing a letter). An Intention is a type of intentional moment (other examples
of intentional moments are belief and desire) that represents an internal commitment
of the agent to act towards that goal and, therefore, causes the agent to perform action
contributions.
The communication ontology considers sender and receiver as the central agents in
the communication process. The Sender is an agent that sends a message through a
communicative act. A Communicative Act is an action contribution that carries out
the information exchanged in the communication process. This communicative act cor-
responds to what Searle names illocutionary act [8]. The exchanged information is here
captured as a Message, which is the propositional content of the communicative act.
AReceiver is an agent that perceives the communicated message. As the communica-
tive act, a Perception is an action contribution that consists in the reception of the
exchanged information (thus, also having a message as propositional content). A Com-
municative Interaction is a complex action composed of exactly one communicative
act and one or more perceptions. In other words, in a communicative interaction, there
is one sender agent and at least one receiver agent. Moreover, the communicative act
and the perceptions involved in a communicative interaction have the same message as
propositional content. Thus, this message is also said to be the propositional content of
the communicative interaction.
In addition, due to the gathering of feedback is seen as a communication process,
we extend the communication ontology with concepts coming from the literature about
the characterisation, elaboration and use of user feedback. Specifically, we look at the
feedback expressed in NL, hence, we rely on the linguistic theory of SAT [8, 9].
3 User Feedback Ontology
In this section we explain the concepts of our ontology. Let’s start with the concepts
of the baseline, which are illustrated with a dark grey colour in Fig. 1. We have dis-
tinguished three key concepts of the communication ontology to be extended. These
concepts are intention, message and communicative act.
We extend the concept of Intention into communicative intention and reflexive in-
tention. The Communicative Intention refers to the internal commitment of a sender
of conveying an information to a receiver or an audience, regardless of having this in-
formation understood. While a Reflexive Intention, according to H.P. Grice, as quoted
in [9], refers to the sender’s intention that is formulated and transmitted with the purpose
of being recognised or understood by a receiver.
A message bears a given Topic, in UFO a topic is an intrinsic moment (i.e. a prop-
erty of the message), that becomes the subject of conversation between agents. The last
4 Itzel Morales-Ramirez, Anna Perini, and Renata Guizzardi
Perception
Message
ReceiverSender
Agent
Communicative
Act
1..*
*
triggered by >
1
*
*
*
1
< propositional
content of
Proposition
Action
Contribution
Intention
(internal
commitment)
Speech
act
*< performance of
1
*
< performance of
11
propositional
content of >
involves >
Intentional
Moment
Belief Desire
inheres in >
Reflexive
Intention
1
3
< performance
of
Communicative
Intention
Topic
< inheres in
Fig. 1. Concepts extending the communication ontology. Baseline concepts in dark grey, new
concepts extending the ontology in white.
concept that is added and central in our ontology is the concept Speech Act that is
per-se an action, i.e. an action contribution, and is the basic unit in a linguistic commu-
nication.
Communication ontology and SAT concepts. A speech act involves three commu-
nicative acts, namely, locutionary, illocutionary and perlocutionary act that we consider
specialisations of a communicative act. These acts are visualised in Fig. 2, in which we
refine the relation between speech act and communicative act and connect the speech
act directly to each one of the acts. A Locutionary Act is the act of “saying something”
(production of words), an Illocutionary Act makes reference to the way in which the
locutions are used and in which sense (intention to motivate the production of words),
and a Perlocutionary Act is the effect the sender wants to accomplish on the receiver or
audience. Let’s consider the utterance “Is there any example code I could look at?”, the
locutionary act corresponds to the utterance of this sentence, the illocutionary act cor-
responds to the speaker’s intention to make the audience aware that she has a request,
and the effect, i.e. the perlocutionary act, is that the speaker got the audience to handle
her request. A speech act involves at least one act, what we mean is that someone could
utter a senseless phrase (e.g.,“one the snow”), accomplishing the locutionary act, but
the illocutionary and perlocutionary acts are not present. The Performative verb refers
to the verb that classifies the illocutionary act into five categories that were introduced
by Searle, and later revised by Bach and Harnish [9].
An intention inheres in the sender, in this case the reflexive intention that causes
the illocutionary act. Then, this illocutionary act triggers the execution of a locutionary
act through the utterance of specific words that will reify such a reflexive intention. The
consequence of the illocutionary act is the sender’s perlocutionary act that the receiver
will perceive as the overall effect of a speech act. We need to clarify that the relation
consequence of between the illocutionary and perlocutionary act is a type of indirect
causation, i.e. if the communication is successful, the receiver will perform the action
intended by the sender. Finally, a speech act is successful if the intention that the sender
expresses is identified by the receiver by means of recognising such a sender’s intention.
Title Suppressed Due to Excessive Length 5
Perception
Message
Receiver
Sender Communicative
Act
1..*
*triggered by >
1
*
*
*
1
< propositional
content of
Intention
(internal
commitment)
Speech
act
*
< performance of
1
1
propositional
content of >
Reflexive
Intention
performance
of >
Communicative
Intention
Perlocutionary
act
(Effects)
Locutionary
act
(Production)
Illocutionary
act
(Intention)
1
< triggers
1..*
consequence
of >1
11
< effect on
1
1
causes >
1
Performative
verb
1
is classified
by >
1
< inheres in
involves >
1
1involves >
1
1involves >
1
1
Intention
1
interpreted by analyst >
1
ConstativeDirective Expressive
... ... ...
Feeback analysis
Clarification
Request
Bug
Report
Provide
Rating
New
Feature
Request Requirements Questions
Fig. 2. Communicative acts involved in each speech act.
SAT concepts. Our user feedback ontology builds on a revised taxonomy of speech
acts proposed in a previous work [11], which considers speech acts commonly used in
online discussions of an open source project. We grouped the selected speech acts in
three main categories: constantive, directive, and expressive. We are currently consol-
idating the part of the ontology that considers concepts that are specific to feedback
collection and analysis. An excerpt may be found in [6]. For reasons of space we only
show the connection of the reflexive intention to the intention interpreted by a software
requirements analyst. Once the analysis of the speech acts is performed, different re-
sults can be presented. For example, the categorisation of the feedback according to the
user’s reflexive intention. As can be found in the recent literature on feedback from soft-
ware users, possible intentions for a user to send a feedback are Clarification Request,
New Feature Request,Bug Report and Provide Rating. For instance, Bug Report
is inspired on corrective or negative feedback [12, 13] as in this case, the user feed-
back refers to information that should be used to correct the software, which means that
this information has a negative connotation. The encouraging [14] and positive [13]
feedback have been turned into Rating (e.g, stars in the AppStore) [2]. Strategic be-
haviour [14] in our ontology refers to a New Feature Request while Clarification
means that the feedback contains questions or extra information (such as critical de-
tails), to make something clearer. These terms are indeed used in different works and
our feedback ontology attempts at unifying all the different concepts in a single classi-
fication.
3.1 Illustrative Example
The following example, see Fig. 3, illustrates how the ontology may support the anal-
ysis of user feedback. We take an excerpt of an e-mail4sent by a user of a software
application called XWiki5. Note that the ontological concepts are highlighted with a
4Source: http://lists.xwiki.org/pipermail/users/attachments/20080220/748039cc/attachment.html
5http://www.xwiki.org/xwiki/bin/view/Main/WebHome
6 Itzel Morales-Ramirez, Anna Perini, and Renata Guizzardi
different font type to facilitate the understanding. This e-mail represents an instance of
unstructured user feedback. The fields Subject: and From: are the concepts Topic and
Sender, respectively, of our ontology. In the body of the e-mail we can distinguish dif-
ferent types of speech acts. In this example we find that the first speech act expresses
the user’s reflexive intention of making questions to be answered. After some other
messages, we find the intention suppose, referring to the speech acts suppositives. The
followed two speech acts requirement and question are also expressed.
Subject: Latex html in editor for math formulas?
| {z }
T opic
From: alan
|{z}
Sender
Hi folks,
(1) Will there be an additional button in the editor, to implement formulas?
| {z }
Question
...
Or how can I use formulas, maybe you know a good latex html live editor.
| {z }
Suppositive
...
it would be nice to have something,
| {z }
Requirement
if there already isnt?
| {z }
Question
...
alan
Fig. 3. Example of unstructured user feedback in the XWiki mailing list.
Now let’s see a detailed explanation of the analysis performed. Taking the previous
example we first see the Message (1) is the propositional content of aCommunica-
tive Act. As highlighted before, this message bears the Topic “Latex html in editor for
math formulas?” and the Sender is alan. The Reflexive Intention –Make a question–
inheres in alan that causes the Illocutionary act Quest. This act produces an effect, i.e.
Perlocutionary act that is a consequence of the Illocutionary act, which together with
the Locutionary act Elaborate a question are the three acts involved in the Speech
Act –Question. The ontology supports the understanding of such an intention expressed
by a sender that must be recognised by a receiver. In this example the understanding
that the sender is expressing a question is that of producing the Perlocutionary act –
Answer –that will have the effect on the receiver, i.e. XWiki community, who through the
Perception –Reading–, which is triggered by the communicative act, will eventually
answer the posed question. However, the other speech acts (i.e. Suppositives, Require-
ments, and Questions) may provide to the analyst indicators for identifying a Feature
Request in this feedback.
3.2 Discussion
At this stage of development the ontology is intended to clarify concepts useful to un-
derstand the nature of explicit feedback elaborated by software’s users and to support
requirements analysts when performing feedback analysis. Due to space limits, we only
discuss one of the proposed CQs, namely (c) what are the speech acts commonly ex-
pressed by users in their explicit feedback? This CQ is answered by querying the part
Title Suppressed Due to Excessive Length 7
of the ontology where the concept speech act is specialised into the different sub-kinds
(see Fig. 2, bottom-centre). Answers to this question are taken into account in tool-
supported analysis technique for explicit unstructured user feedback, which exploits
NLP tools. This tool supports the classification of phrases as instances of concepts rep-
resenting speech acts that commonly appear in user feedback [11]. The ontology allows
us to identify the relation between the types of speech acts (or their combination) to the
type of user feedback (see Fig. 2, left-corner at the bottom). For example if the speech
acts used in the feedback under analysis are classified as Suppositive, Requirement, and
Questions, this may be interpreted as indicators for the analyst towards identifying a
Feature Request.
Other capabilities of the proposed ontology, which can not be illustrated here due to
space limits, concern the support for the identification of analysis techniques, based on
the presentation format of the feedback, i.e. visual, audio and textual, or as a composi-
tion of linguistic and non-linguistic act (e.g. attachment, emoticons).
Concerning the scope of application of the proposed ontology, by construction we con-
sider software requirements analysis and design of feedback collection techniques as
application areas. Since we are building the user feedback ontology extending a foun-
dational ontology we believe that it can help understand the nature of user feedback.
Regarding the current status and known limitations of the work, we are consolidating
the ontology and a proper validation along all the CQs considered so far, is to be per-
formed.
4 Related Work
We briefly recall work addressing the problem that motivates our research, namely how
to collect and analyse explicit user feedback for the purpose of collaborative RE in
software evolution and maintenance. Worth mentioning are tools, which enable users
to give feedback in situ, based on semi-structured collection, e.g. [15, 16]. Focusing on
works that investigate how explicit, indirect user feedback analysis can support software
maintenance and evolution tasks, the following approaches are worth to be mentioned:
[17] presents an approach based on statistical analysis, to exploit explicit, indirect feed-
back by power users, reporting about software defects through the open bug reporting
in the Mozilla project. Statistical analysis is applied by [2] to answer questions about
how and when users provide feedback in the AppStore.
There is a vast literature about the application of ontologies in RE, but it is out of the
scope of this paper to mention all that work. As examples in the case of requirements
elicitation, we can mention [18] that presents an approach to build a domain ontology
used to guide the analyst on domain concepts used for the elicitation of requirements.
Our work differs from it because our ontology aims at supporting designers of feedback
collector tools and requirements analysts to understand why and how user feedback is
provided, as a previous step before determining the requirements.
5 Conclusion
In this paper we introduced a user feedback ontology that we are developing, which
considers feedback as an artefact and as a special type of communication process. We
8 Itzel Morales-Ramirez, Anna Perini, and Renata Guizzardi
described how we are building it by extending and integrating existing ontologies, bor-
rowing concepts from different theories, including SAT. As future work, we intend to
validate it systematically against the whole set of competency questions that it is built
for. The ontology will provide foundation to our ongoing work on conversation analysis
based on the automatic extraction of speech acts [11].
References
1. Kienle, H., Distante, D.: Evolution of Web Systems. In Mens, T., Serebrenik, A., Cleve, A.,
eds.: Evolving Software Systems. Springer (2014) 201–228
2. Pagano, D., Maalej, W.: User Feedback in the Appstore: An Empirical Study. In: RE, IEEE
(2013) 125–134
3. Madhavji, N.H., Fern´
andez-Ramil, J.C., Perry, D.E., eds.: Software Evolution and Feedback:
Theory and Practice. John Wiley and Sons Ltd (2006)
4. Morales-Ramirez, I., Vergne, M., Morandini, M., Siena, A., Perini, A., Susi, A.: Who is
the Expert? Combining Intention and Knowledge of Online Discussants in Collaborative RE
Tasks. In: ICSE Companion, ACM (2014) 452–455
5. Cambria, E., Schuller, B., Xia, Y., Havasi, C.: New Avenues in Opinion Mining and Senti-
ment Analysis. IEEE Intelligent Systems 28(2) (2013) 15–21
6. Guizzardi, R.S.S., Morales-Ramirez, I., Perini, A.: A Goal-oriented Analysis to Guide the
Development of a User Feedback Ontology. In: iStar. CEUR Workshop Proceedings (2014)
7. Oliveira, F.F., Antunes, J.C., Guizzardi, R.S.: Towards a Collaboration Ontology. In: Proc.
of the Workshop on Ontologies and Metamodels for Software and Data Engineering. (2007)
8. Searle, J.R.: Intentionality: An Essay in the Philosophy of Mind. Number 143. Cambridge
University Press (1983)
9. Bach, K., Harnish, R.M.: Linguistic Communication and Speech Acts. MIT Press, Cam-
bridge, MA (1979)
10. Guizzardi, G., de Almeida Falbo, R., Guizzardi, R.S.S.: Grounding Software Domain On-
tologies in the Unified Foundational Ontology (UFO): The case of the ODE Software Process
Ontology. In: CIbSE. (2008) 127–140
11. Morales-Ramirez, I., Perini, A.: Discovering Speech Acts in Online Discussions: A Tool-
supported method. In: CAiSE Forum. CEUR Workshop Proceedings (2014) 137–144
12. Hattie, J., Timperley, H.: The Power of Feedback. Review of Educational Research 77(1)
(March 2007) 81–112
13. Brun, Y., Di Marzo Serugendo, G., Gacek, C., Giese, H., Kienle, H., Litoiu, M., M¨
uller, H.,
Pezz`
e, M., Shaw, M.: Engineering Self-Adaptive Systems through Feedback Loops. In: SE
for Self-Adaptive Syst. Springer (2009) 48–70
14. Mory, E.H.: Feedback Research Revisited. Handbook of Research on Educational Commu-
nications and Technology 45(1) (2004) 745–784
15. Seyff, N., Ollmann, G., Bortenschlager, M.: AppEcho: a User-Driven, In Situ Feedback
Approach for Mobile Platforms and Applications. In: MOBILESoft, ACM (2014) 99–108
16. Schneider, K.: Focusing Spontaneous Feedback to Support System Evolution. In: RE, IEEE
(2011) 165–174
17. Ko, A.J., Chilana, P.K.: How Power Users Help and Hinder Open Bug Reporting. In: Proc. of
the Conference on Human Factors in Computing Systems. CHI ’10, ACM (2010) 1665–1674
18. Omoronyia, I., Sindre, G., Stlhane, T., Biffl, S., Moser, T., Sunindyo, W.: A Domain Ontol-
ogy Building Process for Guiding Requirements Elicitation. In: REFSQ. Springer (2010)
188–202
