ArticlePDF Available

Abstract and Figures

Socially interactive robots are expected to have an increasing importance in human society. For social robots to provide long-term added value to people's lives, it is of major importance to stress the need for positive user experience (UX) of such robots. The human-centered view emphasizes various aspects that emerge in the interaction between humans and robots. However, a positive UX does not appear by itself but has to be designed for and evaluated systematically. In this paper, the focus is on the role and relevance of UX in human-robot interaction (HRI) and four trends concerning the role and relevance of UX related to socially interactive robots are identified, and three challenges related to its evaluation are also presented. It is argued that current research efforts and directions are not sufficient in HRI research, and that future research needs to further address interdisciplinary research in order to achieve long-term success of socially interactive robots.
Content may be subject to copyright.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
1
User experience in social human-robot interaction
Abstract
Socially interactive robots are expected to have an increasing importance in human society and for
social robots to provide long-term added value to people’s lives, it is of major importance to stress
the need for positive user experience (UX) of such robots. The human-centered view emphasizes
various aspects that emerge in the interaction between humans and robots. However, a positive UX
does not appear by itself but has to be designed for and evaluated systematically. In this paper, we
focus on the role and relevance of UX in human-robot interaction (HRI) and identify four trends
concerning the role and relevance of UX related to socially interactive robots, and also present
three challenges related to its evaluation. We argue that current research efforts and directions are
not sufficient in HRI research, and that future research need to further address interdisciplinary
research in order to achieve long-term success of socially interactive robots.
Keywords: socially interactive robots, human-robot interaction (HRI), user experience
(UX), UX evaluation, user experienced design, human-computer interaction (HCI), UX
goals, robotic technology, human-centered HRI, human-technology interaction
Introduction
Socially interactive robots are expected to have an increasing importance in everyday life for
a growing number of people. Lately, there has been an increased number of socially
interactive robots in human environments and their level of participation in everyday
activities are becoming more sophisticated (Dautenhahn, 2007a; Oh & Kim, 2010; Thrun,
2004). For robots – as for all other types of interactive systems, products, and devices –
positive user experience (UX) is necessary in order to achieve the intended benefits. Briefly
stated, user experience is about people’s feelings, as caused and shaped by the use of
technology in a particular context (Hartson & Pyla, 2012; Hassenzahl, 2013), and UX is
therefore essential for user acceptance of social robots (de Graaf & Allouch, 2013). If the
usage of a robot entails a negative experience of the user, it can have negative consequences,
such as reluctance to use that particular robot, or robots in general, erroneous handling, or
spreading bad reputation about robots. Therefore, it is essential for robot developers to put
serious effort into building robots that users experience as positive. By designing a high
quality interaction with the intended users and usage context in mind it is possible to
positively influence that experience (Hartson & Pyla, 2012; Hassenzahl & Tractinsky, 2006).
Positive user experiences underpin the proliferation of social robots in society (Weiss et al.,
2009a), and therefore, the user experience of social robots needs to be a central issue of
concern. However, a positive user experience does not appear by itself but has to be
systematically, thoroughly, and consciously designed for as well as evaluated (Hartson &
Pyla, 2012; Hassenzahl, 2013). This clearly highlights the importance of evaluating the
quality of the interaction, resulting in evaluations of different aspects: including acceptance,
usability, learnability, safety, trust, and credibility. While some of these aspects are covered in
depth, some are just briefly touched upon in human-robot interaction (HRI) research.
Therefore, each specific robot development project needs to take the UX perspective into
account during the whole development process. The field of user experience design (UXD)
offers methods, techniques, and guidelines for creating a positive user experience for all
types of interactive systems intended for human use (Anderson et al., 2010; Hartson &
Pyla, 2012).
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
2
The field of HRI is a young but growing research field that is facing several challenges. For
example, there is a need to build a foundation of theories, models, methods, and tools. There
is a particular need for new evaluation techniques and even though human interactions with
robots differ significantly from interactions with more traditional, and typically more
passive, computer-based artifacts (Dautenhahn, 2007a; Thrun, 2004; Young et al., 2011).
It has been proposed that useful inspiration can be derived from the fields of human-computer
interaction (HCI) and user experience (UX) (Dautenhahn, 2007b). Currently, robot developers
sometimes create their own evaluation methods without sufficient knowledge of appropriate
methodologies, resulting in questionable validity and reliability of these so-called “quick and
dirty” methods (Bartneck et al., 2009). In this paper, we argue that a good way to proceed in
order to address this issue is to adopt existing techniques from HCI and UX, and use these
appropriately adapted to HRI. Hence, practitioners like robot developers need research-
based guidance of how to properly choose and apply UXD techniques and approaches for
the social robotic products.
The aim of this paper is to address the role and relevance of user experience of socially
interactive robots, disentangling several issues related to the evaluation of social human-robot
interaction. We identify four trends in HRI research concerning the role and relevance of UX,
and we also present three related challenges for the effective incorporation of UX evaluation
in HRI. In doing so, we highlight the need for HRI evaluation methods that have
methodological validity and reliability as well as practical applicability. Based on that
framing, additional research directions are addressed, including a wide range of different
perspectives and attributes of UX, the UXD process, and robot products. We advocate an
interdisciplinary approach in HRI that would help improve the societal impact of social robots
in the long run.
The rest of this paper is structured as follows. First, in the background section, the notions of
human-robot interaction (HRI), socially interactive robots, and user experience are
introduced. Next, four trends concerning the role and relevance of UX related to socially
interactive robots are identified as well as three challenges for the effective incorporation of
UX evaluation in HRI and arguments for additional research directions are presented. Finally,
the paper ends with some conclusions.
Background
In this section, the field of HRI is introduced and the type of robot in focus, i.e., socially
interactive robots, is presented. Finally, the notions of UX as well as UXD are characterized
in more detail.
Human-robot interaction (HRI)
HRI is a relatively new and growing research field that is concerned with the ways humans
might work, play and interact with different kinds of robots. According to Dautenhahn and
Saunders (2011), there are several disciplines that contribute to HRI and bring different views
and approaches to the field. They argue that it would be beneficial to consider and incorporate
research from a wider outlook that may challenge and enhance existing frameworks and
embark on new frontiers in HRI. More specifically, Dautenhahn (2007b) emphasizes that
useful inspiration can be derived from the study of HCI and UX, particularly methods and
techniques for designing and evaluating various aspects of HRI.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
3
Robotic technologies can be grouped into three main categories based on their area of
application: industrial robotics, professional service robotics, and personal service robotics
(Thrun, 2004). Industrial robots and professional service robots both manipulate their physical
environment. While the first is strictly computer controlled, and operates in industrial settings,
the second mainly functions outside industrial settings and provides assistance, for example
by cleaning up nuclear waste (Brady et al., 1998) or inspecting abandoned mines (Thrun et al.,
2003). Personal service robots assist or entertain people in domestic settings, for example
robotic vacuum cleaners and robot assistants in elderly care and in therapy (Thrun, 2004)
These types of robots have the highest expected growth rate in society.
The purpose of robotic technology is to make it possible for a person to conduct something
that he or she could not do earlier, facilitate a certain task, or provide entertainment (Goodrich
& Schultz, 2007). Robots can bring different kinds of value to people’s lives, e.g., by
conducting monotonous assembling tasks in manufacturing or keeping the lawn cut
(Dautenhahn & Sanders, 2011). In those cases, often humans do not need to continuously
interact with the robot. Other types of robots and usage situations demand more frequent and
multi-faceted interaction, e.g., robots assisting elderly people. This interplay between robots
and their users has to be carefully taken into account when developing a robot in order for it
to provide added value.
The problem of understanding and designing the interaction between humans and robots is the
core interest in the field of HRI (Goodrich & Schultz, 2007). According to Dautenhahn
(2013), the key challenge and characterization of HRI can be phrased as follows:
HRI is the science of studying people’s behavior and attitudes towards robots in relationship to the
physical, technological and interactive features of the robots, with the goal to develop robots that
facilitate the emergence of human-robot interactions that are at the same time efficient (according
to the original requirements of their envisaged area of use), but are also acceptable to people, and
meet the social and emotional needs of their individual users as well as respecting human values.
The importance of, and the attention attracted to, HRI is increasing in parallel with
technological achievements in robotics. For the same reason, the concept of a robot is
constantly changing (Dautenhahn, 2013). The boundaries for how robots can be constituted,
and the settings in which they can act in, are continually expanding. However, an important
characteristic that separates robotic technology from technological devices in general is that
robotic technology has to, at least to some extent, act autonomously in its environment.
Autonomy, roughly speaking, means that a robot makes its own decisions and adjusts to
current circumstances (Thrun, 2004). Hence, robots can vary along multiple dimensions, e.g.,
the types of task it is intended to support, its morphology, interaction roles, human-robot
physical proximity, and autonomy level (Yanco & Drury, 2004). Moreover, the role of
humans in relation to robots can vary; the human can be a supervisor, operator, mechanic,
teammate, bystander, mentor, or information consumer (Goodrich & Schultz, 2007; Scholtz,
2003). Likewise, robots can have a wide range of manifestations and be used in different
application areas. There are human-like robots (humanoids and androids), robots looking like
animals, and mechanical-appearing robots. Robots can be used for urban search and rescue
tasks, e.g., natural disasters and wilderness search; for assisting and educational purposes,
e.g., therapy for elderly; in military and police forces, e.g., patrol support; as edutainment,
e.g., museum tour guide; in space, e.g., astronaut assistant; at home, e.g., robotic companion;
and in industry, e.g., construction robots (Dautenhahn, 2013; Goodrich & Schultz, 2007).
Consequently, the interactions between user and robot can vary significantly, depending on
user-, task-, and context-based conditions. Generally, interaction can be either remote, i.e., the
human and the robot are spatially, and perhaps also temporally, separated, or proximate, i.e.,
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
4
the human and the robot are collocated (Goodrich & Schultz, 2007). The interaction can be
indirect — which means that the user operates the robot by commanding it — or direct when
the communication is bi-directional between the user and robot and the robot can act on its
own (Thrun, 2004). Dautenhahn (2007a) argues that in many application areas the robots need
also to have social skills; these robots are referred to as socially interactive robots.
Socially interactive robots
The growth in the area of personal robots that act in human environments results in robots that
to a larger extent need to act in relation to social and emotional aspects of interaction. Socially
interactive robots are “robots for which social interaction plays a key role” (Fong et al., 2003,
p. 145). Such robots should display social intelligence, which means that they demonstrate
qualities that resemble human social expressions. Examples of such qualities are emotional
appearance and perception, advanced dialogue capabilities, possibilities to recognize humans
and other robots, as well as be able to make use of for instance gaze and gesture as part of
communication (Fong et al., 2003). When it comes to social interaction with robots, HRI
research can be categorized into three different approaches: robot-centered HRI, robot
cognition-centered HRI, and human-centered HRI (Dautenhahn, 2007a, see Figure 1).
Fig. 1. The conceptual space of approaches for social interaction in HRI research (modified from Dautenhahn, 2007a , p.
685).
While robot-centered HRI views the robot as an autonomous entity and the human as the
robot’s “caretaker” who should identify and respond to the needs of the robot, robot-
cognition-centered HRI views the robot as an intelligent system - and in that case the
fundamental problem is to provide these robots with a cognitive capacity. In the human-
centered HRI, on the other hand, the human perspective is emphasized, and issues related to
design of robot behavior that is suitable for humans are included in this approach. This
involves acceptability and believability, as well as humans’ expectations of, attitudes towards,
and perceptions of robots. In order to get robots to “inhabit our living environments”, the
three approaches need to be synthesized to enhance social interaction (Dautenhahn, 2007a).
However, historically human-centered HRI has not received as much attention as the other
two approaches (Lindblom & Andreasson, to appear).
It is of course not necessary for all robots to be highly socially interactive. Instead, the
purpose and the context in which a robot is supposed to act in sets the requirements for the
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
5
robot’s social skills. For instance, robots that are remotely controlled and separated from the
user spatially and temporally may in many cases require few or even no social skills, whereas
considerably higher social requirements are essential for robots used in nursing care and
therapy (Dautenhahn, 2007a). Dautenhahn (2007a) argues that the social skills required of
robots vary along several dimensions, and, hence, it is important for a robot developer to be
aware of the role and context of the robot to be used in. The dimensions listed by Dautenhahn
(2007a) are:
Contact with humans: from none and remote to repeated long-term physical contact.
Robot functionality: from limited and clearly defined to open and adaptive
functionality that is shaped by learning.
Role of robot: from machine or tool to roles such as assistant, companion, and
partner.
Requirements of social skills: from not required or desirable to essential.
In a similar vein, Breazeal (2003) suggested that there are different modes of social
interaction: socially evocative, social interface, socially repetitive, and sociable. A socially
evocative mode denotes that the robot stimulates humans to anthropomorphize it during
interaction, but goes no further. Robots with the interaction mode of social interface interact
with help of human-like social cues and communication modalities. A socially receptive mode
entails that the robot benefits from interacting with humans, although the benefits are
primarily assessed from the human perspective. A sociable robot is a social participant with
its own internal goals and drives. No matter which approach, dimension, mode, or context that
is focused on, it is important that the human experience the interaction with and the
appearance of the socially interactive robot positively (Lindblom & Andreasson, to appear).
Recently, different aspects related to the emotional quality of the interaction have been
addressed in the HRI literature, including engagement, safety, intentions, acceptance,
cooperation, emotional response, likeability, and animacy. However, user experience (UX)
has largely been omitted. This is the topic we now turn to.
User experience (UX)
Technology is spreading into almost every aspects of daily life and therefore, UX is a concept
that has becoming increasingly important (e.g. Anderson et al., 2010; Hartson & Pyla, 2012;
Hassenzahl, 2010, 2013). Since humans have been using advanced technology for quite a
while, their expectations of and demands on the quality of technological products are going
beyond utility, usability, and acceptance. From the users’ point of view, a product that is
suitable for its purpose, is easy to use, and fits into its intended context are just basic
requirements of the technological product. The users have also started to postulate and
demand a positive and great experience when interacting with the product. Broadly speaking,
UX addresses the feelings created and shaped by the use of technology and how technology
can be designed to create a user experience that evolves the required feeling (e.g. Hartson &
Pyla, 2012; Hassenzahl, 2010, 2013). As pointed out by Hassenzahl and Tractinsky (2006),
UX has become a buzzword in the area of HCI and related fields as a result of more mature
advanced technology that allows for more interactive products, beyond the functional
capabilities that are necessary but not sufficient for high quality technology use. Since the
inception of the term in the mid-1990s, the notion of UX has been embraced by both
practitioners and researchers because it offers a possible alternative to the more traditional and
instrumental HCI. However, many researchers and practitioners use and apply the UX
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
6
concept in a loose and poorly understood manner with negative consequences (Lindblom &
Andreasson, to appear). While UX is hard to characterize, it can be viewed as “the totality of
the effect or effects felt by a user as a result of interaction with, and the usage context of, a
system, device, or product, including the influence of usability, usefulness, and emotional
impact during interaction and savoring memory after interaction” (Hartson & Pyla, 2012, p.
5). This is consistent with ISO defining UX as: “a person's perceptions and responses that
result from the use or anticipated use of a product, system or service" (ISO 9241-210). In
other words, it is not possible to guarantee a certain UX, since it is the subjective inner state
of a human. Although, by designing a high quality interaction with the intended users and the
usage context in mind, it is possible to impact the experience.
The UX is not built in the product itself. Instead, it is an outcome of the interaction that
depends on the internal state of the user, the quality and attributes of the product, and the
particular situation (Hartson & Pyla, 2012; Hassenzahl & Tractinsky, 2006). In effect, UX is
an umbrella term that embraces the totality of the user’s emotions, beliefs, preferences,
perceptions, and accomplishments, that emerge before, during, and after technology use in a
certain situation. Hence, good UX is difficult to define, but easy to identify (Anderson et al.,
2010; Lindblom & Andreasson, to appear).
The concept of UX embraces pragmatic as well as hedonic quality (Hassenzahl & Roto, 2007;
Hassenzahl & Tractinsky, 2006). Pragmatic quality is related to fulfilling the so-called “do-
goals” of the user, which means that the interactive product makes it possible for the user to
reach the task-related goals in an effective, efficient, and secure way. Hence, pragmatic
aspects relate to the usability component of UX and have their roots in HCI. Usability is
defined in ISO 9241-11 as follows: “the extent to which a system, product or service can be
used by specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use” (ISO 9241-11). It should be stressed that usability
relates to the outcome of interacting with a system and as defined in the ISO standard,
usability is not an attribute of a system although appropriate attributes of the system can
contribute to being usable in a particular context of use.
Hedonic quality is about the so-called “be-goalsof the user, beyond the instrumental aspects
addressed in HCI. This relates to psychological and emotional needs of the user, which can
have great impact on how an interactive product is experienced. The user can, for instance,
find the product cool, awesome, beautiful, trustworthy, satisfying, or fun, and the hedonic
quality should therefore be addressed by the interactive product. The hedonic aspects are
usually portrayed as the emotional impact that emerges when the user interacts with the
system (Hartson & Pyla, 2012). The product can, for example, evoke feelings of autonomy,
competence, and relatedness to others (Hartson & Pyla, 2012; Hassenzahl & Roto, 2007;
Partala & Kallinen, 2012). In later work, Hassenzahl, Diefenbach and Göritz (2010) noticed
that “be-goals” are often related to positive affect. Negative UX can have its cause in poor
interface design or a perceived lack of functionality, resulting in a negative emotional
experience during interaction with the system. Accordingly, the positive expectations and the
excitement when a new, cool, high-tech system is launched can very quickly shift from
amazement to annoyance if the usage of the technology fails since beinghigh-tech’ is not a
causative factor of positive UX (Hartson & Pyla, 2012). Thus, positive UX is not built in the
product but is an outcome of the interaction between the user, including the user’s internal
state, the quality and attributes of the product, and the context of use (Hartson & Pyla, 2012;
Hassenzahl & Tractinsky, 2006; Kaasinen, et al, 2015). Nonetheless, by designing for a high
quality interaction with the intended users and the usage context in mind, it is possible to
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
7
positively impact the experience of the interaction and contribute to the quality of the users’
accomplishments of their “do-goals” and “be-goals”. In fact, Hassenzahl and Tractinsky
(2006) argue that a main objective of the UX field should be to contribute to the quality of
human life.
The basis of UXD begins with an investigation phase in order to understand the needs, goals,
meanings, and emotions related to the activities the technology is intended to support. Once
this phase is completed, the What” (i.e., the functionality), and the “How” (i.e., the design of
interaction) can be developed. During the investigation phase, it is necessary to identify the
business needs as well as the users’ needs together with technical and infrastructural
constraints (Hassenzahl, 2013). The user goals have to be connected to the business goals, and
the business goals to user behaviors (Anderson et al., 2010; Kaasinen, et al., 2015). Because
it is not possible to design a product that satisfies everybody, the intended users have to be
identified and described, and focused upon during the whole UX design process. It is also
essential that the UXD process is aligned with the product mission (Anderson et al., 2010).
Another central principle of the UXD process is the iterative and incremental nature of the
development process. It is not possible to have all the answers from the very beginning.
Instead, the answers will be discovered and evolved during the iterative development process
(Anderson et al., 2010; Hartson & Pyla, 2012; Kaasinen, et al., 2015).
The iterative UXD cycle consists of four key elements of UX activities: analyze, design,
implement, and evaluate. Hartson and Pyla (2012) describe these activities as the “UX
wheel”. Briefly stated, “analyze” refers to understanding the users’ work and needs. “Design”
refers to creating conceptual design ideas and the fundamental “look and feel” of the
interaction between the user and the intended product. “Implementation” refers to the more
detailed interaction situations with the use of different kinds of product prototypes, which
vary from low fidelity to high fidelity of details. Finally, “evaluation” refers to the different
methods and techniques that can be used to investigate and analyze to what extent the
proposed design meet the users’ needs, requirements, and expectations. The whole “wheel”
corresponds to an iterative UX lifecycle that is accompanied by defined UX goals that
concretize the intended experience (Hartson & Pyla, 2012). UX goals are high-level
objectives, which should be driven by representative use of an interactive system and should
identify what is important to the users, stated in terms of anticipated UX of an interaction
design. The UX goals are expressed as desired effects, e.g. ease-of-use, learnability,
acceptance, and emotional arousal (Hartson & Pyla, 2012; Kaasinen, et al., 2015).
Like all other interactive products for human use, user interaction with and perception of
socially interactive robots evoke feelings of different nature and intensity. The user can feel
motivated to approach and use a robot. He or she can experience a weak distrust of the robot
and at the same time be curious of it. A user can find a robot to be well adapted and highly
useful after a long-term use, although, initially experienced it to be a bit strange and difficult
A robot can be found to be really fun and entertaining for young children, but boring for
teenagers. Thus, the user experience has many facets and it is complex. Therefore, robot
developers need to identify and characterize what kind of feeling that is especially important
for this particular socially interactive robot to arouse in the users. Once this is done, it is
possible to consciously design the robot with those feelings as the target and it is also possible
to evaluate if the robot can be expected to create or elicit the intended experience in the user.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
8
Role and relevance of user experience in socially interactive robots
It is becoming increasingly important to design technologies that ensure the interaction is
experienced by the user as not only acceptable and safe, but also as positive. Just as with all
interactive systems, positive UX is necessary for robots to achieve the intended benefits. The
notion of user experience is frequently used in research of socially interactive robots but also
in HRI in general. A positive user experience does not appear by itself but has to be
systematically designed for and evaluated (Hartson & Pyla, 2012; Hassenzahl, 2010, 2013)
and, therefore, the UX of social robots needs to be a central issue of concern when developing
robots (Lindblom & Andreasson, to appear). We have conducted a literature study, and the
analysis of the findings resulted in the following four trends in HRI research concerning the
role and relevance of UX related to socially interactive robots. The identified trends are listed
below, and are then addressed in more detail.
UX is considered to be of major importance and is used as an argument for stating
that something is positive. Accordingly, user experience is used as a catch phrase for
highlighting that certain aspects have to be studied or to what degree a specific feature
has a positive impact on the users’ experience of the robot system.
UX is often treated superficially and its deeper aspects are seldom addressed in HRI
research. Unless UX is problematized by unpacking the concept to expose its many
complex facets, there is a significant risk of reducing the validity of the outcome of the
design. That is, the robot is likely to fail to deliver the desired user experience if the
complexity is not properly considered.
UX aspects are often omitted in favor of studying robot-related aspects. Usually, the
focus is on studying how the robot as such affects the user instead of studying the
interaction between the two. It should be noted, however, that if such studies do not
address the many facets of user experience, the outcome may be less useful than it
otherwise could be. Hence, the recognition of the complexity of UX may contribute to
more valuable research results.
UX evaluation is of major importance although the investigations often are conducted
after the actual interaction. This makes the subjects reflect on their experience of
interacting with the robot afterwards, at the very end of the study, and that might bias
their responses.
The first identified trend is that UX is considered to be of major importance and is used as an
argument for stating that something is positive. Examples of this trend in the literature are as
follows. Tritton et al. (2012) claimed that it is important to examine the impact of
communicative robot gestures on UX in order to push the introduction of domestic social
robots. Kuo et al. (2011) expressed that the UX would be more satisfying if the robot can
communicate its internal status and intention during interaction so the expectations of the user
can be attuned. Datta et al. (2011) said that to actively deliver shopping-related information
via a shopping mall robot can counteract the user experience. Juarez et al. (2011) pointed out
that the long-term success on their robot will depend on the UX and, therefore, they chose
heuristic evaluation (e.g., Nielsen, 1994) as part of their study. Callejas et al. (2014)
developed an assessment framework that constitutes a novel instrument to evaluate
personality, as something desirable and related to UX, in three main dimensions. They
concluded that the proposed framework enables comparison and rankings of different agents
with respect to the user's perception of the rendered personality. Carcagnì et al. (2014) argued
that positive UX is desirable, addressing how gender recognition can significantly improve
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
9
the overall UX quality in HRI. Hall et al. (2014) investigated the effects of some key
nonverbal gestures on a human’s own engagement and robot engagement experienced by
humans, which is aligned with positive UX. Johnson et al (2016) explored whether the
multimodal behavioral patterns (i.e. combinations of gestures, eye LED patterns, and verbal
expressions) that were developed affected the entertainment value of playing games with a
humanoid Nao robot. Kim and Mutlu (2014) conducted two studies that examined the effects
of distance based on physical proximity (proxemics distance), organizational status (power
distance), and task structure (task distance) on people's experiences with and perceptions of a
humanlike robot. Their findings highlight the importance of consistency between the status
and proxemics behaviors of the robot as well as task interdependency in cultivating
cooperative actions between the robot and its users for obtaining a positive UX. Matsumoto et
al. (2015) focused on emotions and proposed a new application scenario for the affective
robot that shares the user’s experience. They conducted an experiment in which the user’s
experience is altered by the presence of the affective robot. Based on the obtained results, they
formulated some design points of affective robot behavior for enhancing a positive UX. These
examples illustrate that UX is commonly used as an axiom for underpinning that a certain
aspect has to be investigated and whether or not a feature or attribute should be used.
The second identified trend is that UX is often treated superficially and its deeper aspects are
seldom addressed or problematized in HRI research (see e.g., Callejas et al., 2014; Carcagnì
et al, 2014; Datta et al., 2012; De Carolis et al. 2010; Holthaus & Wachsmuth, 2014; Huang et
al., 2011; Huang & Mutlu, 2014; Jia et al., 2013; Johnson et al., 2014; Jokinen & Wilcock,
2015; Karreman et al., 2015; Körtner et al., 2014; Lohse et al.,2014; Loth et al., 2015;
Matsumoto et al., 2015; Moon et al., 2014; Nunez et al., 2015; Rabbitt et al., 2015; Šabanović
& Chang, 2015; dos Santos et al., 2014; Schlögl et al., 2015; Schroeter et al., 2013; Strasser et
al., 2012; Sumioka et al., 2014; Tritton et al., 2012; Xu et al., 2012; Xu et al., 2015a; Xu et al.,
2015b). Consequently, although UX is mentioned in these papers, no references are provided
to UX-focused work. Since UX is a multi-faceted and complex phenomenon there is a risk of
reducing the precision of results if the complexity is not considered properly. For instance, it
is possible that a certain robot feature or characteristic can positively affect some dimension
of UX and at the same time negatively influence other dimensions of UX, e.g., finding the
robot awesome and curiosity-rising, and at the same time experiencing the robot as difficult to
cooperate with. There are of course also some papers in which the UX concept is in fact
problematized or at least briefly described (see e.g., Anzalone et al., 2015; Cesta, 2016; de
Graaf & Allouch, 2013; de Graaf et al. , 2015a; de Graaf et al., 2015b; Jung et al, 2013; Kim
& Mutlu, 2014; Oh & Kim, 2010; Syrdal et al., 2008; Weiss et al., 2009a; Weiss et al., 2011;
Xu et al, 2013; Young et al., 2011).
The third identified trend is that UX aspects are often omitted in favor of studying robot-
related aspects instead of studying the interaction between the robot and the users. Some
representative examples of this are as follows. He et al. (2014) proposed techniques, involving
a robotic head for social robots to attend to scene saliency with bio-inspired saccadic
behaviors, that are intended to improve the UX. Saini et al. (2005) investigated how the user
experience can benefit from a more socially rich and coherent home dialogue system that is
perceived as socially intelligent. Huang et al. (2011) analyzed the long-term UX of a
particular robot in order to understand the failure of it, with the purpose of concluding some
implications for future design. The result presented by Xu et al. (2013) demonstrated that an
agent’s engagement-aware behaviors improve the UX. Beňuš (2014) reviewed selected
advances and concepts in the area of speech entrainment, arguing that the naturalness and
effectiveness of speech applications in social robotics could be enhanced, as well as the social
bonds between human and robot were better controlled and efficient if they exploit speech
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
10
entrainment. Cohen (2014) investigated whether a humanoid Nao robot (lacking most human-
like facial features, but with a body and colored eyes) is able to express emotions in a manner
recognizable to children, by comparing the recognition rates of the emotions between the Nao
and the iCat robot. Their findings revealed that it was possible to design some recognizable
dynamic emotional expressions for the Nao robot, although these emotional expressions were
better recognized when placed in a context and repeated later on. Hall et al. (2014)
investigated the perceptions and effects of some key nonverbal gestures on a human’s own
engagement with a robot as well as the robot’s engagement with the users experienced by
humans in HRI. Participants who experienced greater robotics knowledge reported higher
overall engagement and greater success at developing a relationship with the robot, suggesting
that greater familiarity with robotics may help to enhance positive UX for humans involved in
interactions with robots. de Graaf et al. (2015b) explored the acceptance of social robots in
domestic environments by studying how older adults perceive and use a social robot for
health purposes. Moon et al. (2014) provide empirical evidence that using humanlike gaze
cues during human-robot handovers can improve the timing and perceived quality of the
handover event. They claim that their work demonstrates that gaze can play a key role in
improving UX of human-robot handovers, by enabling it to be fast and fluent. Sumioka et al.
(2014) focused on the feeling of human presence via a Telenoid robot, investigating the
minimal requirements to enhance the feeling of human presence (so-called sonzai-kan). Their
findings revealed the feeling of sonzai-kan is enhanced if information is presented from at
least two different modalities where the combination of human-like touch and a human voice
seems of major importance. The work conducted by de Graaf et al. (2015b) provides a set of
social behaviors and certain specific features which they argued that social robots should
possess. They also discussed whether robots can actually be social, and then present some
recommendations in order to build better social robots. Jokinen and Wilcock (2015)
thoroughly investigated how different combinations of multimodal behaviors (e.g. gazing,
facial expressions, body posture) may predict the user’s experience and evaluation of spoken
interaction with respect to various evaluation categories. This kind of studies is essential for
building a coherent body of knowledge concerning how to design socially interactive robots
that feel good for the user to interact with and thereby increase the possibilities of user
acceptance. However, if such studies are not combined with addressing the many facets of
UX, the precision of the results may be lower than it could be. Recognizing the complexity of
UX is likely to make contributions even more valuable to both theory and practice.
The fourth identified trend is that UX evaluation is of major importance although the
investigations often are conducted after the actual interaction, making the subjects reflect on
their experience after the interaction occured, which might bias their responses. Examples of
UX evaluation are as follows: Young et al. (2011) carried out a survey of HCI evaluation
techniques with the focus on holistic interaction experience. They presented an appropriate
framework for evaluating UX holistically, emphasizing the importance of hedonistic qualities
of UX. However, this work lacks concrete guidelines on which methods are appropriate and
how they should be applied in practice. Their framework provides a UX lens rather than
concrete guidelines for the evaluation of a positive UX. Weiss et al. (2009b, 2011) developed
an evaluation framework called USUS (Usability, Social acceptance, User experience, and
Societal impact) that provides a promising, comprehensive and holistic view of the aspects
that can affect both usability and UX in HRI. In contrast to the work developed by Young et
al. (2011), USUS provides instructions on the methods and techniques that are appropriate to
use when evaluating single aspects of HRI. Their detailed descriptions of what and how to
evaluate provide clear guidance on which evaluation method is suitable for evaluating certain
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
11
usability and UX aspects that are useful for both experts and novices. It also provides the
possibility of easy extension to include other relevant methods or techniques depending of
what needs to be evaluated. One disadvantage of USUS is that it may be too time consuming
to apply if all usability and UX factors included in the framework are to be evaluated. Another
disadvantage relates to the fact that the USUS framework does not explicitly address the need
to specify UX goals. This is a major shortcoming although it would not be difficult to modify
and develop the framework to address these identified deficiencies (Lindblom & Andreasson,
to appear). de Graaf and Allouch (2013) have examined utilitarian and hedonic variables that
can enable the evaluation of affective factors of interaction. Anzalone et al. (2015) evaluated
the engagement with social robots, emphasizing that evaluation of UX is important, and they
argue that engagement is particularly important in interaction with social robots. They
developed a methodology to evaluate engagement aroused during interactions between social
robots and humans. Bevilacqua et al. (2015) evaluated the services of a so-called Robot-Era
system, including usability and UX aspects. Cesta (2016) conducted a long-term evaluation of
a telepresence robot for elderly people, proposing a methodology in the area of elderly
support, which they call MARTA (Multi-dimensional Assessment of telepresence RoboT for
older Adults). Xu et al. (2015a) addressed methodological issues in scenario-based evaluation
of HRI, by investigating how various scenario media may influence user evaluation of social
robots. Johnson et al. (2014) studied how robots could assist elderly people by investigating
the users’ needs and then tried to identify proper metrics for the context. It should be noted
that some UX studies are conducted to validate and verify robotic prototypes as well as
robots-in-use (see e.g., De Carolis et al., 2010; Huang & Mutlu, 2014; Johnson et al., 2014;
Schroeter et al., 2013; Xu et al., 2012). UX evaluation is fundamental and a necessity for the
development of socially interactive robots that are experienced positively and accepted by the
intended users. Nonetheless, it is possible to widen the scope of research interests and there
are other research directions that should be added to the research agenda concerning UX for
socially interactive robots.
We now leave the four identified trends, and focus on the need for more theoretical as well as
methodological knowledge about methods and techniques that are appropriate for evaluating
UX in HRI (Lindblom and Andreasson, to appear). Bartneck et al. (2009) noted that many
robot developers are unaware of the extensive knowledge about methodologies and
techniques for studying various aspects of human cognition and interaction, and therefore
sometimes run rather naïve user studies and experiments to verify their robot design.
Although disentangling the many interdependent factors in UX is an essential step in the right
direction, we want to address the need for conducting valid usability and UX studies in order
to exploit UX to its full effect and to improve positive UX of socially interactive robots. HRI
research faces complex challenges regarding UX evaluation which do not have easy solutions.
Some of these challenges are listed below. The list is not exhaustive, but provides a useful
starting point in order to close the gap between the different approaches to social interaction
in HRI identified by Dautenhahn (2007a, see Figure 1). A truly interdisciplinary perspective
will require researchers to adopt a wider set of concepts, theories, and methods in their own
research, which implies the need to read a broader spectrum of literature as well as correctly
applying the methods in practice. Lindblom and Andreasson (to appear) present some
identified challenges for HRI research as the following:
The need to adopt an iterative UX design process in HRI. This poses a dilemma
because of the high cost of rapid prototyping in robotics. From the robot developers’
perspective, Lohse et al. (2014), for example, emphasize that long-term studies in
HRI are mostly exploratory and they argue that iterative design is necessary to
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
12
optimize the robot behaviors. They identified some reasons for the lack of iterative
design approaches in HRI when they conducted such a process themselves.
The need to incorporate UX goals to ensure positive UX. A positive UX does not
appear by itself, but has to be systematically designed for and evaluated. Hartson and
Pyla (2009), as well as Kaasinen et al. (2014), among others, point out that defining
UX goals is a key part for working correctly throughout the UX design cycle. In order
to evaluate UX optimally and reflect upon the obtained evaluation results, it is
essential to identify specified UX goals which can be used as evaluation criteria. It
should be noted, however, that stating UX goals is not the same as assessing the
robot’s behavior and functionality. Instead, UX goals focus on the interaction quality
between the human and the robot. Unfortunately, the fundamental activity of
specifying relevant UX goals is often overlooked in HRI, either because of lack of
knowledge or because of lack of time.
The need for robot developers to acquire knowledge about proper UX evaluation, in
theory and in practice. Because many such techniques arrive from other fields, e.g.
HCI, cognitive psychology, artificial intelligence, and human factors (e.g., Goodman
et al., , 2013), it should be noted that the methods need to be adapted and modified in
order to suit UX evaluations in HRI. A related aspect is the possible gap between the
objectives and principle of different methods and their practical application. This
creates a risk for misunderstanding the factors evaluated, resulting in biased
outcomes.
We note that these challenges can be met by drawing on the fields of HCI and UX, for
example with design processes, theories, models, methods, tools, and evaluation approaches
(e.g. Hartson & Pyla, 2012) that may provide starting points for the design, analysis, and
evaluation of HRI studies (Lindblom & Andreasson, to appear).
Arguments for additional research directions
UX is acknowledged as important for achieving socially interactive robots that are widely
used by humans. Hence, it is necessary for robot developers to be knowledgeable of the wide
range of factors or elements that affect UX. It is also necessary for robot developers to
evaluate the actual UX in order to ensure that positive feelings arise in the intended users. The
importance of research concerning establishment of elements that affect the UX and UX
evaluation should not be underestimated, as addressed in the three identified challenges
presented in the prior section. However, in order to develop successful socially interactive
robot products for real-world use, addressing these challenges is not enough. Therefore,
additional research efforts are needed in order to provide robot developers with proper
research-based guidance, so that the launched robotic products are accepted and experienced
by the users in an intended and desirable way.
Due to limited resources, it is not always possible to put equally high efforts in every aspect
that potentially can impact the UX. Instead, the developer has to decide which particular
experiences are most vital to elicit in the intended end-users, and carefully design for and
evaluate the interaction with those users in mind. For instance, is it more important for a
particular robot to evoke feelings of curiosity and fascination than a sense of competence in
the user? Is it more central for a certain robot to make the user feel related to others and that
the user finds the robot elegant than experiencing it as smooth and transparent? The nuances
of UX need to be recognized and carefully considered. Robot developers should be able to
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
13
determine which factors or elements that affect UX are appropriate for a certain purpose, i.e.
which factors that should be avoided and which ones that are needed for enhancing the user’s
motivation to walk up and interact with the robot. If researchers have not considered the
dimensions of UX and, as a consequence, not reported what kind of experiences the studied
factors are intended to shape and which kind of feelings should be achieved, then the actual
influence of the factors or elements that are affecting the UX remains unclear. Hence,
research including UX studies or UX evaluations should go beyond basic feelings, i.e.,
beyond stating that the overall UX is more or less positive or negative.
When developing socially interactive robots for real-world use it is not enough to just imagine
which kinds of UX are important and then hope for the best. Instead, it is vital to make the
decisions concerning which experiences to strive for based on a firm understanding of the
intended user groups, their needs, and the usage context (Anderson et al., 2010; Hartson &
Pyla, 2012; Kaasinen, 2015). With this knowledge in mind, the robot developers can
concentrate their efforts on achieving the desired UX. This implies that the development
process for socially interactive robots should include the whole UXD process, i.e., embracing
the major activities of analysis, design, implementation, and evaluation (Hartson & Pyla,
2012). There is a rich body of knowledge of those activities for interactive systems in general
(e.g. Anderson et al., 2010; Hartson & Pyla, 2012) but for socially interactive robots the major
emphasis so far has been on UX evaluation. To the best of our knowledge, research relating to
the other UXD phases is scarce in socially interactive robots. Lohse et al. (2014) highlighted
that robots are the result of an engineering process that includes a large number of hardware
and software components integrating aspects from various research areas such as navigation,
social perception, computer vision, and cognitive modeling of social skills. This further shows
some challenges for conducting an iterative UXD process in HRI. A related issue is what the
authors refer to as ‘reusability’, i.e. enabling easy integration of different software
frameworks between different robotic platforms. While the work of Lohse et al. (2014) seems
to be from a human-centered perspective, they primarily view robotic developers as users
rather than the targeted end-users. Hence, there is a need to develop research-based guidance
for robot developers concerning how to properly carry out UXD activities that are specific for
socially interactive robots. This is much needed since the human-robot interaction differs
significantly from more traditional and passive human-computer interaction (de Graaf &
Allouch, 2013; Young, et al. 2011). As a consequence, the transferability of methods,
techniques, and guidelines from the field of UXD is an open issue that calls for further
theoretical and empirical studies (Dautenhahn, 2007b; Syrdal et al., 2008; Weiss et al., 2009a;
Weiss et al., 2011). There is also a need for more research on UX analysis, design, and
prototyping for socially interactive robots with the purpose of supporting the practice of robot
developers. Of course, some of these activities are already carried out by robot researchers,
e.g., making prototypes of socially interactive robots, but this is often done more as a mean to
investigate something else than to study UX prototyping in itself. With the exception of the
papers by Bartneck and Hu (2004), as well as Syrdal et al. (2009), this aspect is rather absent
in HRI. Bartneck and Hu (2004) advocated the need for prototyping techniques in robot
development, including scenarios, paper and mechanical mock-ups, and Wizard of OZ. They
do not explicitly mention UX but discuss related aspects. Syrdal et al. (2009) have studied the
use of video prototyping for evaluation purposes. We have not been able to find research that
focuses on how UX aspects should be suitably prototyped, pros and cons of certain types of
prototypes, or under which circumstances a particular kind of prototype is better than another
one in the field of socially interactive robots. The same inclination can be observed for
research into the other phases of the user experience design process, i.e., analysis and design,
for socially interactive robots. It should be noted, however, due to the high cost of rapid
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
14
prototyping in robotics, iterative work is rarely compatible with the iterative view on UXD
(Lindblom & Andreasson, to appear).
There is a need to define prioritized UX goals, UX requirements and validation criteria in HRI
projects (Benyon, 2010; Pressman, 2000) but establishing requirements, determining their
priority, as well as defining validation criteria, are not easy tasks to perform properly. There is
a lot of research on how to establish requirements, prioritize, and set validation criteria (e.g.,
Berander & Andrews, 2005; Kotonya & Sommerville, 1998; Sutcliffe, 2002; Zowghi &
Coulin, 2005). However, as pointed out by Lindblom and Andreasson (to appear), stating UX
goals is not the same as assessing the robot’s behavior and functionality. Instead, UX goals
focus on the interaction quality between the human and the robot. Furthermore, UX goals
offer support throughout the development lifecycle by defining quantitative and qualitative
metrics, which provides the basis for knowing when the required quality of interaction has
been achieved. During the UXD cycle, it is possible to conduct both formative and summative
evaluations. Briefly stated, formative evaluation is typically performed during the early
development of a system, while summative evaluation is performed at the end of a design
process. By performing several formative evaluations during the iterative design process, it
may be possible to compare and contrast the evaluation results obtained during the whole
development process (for more details, see Hartson & Pyla, 2012; Kaasinen et al, 2015).
However, to the best of our knowledge, there is a lack of research focusing on the intersection
of the aspects of UX goals, UX requirements in socially interactive robots.
Furthermore, as a UX-focused robot developer it is not enough to focus on the user needs and
usage context. For commercial development of socially interactive robots there is also a
business case to take into account when establishing and prioritizing UX goals and UX
requirements (Anderson et al., 2010; Hartson & Pyla, 2012; Kaasinen et al, 2015). In line with
previous arguments, robot developers would also be likely to benefit from research-based
guidance of how to fruitfully consider business cases of the socially interactive robots when
making decision of, e.g., requirements and design elements.
Conclusions
In this paper, we have examined the role and relevance of UX for socially interactive robots
and proposed additional research directions. The trends in current research indicate an
increasing awareness of the importance of positive UX for socially interactive robots. UX is
frequently used for stating that some robot-related aspect is positive; almost in an axiom-like
way. Noticeably often, the notion of UX is neither described in any detail nor problematized,
and, consequently, the complex and multi-faceted nature of the concept is not addressed
properly. This can have consequences for the establishment of a coherent body of knowledge
concerning user experience-affecting aspects and elements for socially interactive robots.
There are a lot of studies focusing on effects of various robot features and characteristics on
UX. However, if the results do not identify what kind of experience that is intended to be
achieved, (e.g., astonishing first-time impression, satisfying long-term usefulness, stimulating
communication, neat appearance, impressive adaptability, etc.) there is a risk that the
contributions are not as valuable as they could be. Many of the available methods and
techniques for evaluating UX in HRI have roots in HCI but are modified or adjusted to better
fit the pragmatic and hedonic qualities. Commonly used evaluation methods in HRI include
scenario-based evaluations (e.g. Lohse et al., 2008; Strasser et al., 2012; Syrdal et al.,2008;
Xu et al., 2012, 2015a, 2015b), questionnaires (e.g. Baddoura & Venture, 2013; Lohse et al.,
2008; Weiss et al, 2009b, 2011; Xu et al., 2012, 2015a), interviews and focus groups (e.g.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
15
Cesta et al., 2016; Syrdal et al.,2008; Weiss et al., 2009b, 2011), Wizard of OZ (e.g. Weiss et
al., 2009c), expert evaluations (e.g. Clarkson & Arkins, 2007; Juarez et al., 2011; Weiss et al.,
2010, 2011), and physiological measurements (Anzalone et al., 2015; Baddoura & Venture,
2013;Weiss et al., 2011). Many of these methods and techniques are often applied without
first-hand experience of the interaction situation and they often are conducted afterwards. This
might bias the validity of the conclusions. The interpretation of the physiological
measurements poses problems because you cannot be sure that what is being measured is
causatively connected with what is being assessed, e.g. pupil dilation can indicate many
different things. Another related aspect is the possible gap between the objectives and
principles of different UX evaluation methods and their practical application. This creates a
risk for misunderstanding the factors evaluated, resulting in biased outcomes. There are two
aspects of this: on the one hand, the evaluator needs to have proper knowledge and
understanding about the method and its intended use. On the other hand, the participants in
the UX study need to have an understanding of the criteria they are evaluating. This means
that each evaluation method or technique, no matter how trusted, needs to be regularly
assessed to ensure validity and reliability, irrespective of whether hedonic or pragmatic
qualities are being evaluated (Lindblom & Andreasson, to appear).
To design for a high quality interaction as the basis of positive UX, the design process should
include the whole cycle of central activities; these are analysis, design, implementation, and
evaluation (Anderson, et al., 2010; Hartson & Pyla, 2012). A current trend in HRI research
concerning UX of socially interactive robots is to focus on UX evaluation and examination of
UX. Evaluation is a crucial activity in the UXD process, and research-based guidance for
robot developers of socially interactive robotic products can be valuable. Likewise, similar
guidance regarding the other UXD activities should be beneficial. Therefore, more theoretical
as well as empirical research is needed to provide a proper toolbox to robot developers. Many
open issues still remain to be addressed (Dautenhahn, 2007b, 2013; Lindblom & Andreasson,
to appear).
Product development projects often have limited resources and development projects for
robotic products are no exception in that respect. Therefore, it is important to decide what is
most critical to achieve in terms of UX so the efforts are concentrated upon the key aspects as
well as how to determine when the goals are sufficiently met. To the best of our knowledge,
no research has been conducted on establishment and prioritization of UX requirement for
socially interactive robots. Moreover, for robotic products one has to consider the business
perspective in the UXD process, i.e., to stay attuned to the business case in the development
project (Anderson, et al., 2010; Hartson & Pyla, 2012; Kaasinen et al., 2015). Similarly, to our
knowledge, there is no research that has addressed the problem of including the business
perspective in UXD of socially interactive robotic products.
To conclude, socially interactive robots are expected to be increasingly important in daily life
of more and more people. The experiences that humans have when interacting with such
robotic products have the power to enable, or disable, the robots’ acceptance rate in society.
The current research directions for studying UX of socially interactive robots are not enough.
By employing various existing methods and techniques in HCI and UX evaluation, we can
narrow the identified gap and synthesize the different approaches to social interaction in HRI
identified by Dautenhahn (2007a, see Figure 1). More research directions are needed in order
to accomplish long-term, wide-spread success of socially interactive robots. If the issues and
the challenges that we have raised in this paper are addressed and we believe that they can
be there are good reasons to be optimistic that what we know about UX can successfully
be applied to HRI. As a result, HRI research will benefit and the future users of social robots
will benefit even more.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
16
Acknowledgments
The authors wish to thank Professor David Vernon for input and valuable comments on the
topics addressed in this paper. This work is financially supported by the Knowledge
Foundation, SIDUS project AIR (Action and intention recognition in human interaction with
autonomous systems).
References
Anderson, J., McRee, J., Wilson, R., & the Effective UI Team (2010). Effective UI.
Sebastopol, CA: O'Reilly.
Anzalone, S.M., Boucenna, S., Ivaldi, S. & Chetouani, M. (2015). Evaluating the engagement
with social robots. International Journal of Social Robotics, 7(4), 465-478.
Baddoura, R. & Venture, G. (2013). Social vs. Useful HRI: Experiencing the familiar,
perceiving the robot as a sociable partner and responding to its actions. International
Journal of Social Robotics, 5(4), 29–547.
Bartneck, C., & Hu, J. (2004). Rapid prototyping for interactive robots. In The 8th conference
on intelligent autonomous systems (IAS-8) (pp. 136-145).
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009). Measurement Instruments for the
Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of
Robots. International Journal of Social Robotics, 1(1), 71-81.
Beňuš, Š. (2014). Social aspects of entrainment in spoken interaction. Cognitive Computation,
6(4), 802-813.
Benyon, D. (2010). Designing interactive systems: A comprehensive guide to HCI and
interaction design. Harlow, England: Pearson.
Berander, P., & Andrews, A. (2005). Requirements prioritization. In: A. Aurum & C. Wohlin
(Eds.), Engineering and managing software requirements (pp. 69-94). Berlin, Germany:
Springer.
Bevilacqua, R., Felici, E., Marcellini, F., Glende, S., Klemcke, S., Conrad, I., Esposito, R.,
Cavallo, F., & Dario, P. (2015). Robot-era project: Preliminary results on the system usability.
In: A. Marcus (Ed.): DUXU 2015, Part III, LNCS 9188, pp. 553–561.
Breazeal C. (2003). Toward sociable robots. Robotics and Autonomous Systems, 42(3-4), 167-
175.
Callejas, Z., Griol, D. & López-Cózar, R. (2014). A framework for the assessment of
synthetic personalities according to user perception. International Journal of Human-
Computer Studies, 72(7), 567-583.
Carcagnì, P., Cazzato, D., Del Coco, M., Leo, M., Pioggia, G. & Distante, C. (2014). Real-
time gender based behavior system for human-robot interaction. In: M. Beetz et al. (Eds.):
ICSR 2014, LNAI 8755, pp. 74–83. Springer International Publishing.
Cesta, A., Cortellessa, G., Orlandini, A. & Tiberio, L. (2016). Long-term evaluation of a
telepresence robot for the elderly: Methodology and ecological case study. International
Journal of Social Robotics, 1-21.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
17
Clarkson, E. & Arkin, R. C. (2007). Applying heuristic evaluation to human-robot interaction
systems (pp. 44-49). In: FLAIRS Conference, May, Key West, Florida, US.
Cohen, I., Looije, R. & Neerincx, M.A. (2014). Child’s perception of robot’s emotions:
Effects of platform, context and experience. International Journal of Social Robotics, 6(4),
507-518.
Datta, C., Kapuria, A., & Vijay, R. (2011). A pilot study to understand requirements of a
shopping mall robot. 6th ACM/IEEE International Conference on Human-Robot Interaction
(HRI) (pp 127-128). March 6-9, 2011, Lausanne, Switzerland.
Datta, C., MacDonald, B.A., Jayawardena, C., & Kuo I-H. (2012). Programming behaviour of
a personal service robot with application to healthcare. In: S.S. Ge, O. Khatib, J-J. Cabibihan,
R. Simmons, M-A. Williams (Eds.) Social robotics: Proceedings of 4th International
Conference, ICSR 2012. Chengdu, China, October 29-31, 2012. Lecture Notes in Computer
Science, 7621, 228-237.
Dautenhahn, K. (2007a). Socially intelligent robots: Dimensions of human-robot interaction.
Philosophical Transactions of the Royal Society B, 362(1480), 679-704.
Dautenhahn, K. (2007b). Methodology and themes of human-robot interaction: a growing
research field. International Journal of Advanced Robotic Systems, 4(1), 103-108.
Dautenhahn, K. (2013). Human-Robot Interaction. In: M. Soegaard & R.F. Dam (Eds.). The
Encyclopedia of Human-Computer Interaction, 2nd Ed. Aarhus, Denmark: The Interaction
Design Foundation. Retrieved August 25, 2013, http://www.interaction-
design.org/encyclopedia/human-robot_interaction.html
Dautenhahn, K., & Sanders, J. (2011). Introduction. In: K. Dautenhahn & J. Sanders (Eds.).
New Frontiers in Human-Robot Interaction. Amsterdam, Netherlands: John Benjamins
Publishing Company.
De Carolis, B., Mazzotta, I., Novielli, N., & Pizzutilo, S. (2010). Social robots and ECAs for
accessing smart environments services. Proceedings of the International Conference on
Advanced Visual Interfaces, AVI '10 (pp. 275-278). 25-29 May, 2010, Rome, Italy.
Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003). A survey of socially interactive robots.
Robotics and Autonomous Systems, 42(3-4), 143-166.
Goodman, E., Kuniavsky, M. & Moed, A. (2013). Observing the User Experience: A
Practitioner's Guide to User Research. San Francisco, CA: Morgan Kaufmann Publishers,
Goodrich, M.A., & Schultz, A.C. (2007). Human-robot interaction: A survey. Foundations
and Trends in Human–Computer Interaction, 1(3), 203-275.
de Graaf, M.M.A., & Allouch, S.B. (2013). Exploring influencing variables for the acceptance
of social robots. Robotics and Autonomous Systems, 61(12), 1476-1486.
de Graaf, M.M.A., Allouch, S.B. & Klamer, T. (2015a). Sharing a life with Harvey: Exploring
the acceptance of and relationship-building with a social robot. Computers in Human
Behavior, 43, 1-14.
de Graaf, M.M.A., Allouch, S.B. & van Dijk J.A.G.M. (2015b). What makes robots social?: A
user’s perspective on characteristics for social human-robot interaction In: A. Tapus et al.
(Eds.): ICSR 2015, LNAI 9388, pp. 184–193.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
18
Hall, J., Tritton, T., Rowea, A., Pipe, A., Melhuish, C. & Leonards, U. (2014). Perception of
own and robot engagement in human–robot interactions and their dependence on robotics
knowledge. Robotics and Autonomous Systems, 62(3), 392-399.
Hartson, R., & Pyla, P.S. (2012). The UX Book: Process and guidelines for ensuring a quality
user experience. Morgan Kaufmann, Amsterdam.
Hassenzahl, M. (2010). Experience Design – Technology for All the Right Reasons. San
Rafael, CA: Morgan & Claypool.
Hassenzahl, M. (2013). User Experience and Experience Design. In: M. Soegaard & R.F.
Dam (Eds.). The Encyclopedia of Human-Computer Interaction, 2nd Ed. Aarhus, Denmark:
The Interaction Design Foundation. Retrieved September 15, 2013, from
http://www.interaction-
design.org/encyclopedia/user_experience_and_experience_design.html
Hassenzahl, M., Diefenbach, S., & Göritz, A. (2010). Needs, affect, and interactive products–
Facets of user experience. Interacting with computers, 22(5), 353-362.
Hassenzahl, M., & Roto, V. (2007). Being and doing: A perspective on user experience and
its measurement. Interfaces,72, 10-12.
Hassenzahl, M., & Tractinsky, N. (2006). User experience a research agenda. Behaviour &
Information Technology, 25(2), 91-97.
He, H., Ge, S.S., & Zhang, Z. (2014). A saliency-driven robotic head with bio-inspired
saccadic behaviors for social robotics. Autonomous Robots, 36(3), 225-240.
Holthaus, P. & Wachsmuth, S. (2014). The Receptionist Robot. HRI’14 (p. 329), March 3–6,
2014, Bielefeld, Germany.
Huang, C-M. & Mutlu, B. (2014). Multivariate evaluation of interactive robot systems.
Autonomous Robots, 37(4), 335-349
Huang, C-C., Bardzell, J., & Terrell, J. (2011). Can your pet rabbit read your email? A critical
analysis of the Nabaztag Rabbit. Proceedings of the 2011 Conference on Designing
Pleasurable Products and Interfaces, DPPI '11, June, 22-25, Milano, Italy.
ISO 9241-11 (1998). Guidance on Usability, International Organization for Standardization:
Geneva, Switzerland.
ISO 9241-210 (2010). Human-Centred Design for Interactive Systems, International
Organization for Standardization: Geneva, Switzerland.
Jia, H., Wu, M., Jung, E., Shapiro, A., & Sundar, S.S. (2013). When the tissue box says
“Bless you!”: Using speech to build socially interactive objects. CHI '13 Extended Abstracts
on Human Factors in Computing Systems, CHI EA '13 (pp. 1635-1640). April 27 – May 2,
2013, Paris, France.
Johnson, D.O., Cuijpers, R.H., Juola, J.F., Torta, E., S. M., Frisiello, A., Bazzani, M., Yan,
W., Weber, C., Wermter, S., Meins, N., Oberzaucher, J., Panek, P., Edelmayer, G., Mayer, P.,
& Beck, C. (2014). Socially assistive robots: A comprehensive approach to extending
independent living. International Journal of Social Robotics, 6(2), 195-211.
Johnson, D.O., Cuijpers, R.H., Pollmann, K. & van de Ven, A.A.J. (2016). Exploring the
entertainment value of playing games with a humanoid robot. International Journal of Social
Robotics, 8(2), 247-269.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
19
Jokinen, K. & Wilcock, G. (2015). Modelling user experience in human-robot interactions. In:
R. Böck et al. (Eds.): MA3HMI 2014 Workshop, LNAI 8757, pp. 45-56.
Juarez, A., Bartneck, C., & Feijs, L. (2011). Using semantic technologies to describe robotic
embodiments. Proceedings of the 6th ACM/IEEE International Conference on Human-Robot
Interaction (HRI), March 6-9, 2011, Lausanne, Switzerland (pp 425-432).
Jung, J., Bae, S-H. Lee, J.H., & Kim, M-S. (2013). Make it move: A movement design
method of simple standing products based on systematic mapping of torso movements &
product messages. Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, CHI '13 (pp. 1279-1288). April 27 – May 2, 2013, Paris, France.
Kaasinen, E., Roto, V., Hakulinen, J., Heimonen, T., Jokinen, J. P. P., Karvonen, H.,
Keskinen, T., Koskinen, H., Lu, Y., Saariluoma, P., Tokkonen, H. & Turunen, M. (2015).
Defining user experience goals to guide the design of industrial systems. Behaviour &
Information Technology, 34(10), 976-991.
Karreman, D., Ludden, G. & Evers, V. (2015). Visiting cultural heritage with a tour guide
robot: A user evaluation study in-the-wild. In: A. Tapus et al. (Eds.): ICSR 2015, LNAI 9388,
pp. 317-326.
Kim, Y. & Mutlu, B. (2014). How social distance shapes human–robot interaction.
International Journal of Human-Computer Studies, 72(12), 783-795.
Kotonya, G. & Sommerville, I. (1998). Requirements engineering: Processes and techniques.
Chichester, England: John Wiley and Sons.
Körtner, T., Schmid, A., Batko-Klein, D. & Gisinger, C. (2014). Meeting Requirements of
Older Users? Robot Prototype Trials in a Home-like Environment. In: C. Stephanidis and M.
Antona (Eds.): UAHCI/HCII 2014, Part III, LNCS 8515, pp. 660–671.
Kuo, I-H., Jayawardena, C., Broadbent, E., & MacDonald, B.A. (2011). Multidisciplinary
design approach for implementation of interactive services: Communication initiation and
user identification for healthcare service robots. International Journal of Social Robotics,
3(4), 443-456.
Lindblom, J. & Andreasson, R. (to appear). Current challenges for UX evaluation of human-
robot interaction. To be presented at the 7th International Conference on Applied Human
Factors and Ergonomics (AHFE), 27-31 July, 2016, Orlando, Florida, USA. To appear in the
7th AHFE Conference Proceedings as well as in Lecture Notes in Computer Science.
Lohse, M., Hanheide, M., Wrede, B., Walters, M. L., Koay, K. L., Syrdal, D. S. &
Severinson-Eklundh, K. (2008). Evaluating extrovert and introvert behaviour of a domestic
robot—a video study (pp. 488-493). In: RO-MAN 2008: The 17th IEEE International
Symposium on Robot and Human Interactive Communication, 2008, 1-3 August, 2008,
Münich, Germany.
Lohse, M., Siepmann, F. & Wachsmuth, S. (2014). A Modeling framework for user-driven
iterative design of autonomous systems. International Journal of Social Robotics, 6(1), 121-
139.
Loth, S., Jettka, K., Giuliani, M. & de Ruiter, J.P (2015). Ghost-in-the-Machine reveals
human social signals for human–robot interaction. Frontiers in Psychology, 6, 1-20.
Matsumoto, T., Seko, S., Aoki, R., Miyata, A., Watanabe, T. & Yamada, T. (2015). Towards
enhancing human experience by affective robots: Experiment and discussion. Proceedings of
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
20
the 24th IEEE International Symposium on Robot and Human Interactive Communication
(pp. 271-276), Kobe, Japan, Aug 31 - Sept 4, 2015.
Moon, A.J., Zheng, M., Troniak, D.M., Blumer, B., Gleeson, B., MacLean, K., Pan, M.K.X.J.
& Croft, E.A. (2014). Meet me where I’m gazing: How shared attention gaze affects human-
robot handover timing. HRI'14 (pp. 334-341), March 3–6, 2014, Bielefeld, Germany.
Nielsen, J. (1994). Heuristic evaluation. In: J. Nielsen and R.L. Mack (Eds.), Usability
Inspection Methods (pp. 25-62). New York: John Wiley & Sons.
Nunez, E., Matsuda, S., Hirokawa, M. & Suzuki, K. (2015). Humanoid robot assisted training
for facial expressions recognition based on affective feedback. In: A. Tapus et al. (Eds.): ICSR
2015, LNAI 9388, pp. 492–501.
Oh, K., & Kim, M. (2010). Social attributes of robotic products: Observations of child-robot
interactions in a school environment. International Journal of Design, 4(1), 45-55.
Partala, T., & Kallinen, A. (2012). Understanding the most satisfying and unsatisfying user
experiences: Emotions, psychological needs, and context. Interacting with Computers, 24, 25-
34.
Pressman, R.S. (2000). Software engineering: A practitioner’s approach (5th ed.). London:
McGraw Hill.
Rabbitt, S.M., Kazdin, A.E. & Scassellati, B. (2015). Integrating socially assistive robotics
into mental healthcare interventions: Applications and recommendations for expanded use.
Clinical Psychology Review, 35, 35–46
Saini , P., de Ruyter, B., Markopoulos, P., & van Breemen, A. (2005). Benefits of social
intelligence in home dialogue systems. In: M.F. Costabile & F. Paternò (Eds.) Human-
Computer Interaction - INTERACT 2005: Proceedings IFIP TC13 International Conference,
September 12-16, 2005 Rome, Italy. Lecture Notes in Computer Science, 3585, 510-521.
Šabanović, S. & Chang W-L. (2015). Socializing robots: constructing robotic sociality in the
design and use of the assistive robot PARO. AI & Society, 1-15.
dos Santos, T.F., de Castro, D.G., Masiero, A.A. & Aquino Junior, P.T. (2014). Behavioral
persona for human-robot interaction: A study based on pet robot. In: M. Kurosu (Ed.):
Human-Computer Interaction, Part II, HCII 2014, LNCS 8511, pp. 687–696.
Schlögl, S., Doherty, G. & Luz, S. (2015). Wizard of Oz experimentation for language
technology applications: Challenges and tools. Interacting with Computers, 27(6) 592-615.
Scholtz, J. (2003). Theory and evaluation of human robot interaction. Proceedings of the 36th
Hawaii International Conference of System Sciences (HICSS’03) - Track 5 - Volume 5.
Schroeter, C., Mueller, S., Volkhardt, M. Einhorn, E., Huijnen, C., van den Heuvel, H., van
Berlo, A., Bley, A., & Gross, H-M. (2013). Realization and user evaluation of a companion
robot for people with mild cognitive impairments. IEEE International conference on Robotics
and Automation (ICRA) (pp. 1153-1159). Karlsruhe, Germany, May 6-10, 2013.
Strasser, E., Weiss, A., & Tscheligi, M. (2012). Affect misattribution procedure: An implicit
technique to measure user experience in HRI. Proceedings of the seventh annual ACM/IEEE
international conference on Human-Robot Interaction HRI '12 (pp. 243-244). March 5-8,
2012, Boston, Massachusetts, USA.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
21
Sumioka, H., Nishio, S., Minato, T., Yamazaki, R. & Ishiguro, H. (2014). Minimal human
design approach for sonzai-kan media: Investigation of a feeling of human presence.
Cognitive Computation, 6(4), 760-774.
Sutcliffe, A. (2002). User-centred requirements engineering: Theory and practice. London:
Springer.
Syrdal, D.S., Otero, N., & Dautenhahn, K. (2008). Video prototyping in human-robot
interaction: Results from a qualitative study. Proceedings of the 15th European conference on
Cognitive ergonomics: the ergonomics of cool interaction, ECCE '08, September 16-19, 2008,
Madeira, Portugal.
Thrun, S. (2004). Toward a framework for human-robot interaction, Human-Computer
Interaction, 19(1), 9-24.
Tritton, T., Hall, J., Rowe, A., Valentine, S., Jedrzejewska, A., Pipe, A.G., Melhuish, C. &
Leonards, U. (2012). Engaging with robots while giving simple instructions. In: G. Herrmann,
M. Studley, M. Pearson , A. Conn, C. Melhuish, M. Witkowski, J-H.n Kim & P. Vadakkepat
(Eds.) Advances in Autonomous Robotics: Joint Proceedings of the 13th Annual TAROS
Conference and the 15th Annual FIRA RoboWorld Congress, Bristol, UK, August 20-23,
2012, Lecture Notes in Computer Science Volume 7429, 176-184.
Weiss, A., Bernhaupt, R., & Yoshida, E. (2009a). Addressing user experience and societal
impact in a user study with a humanoid robot. Proceedings of the Symposium on New
Frontiers in Human-Robot Interaction, AISB2009 (pp. 150-157).
Weiss, A., Bernhaupt, R., Laukes, M., & Tscheligi, M. (2009b). The USUS evaluation
framework for human-robot interactions. Proceedings of the symposium on new frontiers in
human-robot interaction, AISB 2009 (pp. 158–165). April 8-9, 2009, Edinburgh, Scotland.
Weiss, A., Bernhaupt, R., Schwaiger, D., Altmaninger, M., Buchner, R. & Tscheligi, M.
(2009c). User experience evaluation with a Wizard of Oz approach: Technical and
methodological considerations (pp. 303-308). In: 9th IEEE-RAS International Conference on
Humanoid Robots, 2009, December, 2009, Paris, France.
Weiss, A., Wurhofer, D., Bernhaupt, R., Altmaninger, M. & Tscheligi, M. (2010). A
methodological adaptation for heuristic evaluation of HRI (1-6).In: RO-MAN 2010:
Proceedings of the 19th IEEE International Symposium on Robot and Human Interactive
Communication, September, Viareggio, Italy.
Weiss, A., Bernhaupt, R., Tscheligi, M. (2011). The USUS evaluation framework for user-
centered HRI. In: K. Dautenhahn and J. Saunders (Eds.), New Frontiers in Human–Robot
Interaction (pp. 89-110), Amsterdam: John Benjamins.
Xu, Q., Ng, J., Cheong, Y. L., Tan, O., Wong, J. B., Tay, T. C., & Park, T. (2012). The role of
social context in human-robot interaction (pp. 1-5). In: Network of Ergonomics Societies
Conference (SEANES), July, 2012, Langkawi, Kedah, Malaysia.
Xu, Q., Li, L., & Wang, G. (2013). Designing engagement-aware agents for multiparty
conversations. CHI 2013 (pp. 2233-2242). April 27 – May 2, 2013, Paris, France.
Xu, Q., Ng, J., Cheong, Y.L., Tan, O., Wong, J.B., Tay, T.C., & Park, T. (2012). The role of
social context in human-robot interaction. Southeast Asian Network of Ergonomics Societies
Conference (SEANES), Langkawi, Kedah, July 9-12, 2012.
Alenljung, B., Lindblom, J., Anderasson, R. & T. Ziemke
International Journal of Ambient Computing and Intelligence (IJACI), 2017, 8(2), 12-31.
22
Xu, Q., Ng, J., Tan, O., Huang, Z. Tay, B. & Park, T. (2015a). Methodological issues in
scenario-based evaluation of human-robot interaction. International Journal of Social
Robotics, 7, 279-291.
Xu, Q., Ng, J.S.L., Tan, O.Y. & Huang, Z. (2015b). Needs and attitudes of Singaporeans
towards home service robots: A multi-generational perspective. Universal Access in the
Information Society, 14(4), 477-486.
Yanco, H.A., & Drury, J. (2004). Classifying human-robot interaction: an updated taxonomy.
IEEE International Conference on Systems, Man and Cybernetics 2004, Vol. 3 (pp. 2841-
2846). Oct. 10-13, 2004.
Young, J.E., Sung, J.Y., Voida, A., Sharlin, E., Igarashi, T., Cristensen, H.I., & Grinter, R.E.
(2011). Evaluating human-robot interaction: Focusing on the holistic interaction experience.
International Journal of Social Robotics, 3, 53-67.
Zowghi, D., & Coulin, C. (2005). Requirements elicitation: A survey of techniques,
approaches, and tools. In: A. Aurum and C. Wohlin (Eds.), Engineering and managing
software requirements (pp. 21-46). Berlin: Springer.
... The ongoing increase of new interactive technologies in our daily lives results in growing expectations of these technologies from their intended users [15]. One such interactive technology is social robots, and their degree of participation in everyday activities in society are becoming more sophisticated [2,5,20]. Social robots have several characteristics that separate them from other kinds of more traditional technologies; they occupy space, act rather autonomously, and users have to respond to them "here and now." ...
... A relevant HRI angle is the intersection with the study of UX, which allows for a user-centered perspective, including studying users' expectations. Although this intersection of UX within HRI exists, and is getting more traction in the literature [2,17,31,36], it is often overlooked [2]. There are also overlaps between users' experience and users' expectations when interacting with technology [33]. ...
... A relevant HRI angle is the intersection with the study of UX, which allows for a user-centered perspective, including studying users' expectations. Although this intersection of UX within HRI exists, and is getting more traction in the literature [2,17,31,36], it is often overlooked [2]. There are also overlaps between users' experience and users' expectations when interacting with technology [33]. ...
Chapter
Full-text available
Social robots are designed in manners that encourage users to interact and communicate with them in socially appropriate ways, which implies that these robots should copy many social human behaviors to succeed in social settings. However, this approach has implications for what humans subsequently expect from these robots. There is a mismatch between expected capabilities and actual capabilities of social robots. Expectations of social robots are thus of high relevance for the field of Human-Robot Interaction (HRI). While there is recent interest of expectations in the HRI field there is no widely adapted or well formulated evaluation framework that offers a deeper understanding of how these expectations affect the success of the interaction. With basis in social psychology, user experience, and HRI, we have developed an evaluation framework for studying users’ expectations of social robots. We have identified three main factors of expectations for assessing HRI: affect, cognitive processing, and behavior and performance. In our framework, we propose several data collection techniques and specific metrics for assessing these factors. The framework and its procedure enables analysis of the collected data via triangulation to identify problems and insights, which can grant us a richer understanding of the complex facets of expectations, including if the expectations were confirmed or disconfirmed in the interaction. Ultimately, by gaining a richer understanding of how expectations affect HRI, we can narrow the social robot expectation gap and create more successful interactions between humans and social robots in society.
... Naturalistic field studies are recommended to understand how users respond to the technology in real-life conditions. Alenljung et al. [4] explain that common methods of user evaluation in HRI include scenario-based evaluation, questionnaires, interviews and focus groups, Wizard-of-Oz studies, expert evaluations, and physiological measurements. ...
... Since some of the richest data collected regarding the user experience cannot be collected via surveys, and since both Lindblom et al. [45] and Alenljung et al. [4] mention the advantages of open-ended questions and interviews, we supplement our surveys with both. Users in both of our studies wrote responses to open-ended questions and then called over an experimenter and verbally explained their answers. ...
Preprint
Full-text available
Hugs are complex affective interactions that often include gestures like squeezes. We present six new guidelines for designing interactive hugging robots, which we validate through two studies with our custom robot. To achieve autonomy, we investigated robot responses to four human intra-hug gestures: holding, rubbing, patting, and squeezing. Thirty-two users each exchanged and rated sixteen hugs with an experimenter-controlled HuggieBot 2.0. The robot's inflated torso's microphone and pressure sensor collected data of the subjects' demonstrations that were used to develop a perceptual algorithm that classifies user actions with 88\% accuracy. Users enjoyed robot squeezes, regardless of their performed action, they valued variety in the robot response, and they appreciated robot-initiated intra-hug gestures. From average user ratings, we created a probabilistic behavior algorithm that chooses robot responses in real time. We implemented improvements to the robot platform to create HuggieBot 3.0 and then validated its gesture perception system and behavior algorithm with sixteen users. The robot's responses and proactive gestures were greatly enjoyed. Users found the robot more natural, enjoyable, and intelligent in the last phase of the experiment than in the first. After the study, they felt more understood by the robot and thought robots were nicer to hug.
... In other words, to increase user acceptance of MARs, what functionality should be additionally developed and from what points of view the functionality should be evaluated. The importance of user experiences (UX) and social interactions between the MARs and users have been emphasized on a gradual basis (Alenljung et al., 2017;Bhattacharjee et al., 2020;De Graaf & Allouch, 2013;Nishiwaki & Yano, 2008;Tanaka et al., 2010). ...
... According to Dautenhahn (2007), HRI research is divided into three categories: (a) a robot-centered view that uses an autonomous robot, (b) a robot cognition-centered view that solves problems by considering a robot as an intelligent system, and (c) a human-centered view that adapts robot behavior to humans. Alenljung et al. (2017) emphasized that social HRI should be designed, especially in a human-centered view; it is most important to provide a positive UX. Clarkson and Arkin (2007) proposed eight new heuristics in HRI research by using Nielsen's (1994, April) canonical list, HRI guidelines suggested by Scholtz (2002), and elements of the ambient and CSCW (Computer-Supported Cooperative Work) heuristics (Baker et al., 2002;Mankoff et al., 2003). ...
Article
Meal-assistance robots (MARs) help feed meals to users with disabilities or who need help consuming their meals. Although such MARs have been introduced in various fields, guidelines for evaluating MARs are limited. This study aims to develop comprehensive guidelines to assess the user experience (UX) of MAR designs considering its social interaction characteristics between humans and robots. Participants from three groups (patients, doctors, and caregivers) with different perspectives on MARs were recruited and a focus group interview was conducted to collect their UXs with MARs. The three groups showed different UXs with MARs in user interface design, robot arm motion, and safety and mobility. In addition, based on the literature review, eight UX features (usability, emotion, value, naturalness, assistance, acceptance, personality, and culture) are proposed to evaluate MAR interfaces. The proposed comprehensive design guideline will be particularly useful in evaluating and designing the UX of MARs.
... Robotics is defined as a computer or computer program that is built to perform human tasks [8,17]. A positive user experience is achieved when people can interact with robots in the same way that they interact with each other [18]. Alenljung et al. [20] concluded that user experience is influenced by the responsiveness, robustness and trickiness of the interaction with a robot. ...
... User experience is critical to the success of any product. Technological products, for example, must provide a positive user experience so that users will want to use them and obtain the intended benefits. To achieve positive user experience, a thoughtful design process and systematic evaluation are needed (Garrett, 2011:p.9-17;Alenljung et al., 2017). Garrett's framework is a guide to prototype design and feedback and provides a wide scope for user experience analysis (compared to measuring how "easy to use" a product is). The basic structure of his framework includes visual design, information and interaction design, functional specifications, and user needs (Garrett, 2011). Accor ...
Conference Paper
Full-text available
Cultural background of end users is usually neglected in the interface and product design as well as in the development process. This has led to poor usability and reduced quality of user experience of early products. This paper presents the effect of the cultural issues on the user experience of older adults from the Philippines and Portugal when using software, in particular mobile games. In our research, we investigated (i) what user experiences the two prototype mobile games evoke in elderly users; (ii) which features of the prototypes influence the participants' user experience; and (iii) how the cultural dimensions of the two countries affect the implementation of the study and the user experience of the participants with poor exposure to technology. In this qualitative study, we analyze and elaborate on the cultural dimensions of Portugal and the Philippines. In our discussion, we also refer to the findings of the previous studies on similar games conducted in Finland and China. We observed that gaming provided new experience for older adults and improved their confidence related to technology, along with the similarity of competitiveness, cheering, and usability during the trials. An unexpected finding was the negative effect that inappropriate assistive aids had on user experience among Filipino participants. The main goal of this paper is to propose a guide for developers, designers, and creators for a more user-centred design approach that views culture as an important part of the development process. In line with this, we highly encourage to consider cultural background in game design including the implementation of gaming into elderly care routines.
... Robotics is defined as a computer or computer program that is built to perform human tasks [8,17]. A positive user experience is achieved when people can interact with robots in the same way that they interact with each other [18]. Alenljung et al. [20] concluded that user experience is influenced by the responsiveness, robustness and trickiness of the interaction with a robot. ...
Chapter
Robotics, robotic processes, artificial intelligence and machine learning are concepts that create uncertainty among people. Technological expansion influences the preconceived, adverse feelings of dislike that users are harboring. This uncertainty influences the acceptance of software robots that are implemented in systems used by employees. This may also be the case regarding the interaction of banking employees with automated information systems used to perform their daily work activities. It is against this background that a framework for interaction design is required. Such a framework could guide designers to consider employed users and establish a positive, accepting interaction. Due to the lack of existing interaction design frameworks aimed at addressing the implementation of software robots, the study aims to answer the question: Which core components will contribute to a framework that will guide the interaction design of a software robotic system in the banking sector? An initial systematic review approach was performed to identify the influence of robotics on user experience. This literature review has identified principles in other industries that were pulled through to financial institutions. This study contributes to filling the gap in academic knowledge by exploring related literature to identify and evaluate applicable principles and integrating these into a new synthesized framework. The result of the study is a new framework called an interaction design dome. The dome can be used to guide software robot design and development from a human-computer interaction viewpoint, with specific reference to the financial sector.
Article
Higher education (HE) in Kuwait suffers from high dropout rates, and one of the leading causes of this desertion is a lack of academic advising. Effective academic advising systems must fulfill its user’s needs and provide a positive user experience (UX), which allows users to successfully and effectively fulfill their goals. This study used quantitative and qualitative approaches to evaluate an electronic advising system (e-Advisor) used by the Public Authority for Applied Education and Training (PAAET). Survey data from 1,095 PAAET students were analyzed to determine students' perceptions and evaluate their experience using e-Adviser. The findings indicated that students were aware of the features and advantages of e-Advisor. Overall, the participants had a moderately positive impression of e-Advisor’s UX. The system’s attractiveness, efficiency, and perspicuity were rated slightly higher than its dependability, stimulation, and novelty. Moreover, female students’ perceptions of e-Advisors UX were significantly more favorable than their male counterparts.
Chapter
Crossover is an important operator in genetic algorithms. Although hundreds of application dependent and independent crossover operators exist in the literature, this chapter provides holistic, but by no means an exhaustive, overview of different crossover techniques used in different variants of genetic algorithms. We will review some of the commonly used crossover operators in binary-coded genetic algorithms as well as in real-coded genetic algorithms and explore the use cases and performance of different techniques for different applications to provide a better understanding of the types of bias exhibited by different crossover operators. This knowledge can be useful when designing an algorithm for a specific problem, particularly if there are known patterns or dependencies in the selected representation.
Chapter
This chapter presents the concept of evolutionary optimally designed machine learning models for end devices monitoring the operational condition of agriculture vehicles (AgVs). The ultimate target is an App usable in normal smartphones as IoT sensory device for monitoring healthy condition of agricultural machinery. Since smartphones use a small integrated computing unit with limited computational resources, the lighter system, the more favorable. Due to that, in evolutionary machine learning, a meta-heuristic optimization is deployed for the optimal choice of machine learning model structure to construct a proper lightweight model usable on smartphones. The chapter reviews some very recent works of evolutionary lightweight machine learning models in health monitoring of agriculture machinery.
Chapter
This chapter illustrates the role of evolutionary optimization in designing AI end-devices to monitor the efficiency of agriculture vehicles (AgVs) mainly on the field via economic sound-based IoT sensors. Due to the application of AI on end-devices, there is a certain limitation for memory and complexity of the deployed algorithms. In such a condition, a machine learning model with optimal structure is of favorite. Lightweight is an aspect of the model as its model structure is optimized for operation of minimum complexity but an acceptable efficiency. The chapter explains that how this target can be achieved by the deployment of metaheuristic evolutionary optimizers. The AI model with optimum complexity and structure is suitable especially for deployment on smartphones. The optimization assists the designer in achieving not only the lightweight structure but also a maximized efficiency for recognition via built-in economic sensors such as smartphone microphones.
Article
Full-text available
If you have picked up this book and are browsing the Preface, you may well be asking yourself"What makes this book different from the large number I can find on amazon. com?". Well, the answer is a blend of the academic and the practical, and views of the subject you won't get from anybody else: how psychology and linguistics influence the field of requirements engineering (RE). The title might seem to be a bit of a conundrum; after all, surely requirements come from people so all requirements should be user-centred. Sadly, that is not always so; many system disasters have been caused simply because requirements engineering was not user-centred or, worse still, was not practised at all. So this book is about putting the people back into com­ puting, although not simply from the HCI (human-computer interaction) sense; instead, the focus is on how to understand what people want and then build appropriate computer systems.
Article
Full-text available
To improve the way humans are interacting with robots various factors have to be taken into account. An evaluation framework for Human-Robot Collaboration with humanoid robots addressing usability, social acceptance, user experience, and societal impact (abb. USUS) as evaluation factors is proposed (see figure 1). The theoretical framework USUS is based on a multi-level indicator model to operationalize the evaluation factors. Evaluation factors are described and split up into several indicators, which are extracted and justified by literature review. The theoretical factor indicator framework is then combined with a methodological framework consisting of a mix of methods derived and borrowed from various disciplines (HRI, HCI, psychology, and sociology). The proposed method mix allows addressing all factors within the USUS framework and lays a basis for understanding the interrelationship of the USUS Factors.
Chapter
Full-text available
The development of socially interactive robots is expected to have an increasing importance in everyday life for a growing number of people. For social robots to provide long-term added value to people’s lives, it is of major importance to stress the need for developing a positive user experience (UX). The human-centered view emphasizes various aspects including acceptance, usability, and credibility, as they emerge in the interaction between humans and robots. In current human-robot interaction (HRI) research, UX is reckoned to be important, but is often taken for granted. However, a positive user experience does not appear by itself but has to be systematically designed and evaluated. In this paper, we focus on the role and relevance of UX in HRI and present three challenges related to the evaluation of UX in interaction with robots, addressing the need for interdisciplinary research in order to achieve long-term success of socially interactive robots.
Article
Full-text available
Telepresence robotic systems are proposed in different contexts and specifically in the area of social robotics for assisting older adults at home. Similarly to other robotic systems, such robots are often designed and then evaluated in laboratory settings for a limited period of time. Lab-based evaluations present limitations because they do not take into account the different challenges imposed by the fielding of robotic solutions into real contexts for longer periods. In order to perform long-term experiments in real ecological settings it is very important to define a structured approach to assess the impact of a prolonged and constant use of the telepresence robot. This paper proposes a methodology in the area of elderly people support, called MARTA, for M ultidimensional A ssessment of telepresence R obo T for older A dults. It introduces the main variables of interest as well as the instruments and administration timeline for assessing relevant changes that may occur over time. MARTA is also validated in a one year-long case study during which a telepresence robot, called Giraff, has been deployed and iteratively assessed. The paper also provides remarks on the technology readiness and suggestions for its improvements.
Conference Paper
In human-human interaction, the participants’ multimodal behaviour has impact on the interaction as a whole, and similarly in spoken human-robot interactions, the interlocutors’ multimodal signals seem to correlate with the user’s experience and impressions of the interaction. We explored in more detail how some aspects of multimodal behaviour (gazing, facial expressions, body posture) can predict the user’s evaluation of the robot’s behaviour (Responsiveness, Expressiveness, Interface, Usability, Overall impression). The results indicate that the user’s assessment concerning the evaluation categories Interface and Usability, and to some extent the categories Expressiveness and Overall correlate with their behaviour in a statistically significant manner. The work contributes to our understanding of how the interlocutors’ engagement and active participation relate to their assessment of the success of communication, and points towards automating evaluation of human-robot interactions.
Conference Paper
A prototype of an assistive robot for older people was tested in three different countries in life-like lab settings. A sample of potential older users with different grades and types of age-related impairments completed a sequence of tasks with the robot. Subsequently, usability issues, user acceptance, and their willingness to pay for such a robot (affordability) were assessed to find out if the robot caters to the needs of the impairment groups. Main results of the data analyses were: ease of use was deemed satisfactory by the majority of participants. Task speed was considered to be rather slow. Additionally, it could be shown that participants were sceptical of buying a robot for their own use, but would be willing to rent one. A significant difference in classifying the robot prototype as helpful for the home was found in participants with mobility impairments compared to participants without mobility impairments.
Conference Paper
The European project Robot-Era is an ambitious integrated project (FP7-ICT-2011.5.4), which objective is the development of advanced robotic services, integrated in intelligent environments, to provide independent living to older people. In order to guarantee the matching of the users’ need and the demands, two loops of experimentation were conceived, in realistic and real setting. The aim of the paper is to described the methods applied and the main results coming from the first experimental loop, concerning the degree of usability of the interfaces and provide guidelines for testing socially assistive robots with older people.
Conference Paper
In this paper we present a user evaluation study on location at the Royal Alcázar in Seville, Spain, with the fully autonomous tour guide robot FROG. In this robot, technological innovations in navigation and vision were integrated with state-of-the-art design for robot behavior in order to provide interactive tours and adaptive content to visitors. In our user evaluation study we aimed to gain insights in user experiences of and attitudes and responses towards this fully autonomous social robot. Such studies are important, because they provide information about how people interact with social robots outside a controlled setting. Invited as well as spontaneous visitors followed tours guided by FROG and were interviewed about their opinions and experiences. Our findings indicate that even if isolated technical features work perfectly in controlled settings, they might not work well in the integrated system, because naïve people interact with the system in an unforeseen manner.