Content uploaded by Tom Williams
Author content
All content in this area was uploaded by Tom Williams on Jan 20, 2022
Content may be subject to copyright.
Practical, Ethical, and Overlooked:
Teleoperated Socially Assistive Robots
in the Quest for Autonomy
Saad Elbeleidy
MIRRORLab
Colorado School of Mines
Golden, CO, USA
selbeleidy@mines.edu
Terran Mott
MIRRORLab
Colorado School of Mines
Golden, CO, USA
terranmott@mines.edu
Tom Williams
MIRRORLab
Colorado School of Mines
Golden, CO, USA
twilliams@mines.edu
Abstract—Socially Assistive Robots (SARs) show significant
promise in a number of domains: providing support for the
elderly, assisting in education, and aiding in therapy. Perhaps un-
surprisingly, SAR research has traditionally focused on providing
evidence for this potential. In this paper, we argue that this focus
has led to a lack of critical reflection on the appropriate level of
autonomy (LoA) for SARs, which has in turn led to blind spots
in the research literature. Through an analysis of the past five
years of HRI literature, we demonstrate that SAR researchers are
overwhelmingly developing and envisioning autonomous robots.
Critically, researchers do not include a rationale for their choice
in LoA, making it difficult to determine their motivation for fully
autonomous robots. We argue that defaulting to research fully
autonomous robots is potentially short-sighted, as applying LoA
selection guidelines to many SAR domains would seem to warrant
levels of autonomy that are closer to teleoperation. We moreover
argue that this is an especially critical oversight as teleoperated
robots warrant different evaluation metrics than do autonomous
robots since teleoperated robots introduce an additional user, the
teleoperator. Taken together, this suggests a mismatch between
LoA selection guidelines and the vision of SAR autonomy found
in the literature. Based on this mismatch, we argue that the next
five years of SAR research should be characterized by a shift in
focus towards teleoperation and teleoperators.
Index Terms—robot teleoperation, wizard of oz, autonomy,
socially assistive robots
I. INTRODUCTION
Socially Assistive Robots (SARs) show significant promise
in supporting individuals across a variety of domains. SARs
can support people receiving therapy [1], [2], [3], [4], students
in education [5], [6], [7], and elderly individuals [8], [9].
Across these fields, SARs are able to provide support through
social interactions with individuals in need of assistance.
Much of the foundational work on SARs has been motivated
by a vision of a future with fully autonomous SARs [10], [11]
but this is not the only path that SAR researchers could have
chosen. Alternatively, SAR researchers could have chosen
to focus their research efforts on developing teleoperation
interfaces that were accessible to non-expert users, and that
were better suited to handle the complexity of SAR domains.
These non-expert users are often caregivers who are already
providing the assistance that individuals need. The distinction
between developing better autonomy or better teleoperation
interfaces is critical since these two research paths suggest
fundamentally different futures for caregivers, with caregivers
as teleoperators or supervisors.
Fully autonomous SARs and teleoperated SARs represent
two distinct endpoints on the scale of possible levels of auton-
omy. Over the past decade, frameworks have been developed
for selecting the ideal Level of Autonomy (LoA) in a given
domain [12]. We argue that these frameworks can be leveraged
to facilitate critical reflection on the field’s choice to focus on
autonomous rather than teleoperated SARs.
LoA researchers argue that for each domain, a robot’s LoA,
from fully teleoperated to fully autonomous, must be carefully
selected. LoA selection guidelines suggest that three primary
dimensions should be considered when deciding what LoA
is appropriate: task criticality, task accountability, and envi-
ronmental complexity [12]. When robots are used for highly
critical tasks that have potential for human safety concerns,
less autonomous robots are recommended due to these safety
concerns. When robots are used in tasks which require a clear
chain of accountability, less autonomous robots are recom-
mended so that blame can be appropriately attributed. When
the robot’s environment is dynamic and complex, the use
of autonomy needs to be carefully considered because using
autonomous robots requires advanced sensing capabilities, and
because a human supervisor may be needed anyway when
important aspects of the environment are unpredictable.
These guidelines should be used to reflect on the field’s LoA
choices. Since LoAs are domain-specific, this first requires
an understanding of the domains in which SARs are used
in recent literature, which are broader than those that were
considered in foundational SAR research. With an understand-
ing of the domains for which SARs are being developed, one
could determine the recommended range of LoAs for SARs
and compare them to the LoAs chosen in current research.
In this paper we follow precisely this research plan to
critically reflect on the LoA choices made in recent SAR
literature. Our results show that the SARs community (as
viewed through the lens of work published in the Human-
Robot Interaction field’s primary conference and journal) is
overwhelmingly focused on fully autonomous SARs, while
working in domains in which teleoperation would be better
justified according to LoA selection guidelines. Researchers
generally do not include a rationale for their choice in LoA and
seem to instead be defaulting to a vision of fully autonomous
SARs. Based on these findings, we argue that the next five
years of SAR research should be characterized by a shift
in focus towards teleoperation. Moreover, we provide recom-
mendations for all SAR researchers to ensure that not only
are researchers acknowledging and specifying their choice in
LoAs, but that those LoAs are appropriately motivated.
II. MOT IVATIO N
A. Levels of Autonomy (LoA)
In this paper we use Beer et al.’s definition of autonomy:
“The extent to which a robot can sense its envrionment, plan
based on that environment, and act upon that environment with
the intent of reaching some task-specific goal without external
control” [12]. Describing autonomy as an ”extent” implies
the existence of a continuous range of autonomy containing
multiple Levels of Autonomy (LoAs). This theory of levels of
autonomy builds on the theory of levels of automation [13] by
additionally considering robot capabilities such as social inter-
action [12]. To understand appropriate LoA choice for SARs,
we used the guidelines proposed by Beer et al. [12]. These
guidelines were designed to help robot developers identify the
ideal Level of Autonomy (LoA) for their application domain,
from fully autonomous to fully teleoperated1. Following these
guidelines, researchers determine the appropriate LoA for a
robot in a particular application domain by making several
key considerations.
As described earlier, these guidelines suggests researchers
examine three dimensions of the domain in which the robot
will be deployed: task criticality, task accountability, and
environment complexity. Importantly, these three decision
dimensions correspond to the three central robot capabilities
of sensing, planning, and acting. That is, environmental com-
plexity mainly influences sensing; task accountability mainly
influences planning; and task criticality mainly influences
acting. And, just as sensing, planning, and acting are invariably
entwined, so too these factors must be considered in tandem
when evaluating a particular domain, as they are certain to
influence each other. We will now consider how each of these
three dimensions influences the choice of an ideal LoA.
1) Task Criticality: Task criticality is central to choosing a
robot’s LoA because of the relationship between automation
and task failure. Automation has direct consequences for task
failure rates [14], and increased automation can introduce
unique risks in the context of highly critical tasks [15], [16].
Moreover, even when autonomy is otherwise low, introduc-
ing temporary increases in autonomy can negatively impact
an operator’s Situation Awareness and their ability to exert
1Or, at limit (though not considered in this work), perhaps no robot at all.
control [17]. Robots with less autonomy are recommended in
domains with highly critical tasks.
2) Task Accountability: When task errors occur, it is impor-
tant to identify who should be held accountable so that future
errors can be mitigated. Task accountability can therefore also
influence the appropriate LoA. When robots are perceived as
more autonomous, people give them more credit or blame for
the resulting outcomes of a task [18]. In clinical environments,
for example, some have been reluctant to adopt automated
technologies since they would be liable for errors caused by
those technologies [19]. Robots with less autonomy are rec-
ommended within domains where accountability is important.
3) Environmental Complexity: Finally, a robot’s environ-
mental context can also influence its ideal LoA. Robots
deployed in complex and dynamic environments would require
higher sensing capabilities if they are to be autonomous [20].
However, even given high sensing capabilities, a high LoA
may only be justifiable when a complex environment is
predictable. When an environment is unpredictable, a robot
may need to be teleoperated or, at minimum, supervised [21].
These three dimensions can be used to carefully guide the
selection of a robot’s Level of Autonomy with respect to
sensing, planning, and acting, and in turn, an overall LoA
for the robot as a whole [12].
B. Perspectives on SAR Autonomy
Much of the foundational work on SARs has been moti-
vated by a vision of a future with fully autonomous robots.
This vision was influenced by the challenges of practically
deploying teleoperated SARs and the perceived limitations
of teleoperation interfaces. Feil-Seifer and Matari´
c [10] ar-
gued for autonomous SARs in order to minimize training
and operation difficulty for non-expert users. Similarly, Scas-
sellati et al. [11] argued that SAR teleoperation might be
infeasible given the complexity of the domains in which SARs
were to be deployed. As such, they viewed teleoperation as a
short-term solution but impractical for wide-scale deployment
and adoption. And at first glance, the field has overwhelmingly
continued within this autonomous tradition. In fact, some
have recently argued that SAR research continues to rely
on too much teleoperation [22]. However, some of those
same researchers who advocated for a vision for autonomous
SARs, have published arguments that could suggest a need
for teleoperation. Specifically, Matari´
c [23] has advocated
for SARs to support caregivers through human augmentation
rather than automation.
Importantly, researchers using teleoperated SARs today may
be using them for vastly different reasons and motivated by
vastly different futures. One researcher may use teleopera-
tion as a present-time cost reduction approach to research
that is motivated by an autonomous future. Whereas another
researcher may be using teleoperated SARs because of a
belief that teleoperated SARs are the appropriate LoA for
their domain both in the present and the future. Similarly, this
applies to researchers currently developing autonomous SARs.
One may be developing autonomous SARs now motivated by
a vision for future autonomous SARs. However, others may be
developing autonomous SARs now solely to overcome present
challenges in teleoperation. As such, it is unclear whether the
SAR community is still working towards a vision of fully
autonomous robots; and moreover, it is unclear whether, based
on the LoA selection guidelines described above, researchers
should continue to move in this direction.
In this paper, we seek to answer these open questions by
analyzing the state of SAR research with regards to Levels
of Autonomy. Since LoA choice is domain and application-
specific, we first review recent SAR literature to answer (RQ1)
What SAR domains are most prevalent in recent research?
We then examine recent work in those domains to answer
(RQ2) What LoAs are researchers currently applying to SARs
and envisioning for the future of SARs? Next, following LoA
selection guidelines, we then answer (RQ3) What range of
LoAs would be recommended for SARs in the most commonly
researched domains? Finally, using our answers to these ques-
tions, we address a final overarching question: (RQ4) Is there
a mismatch between LoAs chosen in the literature and LoAs
recommended by LoA selection guidelines?
III. QUAL ITATI VE APPROACH
To answer these questions, we conducted a literature review
in which we examined all papers mentioning Socially Assistive
Robots that were published within the past five years in the
Proceedings of the ACM/IEEE International Conference on
Human-Robot Interaction (HRI) or in the ACM Transactions
on Human-Robot Interaction (T-HRI). This literature review
identified 20 papers from HRI and 26 papers from T-HRI that
matched our criteria.
To maintain sensitivity to the fact that different types of
perspectives might be held in different parts of the HRI
community, we first coded these papers into four categories:
•User studies - Papers that focus on a human subjects /
user study and its results.
•Design - Papers that focus on design approaches or
introducing a novel robot design.
•Technical Advances - Papers that focus on novel com-
putational techniques.
•Ethics / Analysis - Papers that focus on applying an
ethical framework or presenting an argument through
analysis without experimentation.
Within each of these categories we coded for themes that
could answer our research questions. To answer RQ1, we
coded papers for the domains in which the presented SARs
were intended to be deployed. To answer RQ2, we coded
papers for the current LoA used for their robots (and if
provided, why), and the future LoA motivating the work
(if specified). To answer RQ3, we applied LoA selection
guidelines to the domains identified in answering RQ1 (see
Section V). To answer RQ4, we compare our answers from
RQ2 and RQ3 and present our discussion in Section VIII.
We began with a set of codes we expected to find in the
literature as determined by the first and third author. The first
author then conducted an initial coding of all papers, creating
additional labels as necessary throughout the coding process.
Finally, the first and third author discussed all labels and paper
codes and arrived at a consensus. Eleven of the 46 papers
were excluded for lack of relevance. The remaining papers are
referenced in Table I at the end of this document. As shown in
Figure 1, the remaining 35 papers consisted of 20 User Studies
papers, 7 Design papers, 5 Technical Advances papers, and 3
Ethics / Analysis papers. These numbers are proportional to
what we might expect, given that user studies have comprised
about 50% of papers in recent HRI conferences.
IV. SAR DOMAINS IN REC EN T RESEARCH
Across paper types, SAR research papers covered diverse
domains in which robots provided social assistance. We identi-
fied 16 themes of SAR domains that fell into six major groups:
•Target Age - Papers that focused on assisting individuals
based on their age. Subthemes: Children, Elderly
•Activities - Papers that focused on assisting individuals
with a particular activity. Subthemes: Education, Therapy,
Fitness, Interventions, Playing Games, Activities of Daily
Living
•Needs or Disabilities - Papers that focused on assist-
ing with resulting needs. Subthemes: Autism, Dementia,
Parkinson’s, Depression, Loneliness, Non-verbal Commu-
nication
•Location - Papers that focused on assisting individuals
based on their location. Subthemes: Home, Work
•General - Miscellaneous papers that targeted general
SAR applications
Based on our grouping shown in Figure 2, we can see
that recent SAR research has been predominantly focused
on providing assistance in conducting particular activities or
assistance for target age groups. Figure 3 shows that SARs
in recent research cover a wide range of domains. Assisting
elderly individuals, children with their education, and individ-
uals in therapy were the most prevalent domains. As such, we
have chosen to focus our analysis on these three domains.
Fig. 1: Paper frequency by research category.
Fig. 2: Paper frequency by SAR domain theme.
Fig. 3: Paper frequency by SAR domain subtheme
V. APPLYING LOA SEL EC TI ON GUIDELINES
Now that we have identified the most prevalent SAR do-
mains in recent research, we can determine recommended
robot LoAs in these domains. Our aim here is to determine
broad ranges of recommended LoAs for each of these three
domains; either closer to teleoperation or closer to full auton-
omy. Because we are specifying ranges, when we state low
LoAs we mean a focus on teleoperation with possibility for
some autonomy, and when we state high LoAs we mean a
focus on autonomy with possibility for some teleoperation.
As described above, LoA choice is not binary.
A. The Elderly
When SARs are used to support the elderly, they can be
used for companionship, engagement, supporting activities of
daily living, health guidance, initial health evaluations, and
therapy [8], [9]. When interacting with a companion SAR,
elderly individuals experience high levels of attachment [24]
and decreased levels of loneliness [25], [26]. Moreover, SARs
have been shown to reduce many symptoms of dementia
[27], [28], [29] and increase assisted individuals’ cognitive
activity [30], decrease response time [31], increase social
engagement [32], and improve overall quality of life [33].
These are especially important benefits as SARs are often used
to support elderly individuals who suffer from Dementia or
Mild Cognitive Impairment (MCI).
When supporting the elderly in companionship, task crit-
icality is fairly low, and the need for accountability is also
fairly low. However, when supporting elderly individuals with
MCI or Dementia, task criticality may increase since these
individuals are more vulnerable. Task accountability becomes
important as well since if an issue were to occur, it would be
important to identify its cause so that it can be prevented in the
future. When working with the elderly, stationary robots are
often used and these robots generally exist in a controlled or
static environment; therefore environmental complexity is low.
Based on this brief analysis we would expect companionship
robots to have higher LoA and robots supporting individuals
with MCI or Dementia to have low LoAs.
B. Education
SARs have also been successfully deployed in educational
domains, which traditionally rely on social interaction [5], [6],
[7]. SARs can help children learn a variety of different skills,
including handwriting [34], sports [35], drama [36], arithmetic,
mathematics, and science [37], [6] sign language [38] and
spoken second languages [39]. Not only are robots capable
of teaching these skills, but in some cases the introduction of
a SAR can result in increased engagement [37], [39], more
learning gains [40], [41] and more efficient learning [42].
Children’s education is a crucial part of their upbringing
and socialization. When SARs are deployed in education
contexts they may have varying roles. When robots are dis-
pensing educational content, task criticality may vary greatly
based on educational topic and based on the likelihood of
erroneously communicating information. For example, content
from quantitative topics such as mathematics may be easier to
correct or verify than content from socially-oriented subjects
such as history. Similarly, task accountability is therefore
dependent on the topic of choice. Since education involves
children, and children are a vulnerable population, parents will
often view task accountability as high. Finally, a classroom
environment may vary in its complexity. Some classrooms
are highly dynamic environments that are fairly unpredictable.
Autonomous robots may require high sensing capabilities to
navigate classroom interactions but even these robots, based
on the suggestions by LoA researchers, would benefit from
a human supervisor (likely the teacher or teacher’s assistant
in this case). However, when educational SARs are used in
the home, environmental complexity may be lower. Based
on our analysis of the three dimensions, we would expect
SARs used in education to vary greatly in their level of
autonomy. While in many educational contexts a high LoA
might be recommended, we expect SARs used in teaching
highly sensitive content to have a low LoA.
C. Therapy
Similar to education, therapy is another domain that may in-
volve a high degree of social interaction. SARs have been used
in a variety of different therapies, such as rehabilitation [1], [2]
and mental health [3], [4] SARs have also been used with great
success in therapy with specific populations, including autistic
individuals [11], [43], [44], children in pain or distress [45],
[4], and children with cerebral palsy [46]. Therapy is an
umbrella domain that covers a variety of different needs.
By the very nature of therapy, task criticality is high since it
is likely supporting a recovering and/or vulnerable individual.
For example, if a SAR is supporting an individual receiving
rehabilitative physical therapy by motivating them, motivating
the individual to continue with an exercise that they are poorly
executing may have severe consequences on rehabilitation.
Task accountability is also high since trust and accountability
in clinical settings are crucial. As we mentioned previously,
clinicians are sometimes reluctant to adopt automated tech-
nologies for fears of misplaced blame [19]. Environmental
complexity, however, may vary greatly depending on the
type of therapy. A physical therapy or gym environment
may contain substantial complexity and dynamic interactions,
however a talk therapy environment may be fairly static. Given
the high task criticality and accountability we would expect
SARs in therapy to have low LoAs resulting in a fair amount
of research discussing teleoperated robots.
Our brief analysis of the most prevalent SAR domains
suggests that in many cases an LoA that involves at least some
teleoperation would be recommended. Next, we will present
our findings about LoAs chosen in the literature to see whether
the LoAs chosen align with this recommendation.
VI. CURRENT AND ENVISIONED FUTURE LOAS I N
REC EN T SAR RESEARCH
Through our literature analysis, we identified the LoAs
of the robots used in the papers examined and researchers’
rationale for their choice in LoA. We also identified, when
possible, the LoAs that served as motivation for researchers;
that is, whether researchers envisioned future SARs to be
teleoperated, autonomous, or somewhere in between. Note that
as previously discussed, the LoA of a robot used in a research
paper does not imply the same LoA for the researchers’
envisioned future. Each paper was coded with respect to
whether the robots used were described as fully autonomous,
semi-autonomous,teleoperated,both autonomous and tele-
operated, not explicit or unclear LoA, whether the authors
were ambivalent about the robot’s LoA, or whether the LoA
was not applicable because no robot was used in the paper.
A. Current LoA in SAR Research
A plurality (15 out of 35 papers) of recent research on
SARs investigates the use of autonomous robots as shown in
Figure 4. By comparison, eight papers cover teleoperation,
with six of them using Wizard of Oz style teleoperation and
two using clearly teleoperated robots. The clearly teleoperated
robots were used in domains that necessitated teleoperation,
since they were teleoperated by the assisted individual, which
we will discuss in more detail in Section VII-B. Importantly,
a large number of papers (5) do not explicitly mention their
chosen LoA. This shows how some SAR researchers are often
focused on evaluating the assistive impact that robots have
without discussing the robots themselves.
Fig. 4: Current LoA of SARs in Recent Research
B. Researcher’s Rationale for LoA in SAR Research
The plurality of papers (17 out of 35), as shown in Figure 5,
did not include a rationale for their LoA choice. The 10
papers labeled as N/A for the presence of LoA rationale are
comprised of the five papers labeled as not explicit or unclear,
and the five papers labeled as N/A (with respect to current
LoA). Since only 8 papers presented their rationale for LoA,
we will briefly detail each rationale. Of the eight papers that
mentioned LoA rationale, four used autonomous robots, three
used Wizard of Oz teleoperation and one was not explicit about
their LoA choice but still presented a rationale surrounding
robot autonomy. The papers’ rationales are as follows:
Autonomous 1: A user study conducted to compare the
efficacy of autonomous robots to teleoperated robots when
used with elderly individuals suffering from Dementia. This
study showed that autonomous robots may be superior in that
context [47].
Autonomous 2: A design methods paper that showed that
therapists and technology developers wanted a stand-alone
solution that did not require technical expertise and did not
hinder therapist activities [48].
Autonomous 3: A user study on autonomous robots for cog-
nitive stimulation of elderly individuals that used autonomous
robots to avoid human facilitation for adults living alone [49].
Autonomous 4: A technical advances in robot autonomy pa-
per about autonomous SARs for individuals with Parkinson’s
Disease that briefly states that autonomy will be necessary
when robots are deployed in homes [50].
Teleoperation 1: A user study that described Wizard of Oz
as a way of verifying robot effects before investing in robot
autonomy [51].
Teleoperation 2: A novel robot design paper that presented
teleoperation as an accessible method for robot control by non-
technical HRI researchers [52].
Teleoperation 3: A novel robot design paper that used
Wizard of Oz to enact realistic system responsiveness to
demonstrate the robot’s effectiveness [53].
Not explicit LoA 1: A user study that was not explicit about
their robot’s LoA but argued that while there is a common
Fig. 5: Presence of Rationale for SAR LoA Choice
Fig. 6: Future LoA of SARs Motivating Recent Research
vision for autonomous robots in the SAR community, their
necessity is questionable [54].
C. Future LoA in SAR Research
As described previously, we also identified researchers’
vision for LoAs of future SARs. Researchers are mostly either
not explicit (16 of 35) or ambivalent (11 of 35) about the
LoA they envision for future SARs within their domain. When
researchers are explicit, most (11 out of 13) expect a future
with autonomous robots. Of the remaining two papers that
expect a future of teleoperated systems, one involved telep-
resence robots and the other involved a robot that delivered
its assistance through teleoperation by the assisted individual;
both applications that necessitate at least some teleoperation.
VII. TRENDS IN LOA DE PL OYM EN T
In this section, we detail patterns we identified about the
two major LoAs (autonomous or teleoperated). Specifically we
describe how robots are developed, how they are used and by
whom, and examples of domains in which they are deployed.
Importantly, in this section, we focus on the users of SARs and
how they differ across chosen LoAs. We present a diagram of
the different modes of deployment in Figure 7.
(a) Deploying fully autonomous SARs: The robot interacts with the
assisted individual directly providing assistance.
(b) Deploying teleoperated SARs: The teleoperation mode where the
SAR is teleoperated by a caregiver who delivers assistance through
a teleoperating interface that allows them to control the robot.
(c) Deploying teleoperated SARs: The teleoperation mode where the
assisted individual uses a teleoperation interface to operate the SAR
and receive assitance.
Fig. 7: Three modes of deploying SARs.
A. Trends in Autonomous SAR Deployment
When using a fully autonomous SAR, the robot is often
left with the assisted individual and can autonomously provide
the assistance the individual needs. These robots are often left
with the elderly or with children to support them with compan-
ionship, cognitive stimulation, or education. To develop these
robots, developers usually collaborate with an assistive expert
to ensure that the robot is capable of meeting these needs.
As part of developing these robots, researchers verify the
robot’s autonomy by evaluating its assistive capabilities. Tech-
nical advances papers about autonomous SARs introduce ways
of implementing the robot autonomy. In one paper, however,
instead of describing the development of the autonomous
robot, researchers propose a development language for non-
expert coders to program robot autonomy [55].
Limitations: When using this approach, developers often
have to limit the scope of activities of the robots in order to
ensure they are effective.
User spotlight: For autonomous robots, the assisted indi-
vidual is the primary user of the robot and other stakeholders
have minimal interactions with the robot.
B. Trends in Teleoperated SAR Deployment
When SARs are teleoperated they can be characterized as
either of two teleoepration modes: teleoperated by a caregiver,
or teleoperated by the assisted individual.
Mode 1: Teleoperation by Caregiver
When the assisted individual is receiving assistance from
a teleoperated robot, the robot is commonly teleoperated by
a caregiver in real-time. Research implementing this mode
mostly focused on evaluating the assistive impact of the robot.
These papers implied that the real desired outcome would be to
have a fully autonomous SAR. However, because the desired
technology was not yet available or was too costly for the
purpose of the experiment, researchers teleoperated the robots
instead. Examples of these papers explored interactions such
as incorporating a robot mediator for conflict in children, or
controlling a plant robot that can support individuals dealing
with depression. Some papers describing their implementation
of this mode were design papers introducing a novel robot de-
sign. In those papers, the teleoperation interface was described,
almost as a requirement, to explain to other researchers how
to use the robot as motivation for the robot’s adoption. No
papers implementing this mode had focused discussion of how
their SAR was teleoperated, suggesting that these researchers
approached SAR teleoperation as an undesirable necessity.
Limitations: This approach requires teleoperation interfaces
that enable caregivers to achieve the necessary control over
the robots. This approach also requires the presence and
availability of another human; the caregiver.
User spotlight: In this mode, the interaction is really
between the caregiver and the assisted individual as mediated
by the physical robot and the teleoperation interface. The
physical robot is designed to meet the assisted individual’s
needs whereas the teleoperation interface is designed to meet
the caregiver’s teleoperation needs.
Mode 2: Teleoperation by Assisted Individual
In the second teleoperation mode, SARs were operated by
the individual receiving assistance. Examples of this mode
include when robots were used as communication devices for
non-vocal individuals or as telepresence robots controlled by
the assisted individual. Papers that implemented this mode
present robots that are accessible to the individual receiving
the assistance and are effective at creating behaviors that
meet their assistive needs. In these cases, teleoperation is a
necessary component of the robot; while some autonomy may
be introduced, these robots could not be fully autonomous.
Limitations: When using this approach, developers have
to design and build a teleoperation interface for the assisted
individual to achieve the necessary control over the robot.
User spotlight: In this mode, the interaction is between the
assisted individual and other individuals or the environment as
mediated by the physical robot and the teleoperation interface.
Both the robot and the teloperation interface in this case are
designed to meet the assisted individual’s needs.
VIII. DISCUSSION
A. SAR researchers are clearly interested in fully autonomous
SARs but rarely present a rationale for autonomy
Current SAR research is predominantly focused on fully
autonomous SARs. Researchers seem confident in the ability
of autonomous technologies to deliver social assistance. Most
researchers already explicitly describe how their robots are
controlled; however, it is also necessary that they explain why.
Researchers may use teleoperation as a proof of concept
approach to investigate assistive capabilities or because it
is the more appropriate solution. We encourage researchers
conducting WoZ style experiments to describe why they are
using a WoZ system and clarify if they expect that to be the
eventual real world usage of these robots.
Autonomy may sometimes be the appropriate choice for
a SAR. Autonomous robots can provide assistance in sce-
narios when no human is available. Autonomy may also
be a reasonable choice until teleoperation capabilities are
available [11]. In some cases, autonomous robots may be
used in comparison to teleoperated robots to determine which
LoA is more assistive [47]. However, this research would
then raise important questions for the community to answer,
such as: (1) How are we measuring effectiveness of SARs?
(2) Which stakeholders are we considering (or not consider-
ing)? (3) When autonomous robots are evaluated positively in
comparison to teleoperated robots, is this due to the benefits
of the particular autonomy used, or the shortcomings of
the teleoperation interface used? (4) How do we compare
autonomous robots and teleoperated robots on equal footing?
Papers where researchers use the same LoA may appear to
envision the same future for SARs. However, as we have said
there are multiple reasons for a specific choice in LoA, whose
rationales might be motivated by vastly different visions for
the future. Researchers need to explicitly state both their LoA
choices and their visions for the future, not only to allow other
researchers to critically assess and evaluate their research, but
also because of the impact these explicit statements can have
on the field. By explaining their rationale, researchers motivate
and encourage additional research by their colleagues. This
effect can be seen years later as we have demonstrated in this
paper: early SAR research suggesting autonomy was followed
by the majority of SAR papers a decade later implementing
autonomous robots in their research.
B. Autonomy is not always the right answer, and teleoperation
is often the recommended LoA
As discussed in Section V, lower LoAs are often more
appropriate in commonly studied SAR domains. Even in
domains where full autonomy is achievable, teleoperation by
a caregiver may often be a more appropriate choice. This
is especially true in assistive contexts like children’s therapy
which are highly sensitive and in which the consequences of
mistakes can be dire. If mistakes happen in these domains,
it is critical for blame and accountability to be appropriately
ascribed. Questions of accountability for autonomous robots
are longstanding in academic literature and do not have easy
answers. When teleoperation is used, the ascription of blame is
more obvious. Additionally, there are already human experts in
these domains who are explicitly trained to adapt to unusual
situations and mitigate potential harms. Roboticists can and
should rely on these human experts, rather than replacing
them [23]. In this way, teleoperated robots can leverage human
expertise while keeping power in their hands.
SARs are in practice teleoperated by caregivers [56]. Not
only is this an effective way of delivering assistance but it
is also a practical step towards the widespread adoption of
robots in assistance contexts. That is, the caregivers who would
be teleoperating these robots are the ones already providing
assistance to individuals today. Deploying teleoperated SARs
is a more feasible way of deploying robots in the near future
and to immediately benefit vulnerable communities.
C. Teleoperation changes design objectives and target users
When SARs are mostly autonomous, their target user is the
assisted individual. Interactions with caregivers are generally
limited to extracting their expertise ahead of time and trans-
lating that into robot autonomy. When SARs are teleoperated,
they introduce additional stakeholders and design factors. Most
importantly, teleoperated SARs introduce a teleoperation inter-
face that a caregiver can use to control the robot. Since evalua-
tion of SAR assistance includes the impact on caregivers [57],
and caregivers would be more involved if they are teleoperat-
ing the robot, factors surrounding interface evaluation should
also be considered such as Situation Awareness, workload, and
latency. In summary, researchers’ choice of robot LoA requires
careful consideration, and requires prioritizing different users
and understanding these users’ needs.
D. Ethical Implications of LoA Choices
By considering the users of teleoperated SARs, we identify
new implications for the decision to develop fully autonomous
SARs. In most scenarios, the SAR teleoperator is a caregiver;
an expert trained to provide assistance who can use the SAR
to deliver assistance more effectively. By replacing caregivers
with fully autonomous robots, developers potentially cause
harm by shifting power away from domain experts. The
replaced caregiver is expertly equipped to handle critical
tasks, complex environments, and offer accountability for
their actions; the precise scenarios for which LoA guidelines
suggest human reliance. By replacing the human in the loop,
roboticists not only make it difficult for these individuals to
earn a living, but also lower the quality of assistive interaction
for the robots’ potentially vulnerable users.
IX. LIMITATIONS
In this paper, we used LoA selection guidelines to determine
appropriate ranges of LoA for several domains. However, the
process for LoA selection as outlined by Beer et al. [12] does
not end with the selection of a range. Researchers identifying
the recommended LoA for an application would need to
continue following the remaining guidelines. Additionally,
we used guidelines for levels of robot autonomy, whereas,
more recent research suggests using levels of human control
abstractions (LHCA) [58]. Since LoA selection guidelines
suggest more teleoperation in SAR domains and therefore
more attention given to teleoperators (caregivers), it could be
advantageous to use that same lens to determine the recom-
mended level for human control. However, our choice in using
an LoA framework rather than the LHCA framework was
due to the LHCA framework being framed primarily around
missions involving unmanned aerial drones. More research
is needed into alternative frameworks like LHCA to make
clear how they can be applied within broader Human-Robot
Interaction contexts. Finally, we only examined work from
HRI and T-HRI in the last five years. Whereas SAR research
is published across a wide variety of venues [5], [2], [59],
[60], [61], [62], [63].
X. CONCLUSION
Most Socially Assistive Robotics research assumes we are
working towards a future where SARs are fully autonomous.
Our analysis of recent Socially Assistive Robotics research
shows a mismatch between the recommended LoA for SAR
domains and the chosen LoA in research about these domains.
We show that while the research community is interested in
fully autonomous SARs, they do not present clear justification
for this choice, which research has shown to not necessarily
be the appropriate choice. Following guidelines from LoA
researchers, we show that teleoperated SARs are the appro-
priate choice for the majority of SAR domains. We argue
that in addition to being a more appropriate LoA choice to
serve individuals in many SAR domains, teleoperation is also
more respectful to caregivers currently providing assistance
and support, and more likely to lead to widespread SAR
adoption that can transfer the researched benefits of SARs to
a wider audience. We therefore argue that the next five years
of SAR research should be characterized by a shift in focus
towards teleoperation and teleoperators.
TABLE I: Papers Included in Literature Analysis
Venue Year Publications
HRI 2017 [64], [65], [54]
HRI 2018 [66], [51], [67], [68], [69], [70], [71]
HRI 2019 [72], [73]
HRI 2020 [47], [74], [75], [55], [76], [77], [78]
HRI 2021 [79], [80], [81], [82]
T-HRI / JHRI22017 [36]
T-HRI 2018 [48], [83], [84]
T-HRI 2019 [52], [85], [86]
T-HRI 2020 [50], [87]
T-HRI 2021 [49], [53], [88]
2NB, in 2017 T-HRI was the Journal of Human-Robot Interaction (JHRI).
REFERENCES
[1] J. Butchart, R. Harrison, J. Ritchie, F. Mart´
ı, C. McCarthy, S. Knight,
and A. Scheinberg, “Child and parent perceptions of acceptability and
therapeutic value of a socially assistive robot used during pediatric
rehabilitation,” Disability and rehabilitation, vol. 43, no. 2, pp. 163–
170, 2021.
[2] J. Casas, E. Senft, L. F. Gutierrez, M. Rincon-Rocancio, M. Munera,
T. Belpaeme, and C. A. Cifuentes, “Social assistive robots: assessing
the impact of a training assistant robot in cardiac rehabilitation,”
International Journal of Social Robotics, vol. 13, no. 6, pp. 1189–1203,
2021.
[3] S. M. Rabbitt, A. E. Kazdin, and B. Scassellati, “Integrating socially
assistive robotics into mental healthcare interventions: Applications and
recommendations for expanded use,” Clinical psychology review, vol. 35,
pp. 35–46, 2015.
[4] K. Kabaci ´
nska, T. J. Prescott, and J. M. Robillard, “Socially assistive
robots as mental health interventions for children: a scoping review,”
International Journal of Social Robotics, vol. 13, no. 5, pp. 919–935,
2021.
[5] T. Belpaeme, J. Kennedy, A. Ramachandran, B. Scassellati, and
F. Tanaka, “Social robots for education: A review,” Science robotics,
vol. 3, no. 21, 2018.
[6] F. B. V. Benitti, “Exploring the educational potential of robotics in
schools: A systematic review,” Computers & Education, vol. 58, no. 3,
pp. 978–988, 2012.
[7] L. P. E. Toh, A. Causo, P.-W. Tzuo, I.-M. Chen, and S. H. Yeo, “A
review on the use of robots in education and young children,” Journal
of Educational Technology & Society, vol. 19, no. 2, pp. 148–163, 2016.
[8] M. Ghafurian, J. Hoey, and K. Dautenhahn, “Social robots for the care
of persons with dementia: A systematic review,” ACM Transactions on
Human-Robot Interaction (THRI), vol. 10, no. 4, pp. 1–31, 2021.
[9] R. Van Patten, A. V. Keller, J. E. Maye, D. V. Jeste, C. Depp, L. D.
Riek, and E. W. Twamley, “Home-based cognitively assistive robots:
maximizing cognitive functioning and maintaining independence in
older adults without dementia,” Clinical Interventions in Aging, vol. 15,
p. 1129, 2020.
[10] D. Feil-Seifer and M. J. Mataric, “Defining socially assistive robotics,” in
9th International Conference on Rehabilitation Robotics, 2005. ICORR
2005. IEEE, 2005, pp. 465–468.
[11] B. Scassellati, H. Admoni, and M. Matari´
c, “Robots for use in autism
research,” Annual review of biomedical engineering, vol. 14, 2012.
[12] J. M. Beer, A. D. Fisk, and W. A. Rogers, “Toward a framework for
levels of robot autonomy in human-robot interaction,” Journal of human-
robot interaction, vol. 3, no. 2, p. 74, 2014.
[13] M. R. Endsley and D. B. Kaber, “Level of automation effects on
performance, situation awareness and workload in a dynamic control
task,” Ergonomics, vol. 42, no. 3, pp. 462–492, 1999.
[14] J. Carlson, R. R. Murphy, and A. Nelson, “Follow-up analysis of
mobile robot failures,” in IEEE International Conference on Robotics
and Automation, 2004. Proceedings. ICRA’04. 2004, vol. 5. IEEE,
2004, pp. 4987–4994.
[15] R. Parasuraman, T. B. Sheridan, and C. D. Wickens, “A model for types
and levels of human interaction with automation,” IEEE Transactions
on systems, man, and cybernetics-Part A: Systems and Humans, vol. 30,
no. 3, pp. 286–297, 2000.
[16] R. Parasuraman and C. D. Wickens, “Humans: Still vital after all these
years of automation,” Human factors, vol. 50, no. 3, pp. 511–520, 2008.
[17] M. R. Endsley, “Situation awareness in future autonomous vehicles:
Beware of the unexpected,” in Congress of the International Ergonomics
Association. Springer, 2018, pp. 303–309.
[18] T. Kim and P. Hinds, “Who should i blame? effects of autonomy and
transparency on attributions in human-robot interaction,” in ROMAN
2006-The 15th IEEE International Symposium on Robot and Human
Interactive Communication. IEEE, 2006, pp. 80–85.
[19] P. Tiwari, J. Warren, K. J. Day, and B. MacDonald, “Some non-
technology implications for wider application of robots to assist older
people,” Health Care and Informatics Review Online, 2010.
[20] S. Thrun, “Toward a framework for human-robot interaction,” Human–
Computer Interaction, vol. 19, no. 1-2, pp. 9–24, 2004.
[21] M. Desai, K. Stubbs, A. Steinfeld, and H. Yanco, “Creating trust-
worthy robots: Lessons and inspirations from automated systems,” in
Proceedings of AISB ’09 Convention: New Frontiers in Human-Robot
Interaction, April 2009.
[22] C. Clabaugh and M. Matari ´
c, “Escaping oz: Autonomy in socially
assistive robotics,” Annual Review of Control, Robotics, and Autonomous
Systems, vol. 2, pp. 33–61, 2019.
[23] M. J. Matari´
c, “Socially assistive robotics: Human augmentation versus
automation,” Science Robotics, vol. 2, no. 4, p. eaam5410, 2017.
[24] M. R. Banks, L. M. Willoughby, and W. A. Banks, “Animal-assisted
therapy and loneliness in nursing homes: use of robotic versus living
dogs,” Journal of the American Medical Directors Association, vol. 9,
no. 3, pp. 173–177, 2008.
[25] H. Robinson, B. MacDonald, N. Kerse, and E. Broadbent, “The psy-
chosocial effects of a companion robot: a randomized controlled trial,”
Journal of the American Medical Directors Association, vol. 14, no. 9,
pp. 661–667, 2013.
[26] M. Kanamori, M. Suzuki, H. Oshiro, M. Tanaka, T. Inoguchi, H. Taka-
sugi, Y. Saito, and T. Yokoyama, “Pilot study on improvement of quality
of life among elderly using a pet-type robot,” in Proceedings 2003 IEEE
International Symposium on Computational Intelligence in Robotics and
Automation. Computational Intelligence in Robotics and Automation for
the New Millennium (Cat. No. 03EX694), vol. 1. IEEE, 2003, pp. 107–
112.
[27] N. Jøranson, I. Pedersen, A. M. M. Rokstad, and C. Ihlebaek, “Change
in quality of life in older people with dementia participating in paro-
activity: A cluster-randomized controlled trial,” Journal of advanced
nursing, vol. 72, no. 12, pp. 3020–3033, 2016.
[28] T. Shibata, “Therapeutic seal robot as biofeedback medical device:
Qualitative and quantitative evaluations of robot therapy in dementia
care,” Proceedings of the IEEE, vol. 100, no. 8, pp. 2527–2538, 2012.
[29] G. W. Lane, D. Noronha, A. Rivera, K. Craig, C. Yee, B. Mills, and
E. Villanueva, “Effectiveness of a social robot,“paro,” in a va long-term
care setting.” Psychological services, vol. 13, no. 3, p. 292, 2016.
[30] K. Wada, T. Shibata, T. Musha, and S. Kimura, “Robot therapy for
elders affected by dementia,” IEEE Engineering in medicine and biology
magazine, vol. 27, no. 4, pp. 53–60, 2008.
[31] A. Tapus, “Improving the quality of life of people with dementia through
the use of socially assistive robots,” in 2009 Advanced Technologies for
Enhanced Quality of Life. IEEE, 2009, pp. 81–86.
[32] M.-T. Chu, R. Khosla, S. M. S. Khaksar, and K. Nguyen, “Service
innovation through social robot engagement to improve dementia care
quality,” Assistive Technology, vol. 29, no. 1, pp. 8–18, 2017.
[33] M. Valent´
ı Soler, L. Ag¨
uera-Ortiz, J. Olazar´
an Rodr´
ıguez, C. Men-
doza Rebolledo, A. P´
erez Mu˜
noz, I. Rodr´
ıguez P´
erez, E. Osa Ruiz,
A. Barrios S´
anchez, V. Herrero Cano, L. Carrasco Chill ´
on et al., “Social
robots in advanced dementia,” Frontiers in aging neuroscience, vol. 7,
p. 133, 2015.
[34] D. Hood, S. Lemaignan, and P. Dillenbourg, “When children teach
a robot to write: An autonomous teachable humanoid which uses
simulated handwriting,” in Proceedings of the Tenth Annual ACM/IEEE
International Conference on Human-Robot Interaction, 2015, pp. 83–90.
[35] A. Litoiu and B. Scassellati, “Personalized instruction of physical
skills with a social robot,” in Workshops at the Twenty-Eighth AAAI
Conference on Artificial Intelligence, 2014.
[36] F. ´
A. Bravo S´
anchez, A. M. Gonz´
alez Correal, and E. G. Guerrero,
“Interactive drama with robots for teaching non-technical subjects,”
Journal of Human-Robot Interaction, vol. 6, no. 2, pp. 48–69, 2017.
[37] J. B. Janssen, C. C. van der Wal, M. A. Neerincx, and R. Looije,
“Motivating children to learn arithmetic with an adaptive robot game,”
in International conference on social robotics. Springer, 2011, pp.
153–162.
[38] H. K ¨
ose, P. Uluer, N. Akalın, R. Yorgancı, A. ¨
Ozkul, and G. Ince, “The
effect of embodiment in sign language tutoring with assistive humanoid
robots,” International Journal of Social Robotics, vol. 7, no. 4, pp. 537–
548, 2015.
[39] T. Schodde, K. Bergmann, and S. Kopp, “Adaptive robot language
tutoring based on bayesian knowledge tracing and predictive decision-
making,” in Proceedings of the 2017 ACM/IEEE International Confer-
ence on Human-Robot Interaction, 2017, pp. 128–136.
[40] J. Kennedy, P. Baxter, E. Senft, and T. Belpaeme, “Higher nonverbal
immediacy leads to greater learning gains in child-robot tutoring inter-
actions,” in International conference on social robotics. Springer, 2015,
pp. 327–336.
[41] D. Leyzberg, S. Spaulding, M. Toneva, and B. Scassellati, “The physical
presence of a robot tutor increases cognitive learning gains,” in Pro-
ceedings of the annual meeting of the cognitive science society, vol. 34,
no. 34, 2012.
[42] M. Saerbeck, T. Schut, C. Bartneck, and M. D. Janse, “Expressive
robots in education: varying the degree of social supportive behavior
of a robotic tutor,” in Proceedings of the SIGCHI conference on human
factors in computing systems, 2010, pp. 1613–1622.
[43] L. A. Dickstein-Fischer, D. E. Crone-Todd, I. M. Chapman, A. T.
Fathima, and G. S. Fischer, “Socially assistive robots: current status and
future prospects for autism interventions,” Innovation and Entrepreneur-
ship in Health, vol. 5, pp. 15–25, 2018.
[44] E. Martinez-Martin, F. Escalona, and M. Cazorla, “Socially assistive
robots for older adults and people with autism: An overview,” Electron-
ics, vol. 9, no. 2, p. 367, 2020.
[45] M. J. Trost, A. R. Ford, L. Kysh, J. I. Gold, and M. Matari´
c, “Socially
assistive robots for helping pediatric distress and pain: a review of
current evidence and recommendations for future research and practice,”
The Clinical journal of pain, vol. 35, no. 5, p. 451, 2019.
[46] M. M. Blankenship and C. Bodine, “Socially assistive robots for children
with cerebral palsy: A meta-analysis,” IEEE Transactions on Medical
Robotics and Bionics, vol. 3, no. 1, pp. 21–30, 2020.
[47] D. Cruz-Sandoval, A. Morales-Tellez, E. B. Sandoval, and J. Favela, “A
social robot as therapy facilitator in interventions to deal with dementia-
related behavioral symptoms,” in Proceedings of the 2020 ACM/IEEE
International Conference on Human-Robot Interaction, 2020, pp. 161–
169.
[48] F. Mart´
ı Carrillo, J. Butchart, S. Knight, A. Scheinberg, L. Wise,
L. Sterling, and C. McCarthy, “Adapting a general-purpose social robot
for paediatric rehabilitation through in situ design,” ACM Transactions
on Human-Robot Interaction (THRI), vol. 7, no. 1, pp. 1–30, 2018.
[49] N. Gasteiger, H. S. Ahn, C. Gasteiger, C. Lee, J. Lim, C. Fok,
B. A. Macdonald, G. H. Kim, and E. Broadbent, “Robot-delivered
cognitive stimulation games for older adults: Usability and acceptability
evaluation,” ACM Transactions on Human-Robot Interaction (THRI),
vol. 10, no. 4, pp. 1–18, 2021.
[50] J. R. Wilson, L. Tickle-Degnen, and M. Scheutz, “Challenges in de-
signing a fully autonomous socially assistive robot for people with
parkinson’s disease,” ACM Transactions on Human-Robot Interaction
(THRI), vol. 9, no. 3, pp. 1–31, 2020.
[51] S. Shen, P. Slovak, and M. F. Jung, “” stop. i see a conflict happening.”
a robot mediator for young children’s interpersonal conflict resolution,”
in Proceedings of the 2018 ACM/IEEE International Conference on
Human-Robot Interaction, 2018, pp. 69–77.
[52] M. Suguitan and G. Hoffman, “Blossom: A handcrafted open-source
robot,” ACM Transactions on Human-Robot Interaction (THRI), vol. 8,
no. 1, pp. 1–27, 2019.
[53] A. S. Bhat, C. Boersma, M. J. Meijer, M. Dokter, E. Bohlmeijer, and
J. Li, “Plant robot for at-home behavioral activation therapy reminders
to young adults with depression,” ACM Transactions on Human-Robot
Interaction (THRI), vol. 10, no. 3, pp. 1–21, 2021.
[54] A. M. Rosenthal-von der P¨
utten, N. Bock, and K. Brockmann, “Not your
cup of tea? how interacting with a robot can increase perceived self-
efficacy in hri and evaluation,” in 2017 12th ACM/IEEE International
Conference on Human-Robot Interaction (HRI. IEEE, 2017, pp. 483–
492.
[55] A. Kubota, E. I. Peterson, V. Rajendren, H. Kress-Gazit, and L. D.
Riek, “Jessie: Synthesizing social robot behaviors for personalized
neurorehabilitation and beyond,” in Proceedings of the 2020 ACM/IEEE
International Conference on Human-Robot Interaction, 2020, pp. 121–
130.
[56] S. Elbeleidy, D. Rosen, D. Liu, A. Shick, and T. Williams, “Analyzing
teleoperation interface usage of robots in therapy for children with
autism,” in Proceedings of the ACM Interaction Design and Children
Conference, 2021.
[57] D. Feil-Seifer, K. Skinner, and M. J. Matari´
c, “Benchmarks for evalu-
ating socially assistive robotics,” Interaction Studies, vol. 8, no. 3, pp.
423–439, 2007.
[58] C. D. Johnson, M. E. Miller, C. F. Rusnock, and D. R. Jacques, “A
framework for understanding automation in terms of levels of human
control abstraction,” in 2017 IEEE International Conference on Systems,
Man, and Cybernetics (SMC). IEEE, 2017, pp. 1145–1150.
[59] H. Bradwell, R. Winnington, S. Thill, and R. B. Jones, “Prioritising
design features for companion robots aimed at older adults: Stakeholder
survey ranking results,” in International Conference on Social Robotics.
Springer, 2021, pp. 774–779.
[60] M. Pino, M. Boulay, F. Jouen, and A. S. Rigaud, ““are we ready for
robots that care for us?” attitudes and opinions of older adults toward
socially assistive robots,” Frontiers in aging neuroscience, vol. 7, p. 141,
2015.
[61] D. Silvera-Tawil and C. Roberts-Yates, “Socially-assistive robots to
enhance learning for secondary students with intellectual disabilities
and autism,” in 2018 27th IEEE International Symposium on Robot and
Human Interactive Communication (RO-MAN). IEEE, 2018, pp. 838–
843.
[62] C. Recchuto, L. Gava, L. Grassi, A. Grillo, M. Lagomarsino, D. Lanza,
Z. Liu, C. Papadopoulos, I. Papadopoulos, A. Scalmato et al., “Cloud
services for culture aware conversation: Socially assistive robots and
virtual assistants,” in 2020 17th International Conference on Ubiquitous
Robots (UR). IEEE, 2020, pp. 270–277.
[63] N. Randall, C. C. Bennett, S. ˇ
Sabanovi´
c, S. Nagata, L. Eldridge,
S. Collins, and J. A. Piatt, “More than just friends: in-home use and
design recommendations for sensing socially assistive robots (sars) by
older adults with depression,” Paladyn, Journal of Behavioral Robotics,
vol. 10, no. 1, pp. 237–255, 2019.
[64] A. Ramachandran, C.-M. Huang, and B. Scassellati, “Give me a break!
personalized timing strategies to promote learning in robot-child tutor-
ing,” in Proceedings of the 2017 ACM/IEEE International Conference
on Human-Robot Interaction, 2017, pp. 146–155.
[65] H. R. Lee, S. ˇ
Sabanovi´
c, W.-L. Chang, S. Nagata, J. Piatt, C. Bennett,
and D. Hakken, “Steps toward participatory design of social robots:
mutual learning with older adults with depression,” in Proceedings of the
2017 ACM/IEEE international conference on human-robot interaction,
2017, pp. 244–253.
[66] A. Ramachandran, C.-M. Huang, E. Gartland, and B. Scassellati, “Think-
ing aloud with a tutoring robot to enhance learning,” in Proceedings of
the 2018 ACM/IEEE international conference on human-robot interac-
tion, 2018, pp. 59–68.
[67] X. Z. Tan, M. V´
azquez, E. J. Carter, C. G. Morales, and A. Steinfeld,
“Inducing bystander interventions during robot abuse with social mecha-
nisms,” in Proceedings of the 2018 ACM/IEEE international conference
on human-robot interaction, 2018, pp. 169–177.
[68] K. Jeong, J. Sung, H.-S. Lee, A. Kim, H. Kim, C. Park, Y. Jeong, J. Lee,
and J. Kim, “Fribo: A social networking robot for increasing social
connectedness through sharing daily home activities from living noise
data,” in Proceedings of the 2018 ACM/IEEE International Conference
on Human-Robot Interaction, 2018, pp. 114–122.
[69] S. You and L. P. Robert Jr, “Human-robot similarity and willingness to
work with a robotic co-worker,” in Proceedings of the 2018 ACM/IEEE
International Conference on Human-Robot Interaction, 2018, pp. 251–
260.
[70] K. Winkle, P. Caleb-Solly, A. Turton, and P. Bremner, “Social robots
for engagement in rehabilitative therapies: Design implications from a
study with therapists,” in Proceedings of the 2018 acm/ieee international
conference on human-robot interaction, 2018, pp. 289–297.
[71] R. Gomez, D. Szapiro, K. Galindo, and K. Nakamura, “Haru: Hardware
design of an experimental tabletop robot assistant,” in Proceedings of the
2018 ACM/IEEE international conference on human-robot interaction,
2018, pp. 233–240.
[72] S. Joshi and S. ˇ
Sabanovi´
c, “Robots for inter-generational interac-
tions: implications for nonfamilial community settings,” in 2019 14th
ACM/IEEE International Conference on Human-Robot Interaction
(HRI). IEEE, 2019, pp. 478–486.
[73] K. Winkle, S. Lemaignan, P. Caleb-Solly, U. Leonards, A. Turton, and
P. Bremner, “Effective persuasion strategies for socially assistive robots,”
in 2019 14th ACM/IEEE International Conference on Human-Robot
Interaction (HRI). IEEE, 2019, pp. 277–285.
[74] M. E. Ligthart, M. A. Neerincx, and K. V. Hindriks, “Design patterns
for an interactive storytelling robot to support children’s engagement
and agency,” in Proceedings of the 2020 ACM/IEEE International
Conference on Human-Robot Interaction, 2020, pp. 409–418.
[75] D. P. Davison, F. M. Wijnen, V. Charisi, J. van der Meij, V. Evers, and
D. Reidsma, “Working with a social robot in school: a long-term real-
world unsupervised deployment,” in Proceedings of the 2020 ACM/IEEE
International Conference on Human-Robot Interaction, 2020, pp. 63–72.
[76] A. Dobrosovestnova and G. Hannibal, “Teachers’ disappointment: The-
oretical perspective on the inclusion of ambivalent emotions in human-
robot interactions in education,” in Proceedings of the 2020 ACM/IEEE
International Conference on Human-Robot Interaction, 2020, pp. 471–
480.
[77] M. Natarajan and M. Gombolay, “Effects of anthropomorphism and
accountability on trust in human robot interaction,” in Proceedings of the
2020 ACM/IEEE International Conference on Human-Robot Interaction,
2020, pp. 33–42.
[78] R. Feingold Polak and S. L. Tzedek, “Social robot for rehabilitation:
Expert clinicians and post-stroke patients’ evaluation following a long-
term intervention,” in Proceedings of the 2020 ACM/IEEE International
Conference on Human-Robot Interaction, 2020, pp. 151–160.
[79] D. J. Rea, S. Schneider, and T. Kanda, “” is this all you can do?
harder!” the effects of (im) polite robot encouragement on exercise
effort,” in Proceedings of the 2021 ACM/IEEE International Conference
on Human-Robot Interaction, 2021, pp. 225–233.
[80] N. Tsoi, J. Connolly, E. Ad´
en´
ıran, A. Hansen, K. T. Pineda, T. Adam-
son, S. Thompson, R. Ramnauth, M. V´
azquez, and B. Scassellati,
“Challenges deploying robots during a pandemic: An effort to fight
social isolation among children,” in Proceedings of the 2021 ACM/IEEE
International Conference on Human-Robot Interaction, 2021, pp. 234–
242.
[81] K. Winkle, P. Caleb-Solly, U. Leonards, A. Turton, and P. Bremner, “As-
sessing and addressing ethical risk from anthropomorphism and decep-
tion in socially assistive robots,” in Proceedings of the 2021 ACM/IEEE
International Conference on Human-Robot Interaction, 2021, pp. 101–
109.
[82] S. Valencia, M. Luria, A. Pavel, J. P. Bigham, and H. Admoni, “Co-
designing socially assistive sidekicks for motion-based aac,” in Proceed-
ings of the 2021 ACM/IEEE International Conference on Human-Robot
Interaction, 2021, pp. 24–33.
[83] H. R. Lee and L. D. Riek, “Reframing assistive robots to promote suc-
cessful aging,” ACM Transactions on Human-Robot Interaction (THRI),
vol. 7, no. 1, pp. 1–23, 2018.
[84] C. Moro, G. Nejat, and A. Mihailidis, “Learning and personalizing
socially assistive robot behaviors to aid with activities of daily living,”
ACM Transactions on Human-Robot Interaction (THRI), vol. 7, no. 2,
pp. 1–25, 2018.
[85] M. Bajones, D. Fischinger, A. Weiss, P. D. L. Puente, D. Wolf,
M. Vincze, T. K ¨
ortner, M. Weninger, K. Papoutsakis, D. Michel et al.,
“Results of field trials with a mobile service robot for older adults in
16 private households,” ACM Transactions on Human-Robot Interaction
(THRI), vol. 9, no. 2, pp. 1–27, 2019.
[86] H. Javed, R. Burns, M. Jeon, A. M. Howard, and C. H. Park, “A robotic
framework to facilitate sensory experiences for children with autism
spectrum disorder: A preliminary study,” ACM Transactions on Human-
Robot Interaction (THRI), vol. 9, no. 1, pp. 1–26, 2019.
[87] Y. Noguchi, H. Kamide, and F. Tanaka, “Personality traits for a social
mediator robot encouraging elderly self-disclosure on loss experiences,”
ACM Transactions on Human-Robot Interaction (THRI), vol. 9, no. 3,
pp. 1–24, 2020.
[88] B. R. Schadenberg, D. Reidsma, D. K. Heylen, and V. Evers, ““i see
what you did there” understanding people’s social perception of a robot
and its predictability,” ACM Transactions on Human-Robot Interaction
(THRI), vol. 10, no. 3, pp. 1–28, 2021.
