Content uploaded by Tom Williams
Author content
All content in this area was uploaded by Tom Williams on Jul 05, 2023
Content may be subject to copyright.
Beyond the Session: Centering Teleoperators in Socially Assistive Robot-Child
Interactions Reveals the Bigger Picture
SAAD ELBELEIDY∗,MIRRORLab, Colorado School of Mines, USA
TERRAN MOTT∗,MIRRORLab, Colorado School of Mines, USA
DAN LIU, ATLAS Institute, University of Colorado Boulder, USA
ELLEN DO, ATLAS Institute, University of Colorado Boulder, USA
ELIZABETH REDDY, Engineering, Design & Society, Colorado School of Mines, USA
TOM WILLIAMS, MIRRORLab, Colorado School of Mines, USA
Socially assistive robots play an eective role in children’s therapy and education. Robots engage children and provide interaction
that is free of the potential judgment of human peers and adults. Research in socially assistive robots for children generally focuses
on therapeutic and educational outcomes for those children, informed by a vision of autonomous robots. This perspective ignores
therapists and educators, who operate these robots in practice. Through nine interviews with individuals who have used robots to
deliver socially assistive services to neurodivergent children, we (1) dene a dual-cycle model of therapy that helps capture the domain
expert view of therapy, (2) identify six core themes of teleoperator needs and patterns across these themes, (3) provide high-level
guidelines and detailed recommendations for designing teleoperated socially assistive robot systems, and (4) outline a vision of
robot-assisted therapy informed by these guidelines and recommendations that centers teleoperators of socially assistive robots in
practice.
ACM Reference Format:
Saad Elbeleidy, Terran Mott, Dan Liu, Ellen Do, Elizabeth Reddy, and Tom Williams. 2023. Beyond the Session: Centering Teleoperators
in Socially Assistive Robot-Child Interactions Reveals the Bigger Picture. In Proceedings of ACM Conference On Computer-Supported
Cooperative Work And Social Computing (CSCW’23). ACM, New York, NY, USA, 36 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn
1 INTRODUCTION
Socially Assistive Robots (SARs), robots that provide assistance through social interaction [
33
], have shown much
success across research domains like education [
6
,
8
,
65
] and therapy [
11
,
14
,
16
,
39
,
51
,
57
]. SARs have shown particular
success in interactions with children [
57
,
65
], as they can engage children as peers without the threat of judgment
or ostracism that may accompany interactions with adult authority gures [
15
]. Children often nd robots engaging
and are therefore highly receptive to interactions with robots. It is thus crucial to understand how robots can be best
designed to provide social assistance to children.
SAR researchers often focus on the ecacy of robots as a social assistance tool, measuring therapeutic ecacy or
educational outcomes with the aim of making the case for robot adoption. This research is essential to ensure these
child-robot interactions are eective, and to help us understand how best to design and develop robots to ensure this
∗The rst two authors contributed equally to this work.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not
made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page. Copyrights for components
of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to
redistribute to lists, requires prior specic permission and/or a fee. Request permissions from permissions@acm.org.
©2023 Association for Computing Machinery.
Manuscript submitted to ACM
1
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
eectiveness. However, this research often assumes that SARs should be autonomous, eliding critical questions as to
how these robots are (and should be) operated, and by whom.
Recent research suggests, in fact, that SARs may need to be teleoperated for both practical and ethical reasons [
31
].
SARs are often used with vulnerable populations in critical tasks where accountability is crucial. In domains with
these characteristics, teleoperation is typically recommended, since human control and supervision allow a responsible
individual to intervene, and allow humans to readily identify the causes of adverse events and quickly head them o.
SARs used with young children in education and therapy are especially likely to have these characteristics, and in these
domains teleoperation is typically necessary not only for safety and accountability, but to keep power in the hands of
trained professionals who are best equipped to provide therapy or education.
Indeed, in contrast to the assumptions of most SAR research, robots used with children in therapy and education are
often teleoperated in practice [
12
,
32
,
43
]. Based on the success of SARs in laboratory-based child-robot interaction
experiments [
29
,
55
–
57
], therapists have begun using SARs in their therapeutic practice, and many organizations are
working to make these robots more widely accessible [
37
,
49
,
60
]. Due to the complexity of real-world therapeutic
contexts relative to controlled laboratory environments, therapists have needed to – and desired to – use these
new robotic tools in a teleoperated manner. Unfortunately, the current tools available to therapists and educators to
teleoperate SARs in accordance with their needs and wishes oer extremely limited and rudimentary control. Moreover,
because most SAR researchers have historically assumed that SARs should be autonomous, it is unclear how tools
should be designed to best support therapists and educators in their use of teleoperated robots. In this work, we thus
aim to develop this necessary and foundational understanding, with special attention to the domain of therapy.
In this paper, we share the results of 9 interviews with individuals who have used robots in their daily practice to
deliver therapy or other socially assistive services to neurodivergent children. We then use a grounded theory approach
to analyze these interviews and uncover patterns across our participants’ experiences. We share stories of how these
individuals prepare to work with robots, learn which activities engage children, maintain awareness of a diverse set
of environmental factors, and adapt quickly in the moment. Based on this analysis, we produce (1) a characterization
of therapists’ experience of therapy as a dual-cycle; (2) six themes of teleoperator needs and a set of patterns that
recur across those themes; (3) high-level guidelines and detailed recommendations for designing SAR systems that
center teleoperators; and (4) a vision for the future of robot-assisted therapy that is informed by these guidelines and
recommendations, and that centers the individuals who operate socially assistive robots in practice.
2 RELATED WORK
In this section, we provide necessary background information about neurodiversity, the goals of therapy for neurodi-
vergent children, robots’ appeal to children, and the impact of robots in therapy.
2.1 Neurodiversity
Neurodiversity is the perspective that individuals possess neurological dierences that ought to be respected and
appreciated, similar to other individual dierences in human variation [
3
]. This perspective oers a social model of
disability that aims to de-pathologize individuals while maintaining that neurological classication may still be useful
and meaningful [
19
]. Neurodivergent features include autism, attention-decit / hyperactivity disorder (ADHD), dyslexia,
Down’s syndrome, bipolar disorder, and others [
2
]. Autistic individuals have been a fairly vocal group advocating for
the acceptance of neurodiversity [
44
]. Importantly, a signicant portion of Socially Assistive Robot research focuses on
assistance for autistic individuals, particularly autistic children [29,55–57].
2
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
Historically, neurodivergent features were treated as mental health illnesses that resulted in institutionalization. From
a medical perspective, neurodivergent features have often been treated as illnesses in search of a cure. Neurodivergent
individuals were required to change their behaviors to t in a society that did not accept them. Yet recently, and thanks
in large part to autistic self-advocates, the neurodiversity movement has instead chosen a social model for disability1.
From the social perspective, the neurodiversity movement asks society to accept disabled individuals without an attempt
to cure them [
48
]. Instead, the neurodiversity movement asks communities to support disabled individuals. Some have
proposed alternatives to the social model [
28
], however, the social model for disability remains the more commonly
accepted model [3,13,19] and the preferred model by autistic self-advocates [48].
Neurodivergent individuals may require special support or care to navigate the world. Since society often sets an
expectation of neurotypical behavior, neurodivergent people often benet from learning to adapt their behaviors to
have an easier time going about their lives. Under the guidance of their parents, neurodivergent children commonly
begin therapy and other services at a young age to learn to communicate eectively, unlearn certain self-injurious
behaviors, or receive other forms of support to meet their individual needs.
2.2 Goals of therapy for neurodivergent children
Because therapy for neurodivergent children depends on the individual’s particular therapeutic needs, a variety of
therapeutic modalities are commonly used. These modalities include speech and language pathology (SLP) to improve
speech and communication, occupational therapy (OT) to improve movement, and applied behavior analysis (ABA)
therapy to improve social and communication skills. Within these modalities, therapists may choose from a variety of
experiential approaches to engage with their clients, such as art therapy, music and dance therapy or robot-assisted
therapy.
Personalization is critical for therapy, both to determine the desired outcomes and metrics of success, and to select
the specic interventions to encourage those outcomes. Meta-analysis of personalization in therapy has shown that
personalizing the goals of therapy to the specic needs of individual clients leads to better overall outcomes [
63
]. As such,
aligning on goals by collaborating with clients and achieving consensus has been shown to lead to improved treatment
outcomes [
66
]. Critically, the personalization of therapeutic interventions includes choosing not only the particular
content or intervention, but also the particular ways of delivering that content in order to maintain engagement.
A necessary aspect of therapy’s eectiveness is how well it engages children [
5
]. Studies have shown that success
in therapy is directly attributable to therapists’ success at engaging clients [
64
,
69
]. Accordingly, therapists may try
dierent approaches to engage dierent children. Depending on the child, dierent modalities or approaches to therapy
may be more eective. One therapeutic modality that has been shown to be widely eective for children with dierent
needs is robot-assisted therapy.
2.3 Robots’ appeal to children, especially neurodivergent children
Researchers have studied how robots can be used to support a large variety of child health needs and demonstrated
that when used to support such needs, robots consistently lead to benecial outcomes such as enhanced positive
aect [
38
]. Positive aect is especially important in therapy with children [
5
]. Research shows that children have a high
acceptance of and likability for robots in healthcare contexts [
26
]. Dialogue-based robots, specically, can motivate
children to interact, and increase children’s probability of learning [
47
]. With autistic children, robots are particularly
1
The shift from a medical perspective to a social perspective is a core part of the change in language referring to neurodivergent individuals using
identity-rst language rather than person-rst language [13].
3
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
well received [
52
]. A study analyzing fMRI scans of autistic and non-autistic people suggests that autistic people may
consider robots as social interacting partners in the same way non-autistic people consider other humans [
17
]. This
is consistent with ndings that autistic children lack the bias towards anthropomorphism that is usually present in
non-autistic individuals [18].
2.4 Robots’ Impact on Children
Researchers have demonstrated that socially assistive robots can successfully help neurodivergent children reach
various learning goals, likely as a direct benet of the increased engagement described in the previous section. In a
small study with autistic children learning to program robots, children’s collaborative behaviors increased in relation to
their engagement with robotic content [
67
]. Interacting with robots has also been found to be positively correlated
with autistic children’s verbal expression [
23
,
34
,
40
] and eye contact [
23
,
25
], likely due to an increase in engagement.
Robots can help children, particularly shy children, engage in social play [
36
] Additionally, some autistic children show
an improvement in emotional response and imitation after interacting with a robot [50].
Longitudinal studies have also shown long-term positive impacts of robots on neurodivergent children. In one
10-week study, pairs of autistic children showed increased eye gaze and positive aect after playing imitation games
with a robot [
68
]. Similarly, in a 12-week study with autistic children learning social skills from an adult, incorporating
a NAO robot in some activities lead to an increase in children’s frequency and duration of eye contact with the adult
and an increase in children’s verbalizations [24].
2.5 Socially Assistive Robot Level of Autonomy
There is substantial research showing the impact robots can have on children [
29
,
55
–
57
]. The bulk of this previous
research, however, focuses on child-robot interaction, mostly without emphasis on how these robots are operated. Some
research studies are explicit about using autonomous robots [
68
], or teleoperated/Wizard of Oz style robots [
45
], but
most studies do not specify how the robots they use are operated [
17
,
23
–
26
,
34
,
40
,
50
,
52
,
58
,
59
,
67
], likely since the
focus of these studies is to understand the child’s response and the impact the robot has on the child. Such research
endeavors are valuable because they can provide insight into children’s preferences for robot design features, such as
robot morphology.
In a review of studies in robots and autism research, Scassellati et al
. [57]
suggest a need for robot autonomy due
to a current lack of sophistication in robot control, which would be needed to handle the ”complex, dynamic, and
unpredictable situations” that exist in therapy. They also discuss therapists’ expertise and the fact that therapists are
trained specically to handle these unpredictable scenarios. We agree that there is room for improvement in robot
teleoperation systems and that therapists are the experts in this context. However, we do not conclude that therapists
ought to be replaced by autonomous robots. Instead, we argue that improvements should be made to the teleoperation
capabilities of robots that allow for eective delivery of therapy by a therapist.
To be clear, there are reasonable arguments for the use of autonomous robots in therapy, including on-demand access
and the diculty of teleoperation. However, there are also crucial arguments against the use of autonomous robots in
therapy.
2.6 Issues with Autonomous Robots
Autonomous robots result in several key issues when used in therapy. Autonomous robots distance domain experts
from their subjects. These robots are often expensive and/or widely inaccessible, resulting in minimal deployment in
4
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
practice. When they are able to be deployed, they remove the human in the loop who can bridge the results of therapy
from human-robot interactions to human-human interaction.
The process of automation distances domain experts from robot users. This added distance is generally problematic
since it increases the diculty of domain experts to monitor, review and improve these robots. Moreover, this distance
makes it more dicult for therapists to monitor, review, and improve therapeutic interventions themselves; a critical
component of sustained therapy.
Autonomous robots are typically much more expensive than teleoperated ones, due not only to the cost of developing
the robot autonomy, and the cost of integrating domain expertise into that autonomy, but also due to the suite of sensors
typically needed to sense participants. Teleoperation reduces costs by removing the need both for this autonomy and
by removing the need for the sensors that enable it. The majority of robots that can be used in therapy contexts are
not widely accessible specically because they are expensive. The large cost of robot hardware (typically thousands
of dollars) has led to several eorts to create open-source or aordable alternatives such as the Romibo robot [
60
]
or Peerbots [
49
]. These platforms reduce the cost barrier to widespread adoption of robots in therapy by promoting
solutions that rely on teleoperation of comparatively aordable hardware that people may already own, such as
smartphones and tablets. Most robots currently used in therapy with children are teleoperated by a therapist or educator,
so research intended to benet current therapeutic practice needs to account for this context of use.
Since human connection is a central component of therapy, having a human-in-the-loop facilitates transferring
human-robot interaction outcomes to human-human interaction. Therapy is a sensitive context where children learn
how to improve their speech, communication, or motion, all of which are essential behaviors for human connection.
Therapists and similar specialists are specically qualied to help children navigate this experience. While robots may
be used to increase interest and engagement in activities, having a human in the loop ensures that therapy remains safe
and meaningful for the child.
Above all, teleoperation ensures that a human expert can address the unpredictable, emotionally fraught scenarios
that characterize therapy with children. It is imperative that technologists honor therapists’ expertise and support their
agency in this way. Teleoperation keeps power in the hands of trained professionals who bring empathy, nuanced
judgment, and adaptable decision-making to each situation, instead of placing power in the hands of robotics companies
and their developers. Recent work [
31
] supports this idea that socially assistive robots ought to incorporate teleoperation
despite current research choosing to prioritize autonomous robots.
2.7 Teleoperated Robots in Practice
While most research has either focused on SAR autonomy [
31
], in practice, caregivers have been using teleoperated
SARs to much success [
12
,
32
,
43
]. Teleoperated SARs allow caregivers to incorporate robots into their typical workow
where appropriate. When using teleoperated SARs, caregivers maintain their own agency, autonomy, and control, so
that they can override or interfere with any issues that come up.
Importantly, the level of autonomy chosen for robots has an inuence on the role that caregivers play. When using
teleoperated robots, caregivers control the robot’s exact actions. When some autonomy is used, caregivers switch to a
more supervising role. When full autonomy is expected, caregivers review the robots’ performance. These dierent roles
for caregivers present dierent teleoperation needs and are therefore crucial to consider when designing teleoperation
interfaces [35].
It’s well known that robot teleoperators have specic needs that do not arise when robots are fully autonomous.
There is a long history of research on teleoperation in the human-robot interaction community [
27
,
41
,
46
], which has
5
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
revealed key factors that need to be considered when working with teleoperators [
1
,
21
], including the need to consider
teleoperators’ workload demands and reduced Situation Awareness [1,21].
Because of the way that teleoperation interfaces need to be carefully developed for specic contexts of use and
the particular human factors constraints imposed by those contexts, eective use of robot teleoperation interfaces
often requires expertise both in the relevant subject matter area and in robot teleoperation itself [
62
]. This is true both
for teleoperation interfaces used in traditionally high-technology domains like space exploration, as well as socially
assistive domains like therapy. In both of these domains, however, the teleoperation interfaces used in practice are
not typically designed with consideration for the background technological expertise and skillsets of the workers who
will become teleoperators. Recent research on SARs supports this, showing that caregivers considering the use of
teleoperated SARs in their practice need to dedicate substantial time to learning teleoperation interfaces [30].
Research on SARs has found robots to be eective at engaging children in therapy, resulting in positive outcomes
for children. Most of this research has focused on autonomous robots despite level of autonomy recommendations
suggesting a need for teleoperation when supporting critical tasks with high needs for accountability such as therapy [
31
].
The chosen level of autonomy for SARs heavily inuences the design of the robots and their teleoperation interfaces if
present. Yet, little is known about therapists’ needs when teleoperating a robot in therapy. In this paper, we aim to
answer two key research questions:
RQ1. What are therapists’ needs in conducting therapy with neurodiverse children?
RQ2. How do these needs change when teleoperating a robot in therapy?
To answer these research questions, we interview individuals with experience teleoperating robots to deliver
therapeutic activities for neurodivergent children. We uncover teleoperators’ needs by grounding our investigation in
the individual experiences of our participants. As a result, we provide guidelines for non-therapists to better understand
their experiences.
3 METHODOLOGY
To understand therapists’ experience during therapy, we conducted ethics-board-approved semi-structured interviews
with 9 participants about their experiences conducting therapy with and without robots. All participants had experience
or an accreditation in a therapy practice with children. They all also had experience teleoperating robots during sessions
with neurodivergent children. We share more details about our participants’ experience in Table 1. Interviews were
conducted virtually over a video chat platform and were recorded to allow interviewers to focus on participants and
provide a more engaging conversation. Interviews were around one hour long and covered conducting and preparing
therapeutic social assistance with and without a robot, and how participants would use a particular robot teleoperation
interface shown to them on their screens. Our interviews established a base case of therapy as therapy without a robot.
We then discussed therapy with a robot as an alternative therapy in contrast to that base case. This allowed us to
determine the dierences and changes that occur when using a robot. In the nal portion of our interviews, we asked
our participants about how they would use a specic teleoperation interface (Peerbots [
49
]) to establish a consistent
frame of reference across our participants.
6
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
3.1 Recruitment
Through a collaborator who has built socially assistive robots for several years, we reached out to 22 individuals who
had experience operating socially assistive robots in sessions with neurodivergent children. All approached individuals
had experience controlling robot dialogue during these sessions, and some also had experience controlling robot motion.
We sent each participant a description of the research, and a consent and video release form, allowing them to
provide their level of consent over data use. Upon signing the consent forms, we scheduled a time for an interview via
video chat. Out of 22 potential candidates, 7 did not respond to our initial inquiry, 4 did not respond after research
details were shared, 2 declined to participate, and 9 agreed to be interviewed.
3.2 Participants
In this study, we investigated specic individual experiences in order to illuminate broader patterns in the teleoperation
of robots in therapy. Details about our participants are described in Table 1. Of our nine participants, six are credentialed
therapists, one is an educator, and two are non-therapists who have relevant experience to robot-assisted therapy for
children. While the two “non-therapists” do not have the credentials necessary to be licensed as therapists or educators,
they have extensive experience using the same robots to perform the same activities with neurodiverse children. Our
interviews suggest that they have similar views about and needs from the technology used. All participants had direct
experience using robots in delivering therapeutic content to children. Participants had experience with using at least one
of the following robot systems commonly used for social assistance: Peerbots [
49
], Romibo [
60
], Misty [
54
], Movia [
53
].
A majority of our participants (6 of 9) continue to regularly practice delivering therapy.
We used information power [
42
] as a reference for the quality of our interviewed sample. We chose to use semi-
structured interviews in order to facilitate participants’ reections on specic aspects of their experiences in the eld.
We used these reections to develop better qualitative understanding of their work [
9
]. While the population with
our desired experience is small, we were able to recruit participants who all had experience operating robots in this
context. All our participants were deeply engaged, sharing rich and descriptive stories from their experiences. While
each participant had unique elements to their experience, they all shared consistent narratives about their experience
in therapy and their robot use.
3.3 Study Procedure
After reading and signing our consent forms and data use policy, participants completed a survey of questions about
their background; therapeutic credentials, experience working with children, and experience working with robots. We
summarize participants’ survey responses in Table 1. After completing the survey, participants scheduled a time for an
interview with one of our research team members.
Interviews were conducted over video chat. The interviewer began interviews by asking if participants had any
questions about the study procedure, and conrming their consent to participate. The interviewer then reviewed
participants’ survey responses and provided a summary of the topics they could expect to receive questions about. The
interview was broken up into two portions: (1) therapy in general, and (2) therapy with robots. Our interview was
semi-structured with guiding questions and prompts for each section shown in Appendix A.
During the portion of the interview in which we discussed therapy with robots, we showed participants a screenshot
of Peerbots [
49
] the example teleoperation interface shown in Figure 1. A larger version of this image is provided in
the Appendix in Figure 10. The Peerbots application provides teleoperation capabilities for Misty robots and other
7
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Fig. 1. An example of the Peerbots teleoperation interface that was shown to participants.
8
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
ID Therapeutic Credentials Experience with robots Interview (Time, Words)
Alissa (T1)
Registered Behavior Technician with 3-
5 years of experience delivering behav-
ioral and art therapy to neurodivergent
children in one on one sessions.
Used robots in therapy with children in
the past.
00:55, 9,899 words
Blanche (T2)
Dance and movement therapist with 5-
10 years of experience working with chil-
dren who have various disabilities.
Extensive experience with robots and
regularly uses robots for group social
skills sessions.
01:33, 19,132 words
Caroline (T3)
Therapist with B.S. in Psychology with
5-10 years working with children who
have various disabilities.
Used robots in therapy one on one and
in groups.
01:04, 8,346 words
Emily (T4)
Occupational therapist for 5-10 years
with 3-5 years working with children
with various disabilities.
Used robots for some one on one ses-
sions.
00:36, 6,305 words
Fiona (T5)
Licensed marriage and family therapist
with over 10 years of experience work-
ing with children who have various dis-
abilities.
Regularly uses robots for therapy with
children.
00:47, 7,418 words
Greg (T6)
Behavior interventionist with over 10
years of experience working with chil-
dren who have various disabilities.
Led social skills group sessions for many
years then left to found a social robotics
company.
01:09, 8,931 words
Holly (E1)
Child development specialist with over
20 years of experience in teaching.
One year of using robots in one on one
sessions with children who have various
disabilities.
01:04, 8,456 words
Isaac (N1)
No therapeutic credentials. Adapted
Aquatics Instructor
Early adopter working with children
to delivered social skills content using
robots to groups of children for 5-10
years.
00:58 8,133 words
Jaclyn (N2) No therapeutic credentials.
Organization leader using robots to reg-
ularly deliver therapy for groups of chil-
dren with various disabilities for 5-10
years.
00:58, 10,308 words
Table 1. List of participants and relevant information. Individual codes used start with T for therapists, E for educators, and N for neither.
9
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
devices. As a free application available on various operating systems, it provides an example of a teleoperation interface
that is relatively widely accessible to participants. As was later veried by our participants, the Peerbots interface was
fairly consistent with other teleoperation interfaces our participants had used. The Peerbots interface has (1) buttons to
activate robot dialogue, (2) organization capabilities for organizing robot dialogue, (3) edit capabilities to modify robot
dialogue content, and (4) features to manage the connection to the robot being teleoperated.
Interviewers used the guiding questions across the interview as starting questions to guide the conversation.
Interviewers relied on these questions mostly as fallback questions when they could not build o of participants’
responses to previous questions. Importantly, as we conducted more interviews, we became better informed about
therapists’ experiences and began to ask more targeted questions. As expected, most, if not all, of our interviews shifted
from the scripted questions, as can be seen from participants’ quotes in Section 5.
3.4 Analysis
After we conducted the interviews, we used auto-generated transcripts of the recordings as our starting point for text
analysis. One member of our team listened to all the recordings and veried and edited these transcripts. Interviews
averaged 60 minutes long with outliers at a minimum of 36 and maximum of 93 minutes long. Interview transcripts
averaged 9658 words with outliers at a minimum of 6,305 and maximum of 19,132 words. Distributions of these values
are shown in Figure 2.
We followed the general principles and heuristic devices of grounded theory [
4
,
20
] to identify and analyze key
patterns from our participants’ perspectives about therapy. We selected grounded theory since it is eective at working
in a largely unknown context to produce an explanation of an underlying process [
22
]. While therapeutic practices are
generally well-dened, the use of robotics in these contexts is comparatively novel, the earliest of which was in 1997 [
10
].
Our research thus contributes to increasingly important scholarly conversations about robots in therapy [
29
,
55
–
57
]
with a new focus on the teleoperators of these robots.
Our coding approach was conducted as follows. One coder (the rst author) began by open coding the nalized
transcripts. This coder annotated text with consideration for dominant themes or topics addressed in each passage
by the participant. All text was coded, and some portions were coded with multiple codes where the content covered
more than one idea. Upon completing the open coding of all the transcripts, this coder read through all the open codes
and merged those that presented the same idea but were phrased dierently in the code name. The nal count of open
codes was 528. Two coders (the rst and second authors) then conducted multiple iterations of axial coding in which
they identied patterns across open codes. By aggregating similar codes, axial codes allow us to nd general patterns in
the data and come to conclusions about our participants’ experience. Our goal was to group the large number of open
codes into related concepts with a lens focused on needs and diculties that our participants have experienced. We
arrived at 6 themes with 14 sub-themes that appeared across the open codes. While not all open codes fell in these
themes, our goal was to identify patterns specically relating to therapists’ needs. In our analysis, we focus on the
themes that were repeated across our participants that seemed salient during interviews and were important to them.
The rst and second authors discussed dierent approaches to organize the axial codes and came to an agreement.
We present two ways in which these themes relate to each other: (1) hierarchically, when we discuss each theme and
subtheme in Section 5, and (2) relationally, when we discuss patterns across themes in Section 6.
We note that our theme formulation originates in the perspectives of technologists, and may dier slightly from
strictly therapeutic perspectives. For example, we emphasize the dierence between documentation and evaluation,
10
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
(a) Distribution of interview duration in minutes. (b) Distribution of transcript length in words.
Fig. 2. Distributions of interview statistics.
whereas therapists often perform those two activities concurrently. When technology is involved, the processes of
documentation and evaluation can be separated, since documentation does not require therapeutic expertise, but
evaluation does. We highlight this and other important pieces of our analysis in each theme’s description.
By improving our understanding of therapy and how therapists use robots, we can provide guidelines for developing
interfaces with their operators’ needs in mind. With our identied themes, we are able to describe therapists’ experiences
conducting therapy in an accessible way to non-experts; specically those building technological tools for therapists. In
Section 4, we dene the dual-cycle therapeutic process that stems from our analysis. In turn, this situates readers in
the context that our participants experience. By dening this dual-cycle process, we show how therapists’ needs are
oriented around each cycle and the relationships between these cycles. Following that, in Section 5we discuss each
theme in further detail, providing interface recommendations, and in Section 6we discuss relationships across these
themes.
4 THERAPY AS A DUAL-CYCLE PROCESS
It is essential for developers to recognize that therapy is cyclic and that the cycle of therapy is in fact comprised of two
nested and paralleled cycles. Our analysis of participant interviews revealed this as a key pattern in the way therapists
discussed their work. Therapy involves examining the client, evaluating their needs, preparing to meet those needs,
delivering an intervention, and then repeating that process. This cycle ensures that a client’s changing needs are met
and their goals are achieved over time. Critically, therapy is dened by two cycles at dierent levels; on a long timescale
of months or years across the clients’ sessions (the outer cycle), and a short time scale of minutes or hours within each
session (the inner cycle) 2. We present a visual representation of these cycles in Figure 3.
The outer cycle begins when a new client is identied. They are examined and evaluated by a specialist who
can determine their needs in collaboration with parents, schools, and healthcare institutions. Afterwards, therapists
collaborate with these other stakeholders to outline the clients’ goals and determine the relevant interventions. They
then deliver these interventions during a therapy session and examine the child throughout to determine the success
2
While the inner cycle exists within the outer cycle, when we refer to the outer cycle we generally mean the parts of the outer cycle that are distinct from
the inner cycle —this distinction will become important in future sections.
11
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Fig. 3. The cyclical process that therapists undergo in therapy with a client. The outer cycle involves a variety of stakeholders whereas
the inner cycle within a particular session is focused on the therapist and their client.
of each intervention. These observations are collaboratively reviewed, goals are reconsidered, and the outer cycle of
therapy continues.
Within each individual session (the inner cycle), therapists undergo a similar cycle of (1) examining and evaluating
their client, (2) preparing interventions, and (3) delivering interventions. In this context, therapists are not in direct
collaboration with other stakeholders. The inner cycle involves only the therapist and child working together.
In summary, while both cycles follow a parallel process of examining, evaluating, preparing, then delivering an
intervention, key dierences exist in terms of time-scale and whether other stakeholders are involved.
Therapy is often approached with a focus on either the outer cycle of macro-interactions between stakeholders [
7
,
61
]
or the inner cycle of therapeutic interventions [
23
,
23
,
25
,
40
]. However, these single-cycle perspectives miss a key
piece of the story. By understanding how the outer and inner cycles interact from therapists’ perspectives, we identify
valuable considerations that can improve robotic technology used in therapy. We described earlier in Section 2.5 that
most research focuses on the robot-child interaction. It may in fact be more precise to say that it focuses on the inner
cycle of therapy, often without acknowledging outer-cycle activities like preparation. In contrast, our research takes a
broader view that places enhanced emphasis on the outer cycle of therapy. Moreover, and distinct from the bulk of
prior literature, our research focuses on the therapists who seem to often be ignored as the real users of these systems.
Participants seemed generally appreciative of our interviews. Emily (T4) concluded their interview with, “I’m just
happy that you all, are taking into account therapists’ opinions, so I really appreciate that."
The six themes that emerged from our interview analysis are directly connected to the way that therapists examine,
evaluate, and prepare in the inner and outer cycles of therapy. In the remainder of this paper, we will explore the
details of each theme, outline recommendations for developers building tools used in this context, and describe how
these themes relate to each other. We ground our recommendations in the experiences of the therapists who are the
real-world users of these products. Based on our analysis, we develop recommendations that are directly connected
to our participants’ stories and experiences. Additionally, to facilitate the discussion with developers, we present the
following high level guidelines:
12
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
Fig. 4. The themes and sub-themes derived from our analysis of interviews about delivering services using robots to neurodivergent
children.
(1)
When possible, tools should directly meet an identied need of therapists in the moment it occurs—within the
inner or outer cycle.
(2)
Tools should account for the interdependent relationships between and the consequences of actions that occur
in the inner and outer cycles of therapy.
(3)
Since therapists have more time during the outer cycle, tools should move demanding or time-consuming
activities to the outer cycle when possible.
5 THEMES
We identied six themes that consistently emerged across our interviews (as visualized in Figure 4):
(1) Preparation: therapists need to create or organize the resources they need for therapy.
(2) Variety: therapists rely on a variety of content to deliver eective therapy.
(3) Awareness:
therapists maintain a high level of awareness in therapy to respond to their client and surroundings.
(4) Adaptability: therapists adapt within each session to clients’ needs in the moment.
(5) Documentation: therapists rely on thorough documentation.
(6) Evaluation: therapists continuously evaluate their clients to determine progress and upcoming interventions.
5.1 Preparation
Eective preparation is necessary for eective therapy. Yet preparation is often omitted or disregarded in work focusing
solely on the inner cycle of therapy sessions with children. In contrast, framing therapists as robot users naturally
centers their preparation process. Our participants explained how demanding preparation can be, how the uncertainty
of therapy leads them to over-prepare, and how introducing robots complicates their preparation process. In tension
13
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
with these feelings, therapists also reported that they only feel prepared when they’re over-prepared. When they use a
robot, this need for over-preparation becomes even more salient, since preparation with a robot is more time consuming.
5.1.1 Preparation is demanding. Preparation is a fundamental part of therapy. To prepare, therapists read documentation
about their clients’ goals and previous sessions (see Section 5.5.1) to create or organize the appropriate interventions.
When we discussed preparation with Holly (E1) , they explained that “you’re preparing your lesson. ahead of time, based
on what the goals would be for that session". Blanche (T2) , a music and dance therapist, describing their preparation for
sessions without a robot, says “the rst thing I do is set up my playlist"
3
Blanche (T2) says that when selecting music,
“the music really has to support the goals of the clients as well, so it is a big deal. I spend, a ridiculous amount of time
nerding out about music". For Blanche (T2) , it’s clear that the preparation process is time-consuming and demanding.
Blanche (T2) explains that “For this week, maybe I know we want to work on self regulation ... so I’m going to, I know I’m
planning to play a freeze dance game". Blanche (T2) shows us that preparation isn’t just a one-time occurrence but
instead changes every week or session. Holly (E1) , a teacher with over 20 years of experience, mentioned that “there’s
always preparation, no matter how experienced you are." Preparing for therapy is clearly time-consuming even without
the incorporation of a robot.
When a robot is introduced, it can exacerbate this time burden. Holly (E1) explained: “You know, I wasn’t born and
raised with technology like kids today. So, the technology end of it for me kind of felt like a lot of my focus in preparation".
Therapists are therapy experts, but not necessarily experts in operating robots or other technologies. As Blanche (T2)
puts it, “I’m a therapist, like, I’m not like, like, I don’t know tons of tech stu, like, I am passable at best.. . I can troubleshoot
small things, but when something massive is happening with an app that I know nothing about, I’m bewildered." Holly
(E1) mentions that “it does require skill on the therapist". Even Isaac (N1) , who suggested being technologically savvy
(“ we’re nerds here, and we like to do nerdy things"), mentions that “I took some practice because it was, it was a tedious
process in the beginning." Caroline (T3) shares that “planning for that session (with a robot) for me was harder, really hard
in the beginning", and that was because “I did have to learn how to use the software." Greg (T6) elaborates on the precise
and time-consuming ways that the introduction of a robot exacerbates existing preparation challenges: “you have to
have your content, and your interface pretty well mastered ahead of time to be able to respond as quickly as you need to
respond in a therapy session."
Preparing robot content requires far more detail when compared to not using a robot. Caroline (T3) , a therapist
with over 5 years of experience, described how when using a robot, “ I actually have to plan even more." They clarify: “If
it’s just me talking I have control about myself. If I have to put things in the robot prior then he can only follow whatever
I put there—my preparation needs to be well-polished." Three of our participants (T1, T3, E1) explicitly described how
planning details is a big part of preparing with a robot. Alissa (T1) mentions that with robots they had to “pre-plan
everything really extensively to be successful in those sessions." When asked about what was involved in preparation with
robots their initial response was just “A lot. A lot."
Recommendation:
We recommend that developers building robot teleoperation tools present functionality for
authoring content and operating robots in separate interfaces. Separating these modes of use acknowledges that
authoring (preparing) and operating occur at dierent times and require dierent kinds of mental eort. Authoring
generally occurs in the outer cycle when therapists have more time to be thorough. Teleoperation occurs during the
more time-sensitive inner cycle of therapy sessions. Authoring interfaces ought to support planning for a high level of
detail that simplies teleoperation while incurring minimal inconvenience for the content author.
3In music and dance therapy, music selection is analogous to content and activity selection in other forms of therapy.
14
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
5.1.2 Therapists over-prepare. For therapists, being well-prepared means being over-prepared. When Holly (E1)
described their preparation process with a robot, they emphasized: “I typically over-prepare—I mean have many activities
in the bag. . . in case you pull out an activity and the child is not interested in the activity, you might try another."
Preparing a large variety of activities helps therapists connect with their clients (see Section 5.2.2). An important part
of understanding therapists’ needs around over-preparation is understanding how much they over-prepare. Blanche
(T2) explained that “if the session is going to be 45 minutes or an hour I’m going to have probably 85 minutes worth
of music. . . I always over-prep everything. I’d rather be over-prepared than under-prepared.". That’s close to double the
amount of content used in-session. They add that over-preparedness is necessary because therapy is unpredictable (see
Section 5.4.1): “it might not happen, but I’m planning (for it)". Isaac (N1) arms this when they say “We always had a
backup". Greg (T6) mentioned bringing a “Mary Poppins bag of tricks", explaining that “we bring our big, you know, big
suitcase full of toys and, and reinforcers and gure out what works for (the client)". Although therapists’ over-preparation
is time-consuming, it nevertheless provides them with the tools necessary to form a strong relationship with their
clients (see Section 5.2.2) in order to create an emotionally safe environment (see Section 5.3.2).
While having more content can help therapists feel prepared, it makes it more dicult to organize and access
content easily in sessions. Blanche (T2) revealed that “I would write so much (session content) But then, sometimes in
the moment it’s hard to scroll through screens to nd what it is that you want." Blanche mitigated this issue by opting
for a highly modular system: “So for me, I always like to work with separate palettes for opening and closing and maybe
sometimes separate palettes for transitions. If I know I’m transitioning from this activity to the next one, this is a palette I
can come back to all the time." Our analysis uncovers a clear need for easy content organization but also shows that
therapists use diverse approaches to content organization. Robot teleoperation interfaces typically allow therapists to
group content to be verbalized by the robot into collections. Some teleoperators, such as Jaclyn (N2) , organized all
content chronologically, in a single collection. Some, as Blanche (T2) mentioned, created activity-based collections that
were modular. Others chose to create modular collections according to content theme, such as content for feedback,
redirection, or relationship-building, instead of specic activities, such as a game or book.
Recommendation:
We recommend that tools for robot content authoring and teleoperation support intuitive,
adaptable content organization strategies that allow therapists to plan activities easily and stay organized to minimize
demands on teleoperators’ attention during sessions. We note that by over-preparing, therapists are moving a dicult
process (creating an intervention) from the inner cycle, where they have a short amount of time, to the outer cycle,
where they have more time. This approach inspired some of our recommendations and is one of our high-level guidelines
outlined in section 4.
5.1.3 Preparation is collaborative. Our analysis shows that collaborative preparation is vital to therapy and occurs
during the outer cycle. Our participants described preparation as a matter of iterative input from many stakeholders.
Goal setting, which is a primary component of preparation, for a client or for a session can rely on parents, teachers,
insurance companies, and others.
Information about clients is often consolidated for easy access to a child’s information. For example, US children’s
special education teachers organize their goals into Individual Education Plans (IEPs). Holly (E1) described how they
begin with “knowing what (the client’s) IEP might say, having anecdotes from teachers—that’s preparation." Therapists
often rely on this crucial documentation from others (see Section 5.5.2). Fiona (T5) similarly recounted that “preparation
may include reviewing what kinds of goals are in the IEP, like anger management.. .to see what’s been going in class in
terms of behavior and everything we keep track of."
15
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Parents often provide substantial input into their children’s therapy. Greg (T6) told us how “the rst step of any
therapy is guring out a preference assessment, A lot of times you get that ahead of time from the parents (who) can tell
you what worked for them already.". Because parents’ input is given such weight, they can sometimes play a crucial role
in therapy. In the early stage when “the adult/the parent will typically establish these goals with the therapist", Emily
(T4) mentions that they would follow up with parents to elaborate on ways they can achieve these goals or other goals
that might be important. Several participants shared challenges of working with parents. Alissa (T1) when describing
out-of-session guidelines that parents should follow, “And they don’t want to follow it, and then they wonder why their
child is still (discouraged activity)". Overall, parents play a vital role in therapy through their input and expectations of
therapy.
While parents and therapists have direct interactions with children over the course of therapy, other indirect, yet
signicant stakeholders are also involved such as health insurance companies. In the USA, where all of our participants
practice, insurance approval is often required for children to receive therapy. Participants described coordinating
with insurance companies to approve a robot as a therapy tool or to incorporate it into a treatment plan. Greg (T6)
connected the approval process back to goal-setting: “Most therapy is paid for by insurance or by the state and so it’s
fairly structured. You have to be working towards very specic goals and those goals have to be agreed upon by the team".
Goals and children’s regular evaluations are used to determine whether a client receives funding for their therapy.
These regular evaluations are often handled by the therapists’ employer (see Section 5.6.1 ) so that a child’s records are
easily passed down if therapists change.
Recommendation:
We recommend that developers of robot tools incorporate collaborative communication tools
for various stakeholders to share information about children’s goals and preferences. These tools should account for
the role that insurance companies or other health institutions play in the use of socially assistive robots. Specically, it
is important for developers to understand that their interface design choices can impact, support or hinder, insurance
(or similar) approval.
5.2 Variety
When therapists prepared, they focused on getting a variety of activities ready for clients. All of our participants stressed
that having a variety of activities and interventions for a session helps therapy proceed smoothly and work eectively.
Therapists emphasize that variety is critical in order to customize sessions and build relationships with their clients.
Robots provide access to a variety of content which makes the robots appeal to therapists for use in therapy.
5.2.1 Variety is for customization. Therapists choose dierent interventions for dierent children based on factors
such as children’s age, the challenges they face, their therapy goals, and the features of their IEP or insurance plan.
Emily (T4) reected that the challenges a child faces “present in so many dierent ways; and then what’s important to
one child might not be important to another While “ there are some standard goals, like improving school performance or
improving social interaction, that looks dierent to dierent people."" All interviewed therapists (6 of 6) emphasized that
there is no single criterion by which to categorize children and determine whether they would benet from specic
types of activities.
Participants were reluctant to make generalizations based on children’s age or disability; some even challenged
interview questions that implied such generalizations. Greg (T6) pointed out that “with autism, there’s such variety.
You can have a ten-year-old working on the same skills as a ve-year-old, it’s really dependent on the child." Alissa (T1)
shares that “it’s important to not force kids into a box, that we want them to t in. Like I think it’s important to let a kid be
16
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
who they are, and to meet them where they’re at in any type of therapy that they get. " Technologists ought to be careful
when deciding to structure (or market) products according to disability or age. Therapists themselves rarely rely on
such generalizations, which are, on their own, often unhelpful or inaccurate for individual children. Our centering on
therapists’ perspectives presents the highly individualized process of therapy and shows that tools must work for all
children.
Recommendation:
We recommend tools easily accommodate a large variety of content that can be used in a
variety of activities to better engage with clients. As we have outlined earlier, engagement is a crucial part of therapy’s
ecacy [5].
Recommendation:
We recommend that tools for robot-assisted therapy allow therapists to easily tailor session
content to a child’s specic interests. For teleoperation interfaces, this could include features that allow operators to
populate content with information specic to a given child—favorite colors, sibling’s names, sports teams, and so forth.
By incorporating children’s preferences in teleoperation interfaces, tools can ease therapists’ delivery of potentially
stressful therapy sessions.
5.2.2 Variety is for relationship-building. Content and activity variety provides the structure to build a strong child-
therapist bond. Caroline (T3) described the role that variety plays in the initial formation of relationships with a client:
“Independent of age or developmental issues—I might bring board games, or just painting or drawing or singing or drama.
It could be innite possibilities to play with this child until I feel I have some rapport created. They need to trust in me."
Building a relationship between a therapist and a child is necessary to create a safe space for therapy (see Section 5.3.2).
Greg (T6) brought up similar points about the relationship between variety and trust: “in between all the structured
exercises, you want to maintain the relationship with the child, so you have to nd ways to make it fun and build that trust
relationship. So having a lot of variety is always good " Greg (T6) added that children often have specic interests and
specic media that they’re already partial to: “Trains are really common; a lot of kids get interested in trains. There are
lots of little categories."
Variety not only supports the varied interests of clients but allows therapists to identify these interests. Our
participants describe therapy as an experimental process where therapists try new activities and see what engages
their client. Holly (E1) mentions that when a child is not engaged, “it could take two or three attempts to gure out
why they’re not engaged". As Greg (T6) puts it, therapists are “just constantly trying new things until you gure out
what works and what turns (clients’) mind on". Oering variety fosters trust and promotes a positive client-therapist
relationship. However, having to consider this vast number of tools and activities for preparation likely contributes to
therapists’ over-preparation (see Section 5.1.2).
Recommendation:
We recommend that tools support or provide a variety of activities in a way that is easy to
organize and move between. The goal behind supporting a large variety of activities is to provide therapists with the
tools they need during the inner cycle of conducting a therapy session. However, when tools provide support for a
larger variety of activities, the need to consider dierent content organization approaches becomes even more crucial.
5.2.3 Robots provide variety. Robots’ ability to oer a large variety of activities contributes to their appeal for use in
therapy. Our participants reported that they use robots to facilitate games, read books with children, listen to children
read books, start conversations, and conduct many other activities. Fiona (T5) mentions that “there’s a lot of dierent
options with this robot and software; everything from academics, to social skills, to vocational skills, self, daily living skills,
stories". Isaac (N1) talks about having content for “a basic conversation. And some go-tos like either like Simon Says, I Spy."
17
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Therapists typically rely on a variety of content to administer therapy, whether or not they use a robot. However, with
robots, each individual activity is rigid in terms of customization. Emily (T4) describes without a robot, customization
is simpler “since I’m the one verbalizing everything I can customize it because I’m verbalizing". Holly (E1) describes their
diculty trying to customize activities for children using robots by explaining that “with responses of the robot you only
have one or two keys, you can press". These diculties can slow down therapy as therapists try to gure out how to use
the available tools. Robots add an extra step for the therapists when conducting therapy (see Section 5.3.3) especially
with the diculty of customizing content.
Instead of customizing specic activities, some therapists choose to provide variety by transitioning between dierent
activities. Holly (E1) mentioned that “I don’t think there was a lot of wiggle room for customizing the activity itself, but
you could customize which activity you chose.. . the selection of activities were diverse enough that if the child didn’t feel
comfortable reading a book with me, I could do a coloring book, for example." Alissa (T1) described how they could
prepare two books and “ let (the client) choose between these two books but I already have both of these books typed into the
robot so (the client) can choose between these two." However, if the child asks “can you read me my own book?" Alissa (T1)
would respond with “you know what (robot name) is tired today we’re gonna have to wait till next week to read this book.".
Alissa (T1) later explained that they may then ask to borrow that book and use it to prepare for an upcoming session.
Recommendation:
We recommend that tools allow for easily customizable and modiable content. One way to do
so would be by allowing the authoring of content with placeholders. This would reduce the workload on therapists
within the inner cycle to customize content and move much of the eort for customization to the outer cycle when they
are authoring content.
5.3 Awareness
Within the inner cycle, therapists say they are highly alert to everything going on. During a session, therapists must
monitor their client’s feelings, keep track of the features of the teleoperation interface, and resolve any technical
diculties. Therapists need to maintain the pace of conversation and avoid missing the important moments in a session.
5.3.1 Therapy demands circumstantial awareness. Our participants described dierent environments in which they
conduct their sessions, often homes or school classrooms. Each of these variations comes with dierent needs. That,
coupled with the vast assortment of goals that therapists work on with dierent clients, requires a high level of attention
from therapists. When we asked Alissa (T1) about what they need to be aware of in sessions they said “Everything.
Everything at the same time". They went on to add that “you have to be aware of of everything, because we work on potty
training, we work on how to hold utensils when you eat, we work on .. .". They went on to describe an in-home session
where the parent was stressed and home renovations were happening at the same time during therapy. This example
illustrates that, in addition to being unpredictable in terms of what a child might need (see Section 5.4.1), therapy can be
unpredictable in terms of environmental factors. Alissa (T1) said “I’m trying to, really, you know, do my work, but then
all of those factors denitely, denitely play a factor in, in... You know, what I’m doing". Therapy demands a great deal
from therapists, as Blanche (T2) explains “it is denitely a hard job and some days you go in there, like oh I’m already so
tired how am I going to do this".
Therapists need high awareness to detect factors that may be impacting their clients. When we asked Greg (T6)
about the sorts of things they are aware of during a session, they told us “Everything! You have to be aware of everything
in the environment at all times. You can be missing 90% of the picture if you’re not tuned in." Greg (T6) describes how
something as simple as “some ickering thing in the corner or some loud sound outside or some buzzing" could be “
18
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
absorbing (the client’s) awareness at the moment". Greg (T6) added that they also learned to be aware of contextual
factors in a child’s life (“what their day has been like, what their diet has been like, have they eaten at all today, do they
need to go the bathroom, did they just have a ght with their brother, sister.. .") Alissa (T1) shared a story of one instance
where a child hadn’t “pooped in four days" and how that aected the therapy session. “So you haven’t pooped and you
don’t sleep and now I’m sitting here asking you to to identify this item that you don’t care about so I’m gonna throw some
behavior I’m going to scratch or I’m going to do whatever because... It’s dumb!" Prior to sharing that story Alissa (T1)
mentioned that “I talk about bowel movements. A lot. So, I, forgive me, but.. . We talk about poop constantly." alluding to
the fact that the shared story is not a one-o occurrence. This story illustrates therapists’ need to be aware of not only
what’s going on within a session but also much of what else is going on in their clients’ lives. One way therapists do
that is through “anecdotal remarks from other sta, like So-and-so just got moved into a new foster home this week’" says
Holly (E1) . All these factors inevitably add to the cognitive load that therapists experience in the inner cycle of therapy.
Recommendation:
We recommend that developers avoid introducing any artifacts that create substantial complexity
for therapists. Tools should be as minimal and easy to use as possible so that therapists can continue to focus on what’s
important in therapy—the children.
5.3.2 Therapy depends on emotional safety. A fundamental component of therapy is the client’s feeling of emotional
safety. Fiona (T5) described how most of their awareness goes towards “reading” their clients, “So, you always want
to read them .. . because you want to build rapport. If you don’t, if the child doesn’t feel comfortable, then everything else
you’re trying is not going to be eective." This form of awareness builds the foundation for therapy. Without it, clients
cannot work toward any of their goals. 4 of 6 therapists described how their relationship with clients makes therapy
both possible and meaningful. Caroline (T3) explained that: “The most important thing for (clients) is not to understand
‘oh, I have to change.’ The most important thing for them is ‘I could be here sitting with this lady once a week because I do
feel safe here and I did create a nice relationship.’" Other participants emphasized this same sentiment. Blanche (T2)
armed how “the most important aspect of therapy for anybody is that experience of being seen, being witnessed—being
allowed to be your whole self." Emotional safety precedes all other activities in therapy. Therapists use their awareness
during a session to cultivate a safe space for their clients.
Children may feel emotionally safer with robots than with humans. Caroline (T3) describes the general pressures
that play a role in clients’ expression, “ as any human, we do have some type of social expectations. And some children
really struggled in responding to those social expectations, even if I tried my best." Caroline (T3) went on to discuss how
despite the tools available to them as therapists to help children regulate, since therapists are still human, therapists
present a “demand" on their clients. Caroline (T3) says that “The way the robot presents itself doesn’t come with like
this, many years of social representations you know. It feels safer for some clients." When robots are used in therapy, they
change the client-therapist relationship. Caroline (T3) describes that robots “create some like closeness and distance
between us and the client. In a way that can feel safe". Emily (T4) describes the change as a “perceived power shift. So, the
robot kind of creates this new dynamic that we would never be able to achieve in the session, because I’m bigger, and I’m
older and they know that, so it helps. The robot is less intimidating, not constantly watching every move the child makes,
and so that helps". Blanche (T2) mentions that “The safety that (clients) feel with a non- human entity, you know, they
there’s a level of automatic trust that I saw a lot of our kids have, that was safe for them". This perspective that robots help
in creating a safe space for children is shared by all of our participants. When asked about how a robot changes the
child’s experience of therapy, Alissa (T1) says “it’s denitely a safer environment". Children react to the robot in ways
that they normally wouldn’t with a human. To the point where parents say things like “Oh wow, my kid doesn’t even,
19
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
like, look at how they’re talking to that robot, like, they don’t, they don’t even talk". Jaclyn (N2) explains robots’ impact
saying, “the robot was able to open them up; reduce their inhibition and motivate them to verbalize". This is consistent
with the research [
57
] and is a large factor in why therapists use robots. When asked why they use robots, Jaclyn (N2)
says “because they work, because they absolutely hands down are the best tool we ever have come up with".
Recommendation:
Technologists should carefully ensure that their tools do not compromise therapists’ Situational
Awareness so that therapists can successfully deliver their planned interventions. Any technological tools which are too
distracting or frustrating run the risk of impairing the carefully-built sense of emotional safety that grounds therapy.
5.3.3 Robots require aention. When teleoperated robots are used in sessions, they place further demands on a
therapist’s attention during an already stressful inner cycle. Emily (T4) mentions that “the robot is just kind of like an
added step, so I actually think it adds to what I need to be aware of ", then later says “but I don’t think that’s a hindrance".
This sentiment that robots are dicult to use but are worth it comes up frequently. Robots may be considered worth
it because of how they change power dynamics with children, or due to the increase in variety that robots provide
therapists (see Section 5.2.3) to connect with clients.
Robots can introduce additional logistical concerns for therapists. Isaac (N1) , Alissa (T1) , and Holly (E1) told us
how tracking robots’ battery life complicated their sessions. Isaac (N1) mentioned that “Being aware of how well it was
charged, you know. In the beginning those charges held really well, but towards the end, I mean, it was getting used and
abused. Those products have a shelf life. "
Since therapists aim to have robots match the client’s pace of conversation, teleoperation interfaces themselves
inherently demand attention. Emily (T4) told us about this challenge: “That to me is the biggest struggle as a therapist—to
kind of maintain that that timeliness of response in the conversation when you’re trying to like type out a new button."
Greg (T6) had similar reections about their use of robots; “There’s a denitely a tempo that you have to keep to keep
engagement going... If you’re fumbling with the interface and you can’t gure out how to say the thing that you want
to say, but there’s a moment that’s happening right in front of you, and you want to address it and you want to make it
teachable.". In these moments, the technology should be getting out of the way but instead “ it’s still a lot of scanning
and searching to nd the visual button that you need at the moment. You’re maintaining a tempo and you’re trying not to
miss opportunities."
Recommendation:
We recommend developers build tools that minimize the attention required to teleoperate robots
in sessions, and that are easy to learn how to use. By making tools easier to learn, teleoperators can develop habits
using the teleoperation interfaces that make it easier to keep up with the pace of conversation in sessions.
5.4 Adaptability
As therapists maintain Situational Awareness of their environment in therapy, they must respond to what occurs
within their environment. Therapists describe therapy as an unpredictable, dynamic interaction in which maintaining
a positive relationship is key. As the client’s needs change within a session, therapists have to adapt and respond in
the moment. With robots specically, therapists have developed clever, narrative strategies to make up for technical
diculties and lags in child-robot conversation.
5.4.1 Therapy is unpredictable. In a session, a therapist’s success at delivering therapy heavily relies on how they
respond in the moment. Three participants explicitly mention that therapy rarely goes according to plan. Blanche
(T2) said that “it’s cute to sit there and, like, for me, before session to type in all these dierent possible responses and I do.
But then when I get there in the moment that doesn’t mean that I’m going to have the right response." Blanche (T2) later
20
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
explains that “it is super hard to predict what kids are gonna say". Children may also test the therapist’s knowledge and
preparation as Isaac (N1) illustrates, “I learned the hard way that the kids will test the robot and see if it does actually
know everything". Overall this just leads to more preparation (see Section 5.1.2). Alissa (T1) describes how, in order to
run a game involving colors with a robot, they “had to type in random colors in case somebody said that they wanted... so
I had to pre-plan a lot with the robots."
Regardless of over-preparation, Isaac (N1) also mentions the importance of being exible within a session “I would
kind of go in with more of an open mind . . . and go, you know, this is our goal. We might not reach it". Jaclyn (N2) seems
to agree when they say that “we kind of have to be exible and deal with whatever we get but, but we do try to take
everything into account that we possibly can to make things appropriate." Therapists have to respond to whatever is
happening in the session as illustrated by the inner cycle (examine, evaluate, then respond). Fiona (T5) mentions that
“maybe something else comes up in the conversation, and maybe I want to go to a dierent program that I didn’t pre plan".
When asked if this is stressful, Fiona (T5) said that “it’s not really stressful it’s just as long as they’re patient maybe they’ll
give me a second, I want to do something that’s really fun, .. . so it’s not stressful." Fiona (T5) then alludes to a need for
easier-to-use tools: “I don’t know, maybe there’s a quicker way to even nd it here I’m not sure." Holly (E1) mentions that
“most kids are pretty forgiving so I don’t think you have to be perfect at it". This likely explains why current tools continue
to be used despite their imperfections. The benets of children interacting with robots can outweigh the issues that
therapists have to deal with; but for this positive impact to scale, technologists should simplify their products so they
are more accessible to non-experts in robotics.
Recommendation:
We recommend tools simplify in-session authorship and provide simple organization techniques.
While we generally advocate for minimizing the use of autonomous tools in SARs, if they are to be used, we recommend
autonomous tools be used to support therapists in authoring content. An example is through providing therapists with
hypothetical responses that children might have. Therapists would then author content to respond to the hypothetical
interaction and prepare appropriate content in case that arises in a session.
5.4.2 Therapists adapt cleverly. Therapists adapt their activities according to what they notice in the moment. Holly
(E1) described how adaptation keeps children engaged, “I always focus on getting them back engaged by adapting, you
know, just walking over to them and talking to them about what they’re doing and get getting them back on track." Blanche
(T2) mentions that “You go in there with like a general goal, maybe for the student, but you have to be spontaneous and
you have to be ready to respond to what’s there". Emily (T4) describes a similar experience where therapy with new
clients may start o in a similar way, but over time “you nd out what the child prefers and what works and what doesn’t
that’s when the sessions, get a little, they diverge a little bit there, because you’re going in dierent directions at that point."
Therapists are constantly having to respond to children in therapy. Blanche (T2) highlights that in contrast to simply
delivering a lesson plan, “as a therapist my training is .. . not this is my list of things I’m going to accomplish, no matter
what else happens. That’s not how I operate." In practice, therapists are able to adapt in the moment during therapy
thanks to the variety of information they prepared for all the custom needs of their clients (see Section 5.2.1).
The use of robots introduces another facet of therapy that therapists have to adapt to, especially when technical
diculties arise. Our participants talked about how they manage technical diculties, low batteries, or teleoperation
delays. Emily (T4) described how: “ if the robot is thinking of a response and or is glitching then I’ll obviously step in."
Emily (T4) ’s experience shows the worst case scenario; when therapists are unable to resolve the issue using the
technology. Holly (E1) also described that when problems occur they “certainly can impair the whole process if you’re if
there’s a glitch in the technology".
21
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
With certain technical complications, therapists are able to continue to use the technology to explain away the
problems seamlessly through a narrative. Alissa (T1) says: “I use the power nap one a lot. Like right before I knew (the
robot’s) eyes would die, because I could see like the 10% in the corner, I would have a button that says “oh I’m getting so
sleepy! I need to take a quick nap!” and I would press that button right before it died." Emily (T4) mentioned utterances
they authored specically to compensate for delays: “I did have some buttons that said, like “Hmmmm. . . let me think
about that”, like after the child asks a question. I was doing a lot of free typing in and adding buttons on the y kind of
which denitely got a little stressful." When therapists teleoperate robots, pacing is critical to providing a natural-feeling,
positive experience for the child.
Recommendation:
We strongly recommend that technologists ensure that therapists have full control over robots
used in therapy. Specically, therapists should have the ability to override any robot functionality to regain control of
the situation.
5.5 Documentation
Therapists document their observations since documentation is a crucial component of the planning process and is often
used to justify therapy’s merit or validity to health insurance. Therapists assigned to a particular child may change over
time. Good documentation allows a new therapist to easily pick up where the last therapist left o. When therapists
document sessions, they often automatically include an expertise-based evaluation of the child. As developers, we
emphasize the dierence between documentation (writing down what happened) and evaluation (assigning a valence
of that outcome). When technology is used in therapy, tools may support therapist-authored documentation which
includes evaluations but we emphasize that any evaluation should be shaped and performed by the therapist themselves.
5.5.1 Therapists review documentation before sessions. As part of preparation, therapists review documentation. Ther-
apists explicitly mentioned this review process as a way of getting ready for a session. Blanche (T2) shared that “I
usually review my progress notes from the session before just so I can catch myself up, because details might escape you a
week between sessions". This review is to “ help me assess baseline levels for the top of the session " Caroline (T3) had
a similar experience saying that “before the session I usually kind of like review what happen last time .. . I’m going to
remind myself: this is where we were last week" Holly (E1) described a similar scenario for reviewing documentation,
“when I was in the hospital with children, I mean every morning, we would go over their documentation".
Greg (T6) pointed out that documentation happens both before and after each session, “For two hour sessions, usually
hour and a half doing direct therapy and 30 minutes documentation. 15 at the beginning 15 at the end." Alissa (T1)
mentioned that they “run trials on dierent activities and things of that nature and I collect data on that" implying that
this data collection occurs during the session. It’s clear that therapists have notes and collect data, because they’re
reviewing those at the beginning of the session, but isn’t clear from our interviews if there are explicit standard ways in
which they do so. Our participants mentioned documenting their interactions at the beginning, during sessions and / or
at the end of sessions. Regardless of the format in which they collect this information, we know that it is crucial for
them to review documentation, which plays a role in preparation being demanding (see Section 5.1.1).
Recommendation:
We recommend technologists develop tools that ease therapists’ review of documentation and
make documentation content easily accessible and editable throughout sessions. This is especially important in the US
where therapists are often paid by session hours, not work hours overall [
70
]. A tool that eases the review process is
more likely to be accepted by therapists who want to use their out-of-session time wisely.
22
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
5.5.2 Documentation is collaborative. Our participants described several dierent ways that their teams collaborated to
document therapy. Jaclyn (N2) described how “we have a Google form that all of the sta complete after any kind of
session they provide, whether it’s an art class or art therapy or anything. We call it an activity notes form. So, they ll that
out that gets compiled into a computer program." Alissa (T1) described how “I collect data on pretty much every move
that a child will make on a specic goal. There’s usually multiple goals" Other therapists similarly reconciled notes from
dierent stakeholders. Holly (E1) described how educators are an important source of documentation:“If you’re the head
teacher or a teacher’s aide or you’re in the classroom every day with that child then your homework would be that there’s
documentation. Okay, and everyone should be aware of the documentation, whether the child is you know, has atypical or
typical learning" Holly (E1) also describes how there is informal collaborative documentation: “It could also be that
going by the ... anecdotal remarks from other sta." Fiona (T5) has a similar experience asking others about their client,
“I’ll also talk to teachers, how is so and so doing in this behavior ... how they are in the class, as well as from administrators,
because we all have dierent relationships with them, and we see dierent sides of them". The documentation process
serves to synthesize knowledge and observations from multiple therapists, educators, parents, and other stakeholders in
each child’s life. As Fiona (T5) describes, this collaborative aspect of documentation is essential to see all the dierent
sides of the child.
Recommendation:
We recommend that tools are designed to ensure that therapists have easy ways of collaborating
to create and modify documentation from other stakeholders. Since it is important to understand a child’s needs from
dierent perspectives, we recommend that documentation tools be designed to highlight the individuals contributing
the documentation and their role in the child’s life. Understanding the context of a child’s needs can help therapists
determine the right action to take during therapy.
5.6 Evaluation
Evaluation is a collaborative and iterative process that ties together therapists’ responsibilities within and between each
session. Institutions provide support with evaluation by creating standard procedures to ease therapists’ evaluation
and maintain consistent records about clients. With iterative evaluation, therapists are able to monitor their clients’
progress, and report to other relevant stakeholders.
5.6.1 Evaluation is Institutional. Institutions can provide a lot of support for therapists and maintain a consistent
experience for children over the course of their therapy. As we mentioned earlier, Jaclyn (N2) describes a “Google form
that all of the sta complete after any kind of session". Institutions can provide a standard format for evaluating clients to
ease the burden on therapists. Jaclyn (N2) continues to describe “ summary forms —Those are evaluation forms and
those we have for individuals and we have for groups, and we have those for any kind of robot engagement that we do".
While these per-session evaluation forms are important, being able to analyze the success of therapy over a long period
of time is also crucial. Jaclyn (N2) describes “ evaluations after each program" where programs are six, eight, or ten
weeks. As the director of an institution, Jaclyn (N2) mentions running “a pilot for our multi-sensory session, that was
a six-month pilot but we broke it into like two three-months summers for that" An essential part of these pilots is the
evaluation of the therapy delivered. If the evaluation shows a successful delivery of the therapy, then these programs
are then deployed. Through deployment, the organization creates all the content necessary to deliver the program and
establishes the necessary metrics for a program’s evaluation. Using this structure, new therapists can deliver these
repeatable programs with ease.
23
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Institutions provide continuity when working with the client and other stakeholders. When discussing preparation
for a new client, Greg (T6) mentioned that you’re usually reading the notes from a therapist that worked with the child last.
So you know if they picked up on something denitely want to try to build o of that. The continuity across therapeutic
evaluations is helpful especially when we consider the experimental nature of therapy in identifying what children
respond eectively to. When describing the experimental process to connect with a child, Greg (T6) even mentions a
last resort option to “swap out therapists" for the child. While ideally, children would work with the same therapist over
time, institutions provide an ability for them to work with other therapists as well.
When incorporating robots in therapy through an institution, clients may develop a relationship with the robot and
not the therapist. That means that if the therapist changes, therapy for the children is not signicantly hindered. Alissa
(T1) describes their experience entering an organization that had been working with robots: “I kind of came in. Maybe
in the middle of when they had started, but there were some schools that had already seen (the robot) for like a year and a
half or two years before I started. So when I came in, they already kind of had this like established relationship with this
robot." Kids would respond to the robot by telling other kids things like “hey man, like, look at this, like, I knew him
before" or “oh yeah, I remember him". This experience was despite them having never met Alissa (T1) before.
Recommendation:
We recommend that tools be designed to ensure that it is easy to share and retain information
within an institution. Since documentation about therapy is private client information, this may create added privacy
concerns for tools and require extra caution in the development of such tools.
5.6.2 Evaluation is Iterative. Therapists report that it is crucial to continuously evaluate clients to monitor their
improvement and ensure that they are receiving appropriate interventions. Greg (T6) mentions that a child’s overall
“goals are determined periodically throughout the year" and that they are“very specic to the children" resulting in “a
lot of variety" of goals. Not only do therapists have dierent goals related to dierent children, but their goals for an
individual child usually change over time. Caroline (T3) describes that “through the sessions, we realize, you know, that
goal could shift a little bit more to this or to that". Caroline (T3) later describes that even though these goals may change,
“we have to have goals. We have to know where we want to get. So, based on that. You need a(n) assessment." Caroline (T3)
shows us that evaluation is a central part of therapy.
Therapists describe two key ways in which the results of their assessments are used; insurance and preparation.
Alissa (T1) explains that “insurance requires an updated treatment plan every like four to six months". Alissa (T1) shares
that an update to a child’s treatment plan will be made up of “a whole new set of goals that are put into the treatment plan
that he needs to work on, because he made progress, in these, these are all done, we don’t need to work on these anymore, so
now we’re going to move to this other stu that we need to work on." Insurance needs these updates to ensure that the
funding they are distributing to the child is worthwhile and the therapy is eective. Therapists also want to make sure
they are delivering eective therapy and use assessments to determine upcoming interventions. Fiona (T5) mentions
that “We can measure how much of a goal has been accomplished, and if it’s less than maybe 70 to 80% I may want to
continue with that", showing that evaluation directly feeds into session preparation.
Recommendation:
We recommend that tools display reports of the client’s progress on various activities, as
determined by a therapist. To repeat a critical point: while technology should be used to support therapists, we advocate
for therapeutic evaluation to be entirely performed by therapists and not incorporate any automated tools. Developers
should also consider how their tool’s approach to evaluation may help therapists with insurance approval.
24
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
6 PATTERNS ACROSS THEMES
As we’ve shown through our analysis, many therapists’ needs and diculties with robots are interconnected. In this
section, we highlight some important high-level relationships across our themes that provide an alternative presentation
for technologists and help re-enforce the needs of therapists. For each of these relationships, we present the relevant
sub-themes while highlighting the cycle of therapy to which they generally belong.
6.1 Therapists manage uncertainty to ensure emotional safety
During the time-sensitive inner cycle, therapists maintain awareness of a multitude of factors (Section 5.3.1), especially
their client’s emotional safety (Section 5.3.2). The lack of predictability in therapy (Section 5.4.1) requires therapists to
adapt in the moment to meet their clients’ needs. To do so, therapists use the available time during the outer cycle to
over-prepare by authoring a large amount of interventions (Section 5.1.2) that they rely on to more easily adapt in the
moment. This over-preparation often involves creating a large variety of content that is specic to their client and
eases connecting with the client to build a relationship (Section 5.2.2). Equipped with the large variety of interventions
that therapists over-prepare, they can more easily create an emotionally safe and engaging space for children to receive
the therapy they need. We present a visual representation of this pattern in Figure 5.
Fig. 5. During the inner cycle of therapy, therapists manage the various factors producing uncertainty in order to ensure emotional
safety for their clients. Importantly, they rely on their over-preparation in the outer cycle to deliver eective therapy.
6.2 Therapists adapt to customize therapy
During the inner cycle, therapists have to adapt cleverly to their environment, client’s needs, and any technical issues
that arise from the interventions they deliver (Section 5.4.2). To adapt, therapists rely on a variety of content that
they’ve customized to meet these particular needs (Section 5.2.1). The variety of interventions available to therapists is
what allows them to customize and adapt. Customization ensures that clients are engaged, which is crucial in delivering
eective therapy. We present a visualization of this relationship that considers the dual cycle model of therapy in
Figure 6.
25
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Fig. 6. During the inner cycle, therapists rely on the variety of interventions available in order to adapt in the moment by customizing
interventions to the client’s in-the-moment needs.
6.3 Robots create a Variety-Aention trade-o
Robots’ ability to introduce and support a large variety of interventions is a key reason for therapists’ interest in using
them for support in therapy (Section 5.2.3). However, therapists also explain that robots increase the demands on their
attention during therapy (Section 5.3.3). Therapists describe a trade-o (see Figure 7) between the value a robot brings
in terms of an engaging variety of activities and the cost of having to learn to use a potentially complex system in an
already stressful time. It is therefore crucial for technologists to minimize the attention required to use teleoperating
interfaces in this context.
Fig. 7. Robots create a tradeo between the variety they can provide and the aention they require during a session.
6.4 Therapy constantly changes
As part of preparation for therapy, therapists have to review documentation from previous sessions or other collaborators
(Section 5.5.1). This review increases the demanding nature of preparation (Section 5.1.1) since the client’s evaluation
throughout therapy is iterative (Section 5.6.2). As therapists deliver therapeutic interventions for their clients, their
26
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
clients achieve the established therapeutic goals. After achieving these goals, the clients’ therapeutic needs begin to
change. Through preparation in the outer cycle, therapists adapt sessions to their client’s changing long-term needs.
Preparation is a crucial step that can minimize the eort required and stress endured when delivering therapy. We
present a visualization of this relationship in Figure 8.
Fig. 8. The iterative nature of evaluating a client results in demanding preparation for therapists who review the constantly changing
documentation of the child’s progress.
6.5 Collaboration exists throughout therapy
Therapy is collaborative during preparation (Section 5.1.3), documentation (Section 5.5.2), and evaluation (Section 5.6.1).
Collaboration involves a diverse set of stakeholders including parents or guardians, teachers, therapy institutions, and
insurance companies, each with dierent perspectives about the child. As we have described before, these stakeholders
are mostly involved in the outer cycle. We show how collaboration crosses several themes and is mostly present in the
outer cycle in Figure 9.
Fig. 9. Therapy is collaborative across preparation, documentation, and evaluation.
27
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
7 DISCUSSION
7.1 Recommendations Summary
By better understanding the needs of our participants, we are able to provide both high level and detailed recom-
mendations. We begin with our overarching recommendation to technologists: focus on the teleoperators of robots as
the end-user of teleoperation interfaces. To meet teleoperators’ needs, we group our recommendations under three
high-level guidelines: (1) meet therapists’ needs in the moment, (2) account for the inner and outer cycle and their
relationships, and (3) move actions to the outer cycle when possible. We present a summary of all recommendations in
Table 2with references to the particular problems they address in the relevant sections above.
Recommendation Section Target
(1) Meet therapists’ needs in the moment:
Support both modular and chronological content organization. 5.1.2 General
Support a large variety of content. 5.2.1 General
Support a variety of content accounting for child-specic preferences. 5.2.2 General
Take into consideration and design for the likely non-technical audience of tools.
5.3.3 General
Add minimal distraction in the inner cycle. 5.3.1 Operating
Consider the sensitive context of therapy that requires emotional safety for the
child.
5.3.2 Operating
Design for pace of conversation. 5.3.3 Operating
Incorporate easy on-the-y authoring during the inner cycle. 5.4.1 Operating
Ensure that teleoperators have as much control as possible including overriding
any automations introduced.
5.4.2 Operating
(2) Account for inner and outer cycle and their relationships:
Incorporate collaborative documentation that identies documentation author. 5.5.2 General
Allow institutional content organization and sharing. 5.6.1 General
Support collaborative goal-setting and preparation. 5.1.3 Authoring
Allow access to edit and view documentation in the inner cycle. 5.5.1 Operating
Present previous sessions’ evaluations in the inner cycle. 5.6.2 Operating
(3) Move actions to the outer cycle when possible:
Separate authoring and operating modes in robot-assisted therapy interfaces. 5.1.1 General
Incorporate dynamic placeholders for information that can be gathered in the
inner cycle.
5.2.3 General
Incorporate hypothetical responses during authoring to help therapists imagine
how therapy could go.
5.4.1 Authoring
Table 2. Summary of considerations for technologies supporting robot-assisted therapy.
7.1.1 Meet therapists’ needs in the moment. Robot-assisted technologies need to meet therapists’ needs in the moment.
We recommend that technology for robot-assisted therapy support a large variety of content, that accounts for
child-specic preferences and is easy to organize in several ways based on therapists’ preferences. Technology for robot-
assisted therapy should account for the technical expertise of the therapist operator across tasks like setup, authoring,
and teleoperation. Currently, systems often assume more technological expertise and debugging capabilities than
teleoperators possess. During a session, technology needs to ensure minimal distraction to teleoperators considering
the sensitive context of therapy. Robot-assisted therapy tools ought to ensure it is easy for therapists to incorporate
28
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
in-session authoring and maintain the pace of conversation. While some tools may already oer a form of on-the-y
authoring, the capabilities are often limited to a simple textbox to type in. Our participants highly valued the ability to
author content in the moment and we recommend further improvements. Robot-assisted therapy technologies should
ensure that teleoperators maintain as much control as possible in order to best leverage their expertise. Current systems
often introduce automation or autonomy to support therapists, but instead, we recommend that these robot-assisted
therapy solutions rely on the expertise of therapists who are trained to handle precisely the scenarios that arise during
therapy and ensure therapists have full control to override any automation.
7.1.2 Account for the inner and outer cycle and how they relate. When technologists consider tools for robot-assisted
therapy, they often focus on the inner cycle of sessions without consideration for the outer cycle and how it inuences
therapists. We recommend that technologists account for both the inner and outer cycles and how they relate; specically
the collaboration that occurs across these cycles with multiple stakeholders. Robot-assisted therapy technology should
support collaborative documentation that identies the documentation author. Current systems may allow for simple
content importing and exporting but should consider improved sharing capabilities to support the institutional and
collaborative nature of therapy. Importantly, teleoperators should be able to easily access documentation and evaluation
reports within sessions and edit them with updated information in real time.
7.1.3 Move tasks to the outer cycle when possible. Considering the outer cycle’s inuence on the inner cycle is clearly
important, but technologists should not stop there. We noticed that therapists’ approach to preparation in the outer
cycle is intended to move actions from the inner cycle to the outer cycle. Technologists should follow this approach that
therapists are already taking. For example, we recommend content incorporate dynamic placeholders for information
that can be gathered about individual clients during sessions. Currently, therapists have to duplicate content and make
many small edits to customize content. Incorporating dynamic placeholders would allow therapists to easily duplicate
and customize content to their clients.
It is also critical to consider how the inner and outer cycles dier. While teleoperation tools oer some authoring
capabilities, these capabilities are not sucient for therapists’ authoring needs in the outer cycle. We recommend
technologists develop separate authoring and operating modes and tailor these modes to therapists’ dierent needs
in the outer and inner cycles. While authoring content, therapists often imagine how their clients will respond to
chosen content and develop additional content to accommodate those responses. Therapists may also use content over
many sessions and update content based on unexpected responses during those sessions. We recommend robot-assisted
technology for authoring content to provide hypothetical client responses to therapists that may minimize the time it
takes to prepare content.
7.2 Limitations & Future Work
In this section, we share two key limitations of our study, and provide suggestions for future work. First, although our
sample of participants is sucient to infer many important patterns about robot-assisted therapy for neurodivergent
children, the raw number of participants is quite small. Second, the way we recruited participants could be responsible
for some similarity of perspectives within our sample.
Our sample was small because the population we were investigating was itself quite small and comprised of individuals
with a very specic and narrow area of expertise. To recruit individuals with the specic set of expertise and experience
needed, we needed to leverage our collaborator’s professional network. In turn, this means that there were likely some
network eects within our sample, due to similar experiences shared across our participants. Nevertheless, even within
29
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
our sample, participants show a diverse range of therapeutic expertise, backgrounds, and experience, as shown in
Table 1. As robot-assisted therapy gains further adoption, we expect it to be easier to recruit participants for similar
studies. As systems improve following our recommendations presented in this paper, we also expect therapists’ needs
to change to match the limitations of the systems present at the time.
In the future, it may thus become both more necessary and more feasible for researchers to recruit larger samples,
and to engage in longitudinal work to investigate therapists’ needs as they change over time. Future work should
also evaluate implementations of our recommendations based on therapist feedback, as motivated by our overarching
recommendation: listen to therapists.
7.3 Vision
In this section, we share a vision of robot-assisted therapy that encompasses the principles behind our recommendations.
We use this vision as an example of how to apply our recommendations to robot-assisted therapy in particular, and
as an exemplar for teleoperated socially assistive robot systems in general. We describe the experience of a ctional
therapist, Dana, who uses a ctional “Ideal Robot-Assisted Therapy” (IRAT) solution in their practice.
Dana is taking on a new client in their practice, Chloe. In Dana’s rst meeting with Chloe, they document the client’s
needs and goals in the IRAT system. Dana can also see information provided by the children’s parents and teachers in
the IRAT system ahead of time with an easy way to follow up with them to ask questions. After the rst evaluation,
Danands intervention suggestions in the IRAT system that other trusted therapists had labeled for clients with similar
goals to Chloe. Dana pins some of those for easy access in the future. Dana also notices that some of Chloe’s goals are
similar to another client they are already working with, Moe.
A week later, Dana is preparing for their rst session with Chloe just 10 minutes before the session, they open the
IRAT system to check their notes from the initial evaluation, and any additional comments from teachers and parents.
They nd no comments from teachers or parents. Dana starts preparing for a session based on the main goal that they
prioritized for Chloe; Dana selects three of the pinned activities as a reference to use for the session. Dana then sets up
the robot and IRAT teleoperation interface in just under a minute. When Chloe comes to the session with their teacher,
the teacher mentions that Chloe had a dicult time today with a friend. Dana quickly notes that information from the
teacher in the IRAT system and nds content to support relationship building and communication. Dana begins the
session focusing on what Chloe needed in the moment. After the rst twenty minutes of the session, Dana is able to
switch to Chloe’s originally prioritized goals and make some progress towards them by using the originally pinned
activities. As Chloe’s engagement increases, they begin asking many questions and the conversation veers o into a
topic that Dana does not have content for. Dana uses the IRAT teleoperation interface to easily respond with on-the-y
answers to Chloe’s questions. At the end of the session, the IRAT system displays a summary of the session for Dana to
review. Dana adds some notes in the evaluation section to note how Chloe is doing. Dana packs up the robot and
teleoperation device, which are still at over 90% battery, and goes to the next session.
Dana arrives late to their next session with Moe. They quickly glance over the IRAT system summary from last
session, and select an activity they did that session. Dana gets the robot set up in just under a minute and Moe is
very excited to interact with it. Dana easily selects some relationship-building content to engage with Moe as they
look through content they want to use to best match Moe’s goals. Dana starts the activity that they did last time with
Moe but Moe seems uninterested. Dana needs to choose another topic instead and nds that the activity they just did
with Chloe would be a good t. They select the activity and the IRAT system automatically customizes the activity
based on Moe’s prole of interests that Dana has updated over many previous sessions. Moe is engaged with the robot
30
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
as Dana selects content easily and is able to calmly relax after being late to the session. They start reading through
Moe’s documentation and nd that Moe’s parents had mentioned some tasks that they want Moe to work on. While
the parent’s suggestion is useful, Dana relies on their expertise to pick an activity that matches the underlying goal to
address. Dana nds an activity from the vast collection of content in the IRAT system and pins it to work on next.
As Moe engages with the robot, Dana determines that Moe has progressed in reaching their main goal. Dana notes
that in the IRAT system. They then switch to the new activity and work on that. The session continues smoothly and
Dana reviews the summary at the end, adds additional notes, and updates their evaluation further. As Dana packs up
and is ready to leave, Moe’s parent cheerily comes to pick up Moe and is appreciative of Dana’s work; Moe’s parent had
seen the updated evaluation in the IRAT system from earlier in the session and are very happy with the therapy their
child is receiving. Moe’s family has been very happy with the progress Moe has made and have purchased an aordable
robot to use at home. Dana suggests some activities for them to use at home to follow up on therapy. These activities
were originally designed by Dana’s colleague who ensured that they would be easy for parents to follow and Dana has
made some changes to accommodate Moe’s particular goals.
After leaving, Dana recalls that their manager had asked them to develop some content to support other colleagues.
Through their experience, Dana has become an expert at supporting children in dealing with conict. At home,
Dana selects some session activities from the IRAT authoring system and begins to update and customize activities
based on their experience. The IRAT system oers suggestions for what children may say in response and creates an
engaging authoring experience that makes authoring enjoyable and more eective. The IRAT system’s suggestions
remind Dana of previous sessions and they’re able to look them up for reference. After completing authoring the
activity, Dana shares them with their colleagues and manager who are very grateful for the eort they put in.
8 CONCLUSION
As robots are adopted in therapy for neurodivergent children, it is crucial to understand how these technological tools
are being used by therapists and educators, and the ways that these tools are changing and mediating the practice of
therapy and education. We interviewed nine participants who had all used robots to diering extents in the context of
therapy for neurodivergent children. In each interview, we discussed with participants how they prepare and conduct
therapy with neurodivergent children in their usual practice, how they do so when using a robot, and we asked them to
walk us through an example robot teleoperation interface. We conducted a grounded theory analysis of our interview
transcripts to determine key themes around therapists’ experience in therapy. We guided our analysis with the intention
to summarize and explain a therapist’s experience to non-therapists who are developing tools that support therapists in
delivering therapy.
Our analysis results in several key contributions: (1) a new understanding of the cyclical processes within therapy that
may generalize to all longitudinal social assistance, (2) a taxonomy of six themes that help elucidate the unique needs
and diculties of therapists using robots with neurodivergent children, (3) guidelines and detailed recommendations for
technologists to address therapists’ needs in particular and social assistance experts’ needs in general when operating
robots in their practice, and (4) a vision of the ideal robot-assisted therapy tool informed by therapists’ needs and our
recommendations. Our results demonstrate that therapy can be best understood as a process consisting of two cycles:
the outer and inner cycle shown in Figure 3. The outer cycle occurs between sessions on a time scale of days or weeks,
during which therapists and stakeholders collaboratively examine and evaluate children to prepare and conduct therapy
sessions. The inner cycle occurs within each individual session and exists on a time scale of minutes or hours, during
which therapists re-examine and re-evaluate their clients in the moment in order to adapt, and/or deliver therapeutic
31
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
content. Understanding how tools relate to each of these cycles is crucial to delivering the most eective support for
therapists.
Our analysis produced six themes of needs and diculties that arise during therapy; preparation, awareness, variety,
adaptability, documentation, and evaluation. We described each of these themes, how they relate, and recommendations
to address these needs in robot-assisted therapy technologies. We present a concise collection of our recommendations
as a reference in Table 2. Finally, we outlined a vision of an ideal robot-assisted therapy system describing how a
therapist would use these systems to engage with new clients, existing clients, and colleagues.
Our study, analysis, and approach reect and encourage an overarching belief that developers should approach the
design of therapeutic tools by examining how therapists use current tools, and using such examination to determine
what therapists most need to deliver eective therapy for their clients. This approach may avoid the stumbling blocks
that currently arise from the over-emphasis on the automation of therapeutic activities.
REFERENCES
[1]
George Adamides, Georgios Christou, Christos Katsanos, Michalis Xenos, and Thanasis Hadzilacos. 2014. Usability guidelines for the design of
robot teleoperation: A taxonomy. IEEE Transactions on Human-Machine Systems 45, 2 (2014), 256–262.
[2]
Thomas Armstrong. 2010. Neurodiversity: Discovering the extraordinary gifts of autism, ADHD, dyslexia, and other brain dierences. ReadHowYouWant.
com.
[3] Thomas Armstrong. 2017. Neurodiversity: The Future of Special Education?. Educational Leadership 74, 7 (2017), 10–16.
[4] Paul Atkinson, Amanda Coey, and Sara Delamont. 2003. Key themes in qualitative research: Continuities and changes. Rowman Altamira.
[5] Kenneth Barish. 2004. What Is Therapeutic in Child Therapy? I. Therapeutic Engagement. Psychoanalytic psychology 21, 3 (2004), 385.
[6]
Tony Belpaeme, James Kennedy, Aditi Ramachandran, Brian Scassellati, and Fumihide Tanaka. 2018. Social robots for education: A review. Science
robotics 3, 21 (2018).
[7]
Teal W Benevides, Stephen M Shore, Kate Palmer, Patricia Duncan, Alex Plank, May-Lynn Andresen, Reid Caplan, Barb Cook, Dena Gassner,
Becca Lory Hector, et al
.
2020. Listening to the autistic voice: Mental health priorities to guide research and practice in autism from a stakeholder-
driven project. Autism 24, 4 (2020), 822–833.
[8]
Fabiane Barreto Vavassori Benitti. 2012. Exploring the educational potential of robotics in schools: A systematic review. Computers & Education 58,
3 (2012), 978–988.
[9]
Alain Bernard. 2005. Virtual engineering: methods and tools. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering
Manufacture 219, 5 (2005), 413–421.
[10] M Blamires. 1997. Towards an educational model for pupils with autism and Asperger’s syndrome. Wiley.
[11]
Madeline M Blankenship and Cathy Bodine. 2020. Socially Assistive Robots for Children With Cerebral Palsy: A Meta-Analysis. IEEE Transactions
on Medical Robotics and Bionics 3, 1 (2020), 21–30.
[12]
Laura Boccanfuso, Sarah Scarborough, Ruth K Abramson, Alicia V Hall, Harry H Wright, and Jason M O’Kane. 2017. A low-cost socially assistive
robot and robot-assisted intervention for children with autism spectrum disorder: eld trials and lessons learned. Autonomous Robots 41, 3 (2017),
637–655.
[13]
Kristen Bottema-Beutel, Steven K Kapp, Jessica Nina Lester, Noah J Sasson, and Brittany N Hand. 2021. Avoiding ableist language: Suggestions for
autism researchers. Autism in Adulthood 3, 1 (2021), 18–29.
[14]
Joanna Butchart, Reema Harrison, Jan Ritchie, Felip Martí, Chris McCarthy, Sarah Knight, and Adam Scheinberg. 2021. Child and parent perceptions
of acceptability and therapeutic value of a socially assistive robot used during pediatric rehabilitation. Disability and rehabilitation 43, 2 (2021),
163–170.
[15]
John-John Cabibihan, Hifza Javed, Marcelo Ang, and Sharifah Mariam Aljunied. 2013. Why robots? A survey on the roles and benets of social
robots in the therapy of children with autism. International journal of social robotics 5, 4 (2013), 593–618.
[16]
Jonathan Casas, Emmanuel Senft, Luisa F Gutierrez, Monica Rincon-Rocancio, Marcela Munera, Tony Belpaeme, and Carlos A Cifuentes. 2021.
Social assistive robots: assessing the impact of a training assistant robot in cardiac rehabilitation. International Journal of Social Robotics 13, 6 (2021),
1189–1203.
[17]
Thierry Chaminade, David Da Fonseca, Delphine Rosset, Ewald Lutcher, Gordon Cheng, and Christine Deruelle. 2012. FMRI study of young adults
with autism interacting with a humanoid robot. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive
Communication. IEEE, 380–385.
[18]
Thierry Chaminade, Delphine Rosset, David Da Fonseca, Jessica K Hodgins, and Christine Deruelle. 2015. Anthropomorphic bias found in typically
developing children is not found in children with autistic spectrum disorder. Autism 19, 2 (2015), 248–251.
32
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
[19]
Robert Chapman. 2019. Neurodiversity theory and its discontents: Autism, schizophrenia, and the social model of disability. The Bloomsbury
companion to philosophy of psychiatry 371 (2019).
[20] Kathy Charmaz. 2006. Constructing grounded theory: A practical guide through qualitative analysis. sage.
[21]
Jessie YC Chen, Ellen C Haas, and Michael J Barnes. 2007. Human performance issues and user interface design for teleoperated robots. IEEE
Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37, 6 (2007), 1231–1245.
[22]
Ylona Chun Tie, Melanie Birks, and Karen Francis. 2019. Grounded theory research: A design framework for novice researchers. SAGE open medicine
7 (2019).
[23]
Eva Yin-han Chung. 2019. Robotic intervention program for enhancement of social engagement among children with autism spectrum disorder.
Journal of Developmental and Physical Disabilities 31, 4 (2019), 419–434.
[24]
Eva Yin-han Chung. 2020. Robot-Mediated Social Skill Intervention Programme for Children with Autism Spectrum Disorder: An ABA Time-Series
Study. International Journal of Social Robotics (2020), 1–13.
[25]
Oliver Damm, Karoline Malchus, Petra Jaecks, Soeren Krach, Frieder Paulus, Marnix Naber, Andreas Jansen, Inge Kamp-Becker, Wolfgang Einhaeuser-
Treyer, Prisca Stenneken, et al
.
2013. Dierent gaze behavior in human-robot interaction in Asperger’s syndrome: An eye-tracking study. In IEEE
International Symposium on Robot-Human Interactive Communication (RO-MAN). 368–369.
[26]
Julia Dawe, Craig Sutherland, Alex Barco, and Elizabeth Broadbent. 2019. Can social robots help children in healthcare contexts? A scoping review.
BMJ paediatrics open 3, 1 (2019).
[27] Paulo G De Barros and Robert W Lindeman. 2009. A survey of user interfaces for robot teleoperation. Worcester Polytechnic Institute (2009).
[28]
Guy Dewsbury, Karen Clarke, Dave Randall, Mark Rounceeld, and Ian Sommerville. 2004. The anti-social model of disability. Disability & society
19, 2 (2004), 145–158.
[29]
Joshua J Diehl, Lauren M Schmitt, Michael Villano, and Charles R Crowell. 2012. The clinical use of robots for individuals with autism spectrum
disorders: A critical review. Research in autism spectrum disorders 6, 1 (2012), 249–262.
[30]
Saad Elbeleidy, Terran Mott, Dan Liu, and Tom Williams. 2022. Practical Considerations for Deploying Robot Teleoperation in Therapy and
Telehealth. In 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 977–984.
[31]
Saad Elbeleidy, Terran Mott, and Tom Williams. 2022. Practical, Ethical, and Overlooked: Teleoperated Socially Assistive Robots in the Quest for
Autonomy. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (Sapporo, Hokkaido, Japan) (HRI ’22). IEEE
Press, 577–587.
[32]
Saad Elbeleidy, Daniel Rosen, Dan Liu, Aubrey Shick, and Tom Williams. 2021. Analyzing Teleoperation Interface Usage of Robots in Therapy for
Children with Autism. In Proceedings of the ACM Interaction Design and Children Conference.
[33]
David Feil-Seifer and Maja J Mataric. 2005. Dening socially assistive robotics. In 9th International Conference on Rehabilitation Robotics, 2005. ICORR
2005. IEEE, 465–468.
[34]
Irini Giannopulu. 2013. Multimodal cognitive nonverbal and verbal interactions: the neurorehabilitation of autistic children via mobile toy robots.
IARIA International Journal of Advances in Life Sciences 5 (2013).
[35] Gülay Hasdoˇ
gan. 1996. The role of user models in product design for assessment of user needs. Design Studies 17, 1 (1996), 19–33.
[36]
Flannery Hope Currin, Kyle Diederich, Kaitlyn Blasi, Allyson Dale Schmidt, Holly David, Kerry Peterman, and Juan Pablo Hourcade. 2021. Supporting
Shy Preschool Children in Joining Social Play. In Interaction Design and Children. 396–407.
[37] Fine Art Miracles Inc. 2021. Fine Art Miracles.https://neartmiracles.com/
[38]
Katarzyna Kabacińska, Tony J Prescott, and Julie M Robillard. 2020. Socially Assistive Robots as Mental Health Interventions for Children: A Scoping
Review. International Journal of Social Robotics (2020), 1–17.
[39]
Katarzyna Kabacińska, Tony J Prescott, and Julie M Robillard. 2021. Socially assistive robots as mental health interventions for children: a scoping
review. International Journal of Social Robotics 13, 5 (2021), 919–935.
[40]
Elizabeth S Kim, Lauren D Berkovits, Emily P Bernier, Dan Leyzberg, Frederick Shic, Rhea Paul, and Brian Scassellati. 2013. Social robots as
embedded reinforcers of social behavior in children with autism. Journal of autism and developmental disorders 43, 5 (2013), 1038–1049.
[41] S Lichiardopol. 2007. A survey on teleoperation. Technische Universitat Eindhoven, DCT report 20 (2007), 40–60.
[42]
Kirsti Malterud, Volkert Dirk Siersma, and Ann Dorrit Guassora. 2016. Sample size in qualitative interview studies: guided by information power.
Qualitative health research 26, 13 (2016), 1753–1760.
[43] Maja J Matarić and Brian Scassellati. 2016. Socially assistive robotics. In Springer handbook of robotics. Springer, 1973–1994.
[44] Micki McGee. 2012. Neurodiversity. Contexts 11, 3 (2012), 12–13.
[45]
François Michaud, Tamie Salter, Audrey Duquette, Henri Mercier, Michel Lauria, Helene Larouche, and Francois Larose. 2007. Assistive technologies
and child-robot interaction. In AAAI spring symposium on multidisciplinary collaboration for socially assistive robotics.
[46]
MD Moniruzzaman, Alexander Rassau, Douglas Chai, and Syed Mohammed Shamsul Islam. 2022. Teleoperation methods and enhancement
techniques for mobile robots: A comprehensive survey. Robotics and Autonomous Systems 150 (2022), 103973.
[47]
Junya Nakanishi, Jun Baba, and Hiroshi Ishiguro. 2022. Robot-Mediated Interaction Between Children and Older Adults: A Pilot Study for Greeting
Tasks in Nursery Schools. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (Sapporo, Hokkaido, Japan) (HRI
’22). IEEE Press, 63–70.
[48]
Autistic Self Advocacy Network and L. Berry. 2020. Welcome to the Autistic Community. Autistic Press. https://books.google.com/books?id=
bzRszQEACAAJ
33
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
[49] PEERbots. 2021. PEERbots.https://peerbots.org
[50]
Giovanni Pioggia, ML Sica, Marcello Ferro, Roberta Igliozzi, Filippo Muratori, Arti Ahluwalia, and Danilo De Rossi. 2007. Human-robot interaction in
autism: FACE, an android-based social therapy. In RO-MAN 2007-the 16th IEEE international symposium on robot and human interactive communication.
IEEE, 605–612.
[51]
Sarah M Rabbitt, Alan E Kazdin, and Brian Scassellati. 2015. Integrating socially assistive robotics into mental healthcare interventions: Applications
and recommendations for expanded use. Clinical psychology review 35 (2015), 35–46.
[52]
Ben Robins, Kerstin Dautenhahn, Rene Te Boekhorst, and Aude Billard. 2004. Eects of repeated exposure to a humanoid robot on children with
autism. In Designing a more inclusive world. Springer, 225–236.
[53] Movia Robotics. 2021. Movia Robotics.https://moviarobotics.com/
[54] Misty Robotics. 2022. Misty Robotics.https://www.mistyrobotics.com/
[55]
Mohammed A Saleh, Habibah Hashim, Nur Nabila Mohamed, Ali Abd Almisreb, and Benjamin Durakovic. 2020. Robots and autistic children: a
review. Periodicals of Engineering and Natural Sciences (PEN) 8, 3 (2020), 1247–1262.
[56]
Zohreh Salimi, Ensiyeh Jenabi, and Saeid Bashirian. 2021. Are Social Robots Ready Yet to be Used in Care and Therapy of Autism Spectrum Disorder:
A Systematic Review of Randomized Controlled Trials. Neuroscience & Biobehavioral Reviews (2021).
[57] Brian Scassellati, Henny Admoni, and Maja Matarić. 2012. Robots for use in autism research. Annual review of biomedical engineering 14 (2012).
[58]
Brian Scassellati, Laura Boccanfuso, Chien-Ming Huang, Marilena Mademtzi, Meiying Qin, Nicole Salomons, Pamela Ventola, and Frederick Shic.
2018. Improving social skills in children with ASD using a long-term, in-home social robot. Science Robotics 3, 21 (2018).
[59]
Syamimi Shamsuddin, Hanaah Yussof, Salina Mohamed, Fazah Akhtar Hanapiah, and Luth Idzhar Ismail. 2013. Stereotyped behavior of autistic
children with lower IQ level in HRI with a humanoid robot. In 2013 IEEE Workshop on Advanced Robotics and its Social Impacts. IEEE, 175–180.
[60]
Aubrey Shick. 2013. Romibo Robot Project: An Open-Source Eort to Develop a Low-Cost Sensory Adaptable Robot for Special Needs Therapy and
Education. In ACM SIGGRAPH 2013 Studio Talks (Anaheim, California) (SIGGRAPH ’13). Association for Computing Machinery, New York, NY, USA,
Article 16, 1 pages. https://doi.org/10.1145/2503673.2503689
[61]
Richard L Simpson, Michael McKee, Dixie Teeter, and Alyson Beytien. 2007. Evidence-based methods for children and youth with autism spectrum
disorders: Stakeholder issues and perspectives. Exceptionality 15, 4 (2007), 203–217.
[62]
Leila Takayama. 2022. Putting Human-Robot Interaction Research into Design Practice. In 2022 17th ACM/IEEE International Conference on
Human-Robot Interaction (HRI). IEEE, 1–1.
[63]
Amber Thiessen and Jessica Brown. 2021. Personalization of restorative and compensatory treatments for people with aphasia: A review of the
evidence. Topics in Language Disorders 41, 3 (2021), 269–281.
[64]
Sanna J Thompson, Kimberly Bender, Janet Lantry, and Patrick M Flynn. 2007. Treatment engagement: Building therapeutic alliance in home-based
treatment with adolescents and their families. Contemporary Family Therapy 29 (2007), 39–55.
[65]
Lai Poh Emily Toh, Albert Causo, Pei-Wen Tzuo, I-Ming Chen, and Song Huat Yeo. 2016. A review on the use of robots in education and young
children. Journal of Educational Technology & Society 19, 2 (2016), 148–163.
[66] Georgiana Shick Tryon and Greta Winograd. 2011. Goal consensus and collaboration. (2011).
[67]
Joshua Wainer, Ester Ferrari, Kerstin Dautenhahn, and Ben Robins. 2010. The eectiveness of using a robotics class to foster collaboration among
groups of children with autism in an exploratory study. Personal and Ubiquitous Computing 14, 5 (2010), 445–455.
[68]
Joshua Wainer, Ben Robins, Farshid Amirabdollahian, and Kerstin Dautenhahn. 2014. Using the humanoid robot KASPAR to autonomously play
triadic games and facilitate collaborative play among children with autism. IEEE Transactions on Autonomous Mental Development 6, 3 (2014),
183–199.
[69]
Meng-ning Wang, Jonathan Sandberg, Amy Zavada, Mona Mittal, Anne Gosling, Tziporah Rosenberg, Aaron Jerey, and Justin McPheters. 2006.
“Almost there”... Why clients fail to engage in family therapy: An exploratory study. Contemporary family therapy 28 (2006), 211–224.
[70]
Maureen Werrbach. [n.d.]. Common compensation models for group practices. https://blog.therapynotes.com/common- compensation-models- for-
group-practices
34
Centering Teleoperators in Robot-Child Interactions Reveals the Bigger Picture CSCW’23, October 13-18, 2023, Minneapolis, MN, USA
A INTERVIEW PROCEDURE
As we described in Section 3.3, our interviews were broken up into two portions. The rst portion discussed participants’
experience with therapy in general, when applicable. The second portion discussed participants’ experience working
with robot-assisted therapy technologies. In this section we describe the procedure we followed for each portion of the
interview.
A.1 estions about therapy in general
In the rst half of the interview, the interviewer began by stating that they would ask questions about therapy sessions
in general, and that this portion of the interview was explicitly intended to discuss sessions where participants are
not
using a robot. The interviewer then suggested that participants think of two children with whom they had recently
conducted therapy. Our guiding questions were as follows:
•Could you describe how you conduct a therapy session?
–How does this dier for dierent disabilities that children may have?
–What do you have to maintain awareness of during a therapy session?
–What is most important to you when conducting a therapy session?
–What do you nd is the most dicult in conducting therapy with children?
–What do you sense is most important to the child receiving therapy during the session?
•Could you describe what you do to prepare for a therapy session?
–How does your preparation dier for dierent disabilities that children may have?
–How much customization do you typically do for a particular child receiving therapy?
–How do you establish goals for a child? What kind of goals are there?
–How do you go about preparing materials for the therapy session?
A.2 estions about robot-assisted therapy
In the second half of the interview, the interviewer asked participants about their experience preparing and conducting
for sessions with a robot. Similar to the rst half, the interviewer asked participants to think of the most recent time
they had used robots in a session. Our guiding questions were as follows:
•Could you describe what you do to prepare for a therapy session using a socially assistive robot?
–How does it dier from preparing for sessions without using a robot?
–How much customization for each child are you able to do when using a robot?
–How does using a robot change the child’s experience and response during therapy?
–Are children’s responses to the robot consistent across dierent disabilities and conditions?
–How much do you talk compared to the robot? Do you ever have a conversation with the robot?
Following those questions, the interviewer shared their screen and showed participants the example teleoperation
interface shown in Figure 10 of the Peerbots application [
49
]. We used this example interface to center the conversation
around teleoperation to understand how participants would use an interface, what they expect from interfaces, and
their likes and dislikes with interfaces. We asked participants some questions about that interface following the listed
questions below as a guide:
35
CSCW’23, October 13-18, 2023, Minneapolis, MN, USA Elbeleidy , et al.
Fig. 10. An example of the Peerbots teleoperation interface that was shown to participants.
•Do you recognize the interface shown in this image? Have you used it before?
•Can you talk me through how you’ve used it in the past [or how you would use it]?
•Is there anything in the interface that you’ve never used before?
•What do you like about the interface?
•What do you dislike about the interface?
•What do you wish was easier to do using the interface?
•Is there anything within the interface that you nd useless?
•
How would you rate the interface on a scale of 1 to 10 where 1 is very dicult to use and 10 is very easy to use?
•Do you see any opportunities for using robots in therapy in the future?
•Is there anything else you would like to tell us about using robots in therapy with children?
36
