Conference PaperPDF Available

Conversational Technologies for In-home Learning: Using Co-Design to Understand Children's and Parents' Perspectives

Authors:

Abstract and Figures

Today, Conversational Agents (CA) are deeply integrated into the daily lives of millions of families, which has led children to extensively interact with such devices. Studies have suggested that the social nature of CA makes them a good learning companion for children. Therefore, to understand children's preferences for the use of CAs for the purpose of in-home learning, we conducted three participatory design sessions. In order to identify parents' requirements in this regard, we also included them in the third session. We found that children expect such devices to possess a personality and an advanced level of intelligence, and support multiple content domains and learning modes and human-like conversations. Parents desire such devices to include them in their children's learning activities, foster social engagement, and to allow them to monitor their children's use. This understanding will inform the design of future CAs for the purpose of in-home learning.
Content may be subject to copyright.
Conversational Technologies for In-home Learning: Using
Co-Design to Understand Children’s and Parents’
Perspectives
Radhika Garg
School of Information Studies
Syracuse University, USA
rgarg01@syr.edu
Subhasree Sengupta
School of Information Studies
Syracuse University, USA
susengup@syr.edu
ABSTRACT
Today, Conversational Agents (CA) are deeply integrated into
the daily lives of millions of families, which has led children to
extensively interact with such devices. Studies have suggested
that the social nature of CA makes them a good learning
companion for children. Therefore, to understand children’s
preferences for the use of CAs for the purpose of in-home
learning, we conducted three participatory design sessions. In
order to identify parents’ requirements in this regard, we also
included them in the third session. We found that children
expect such devices to possess a personality and an advanced
level of intelligence, and support multiple content domains
and learning modes and human-like conversations. Parents
desire such devices to include them in their children’s learning
activities, foster social engagement, and to allow them to
monitor their children’s use. This understanding will inform
the design of future CAs for the purpose of in-home learning.
Author Keywords
conversational agents; participatory design; children; parents;
home; learning; learning companion; co-design; learning
technology; cooperative inquiry.
CCS Concepts
Human-centered computing Human computer inter-
action (HCI);
INTRODUCTION
Conversational Agents (CA) such as Google Assistant and
Amazon Alexa have in recent years become deeply interwo-
ven into families’ daily lives through devices such as smart
phones and smart speakers. This has led children increasingly
to interact with CAs for various purposes, such as information
seeking, functional reasons, and entertainment purposes [39].
Furthermore, dialogic reading, which CAs can facilitate, has
the potential to scaffold children’s learning and foster their
interest in long-term learning [62]. Therefore, prior work (e.g.,
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from permissions@acm.org.
CHI’20, April 25–30, 2020, Honolulu, HI, USA
© 2020 ACM. ISBN 978-1-4503-6708-0/20/04.. .$15.00
DOI: 10.1145/3313831.3376631
[8, 25, 31, 3, 59]) has sought to leverage CAs to develop vir-
tual learning companions for children. Researchers (e.g., [3,
59]) have even found that children learn more and report more
satisfaction using learning programs that incorporate virtual
conversational tutors. However, despite the potential of these
programs to simulate a social companion that can engage chil-
dren in conversations for learning purposes [9, 52, 62], less
is known about the needs, preferences, and perceptions of
children using these these this technology for their in-home
learning efforts. Furthermore, parental involvement has been
found to influence children’s use and perception of technology
[30, 41, 48, 56, 53] and their learning performance [21]. How-
ever, there is limited research regarding parents’ preferences
and requirements for the design of technology that can help
with the in-home learning needs of children. Therefore, with
this work, we aim to answer the following research questions:
RQ1: How do children think about in-home conversa-
tional technology that can be used to fulfill their learning
needs?
RQ2: What are the requirements of parents for conversa-
tional technology that can be used to fulfill their children’s
learning needs?
Developing a deep understanding of children’s preferences
regarding intelligent learning technology would be difficult
with traditional modes of qualitative data collection (e.g., in-
terviews, observations, surveys) [60], not only because of the
complex nature of conversational learning companions (e.g.,
learning modes and levels, and human characteristics such as
voice), but also because children can become exhausted in an
interview setting, or have difficulty understanding interview
questions or accurately expressing their mental models [37,
61]. Researchers have demonstrated that Cooperative Inquiry
(CI) enables a researcher to collect rich data from children, as
it enables children to concretely articulate their abstract ideas,
especially compared to conventional methods [16, 17, 27, 58].
Therefore, for this study, we adopted the Collaborative Partici-
patory Design (CPD) method of CI. In CI, adults and children
design technologies for children, with children [17, 18, 26,
65]. To answer our research questions, we conducted three
co-design sessions with an inter-generational group consisting
of 12 children aged 7-12 and their parents (who participated
in the third session with their children).
After transcribing the audio and video recordings of the design
sessions, we analyzed them using affinity diagram [29]. We
found that children expect such devices to possess a high level
of intelligence in terms of human-like conversation, adaptabil-
ity, and emotional intelligence. Children also expected these
devices to take on the role of learning companion, teacher, and
friend. Parents desired such devices to include them in their
children’s learning activities and to allow them to control and
monitor their children’s use.
We contribute in two ways to the discussion and design of
future CAs that could become in-home learning technologies
for children: First, we provide empirical of children’s prefer-
ences for in-home learning technologies and arrive at a set of
recommendations for designers. Second, we provide parental
recommendations for managing learning technologies in the
home, in terms of design features that could help them partici-
pate in learning activities and/or control children’s use when
needed.
RELATED WORK
To orient readers to our work, this section describes two
streams of related research: Children’s use of conversational
technology for learning at home and the role of parents in
children’s learning and in-home technology use.
Children’s Interactions with Conversational Technology
for Learning
The use of conversational agents at home is widespread, both
among the adults and children [52]. Traditionally, children
have been known to use technology at home not only to en-
tertain themselves, but also to learn [22, 33]. Lovato et al.
[38, 39] has showed that children primarily use CAs to seek
information, which is considered a skill children require to
succeed in school, as they are asked to complete homework
and in-class assignments that hinge on search ability [22].
However, researchers [23, 52, 39] have also found that in-
stead of helping children to understand concepts, CAs can
currently only provide direct answers to children’s questions.
Therefore, unsurprisingly, researchers (e.g., [8, 25, 31, 40, 62])
have sought to improve CAs so that they can become virtual
learning companions for children. For example, Mack et al.
[40] conducted a participatory design study with elementary
and middle school children, with the intent to inform the inter-
face design of a social studies educational application. They
concluded that including conversational dialogues in such ap-
plications, such as while communicating with virtual historical
figures, improves children’s perception of the technology. Xu
and Warschauer [62], in designing a storytelling CA, con-
cluded that children enjoyed interacting with the device for
activities such as reading books.
Prior work has also suggested utilizing parasocial relationships
(i.e., one sided emotional bonds that children develop with
familiar media figures) to develop intelligent “characters” that
children can use as learning tools or companions [5, 7, 25, 42].
Based on the experiences gathered from Sesame Workshop,
Gray et al. [25] emphasized the use of personification and
social realism in developing intelligent characters to foster
learning in children. Along similar lines, due to the human-like
characteristics of robots, researchers (e.g., [28, 51]) have found
them to be superior to other educational media and technology
in promoting and improving children’s concentration, interest
in learning, and academic achievement. Purington et al. [49]
have provided evidence of adults and children who already
attribute human-like attributes to CAs and perceive them as
their companions. Researchers (e.g., [25, 7]), however, have
also argued that the development of these relationships is
influenced by parental approval or disapproval – similar to
how parent’s opinions influence children’s relationships with
other people – as parents might selectively encourage or limit
screen time (we go into detail on this in the next section).
Another stream of research (e.g., [47, 35]) has investigated
efficient ways of integrating such CAs into instructional and
educational settings. There has been work around investigat-
ing the potential uses of conversational pedagogical Aritifical
Intelligence (AI) in classrooms, including intelligent tutoring
systems and chatbots [34, 54] and exploring how conversa-
tional agents can support inclusive education [44]. Lester et al.
[36] pointed out that due to ‘persona effect’ children learned
more and reported more satisfaction using learning programs
that incorporated virtual conversational tutors.
Instead of developing instructional materials, our study aims to
inform the design of intelligent conversational technology for
children that can help them in their learning endeavors at home.
Specifically, this work can inform the design of intelligent in-
home learning technologies that cater to children’s preferences
and expectations.
The Role of Parents in Children’s Learning and Technol-
ogy Use
Children’s use and perceptions of a technology are often me-
diated by their parents. Prior work has identified several cate-
gories of parental mediation: in restrictive mediation, parents
set limits on permissible activities; in active mediation, parents
and children discuss appropriate content and use; and in co-
engagement, parents and children consume content together
[30, 41, 48, 56, 53]. Researchers have shown that parents
employ restrictive mediation the most and that they establish
context- and activity-specific rules for children’s media use
[30]. However, the ability to implement parental control is
not straightforward – parental preferences for restricting chil-
dren’s technology use are contextual [41], and if a child’s
access, such as to the Internet, is over-restricted, it may even
compromise his or her ability to use the device to learn, such
as the ability to complete homework [19].
Parents contribute significantly to their children’s in-home
learning by participating in “family learnings”, which gener-
ally refers to the involvement of parents in their children’s nu-
meracy and literacy activities, reading with children at home,
and encouraging homework [14]. Garg and Sengupta [23]
found that while parents in general restrict children’s use of
technology, they encourage children to use devices such as
smart phones and smart speakers as part of “family learning”
activities. Joint media engagement or co-engagement with
technology by parents and children for the purpose of learning
has been found to have a positive effect on children’s grades
[21] and academic achievement [32].
In reference to conversational technology, recent work by
Beneteau et al. [4] and Cheng et al. [11] have examined
children’s interaction with voice-based devices in the home.
Their findings noted that children and parents spend a lot of
time, at least in first few weeks of use, mitigating the break-
down of children’s communication through repair. However,
the current scholarship offers little on parents’ preferences
and requirements for these technologies that become in-home
learning companions of their children (e.g., desired function-
alities, such as the ability to control children’s use, monitor
children’s progress, and jointly engage in learning activities).
Therefore, we aimed to fill this gap by involving parents in
our study to elicit their preferences in regard to the design of
intelligent in-home learning technologies for children.
METHOD
Prior HCI scholarship with children that utilized CI in the
context of complex topics such as intelligent user interfaces,
voice interfaces for the visually impaired, the nature of creepi-
ness and technology, and privacy and online safety [43, 44, 61,
65] has not only shown the importance of including children
as design companions, but has also produced rich data for
improving design of future technology. Therefore, to answer
our research questions, we conducted three CPD sessions in-
volving partnerships between children and adults (i.e., design
researchers and, in the third session, parents as well). We used
the techniques of Stickies [17, 58], Bags of Stuff [17, 58], Big
Props [58], and Layered Elaboration [57].
We structured our design sessions to begin broadly. We
explored what children thought about using conversational
agents for learning activities by having them use different
speech agents that are currently available in the market. Later,
we asked them to design technologies that they thought would
help them in their in-home learning activities. In the last
session, we included parents as design companions so that
parent-child pairs could collaboratively adapt children’s de-
sign(s) to include the preferences of parents as well.
Participants
An intergenerational co-design group, consisting of both child
and adult participants, participated in the design sessions along
with adult design researchers. The first two design sessions
involved the children and researchers; for the third session,
one parent of each child was asked to join as a design partner.
The child participants were recruited through their parents,
who volunteered to participate by responding to recruitment
flyers on Craigslist and in local community centers and li-
braries. When recruiting, we aimed for diversity in gender,
age, and ethnicity to elicit multiple perspectives. Our study
included 12 children between 7 and 12 years of age (see Ta-
ble 1). The adult participants consisted of one undergraduate
student, one graduate student, and one professor [mean age
= 27.6, SD = 4.02]. Between the recruitment of children and
the three design sessions that the paper includes, the research
team (first author and one undergraduate student) had also
conducted twelve other 90-minute co-design sessions over a
period of 6 months (one every two weeks) where children and
researchers engaged in exploratory design practices to build
equal and equitable partnership.
Our protocol for the study was approved by the review board
of the institution. Upon arrival, the children and parents were
informed of the purpose of the research and their role in the
design sessions. We also informed them that we would be
audio- and video-recording our conversations with them, and
that the recordings in their raw form would be used solely by
the research team for analytical purposes and would not be
shared online or with any other audience. We also informed the
participants that we would be collecting any design artifacts. If
they agreed to these conditions, we obtained parental consent
(for their own and their children’s participation) and child
assent (for their own participation).
Name Age Gender Ethnicity Participant
Parent
Milly 7
Female
Asian Mother
Lily 7
Female
White Father
Sam 8 Male Asian Mother
Marianne
8
Female
Asian Father
Chris 8 Male While Mother
Marc 9 Male African Amer-
ican
Father
Juliet 9
Female
White Mother
Susanne 10
Female
Hispanic Mother
Yuling 10
Female
Asian Father
Lauren 11
Female
Asian Mother
John 12 Male White Father
Daniel 12 Male African Amer-
ican
Father
Table 1: Demographic Details of Our Participants. (Note: The
names listed here are pseudonyms.)
Design Sessions
Our study comprised three 90-minute sessions, which were
conducted over two weeks, with the last two design sessions
organized on the same day of the second week. Each day
began with a snack time (15 minutes), which is known to form
bonds with children that lead to trust and teamwork. This was
followed up with circle time (15 minutes), when we asked the
“Question of the Day” to help the participants (i.e., the parents
and children) begin to focus on the day’s design activities
(cf. Table 2). After that, the participants broke into smaller
inter-generational groups for the design session (45 minutes).
Finally, there was discussion time (15 minutes), during which
every small group presented their designs, and all the partic-
ipants then reflected together and discussed common ideas
across different designs. To document the design sessions,
an audio recorder was placed near each small group, and a
volunteer with a video camera moved throughout the room to
capture interactions in different groups.
Design Session 1 (DS 1)
On the first day, we elicited information regarding how chil-
dren think about and use different speech agents for the pur-
pose of learning. To introduce the topic of the day, we asked
the children to share how and for which learning goals they
use technology at home. All our participants had prior expe-
rience using several forms of technology (e.g., tablets, smart
speakers, computers) for the purpose of completing homework
or learning in general. During the design session, we used four
different speech agents – Google Home Assistant, Microsoft
Cortana, Apple Siri, and Amazon Alexa – for a learning ac-
tivity. We did not provide children with examples of learning
activities we were interested in, but we explicitly told them we
wanted to understand what activities they would like to use
the devices for. The design groups evaluated the agents in a
rotating manner, using the Stickies CPD technique [17, 58].
Following this approach, we gave our participants sticky notes
and asked them to write their likes, dislikes, and design ideas
for improving/changing the agents in light of the learning ac-
tivity they tried to use the agent for. We did not in any way
control or constrain the responses given by the agents.
Figure 1: Children thinking about their designs during the
Session 2 co-design activity.
Design Session 2 (DS 2)
The first co-design activity on day 2 utilized the Bags of Stuff
[17] and Big Props [58] techniques. For the first activity,
the children were asked to design technology (e.g., construct
or depict features of technology) that they would like to use
for learning the using technique of Bags of Stuff [17] (see
Figure 1). For this technique, we provided large bags that were
filled with arts and craft supplies, such as glue, colored paper,
markers, styrofoam shapes, pom-pom balls, and scissors. The
children and adults were then supposed to act out a scenario
of using the learning technology they had designed, using the
Big Props technique. The role play began with the children
acting as the system and adults playing the part of a potential
user. After about 15 minutes, they switched roles. In addition
to the material provided in the Bags of Stuff, we gave the
children easel pads to act out a scenario. We did not ask them
to design learning technology that could fulfill a certain social
role, such as learning companion or a teacher, as we did not
want to prime their mental models. Furthermore, we did not
restrict children to speech as the only input/output modality
in their designs, in order to elicit their preferred input/output
modalities.
Design Session 3 (DS 3)
After an hour break, we reconvened on day 2 for the third co-
design session, which included parents as design companions.
At this session, we used the technique of Layered Elaboration
[57], according to which the design teams iteratively generated
ideas by extending existing concepts, while leaving prior ideas
intact. The parents were first presented with the ideas that
the children had developed. Our co-designers – parents and
children – were then asked for ideas about how to make the
existing designs better by adding new features or by modifying
existing ones. We encouraged the parents and children to work
collaboratively when suggesting design ideas.
Design Session Question of the Day
DS 1
How and for which learning goals or
tasks do you use a technology, such as a
computer or tablet for in your home?
DS 2
How do you want technology to be bet-
ter designed for the purpose of in-home
learning?
DS 3
Illustrate an instance when your tech-
nology use was influenced by your par-
ent(s).
Table 2: Question of the Day for the Design Sessions
Data Analysis
The audio and video recordings (totaling 162 minutes of data
– duplicate portions of the recordings were transcribed only
once and snack time was excluded) were transcribed for the
purpose of analysis. The transcribed data was analyzed using
affinity diagramming, an inductive method commonly used to
refine and organize statements into larger themes [29]. The
data were analyzed using grounded, inductive, and qualitative
methods [10, 13]. The authors of this paper collaboratively
constructed the affinity diagram using Miro, formerly referred
to as Realtimeboard [45]. The analysis began by identifying
individual child and parent utterances. Each statement was
then represented as a digital note on the board. Both authors
participated in several rounds of discussions to identify the
theme associated with each utterance. At the end of this pro-
cess, all the notes were categorized into 25 themes. We also
identified various relationships among these themes that led
us to combine them into four major themes: System Output
Modalities and Behavior, User Input Modalities and Behav-
ior, System Intelligence, and Privacy Concerns. We provide
further details about the themes in the sections that follow.
RESULTS
Our analysis of the data revealed children’s and parents’ per-
ceptions and needs in reference to in-home technology that
can be used for learning. We discuss below in detail the major
themes and sub-themes that emerged from our analysis, with
corresponding examples from the co-design sessions.
System Output Modalities and Behavior
In the three design sessions, we identified several themes
where children and parents conceptualized the devices’ inter-
nal logic or functioning and their output modalities. Several
themes that emerged included: Content Domains and Learning
Modes, Output Modalities, Persona, Fostering Social Engage-
ment, and Parental Involvement and Control.
Content Domains and Learning Modes
During the design sessions, it became apparent that children
expected the devices to offer learning activities corresponding
to any subject the user chose. For example, while talking
about his design Marc, during DS 2, stated, So you can learn
whatever you want with it. You can seek general knowledge,
learn subjects they teach you in school like Math and English,
and learn about your favourite sport.
In DS 3, parents and children also included in their designs
the capability to play games and engage in other fun activities
(e.g., sketching or listening to music). For example, Lauren’s
mother said, “I do not think children will continue to use it for
long if there is no fun associated with using the device. So it
should have activities that attract children to interact with it.
Children also included various modes of learning in their
designs, which users could select from. Some of the modes
frequently included in children’s designs were these:
1.
Instead of simply providing direct answers to users’ ques-
tions as children learn a new or revise a previously learned
topic, one of the modes used scaffolding to help users work
out the answers to the queries themselves. This was mostly
done by providing hints in response to users’ questions.
For example, in DS 2, while acting as a system, Daniel
responded to the user’s question in the following way:
Adult: “Where is the Eiffel Tower located ?”
Daniel [speaking as the system]: “That city is
also the capital of France.
Adult: “Is it Paris?”
Daniel [speaking as the system]: “Yes, Good
job!”
2.
Children also included “storytelling” as one of the learning
modes in the designs. In this mode, devices helped users to
learn through engaging stories. The children designed the
device to tell stories through voice, images, or videos on
the screen. Prior work has argued that engaging stories can
support the learner’s intellectual and emotional needs [25],
which in turn can facilitate the development of parasocial
relationships. For instance, during DS 2, Sam noted, “When
children are tired but have a pending homework to complete,
this option will let them learn through stories that is always
fun.
3.
During DS 2, the children included practice drills and for-
mative and summative assessments in the form of quizzes
and competitive games (to be played with other users, who
might be in the same room or in a remote location, or the
device itself), as one of the learning modes in their designs.
The possibility of competing with the device echoes the find-
ings of Woodyard et al. [61], who found that children like
to challenge and compete with the intelligence of intelligent
devices.
During DS 3, parents added the functionality of generating
weekly or monthly progress reports on the assessments that
the children participated in. For example, during DS 3
Juliet’s mother said,
“If my daughter is preparing for some specific test we can
generate reports to track her progress. But, when she is
using it [the device] for day-to-day learning we can disable
this feature. We do not want to watch her progress like a
hawk all the time.
Output Modalities
Children considered a wide range of output modalities for the
systems they designed, with voice, text, images, videos, sound,
facial expressions (in cases where the device was designed to
include a face), and physical output being the most common.
While the fact that we asked the children to evaluate different
speech agents in DS 1 might have influenced the inclusion of
voice in all of their designs, it was more important to under-
stand their reasoning for including multiple output modalities.
Frequently cited reasons were to scaffold users’ interactions
(e.g., if a user is deaf, he or she can view the response in the
form of text), to express emotion (e.g., facial expressions or
gestures were used to convey emotions), and to reward the
user.
For example, in DS 3, the parents and children, while dis-
cussing the need for devices to reward children’s progress,
included physical output in form of merit badges (made from
colored paper in various shapes, such as stars and trophies)
and “candies” (pom-pom balls representing candies). In DS
3, Lauren’s mother observed: “To applaud progress, the de-
vice can reward the child in the form of stars, something very
similar to how it happens in a school setting.
Persona
In DS 1, we observed that, before starting to interact for the
purpose of information seeking or learning, children tried to
attribute identity to the devices, by asking questions such as
“How old are you?” “Where do you live?” and “Can you
understand if I speak in Chinese?” Often, when the children
found the responses to such questions to be unsatisfactory,
they lost interest in learning from the devices. Christine noted,
“If he [device] does not know what his age is, how should I
believe he will tell me true facts about my favourite historical
figure?”
In subsequent design sessions, the children conceptualized
intelligent technology for learning with a personality and a
backstory. In terms of personality, they assigned the devices
physical human features, such as voices and human anatomy
(e.g., faces, hands), and the ability to move; intellectual fea-
tures, such as decision making ability and knowledge; emo-
tional states, which could change based on those of its user,
and human behavior, reflected in its various functionalities.
The backstory of the technology was often based on that of
children’s favorite media character (e.g., the character’s demo-
graphics, expertise, and so forth).
Depending on the assigned personality, children also concep-
tualized these devices to play several roles, with companion,
teacher, and friend being the ones that were most mentioned
during the design sessions. For instance, Lily designed a de-
vice that looked like Robo Baby [55] and expected it to act
as a life-long companion, one who grows old with the user
and helps with learning. She said, “It looks like Robo Baby
from Rescue Bot. It will be same age as that of the user and it
keeps growing too. Just like a companion would. They grow
together and learn together.
Fostering Social Engagement
In DS 3, parents expressed concern about the children’s con-
ceptualization of devices as potential companions or friends.
As a solution, parents built on the children’s designs by adding
the ability to foster social interaction and engagement, which
would serve as a “reality check” (Milly’s mother, DS3) for
children. For example, Lauren’s mother, during DS 3, ex-
plained, “I have added the feature of social games, which
children can play with their siblings or friends, but also col-
laboratively learn in the process. This way they kind of know
a technology is only a technology after all.
This shows that parents required the technology to facilitate
children’s social interactions with other people in the house-
hold, as otherwise it might negatively influence the children’s
social awareness.
Parental Involvement and Control
Prior work has found that parents to influence children’s use
of technology [23, 30, 48]. Therefore, we included parents as
design partners in the third co-design sessions. In this session,
parents refined the children’s designs to reflect their perspec-
tives on parental involvement in children’s use of learning
technology, on controlling and monitoring children’s tech-
nology use, and on privacy concerns. The following section
provides details in reference to these themes.
Parents desired to be able to participate in children’s learning
activities. Considering their current involvement in children’s
learning and educational endeavors, the parents assumed sev-
eral roles – co-learner, motivator, overseer – in children’s use
of learning technology. For many parents, this was driven by
their fear that their children’s continual interactions with con-
nected educational devices would exacerbate their dependence
on technology, which in turn could displace parenting relation-
ships by excluding tasks that would otherwise be considered
primarily a parent’s responsibility. For example, Marc’s father
noted in DS 3,
“I am scared to think, that slowly but surely, technology
can take over all the tasks parents are responsible for.
I can see my child perceives this technology to behave
like a living being. So, I do not want to be excluded. I
want to be able to monitor what my kid is learning and
participate in the activities.
Parents also considered improving children’s social behavior
and interactions to be an important learning activity. There-
fore, in DS 2, while examining children’s designs, parents
refined the devices’ intelligence by giving them the ability to
monitor a user for the use of inappropriate words and tone in
their interactions with the device. Parents designed features
that could inform them when children communicated in an
unacceptable manner and could automatically restrict children
from further use. For example, Yuling’s father, during DS 3,
noted:
“Current technology does not check and restrict children
from using swear words or a rude tone. For example,
while learning math, the device should motivate the use
of polite tone and kind words and also restrict the use if
it happens repeatedly.
Parents who were not technologically savvy shared their fear
of falling behind the technology. Sussane’s mother felt that in
order to participate and be able to monitor what her kids were
actually learning through the device, it is important for such
devices to provide interfaces that explain their functioning and
use in a user- friendly manner. She said, “It is impossible to
monitor or participate if you do not understand how it works.
Children know much more these days and I want to be in
control their use.
Parents also included the option to set daily learning goals
and/or routines for their children and monitor their progress.
Sam’s mother said, “My son has a very fixed routine. I think
that is true for most children. So this option will enable us to
specify her daily routine in terms of learning goals.
User Input Modalities and Behavior
The analysis of the design session data revealed several themes
about the possible ways children desired to use the devices,
and their input modalities. Children also included in their
designs the possibility of customizing a device’s behavior.
Input Modalities
As with the output modalities of the systems, the children
included an array of input modalities for interacting with their
devices. Speech and touch were the most common, probably
because we asked the children to interact with voice-based
agents in DS 1. Other modalities they included in their designs
were gestures (e.g., nodding or shaking the head to indicate
approval or disapproval), facial expressions (e.g., to show
emotion), wireless hardware (e.g., pens, wearable technology,
CD-ROMs), or physical buttons on the device. Also, all the
children’s designs supported more than one input modality.
For example, Juliet, while explaining her device in DS 2, said,
“I can speak to the device, but I can also write on its screen.
Many children also included the ability to remotely access the
device through other devices, specifically when they would
not be at home. The children stated that portable devices like
tablets, phones, and wearables should be able to connect to
the main device, which they envisioned to be always present
in the user’s home. They acknowledged that this might mean
some functionalities or characteristics of the device would not
be available remotely. For example, Marianne noted in DS 2,
If I am out to a friend’s place, my phone can be linked to the
main device that is in my house. So if I quickly need to recall
a fact or learn something while being there, I could use that.
It’s okay if I cannot use a storytelling mode from [my] phone,
but this will allow me to quickly ask a question.
Customization and Selection
In their designs, all the children enabled the user to change
and control any functionality or configuration. As seen be-
fore, the children included multiple personas and learning and
input modes in their designs, and the user could choose and
customize any of these. Furthermore, while the devices had
the ability to change their emotions and personalize learning
modules based on the user’s current mood and characteristics
(as elaborated in the section “System Intelligence”), the users
were also able to control and reconfigure a device’s adaptive
behavior based on their preferences. For example, during DS
2, Chris showed a button to one of the adults that would enable
the user to change the device’s personalization:
Adult: “You selected a module that is too difficult for
me.
Chris [speaking as the system]: “Okay. If you want to
change it, click on this button.
System Intelligence
In DS 1, all the children tested and challenged the intelligence
of the technology by asking questions they already knew an-
swers to, or thought would be difficult to answer. This echoes
the findings of other studies [39, 61] For instance, when talk-
ing to Google Home in DS 1, Marc said, “Do you even know
how far the Sun is?”
In subsequent design sessions, as the children were designing
technology for learning, it became apparent that the children
expected a high level of intelligence in such technology. These
expectations of were manifested in the participants’ designs as
a system’s ability to sense users’ emotions and respond appro-
priately, to evaluate user characteristics such as learning needs
and demographic background, and to maintain a natural con-
versation. Therefore, this theme comprised the sub-themes of:
Human-like Conversation, Adaptability Based on the User’s
Characteristics, Motivating or Initiating Use, and Emotional
Intelligence.
Human-Like Conversation
One common theme that emerged in the designs of all the chil-
dren was the ability of intelligent technology to hold human-
like conversation. In their designs, the devices tailor their
communication to the user by developing an understanding
of the user’s context and characteristics (e.g., mood, inten-
tion, likes or dislikes, level of expertise) from information
gathered from prior interactions (e.g., learning history), or by
sensing current situations (e.g., other people in the immedi-
ate surroundings). Children also designed interactions with
the device to be more social, rather than just being transac-
tional (e.g., asking for clarification, developing a mutual and
common understanding). The following exchange took place
during DS 2, where “the system” (a child) posed a question
in response to a statement from the user (an adult), in order to
understand the user’s characteristics and clarifications:
Adult: “Math is my favourite subject.
Milly [speaking as a system]: “Which topic do like the
most?”
Adult: “Multiplication”
Milly [speaking as the system]: “Did you say multiplica-
tion?”
Adult: “Yes.
Adaptability and Personalization Based on the User’s Charac-
teristics
The children consistently expected learning technology to
adapt itself based on the users’ characteristics (e.g., demo-
graphics, level of expertise in different subject areas). For
example, in DS 2, Lilly explained that the device should be
cognizant of a user’s current expertise in a particular subject
area, to not only better understand the user, but also moti-
vate them to practice and improve in other subject areas. She
further said,
“A three- or four-year-old might have difficulties in prop-
erly talking to the device. So the device can help the child
by asking clarifying questions that one could respond in
a yes or a no, or even write the response down.
During the design activity of DS 2, the children also talked
about the learning profile of the user. The children expected
these devices to remember and adapt based on users’ past
learning activity on the device. For example, Sam explained,
“If you keep on using the device, it will give you more difficult
questions with each subsequent use. If you want to revise
something, you can always ask for revisions, but it will always
give you a new module if you successfully completed the one
before it.
The parents’ desire to facilitate holistic learning led them to
include in their designs the ability to nudge children to practice
a subject (e.g., a school subject), which a child might not have
done by him/herself. In DS 3, Marc’s father designed a device
that “could motivate or tell the kid to ... [work on a] subject
that he or she ignores. I know at least for my kid this is needed,
as otherwise he will just do his math and no other subject.
Motivating or Initiating Use
In their designs, the children and parents gave devices the
ability to activate automatically and motivate a user to partic-
ipate in a learning activity. While parents did this to ensure
that some part of a child’s daily or weekly schedule would be
dedicated to in-home learning activities, the children thought
it could help them when they were supposed to practice or
revise a topic but forgot, or were not motivated, to do so. For
example, in DS 3, the following discussion took place between
John and his father:
John: “If a user clicks on this button, it will start to track
if there is any pending homework or task that needs to be
completed. This way if somebody forgets or is too lazy,
the device can understand it and help the user.
Father: “Actually, it should just get activated at certain
time of the day by itself. That way it ensures the child is
doing his lessons regularly.
Emotional Intelligence
In their designs, the children frequently included the capability
to sense the emotional states of its users (e.g., irritation, fatigue,
sadness). This enabled the systems to adapt their responses
and the learning activities offered to the users. Prior work has
even found this to assist in developing parasocial relationships
[7, 25]. In DS 2, Chris designed a device that looked like Elmo
(a Muppet character on the television show Sesame Street [20])
and could change its emotion based on that of its user: “So
this Elmo look-alike device is more like a learning partner
who can change its emotion and adapt the learning module
to help the person who is using it to learn. For example, if
someone is demotivated, he can begin the learning session
by teaching the person his favorite topic. This might help in
uplifting his morale.
Privacy Concerns
While parents acknowledged that digital technologies can offer
highly personalized and adaptable behavior due to the vast
amount of data they can collect and analyze (e.g., expertise
levels, learning interests, frequency of use, learning progress,
and children’s emotions or moods), they also had privacy
concerns, since the data being collected were primarily related
to children. For example, the parents asked questions like:
What kind of data will be collected from the child user to
support personalization? Besides creating a child’s profile,
for what other purposes will the data collected about a child’s
learning history and preferences be used? How and where will
be the data be stored to prevent its use for malicious reasons?
Half of the parents also expressed concern that because their
children are young, they might not even understand the poten-
tial harm that such a technology could cause. While no child
or parent offered a privacy-conserving mechanisms, many par-
ents desired to be informed about the details of data collection,
and expected the devices to enable users to turn off any sort of
data collection and processing, provided this did not compro-
mise the device’s functionality. For example, Daniel’s father
noted, “When a device collects so much data, it should declare
the data it is collecting, not in the hidden policy documents,
but directly on the screen. The most ideal case will be if the
user is also allowed to approve the data collection and the
reasons for which it is used, without losing the possibility of
using the device for a particular functionality.
DISCUSSION
In this study, we conducted three co-design sessions to better
understand children’s perceptions about in-home conversa-
tional technology that can be used to fulfill their learning
needs. To elicit parental preferences and needs, we included
parents as design partners in the third co-design session. Our
analysis of the audio recordings of the sessions and the de-
sign artifacts generated by the participants revealed four major
themes. Based on these themes we developed a model (cf. Fig-
ure 2) that depicts the major themes along with the sub-themes
and the interrelations among the themes. While many of the
major themes (e.g., System output, modalities, User input,
system behavior, system intelligence) confirm the findings of
Woodward et al.’s [61] model of error detection and correction
for intelligence user interfaces., several details vary for the
context of conversational technologies for in-home learning.
The following section discusses this model and offers rec-
ommendations to support designers in their efforts to design
new technologies that can be used for the in-home learning
activities of children.
User Input Modalities and Behavior
The participants incorporated multiple input modalities in their
designs to convey their intentions to learn and interact with the
devices. While all the designs included speech as the mode of
input, various other modalities (e.g., gestures, connections to
external hardware) were included as well.
How users were to behave and interact with the system was
based on the system’s intelligence and behavior, but users
could also configure and adjust a system’s intelligence and
behavior to suit their preferences. For example, based on users’
emotions, children expected a device to adapt its own emotions
to scaffold learning. This is consistent with the conceptual
model identified by Woodyard et al. [61] in reference to
children’s understanding of intelligent interfaces and the errors
they might encounter.
System Output Modalities and Behavior
As with the input modalities, the participants incorporated mul-
tiple output modalities in their designs, all of which included
speech. They also included multiple learning modes, such as
storytelling, and formative and summative assessments, and
instead of providing direct answers to a question, the device
could scaffold users’ learning by providing hints or prompts,
so that they would be able to arrive at the answer by them-
selves. The children in our study desired technology that could
support learning activities in their homes and would have a
personality similar to that of their favorite media character,
taking on several roles, such as teacher, friend, and companion.
Prior work has also argued that when a technology is perceived
to have a personality and is associated with a social role, it can
promote learning in children [7, 25].
Design Implication 1 - Supporting Different Learning
Modes, Roles, and Personas
The children expected the technology to offer multiple learning
modes, roles, and personas, so that the modes can be selected
and adjusted automatically based on the user’s characteristics,
or controlled manually by the users themselves. Therefore, to
enable adaptability in learning technology, we suggest that de-
signers design technologies that offer different learning modes
and possess different personas. However, while designing a
technology that has a persona (e.g., a human-like personality,
a backstory, and a range of emotions and interests) and can be
associated with a social role, designers will have to critically
examine the impact this will have on the different learning
modes that it offers (e.g., engaging in a competitive game as a
friend versus offering a formative quiz as a teacher). Further-
more, designers should also consider the fact that children can
be fearful of technology that pretends to take on certain roles,
such as that of a parent, or mimic people from their trusted
networks [65] or people they are attached to.
System Intelligence
The children in our study expected to see a high level of intelli-
gence in this technology, in the form of the ability to converse
Figure 2: Model depicting the interrelations among the major themes in the children’s and parents’ expectations regarding in-home
learning technologies.
and to adapt its interactions, emotions, learning content, and
mode based on the users’ characteristics (e.g., demographics,
expertise in a particular area, emotions), which are capabilities
that are considered to be typical of humans.
As was the case in children’s conceptualization of intelligent
user interfaces [61], our participants interpreted a system’s
intelligence through its output. For example, all the designs
supported learning not only by answering questions or asking
the user to solve problems, but also by adapting their interac-
tions to the user’s characteristics.
The current state-of-the art technology does not fulfill the
intelligence preferences of children that our study revealed.
For example, interactions with current conversational agents
(e.g., Google Home Assistant, Amazon Alexa) are sequential,
constrained, and task-oriented [12, 50]. In other words, the
devices are unable to understand the context of interactions,
express emotions, or engage in social talk, which are essen-
tial elements for engaging in a human-like conversation [15,
24, 12] (one of the abilities that the children associated with
intelligent technology). Prior work, however, has found that
when a child trusts a media character and perceives it to be
knowledgeable, when a character is able to converse and ex-
press emotions, or when personalized experience is offered by
a character or a technology based on the user’s context and
characteristics, the strength of the parasocial relationship, and
as a result learning, increases [7, 25].
Design Implication 2 - Including Human-Like Char-
acteristics
Based on our findings, we recommend that designers include in
future devices human-like characteristics such as adaptability
based on the context, the ability to sense and express emotions,
and the ability to converse like a human as much as possible,
in order to foster a parasocial relationship between the device
and the child, which in turn can promote learning [5, 7].
Specifically, designers and system developers can design de-
vices to sense users’ emotions by scanning their facial ex-
pressions [46] or by wirelessly monitoring their heart and
respiration rates [66]. This form of emotional intelligence
can enable devices to adapt their interactions with a child
based on the child’s emotions; if the child is sad and not
motivated to practice his or her lessons on a particular day,
the device can motivate him or her through stories or humor.
However, as already pointed out, it is currently difficult for
conversational/speech agents to completely model a human-
like conversation. Also, it is challenging for children to assist
a device with this process [4, 11, 63], for example, by pro-
viding contextual information [63], which is considered to be
one of the essential components for successfully modeling
human-like conversations [12, 50]. Therefore, an interim solu-
tion could be for a device to develop contextual grounding by
continually learning from users’ prior interactions and asking
them to confirm or clarify input or part of it (e.g., identify a
pronoun used in an input), or asking for follow-up information
if what the user said was unclear or insufficient for the device
to understand the context and generate a response.
Parental Concerns and Role
Prior work has found parents influence children’s use of tech-
nology [30, 41] by monitoring or restricting their use. Further-
more, parents’ communications and attitudes towards technol-
ogy or media characters influence children’s access to them
and shape their direct participation in the children’s activities
[5, 25]. The parents in the study, acting as design partners,
changed the children’s designs by including features that foster
social engagement in children, allow for parental involvement
and control, and cater to parents’ privacy concerns, thereby
influencing the system’s intelligence and the user’s behavior.
The parents were worried about not being able to participate
in or contribute to their children’s learning activities, mon-
itor their children’s progress, or understand the capabilities
of the technology at least as much as their children do. The
parents expressed concern that their children’s complete de-
pendency on technology for the purpose of learning might also
displace them by excluding them from tasks that are otherwise
considered to be primarily their responsibility in the home.
Sciuto et al. [52] also reported that children are highly in-
fluenced by the spoken nature of conversational technology,
which can also influence their social behavior. All of the chil-
dren’s designs included speech as the mode of input and output,
parents also desired the device to facilitate social interaction,
so that the children would converse with other people, rather
than only with the conversational technology. For example,
to foster social interaction, the parents built into their designs
social games that children could play with other people (e.g.,
siblings or friends).
Design Implication 3 - Fostering Social Interaction
and Engagement
Based on our findings, we recommend that designers of future
technology for children incorporate features that can include
several members of a family or friends-circle in children’s
interactions with the device. For example, a device could moti-
vate children to learn as they participate in a device-facilitated
group activity (e.g., a competitive game) with their siblings or
friends, or could include a parent as a co-learner, motivator, or
overseer (e.g., by sending a progress report to the parent) as
the children use the device for learning.
Design Implication 4 - Providing Control to Parents
Based on our findings, it is imperative for parents to be in
control of the technology that their children use. Designers
could provide this control by including features that enable
parents to set learning goals for children, to monitor (e.g., by
informing parents about the daily learning progress of chil-
dren) and restrict (e.g., by setting time-limits) the children’s
daily use and progress, and to approve or disapprove of any
data collection pertaining to the children’s use or behavior.
Ethical Considerations
In this study, we recommend that designers include human-
like characteristics such as the ability to converse, express
emotions, and display personality in future technologies for in-
home learning, as these attributes have been found to establish
a parasocial bond between a child user and the technology,
which might in turn foster learning. We believe this poses
an ethical dilemma – while it will fulfill children’s desire
for technology to have a persona, exhibit intelligence, and
promote learning, it may also lead children to develop asocial
behavior, a feeling of eeriness, revulsion, or disturbed [65, 6],
an overdependence on technology, or an inaccurate sense of
control. For example, a humanoid device designed to support
learning can lead children to attribute more intelligence to the
device than to adults in the family and agree to what it says,
while ignoring others. The parents in our study raised concerns
about a possible decrease in the social awareness of children
due to increased interactions with intelligent technology. Gray
et al. [25] also suggested that a child’s “theory of mind” i.e.,
their attribution of emotions, intent, and knowledge to oneself
and others [2] becomes more complex when they have to also
theorize about AI-driven characters.
Therefore, we believe it is an ethical and a moral responsibility
of the HCI community and parents to critically examine such
technologies in terms of their impact on children to gauge
implications that can go beyond the technologies’ original
intended purpose. In other words, while designing and using
such technology one has to evaluate: How do interactions with
intelligent human-like devices impact children’s development
of social emotion and social cognition?
LIMITATIONS AND FUTURE WORK
Our co-design session included 12 children and their parents
from a single geographic region. While we did not aim to
reach statistical generalizability [64], future work should test
the validity of our themes more broadly – for example, through
large-scale surveys. Prior work has also found that technology
use often differs in families from different backgrounds [1].
Therefore, future work could also compare preferences of
children and parents from different racial and socioeconomic
backgrounds or from different cultures with different rules
for social interaction. It will also be critical in the future to
examine how the preferences and needs differ for children who
struggle with learning disabilities or sensory impairments.
All of our participants’ designs were low-fidelity paper proto-
types rather than interactive high-fidelity prototypes. There-
fore, they might have missed potential challenges presented by
their designs or opportunities to improve the designs. Finally,
future work could produce specific technologies in light of our
findings to prompt designers and parents to critically reflect
on the open questions that we pose throughout this study.
CONCLUSION
In this study, researchers collaboratively designed learning
technology with twelve children in three participatory design
sessions. The third session also included parents as our design
partners,. The participatory design sessions enabled us to
arrive at a deeper understanding of children’s and parents’
preferences and needs for in-home conversational technologies
that can be used for children’s learning endeavors. We found
that while children desire a high level of intelligence, human-
like characteristics, and support for multiple content domains
and learning modes in such technology, parents expect the
devices to foster social interaction and engagement, to allow
them to participate in their children’s activities, and to control
and monitor their children’s use. Designers can utilize our
findings to design future conversational technology that is
tailored to children and their parents.
ACKNOWLEDGMENTS
We would like to thank the children, parents, and volunteers
who participated in this study. We would also like thank
Syracuse University to partially funding this work.
REFERENCES
[1] Morgan G Ames, Janet Go, Joseph’Jofish’ Kaye, and
Mirjana Spasojevic. 2011. Understanding technology
choices and values through social class. In Proceedings
of the ACM 2011 conference on Computer supported
cooperative work. ACM, 55–64.
[2] Janet W Astington, Paul L Harris, and David R Olson.
1990. Developing theories of mind. CUP Archive.
[3] Robert K Atkinson. 2002. Optimizing learning from
examples using animated pedagogical agents. Journal of
Educational Psychology 94, 2 (2002), 416.
[4] Erin Beneteau, Olivia K Richards, Mingrui Zhang,
Julie A Kientz, Jason Yip, and Alexis Hiniker. 2019.
Communication breakdowns between families and
Alexa. In Proceedings of the 2019 CHI Conference on
Human Factors in Computing Systems. ACM, 243.
[5] Bradley J Bond and Sandra L Calvert. 2014. A model
and measure of US parents’ perceptions of young
children’s parasocial relationships. Journal of Children
and Media 8, 3 (2014), 286–304.
[6] Kimberly A Brink, Kurt Gray, and Henry M Wellman.
2019. Creepiness creeps in: Uncanny valley feelings are
acquired in childhood. Child development 90, 4 (2019),
1202–1214.
[7] Kaitlin L Brunick, Marisa M Putnam, Lauren E
McGarry, Melissa N Richards, and Sandra L Calvert.
2016. Children’s future parasocial relationships with
media characters: the age of intelligent characters.
Journal of Children and Media 10, 2 (2016), 181–190.
[8] Justine Cassell. 2001. Embodied conversational agents:
representation and intelligence in user interfaces. AI
magazine 22, 4 (2001), 67–67.
[9] Justine Cassell. 2004. Towards a model of technology
and literacy development: Story listening systems.
Journal of Applied Developmental Psychology 25, 1
(2004), 75–105.
[10] Kathy Charmaz and Liska Belgrave. 2012. Qualitative
interviewing and grounded theory analysis. The SAGE
handbook of interview research: The complexity of the
craft 2 (2012), 347–365.
[11] Yi Cheng, Kate Yen, Yeqi Chen, Sijin Chen, and Alexis
Hiniker. 2018. Why doesn’t it work?: voice-driven
interfaces and young children’s communication repair
strategies. In Proceedings of the 17th ACM Conference
on Interaction Design and Children. ACM, 337–348.
[12] Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle,
Diego Garaialde, Justin Edwards, Brendan Spillane,
Emer Gilmartin, Christine Murad, Cosmin Munteanu,
and others. 2019. What Makes a Good Conversation?:
Challenges in Designing Truly Conversational Agents.
In Proceedings of the 2019 CHI Conference on Human
Factors in Computing Systems. ACM, 475.
[13] John W Creswell and Cheryl N Poth. 2017. Qualitative
inquiry and research design: Choosing among five
approaches. Sage publications.
[14]
Charles Desforges, Alberto Abouchaar, and others. 2003.
The impact of parental involvement, parental support
and family education on pupil achievement and
adjustment: A literature review. Vol. 433. DfES London.
[15] Laurence Devillers, Sophie Rosset,
Guillaume Dubuisson Duplessis, Lucile Bechade, Yucel
Yemez, Bekir B Turker, Metin Sezgin, Engin Erzin,
Kevin El Haddad, Stéphane Dupont, and others. 2018.
Multifaceted Engagement in Social Interaction with a
Machine: The JOKER Project. In 2018 13th IEEE
International Conference on Automatic Face & Gesture
Recognition (FG 2018). IEEE, 697–701.
[16] Christian Dindler, Eva Eriksson, Ole Sejer Iversen,
Andreas Lykke-Olesen, and Martin Ludvigsen. 2005.
Mission from Mars: a method for exploring user
requirements for children in a narrative space. In
Proceedings of the 2005 conference on Interaction
design and children. ACM, 40–47.
[17] Allison Druin. 1999. Cooperative inquiry: developing
new technologies for children with children. In
Proceedings of the SIGCHI conference on Human
Factors in Computing Systems. ACM, 592–599.
[18] Allison Druin. 2002. The role of children in the design
of new technology. Behaviour and information
technology 21, 1 (2002), 1–25.
[19]
Andrea Duerager and Sonia Livingstone. 2012. How can
parents support children’s internet safety? (2012).
[20] Elmo. 2019. https://en.wikipedia.org/wiki/Elmo.
(2019).
[21] Paul G Fehrmann, Timothy Z Keith, and Thomas M
Reimers. 1987. Home influence on school learning:
Direct and indirect effects of parental involvement on
high school grades. The Journal of Educational
Research 80, 6 (1987), 330–337.
[22]
Elizabeth Foss, Allison Druin, Robin Brewer, Phillip Lo,
Luis Sanchez, Evan Golub, and Hilary Hutchinson.
2012. Children’s search roles at home: Implications for
designers, researchers, educators, and parents. Journal
of the American Society for Information Science and
Technology 63, 3 (2012), 558–573.
[23] Radhika Garg and Subhasree Sengupta. 2019. “When
you can do it, why can’t I?”: Racial and Socioeconomic
Differences in Family Technology Use and Non-Use. In
Proceedings of the 22nd ACM conference on
computer-supported cooperative work & social
computing. ACM, to appear.
[24] Emer Gilmartin, Marine Collery, Ketong Su, Yuyun
Huang, Christy Elias, Benjamin R Cowan, and Nick
Campbell. 2017. Social talk: making conversation with
people and machine. In Proceedings of the 1st ACM
SIGCHI International Workshop on Investigating Social
Interactions with Artificial Agents. ACM, 31–32.
[25] James H Gray, Emily Reardon, and Jennifer A Kotler.
2017. Designing for Parasocial Relationships and
Learning: Linear Video, Interactive Media, and Artificial
Intelligence. In Proceedings of the 2017 Conference on
Interaction Design and Children. ACM, 227–237.
[26] Mona Leigh Guha, Allison Druin, Gene Chipman,
Jerry Alan Fails, Sante Simms, and Allison Farber. 2004.
Mixing ideas: a new technique for working with young
children as design partners. In Proceedings of the 2004
conference on Interaction design and children: building
a community. ACM, 35–42.
[27] Mona Leigh Guha, Allison Druin, and Jerry Alan Fails.
2013. Cooperative Inquiry revisited: Reflections of the
past and guidelines for the future of intergenerational
co-design. International Journal of Child-Computer
Interaction 1, 1 (2013), 14–23.
[28]
Jeong-Hye Han, Mi-Heon Jo, Vicki Jones, and Jun-H Jo.
2008. Comparative study on the educational use of home
robots for children. Journal of Information Processing
Systems 4, 4 (2008), 159–168.
[29] Gunnar Harboe and Elaine M Huang. 2015. Real-world
affinity diagramming practices: Bridging the
paper-digital gap. In Proceedings of the 33rd annual
ACM conference on human factors in computing systems.
ACM, 95–104.
[30] Alexis Hiniker, Sarita Y Schoenebeck, and Julie A
Kientz. 2016. Not at the dinner table: Parents’ and
children’s perspectives on family technology rules. In
Proceedings of the 19th ACM conference on
computer-supported cooperative work & social
computing. ACM, 1376–1389.
[31] Mohammed Ehsan Hoque, Matthieu Courgeon,
Jean-Claude Martin, Bilge Mutlu, and Rosalind W
Picard. 2013. Mach: My automated conversation coach.
In Proceedings of the 2013 ACM international joint
conference on Pervasive and ubiquitous computing.
ACM, 697–706.
[32]
William H Jeynes. 2005. A meta-analysis of the relation
of parental involvement to urban elementary school
student academic achievement. Urban education 40, 3
(2005), 237–269.
[33] Yasmin B Kafai and Sharon Sutton. 1999. Elementary
school students’ computer and internet use at home:
Current trends and issues. Journal of Educational
Computing Research 21, 3 (1999), 345–362.
[34] Alice Kerly, Richard Ellis, and Susan Bull. 2007.
CALMsystem: a conversational agent for learner
modelling. In International Conference on Innovative
Techniques and Applications of Artificial Intelligence.
Springer, 89–102.
[35] Daesang Kim and David A Gilman. 2008. Effects of
text, audio, and graphic aids in multimedia instruction
for vocabulary learning. Journal of educational
technology & society 11, 3 (2008), 114–126.
[36] James C Lester, Sharolyn A Converse, Susan E Kahler,
S Todd Barlow, Brian A Stone, Ravinder S Bhogal, and
others. 1997. The persona effect: affective impact of
animated pedagogical agents. In CHI, Vol. 97. Citeseer,
359–366.
[37] Sonia Livingstone and Dafna Lemish. 2001. Doing
comparative research with children and young people.
Children and their changing media environment: A
European comparative study (2001), 31–50.
[38] Silvia Lovato and Anne Marie Piper. 2015. Siri, is this
you?: Understanding young children’s interactions with
voice input systems. In Proceedings of the 14th
International Conference on Interaction Design and
Children. ACM, 335–338.
[39] Silvia B Lovato, Anne Marie Piper, and Ellen A
Wartella. 2019. Hey Google, Do Unicorns Exist?:
Conversational Agents as a Path to Answers to
Children’s Questions. In Proceedings of the 18th ACM
International Conference on Interaction Design and
Children. ACM, 301–313.
[40] Naja A Mack, Robert Cummings, Dekita G Moon
Rembert, and Juan E Gilbert. 2019. Co-Designing an
Intelligent Conversational History Tutor with Children.
In Proceedings of the 18th ACM International
Conference on Interaction Design and Children. ACM,
482–487.
[41] Melissa Mazmanian and Simone Lanette. 2017. Okay,
one more episode: An ethnography of parenting in the
digital age. In Proceedings of the 2017 ACM Conference
on Computer Supported Cooperative Work and Social
Computing. ACM, 2273–2286.
[42]
Sarah J McCarthey. 2000. Home–school connections: A
review of the literature. The Journal of Educational
Research 93, 3 (2000), 145–153.
[43] Brenna McNally, Priya Kumar, Chelsea Hordatt,
Matthew Louis Mauriello, Shalmali Naik, Leyla Norooz,
Alazandra Shorter, Evan Golub, and Allison Druin. 2018.
Co-designing mobile online safety applications with
children. In Proceedings of the 2018 CHI Conference on
Human Factors in Computing Systems. ACM, 523.
[44] Oussama Metatla, Alison Oldfield, Taimur Ahmed,
Antonis Vafeas, and Sunny Miglani. 2019. Voice User
Interfaces in Schools: Co-designing for Inclusion With
Visually-Impaired and Sighted Pupils. In Proceedings of
the 2019 CHI Conference on Human Factors in
Computing Systems. ACM, 378.
[45] Miro. 2019. https://miro.com/. (2019).
[46] Fatma Nasoz and Christine L Lisetti. 2006. MAUI
avatars: Mirroring the user’s sensed emotions via
expressive multi-ethnic facial avatars. Journal of Visual
Languages & Computing 17, 5 (2006), 430–444.
[47] T Newby, D Stepich, J Lehman, and J Russell. 2000.
Instructional technology for teaching and learning:
Designing instruction, integrating computers, and using
media. Educational Technology & Society 3, 2 (2000),
106–107.
[48]
Peter Nikken and Jeroen Jansz. 2006. Parental mediation
of children’s videogame playing: A comparison of the
reports by parents and children. Learning, media and
technology 31, 2 (2006), 181–202.
[49] Amanda Purington, Jessie G Taft, Shruti Sannon,
Natalya N Bazarova, and Samuel Hardman Taylor. 2017.
Alexa is my new BFF: social roles, user satisfaction, and
personification of the amazon echo. In Proceedings of
the 2017 CHI Conference Extended Abstracts on Human
Factors in Computing Systems. ACM, 2853–2859.
[50] Stuart Reeves. 2019. Conversation considered harmful?.
In Proceedings of the 1st International Conference on
Conversational User Interfaces. ACM, 10.
[51] Martin Saerbeck, Tom Schut, Christoph Bartneck, and
Maddy D Janse. 2010. Expressive robots in education:
varying the degree of social supportive behavior of a
robotic tutor. In Proceedings of the SIGCHI conference
on human factors in computing systems. ACM,
1613–1622.
[52] Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I
Hong. 2018. Hey Alexa, What’s Up?: A mixed-methods
studies of in-home conversational agent usage. In
Proceedings of the 2018 Designing Interactive Systems
Conference. ACM, 857–868.
[53]
Lori Takeuchi, Reed Stevens, and others. 2011. The new
coviewing: Designing for learning through joint media
engagement. In New York, NY: The Joan Ganz Cooney
Center at Sesame Workshop.
[54] Stergios Tegos, Stavros Demetriadis, and Anastasios
Karakostas. 2015. Promoting academically productive
talk with conversational agent interventions in
collaborative learning settings. Computers & Education
87 (2015), 309–325.
[55] Transformer Wiki. 2019.
https://tfwiki.net/wiki/Four_Bots_and_a_Baby. (2019).
[56] Patti M Valkenburg, Marina Krcmar, Allerd L Peeters,
and Nies M Marseille. 1999. Developing a scale to
assess three styles of television mediation: Instructive
mediation, restrictive mediation, and social coviewing.
Journal of broadcasting & electronic media 43, 1 (1999),
52–66.
[57]
Greg Walsh, Alison Druin, Mona Leigh Guha, Elizabeth
Foss, Evan Golub, Leshell Hatley, Elizabeth Bonsignore,
and Sonia Franckel. 2010. Layered elaboration: a new
technique for co-design with children. In Proceedings of
the SIGCHI Conference on Human Factors in
Computing Systems. ACM, 1237–1240.
[58] Greg Walsh, Elizabeth Foss, Jason Yip, and Allison
Druin. 2013. FACIT PD: a framework for analysis and
creation of intergenerational techniques for participatory
design. In proceedings of the SIGCHI Conference on
Human Factors in Computing Systems. ACM,
2893–2902.
[59] Wayne Ward, Ron Cole, Daniel Bolaños, Cindy
Buchenroth-Martin, Edward Svirsky, and Tim Weston.
2013. My science tutor: A conversational multimedia
virtual tutor. Journal of Educational Psychology 105, 4
(2013), 1115.
[60] Helen Westcott and Karen Littleton. 2005. Exploring
meaning through interviews with children. (2005).
[61] Julia Woodward, Zari McFadden, Nicole Shiver, Amir
Ben-hayon, Jason C Yip, and Lisa Anthony. 2018. Using
co-design to examine how children conceptualize
intelligent interfaces. In Proceedings of the 2018 CHI
Conference on Human Factors in Computing Systems.
ACM, 575.
[62]
Ying Xu and Mark Warschauer. 2019. Young Children’s
Reading and Learning with Conversational Agents. In
Extended Abstracts of the 2019 CHI Conference on
Human Factors in Computing Systems. ACM, CS10.
[63] Svetlana Yarosh, Stryker Thompson, Kathleen Watson,
Alice Chase, Ashwin Senthilkumar, Ye Yuan, and AJ
Brush. 2018. Children asking questions: speech
interface reformulations and personification preferences.
In Proceedings of the 17th ACM Conference on
Interaction Design and Children. ACM, 300–312.
[64] Robert K Yin. 2015. Qualitative research from start to
finish. Guilford Publications.
[65] Jason C Yip, Kiley Sobel, Xin Gao, Allison Marie
Hishikawa, Alexis Lim, Laura Meng, Romaine Flor
Ofana, Justin Park, and Alexis Hiniker. 2019. Laughing
is Scary, but Farting is Cute. (2019).
[66] Mingmin Zhao, Fadel Adib, and Dina Katabi. 2016.
Emotion recognition using wireless signals. In
Proceedings of the 22nd Annual International
Conference on Mobile Computing and Networking.
ACM, 95–108.
... By observing how children aged 3-12 interacted with smart speakers, researchers found that the majority of children believed that smart speakers were smart, friendly, and trustworthy. While interacting, children often asked questions about their identity and personality and even joked with the device as if it was a real person indeed (e.g., "Google, can I eat you?"; [11,[13][14][15][16]). When asked to draw the smart speaker, elementary school children tended to display anthropomorphic features in their paint-ings (e.g., eyes, limbs, and facial expressions; [17]). ...
... For example, studies have found that children often used loud voice and even intimidation tactics when smart speakers failed to recognize their language [23,49]. Some parents are concerned about whether children would transfer these aggressive verbal expressions into conversations with real people, thus impeding the acquisition and maintenance of good language habits and norms of daily behavior [15,36]. Secondly, children's interactions with smart speakers are immediate (children give commands and smart speakers respond immediately), which is contrary to "delay gratification" [5]. ...
... Thus, too much interaction with and commands to smart speakers may be detrimental to children's development of self-control skills. Third, because smart speakers are equipped with many characteristics of a good partner (e.g., patient listening, no ridicule, and keeping secrets), children may perceive the device as a trustworthy partner with whom they can build emotional and attachment relationships [5,15,23,36,50], which may reduce children's social need in the real world and thus be detrimental to their social development. ...
Article
Full-text available
In recent years, the growing popularity of smart speakers (e.g., Google Home and Alexa) has facilitated young children’s interaction with internet-based devices and provided them with more opportunities to obtain access to online information. This review summarizes the current state of the research by examining smart speakers’ core characteristics, children’s conceptualization and interaction with smart speakers, and the influences on children’s learning and habits. Our review shows that (a) the natural language processing technology and central computing system (Internet) contribute to the uniqueness of smart speakers; (b) although children tend to attribute human characteristics (e.g., smart and friendly) to smart speakers, they might judge these voice assistant devices as neither explicitly living nor nonliving in ontological perception; (c) children’s overattributing certain knowledge (e.g., questions about personal information) to smart speakers does not necessarily mean that this device is believed to be omniscient; and (d) in terms of promoting children’s learning, smart speakers might not be more effective than a real human, and the interaction with smart speakers may not be conducive to children’s maintenance of civilized social norms. Implications for children’s conceptualization and interaction of smart speakers and the design of children-oriented smart agents are also discussed.
... Audial customization could have even broader implications in realworld devices. Examples include in-home devices incorporating voice interaction [79][80][81] and conversational agents more generally (e.g., Alexa [10]) [16,17,93]. For example, it is not well understood what the efects of changing the voices of these devices are (e.g., to be more similar to the user). ...
Conference Paper
Full-text available
Avatar customization is known to positively affect crucial outcomes in numerous domains. However, it is unknown whether audial customization can confer the same benefits as visual customization. We conducted a preregistered 2 x 2 (visual choice vs. visual assignment x audial choice vs. audial assignment) study in a Java programming game. Participants with visual choice experienced higher avatar identification and autonomy. Participants with audial choice experienced higher avatar identification and autonomy, but only within the group of participants who had visual choice available. Visual choice led to an increase in time spent, and indirectly led to increases in intrinsic motivation, immersion, time spent, future play motivation, and likelihood of game recommendation. Audial choice moderated the majority of these effects. Our results suggest that audial customization plays an important enhancing role vis-à-vis visual customization. However, audial customization appears to have a weaker effect compared to visual customization. We discuss the implications for avatar customization more generally across digital applications.
... With the goal of designing user-centered and socially responsible AI technology that could help promote online learners' social connectedness, we take the approach of co-design [52,54] to include online learners as active participants from the beginning of the design process. Co-design has been frequently adopted in prior literature to understand the design of AI agents across various contexts [10,21,48]. Through two co-design workshop studies with 23 online learners, we provide the necessary design techniques and tools for online learners to voice their preferences and concerns freely to envision a future where AI agents could help support their social connectedness. ...
Article
Social emotional skills are foundational competencies upon which children draw throughout their lives. This work investigates current, commercially available experiences for social emotional learning (SEL) through conversational agents (CAs). Specifically, we reviewed 3,767 Skills available in the "Kids" category of the Alexa Skills Marketplace and found 42 working Skills with connections to SEL. We found that the most common scenarios these Skills sought to support were: active listening, emotional wellbeing, conversation with other people, and politeness. The interaction patterns used by these Skills distilled into a taxonomy of styles we labeled: The Delegator, The Lecturer, The Bulldozer, and The One-Track Mind. We found that, collectively, these Skills provide shallow experiences and lack contingent feedback. To examine the gap between current offerings and families' needs, we also conducted 26 interviews with parents to probe parents' ideas about CAs supporting children's SEL. Parents see potential for CAs to support children in four concrete ways, including attuning to others, cultivating curiosity, reinforcing politeness, and developing emotional awareness. Despite their optimism about these opportunities, parents expressed skepticism about CAs' impoverished conversational abilities and worry about CAs advancing values and behavioral norms that are at odds with their own.
Article
Full-text available
There is racial diversity as well as economic inequality in the United States (U.S.). To gain a nuanced understanding of how households from different socio economic and racial backgrounds integrate technology into their lives, we conducted a diary study with 22 parents who were Asian Indian (the fastest-growing immigrant population in U.S.) and 18 who were White American (the largest racial group in U.S.) parents from the working and middle classes. The participants logged in-situ instances of using smart phones and speaker use by, with, and around children for 8 weeks, and were interviewed once every four weeks (two times in total). Our findings reveal differences and similarities in parents' attitudes and practices of using or not using these devices around and with children, in parental restrictions of children's use of technology, and children's daily use patterns. The paper concludes with a discussions of the implications of our findings and suggestions for future design improvements in smart phones and speakers.
Conference Paper
Full-text available
We investigate how families repair communication breakdowns with digital home assistants. We recruited 10 diverse families to use an Amazon Echo Dot in their homes for four weeks. All families had at least one child between four and 17 years old. Each family participated in pre- and post- deployment interviews. Their interactions with the Echo Dot (Alexa) were audio recorded throughout the study. We analyzed 59 communication breakdown interactions between family members and Alexa, framing our analysis with concepts from HCI and speech-language pathology. Our findings indicate that family members collaborate using discourse scaffolding (supportive communication guidance) and a variety of speech and language modifications in their attempts to repair communication breakdowns with Alexa. Alexa's responses also influence the repair strategies that families use. Designers can relieve the communication repair burden that primarily rests with families by increasing digital home assistants' abilities to collaborate together with users to repair communication breakdowns.
Preprint
Full-text available
Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in supporting long-term human-agent relationships, it is paramount that HCI focuses efforts on delivering this promise. We aim to understand what people value in conversation and how this should manifest in agents. Findings from a series of semi-structured interviews show people make a clear dichotomy between social and functional roles of conversation, emphasising the long-term dynamics of bond and trust along with the importance of context and relationship stage in the types of conversations they have. People fundamentally questioned the need for bond and common ground in agent communication, shifting to more utilitarian definitions of conversational qualities. Drawing on these findings we discuss key challenges for conversational agent design, most notably the need to redefine the design parameters for conversational agent interaction.
Conference Paper
Full-text available
In this study, we examine the conversational repair strategies that preschoolers use to correct communication breakdowns with a voice-driven interface. We conducted a two-week deployment in the homes of 14 preschoolers of a tablet game that included a broken voice-driven mini-game. We collected 107 audio samples of these children's (unsuccessful) attempts to communicate with the mini-game. We found that children tried a common set of repair strategies, including repeating themselves and experimenting with the tone and pronunciation of their words. Children were persistent, rarely giving up on the interaction, asking for help, or showing frustration. When parents participated in the interaction, they moved through four phases of engagement: first making suggestions, then intervening, then making statements of resignation, and finally pronouncing that the interaction could not be repaired. Designers should anticipate that in this context, children will borrow behaviors from person-to-person communication, such as pivoting strategies to probe the source of failed communication and structuring communication into turn-taking attempts.
Conference Paper
As a concept, 'conversation' is rife with troublemaking potential. It is not that we should necessarily abandon use of 'conversation' in conversational user interface (CUI) research, but rather treat it with a significant measure of care due to the varied conceptual problems it introduces---problems sketched in this paper. I suggest an alternative, possibly safer articulation and conceptual shift: conversation-sensitive design.
Conference Paper
Children are known to be curious and persistent questionaskers. The pervasiveness of voice interfaces represents an opportunity for children who are not yet fluent readers to independently search the Internet by asking questions through conversational agents such as Amazon Alexa, Apple's Siri, and the Google Assistant. Through a two-week, in-home deployment study involving 18 families (children aged 5-6 and their parents), we report on which questions children choose to ask the conversational agent, the answers the agent provided, challenges in use, and their perceptions of the technology. Based on our analysis, we identify several considerations for the design of voice-based conversational agents that aim to support young children's question-asking behavior and subsequent development.
Conference Paper
In a world that demands independent and cooperative problem solving to address complex social, economic, ethical, and personal concerns, core social studies content is as basic for success as reading, writing, math, and computing. Nationally, only 20% of 4th grade students are at or above Proficient level in U.S. History, the lowest among the core disciplines of social studies. Reaching proficiency requires students to ask more profound questions of the past as well as construct deeper understandings of it. This research explores the intersection of natural language dialogue and intelligent tutoring systems to enhance history learning of upper elementary students in 3rd and 4th grade. Elementary and middle school students were participants in a participatory design study to extract their design needs for the development of an educational learning technology for social studies.
Conference Paper
Young children increasingly interact with voice-driven interfaces, such as conversational agents (CAs). The social nature of CAs makes them good learning partners for children. We have designed a storytelling CA to engage children in book reading activities. This case study presents the design of this CA and investigates children's interactions with and perception of the CA. Through observation, we found that children actively responded to the CA's prompts, reacted to the CA's feedback with great affect, and quickly learned the schema of interacting with a digital interlocutor. We also discovered that the availability of scaffolding appeared to facilitate child-CA conversation and learning. A brief post-reading interview suggested that children enjoyed their interaction with the CA. Design implications for dialogic systems for young children's informal learning are discussed.
Conference Paper
" Voice user interfaces (VUIs) are increasingly popular, particularly in homes. However, little research has investigated their potential in other settings, such as schools. We investigated how VUIs could support inclusive education, particularly for pupils with visual impairments (VIs). We organised focused discussions with educators at a school, with support staff from local authorities and, through bodystorming, with a class of 27 pupils. We then ran a series of co-design workshops with participants with mixed-visual abilities to design an educational VUI application. This provided insights into challenges faced by pupils with VIs in mainstream schools, and opened a space for educators, sighted and visually impaired pupils to reflect on and design for their shared learning experiences through VUIs. We present scenarios, a design space and an example application that show novel ways of using VUIs for inclusive education. We also reflect on co-designing with mixed-visual-ability groups in this space.
Conference Paper
The pervasive availability of voice assistants may support children in finding answers to informational queries by removing the literacy requirements of text search (e.g., typing, spelling). However, most such systems are not designed for the specific needs and preferences of children and may struggle with understanding the intent of their questions. In our investigation, we observed 87 children and 27 adults interacting with three Wizard-of-Oz speech interfaces to arrive at answers to questions that required reformulation. We found that many children and some adults required help to reach an effective question reformulation. We report the common types of reformulations (both effective and ineffective ones). We also compared three versions of speech interfaces with different approaches to referring to itself (personification) and to the participant (naming personalization). We found that children preferred personified interfaces, but naming personalization did not affect preference. We connect our findings to implications for design of speech systems for families.