ChapterPDF Available

Abstract and Figures

A recent UNESCO report reveals that most popular voice-based conversational agents are designed to be female. In addition, it outlines the potentially harmful effects this can have on society. However, the report focuses primarily on voice-based conversational agents and the analysis did not include chatbots (i.e., text-based conversational agents). Since chatbots can also be gendered in their design, we used an automated gender analysis approach to investigate three gender-specific cues in the design of 1,375 chatbots listed on the platform chatbots.org. We leveraged two gender APIs to identify the gender of the name, a face recognition API to identify the gender of the avatar, and a text mining approach to analyze gender-specific pronouns in the chatbot’s description. Our results suggest that gender-specific cues are commonly used in the design of chatbots and that most chatbots are – explicitly or implicitly – designed to convey a specific gender. More specifically, most of the chatbots have female names, female-looking avatars, and are described as female chatbots. This is particularly evident in three application domains (i.e., branded conversations, customer service, and sales). Therefore, we find evidence that there is a tendency to prefer one gender (i.e., female) over another (i.e., male). Thus, we argue that there is a gender bias in the design of chatbots in the wild. Based on these findings, we formulate propositions as a starting point for future discussions and research to mitigate the gender bias in the design of chatbots.
Content may be subject to copyright.
Gender Bias in Chatbot Design
Jasper Feine
(&)
, Ulrich Gnewuch, Stefan Morana,
and Alexander Maedche
Institute of Information Systems and Marketing (IISM),
Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
{jasper.feine,ulrich.gnewuch,stefan.morana,
alexander.maedche}@kit.edu
Abstract. A recent UNESCO report reveals that most popular voice-based
conversational agents are designed to be female. In addition, it outlines the
potentially harmful effects this can have on society. However, the report focuses
primarily on voice-based conversational agents and the analysis did not include
chatbots (i.e., text-based conversational agents). Since chatbots can also be
gendered in their design, we used an automated gender analysis approach to
investigate three gender-specic cues in the design of 1,375 chatbots listed on
the platform chatbots.org. We leveraged two gender APIs to identify the gender
of the name, a face recognition API to identify the gender of the avatar, and a
text mining approach to analyze gender-specic pronouns in the chatbots
description. Our results suggest that gender-specic cues are commonly used in
the design of chatbots and that most chatbots are explicitly or implicitly
designed to convey a specic gender. More specically, most of the chatbots
have female names, female-looking avatars, and are described as female chat-
bots. This is particularly evident in three application domains (i.e., branded
conversations, customer service, and sales). Therefore, we nd evidence that
there is a tendency to prefer one gender (i.e., female) over another (i.e., male).
Thus, we argue that there is a gender bias in the design of chatbots in the wild.
Based on these ndings, we formulate propositions as a starting point for future
discussions and research to mitigate the gender bias in the design of chatbots.
Keywords: Chatbot Gender-specic cue Gender bias Conversational
agent
1 Introduction
Text- and voice-based conversational agents (CAs) have become increasingly popular
in recent years [19]. Many organizations use chatbots (i.e., text-based CAs) in short-
term interactions, such as customer service and content curation [22], as well as in
long-term interactions, such as personal assistants or coaches [21]. Research has found
that chatbots can increase user satisfaction [41], positively inuence perceived social
presence [3], and establish long-term relationships with users [7]. Additionally, large
technology companies have successfully deployed voice-based CAs (e.g., Microsofts
Cortana, Amazons Alexa, and Apples Siri) on many devices such as mobile phones,
smart speakers, and computers.
©Springer Nature Switzerland AG 2020
A. Følstad et al. (Eds.): CONVERSATIONS 2019, LNCS 11970, pp. 7993, 2020.
https://doi.org/10.1007/978-3-030-39540-7_6
Despite the many benecial aspects of this technology, a recent UNESCO report
[43] from 2019 sheds light on the negative implications of the gendered design of most
commercial voice-based CAs. The report reveals that most voice-based CAs are
designed to be female exclusively or female by default[43]. For example, their name
(e.g., Alexa, Cortana, Siri), their voice (i.e., voices of Alexa and Cortana are exclu-
sively female), and how they are advertised (e.g., Alexa lost her voice) often cause
female gender associations. This can lead to the manifestation of gender stereotypes.
For example, since users mostly interact with voice-based CAs using short command-
like phrases (e.g., tell me the weather), people might deem this form of interaction
style as appropriate when conversing with (female) CAs and potentially even (female)
humans [38]. Consequently, the report highlights the urgent need to change gender
expectations towards CAs before users become accustomed to their default (female)
design [43].
While the UNESCO report provides interesting insights on gender-specic cues in
the design of voice-based CAs and its potential implications, the report does not
include an analysis of chatbots since they are not always as clearly gendered because
their output is primarily written text, not speech[43, p. 92]. However, many studies
have shown that the gender of a chatbot can also be manifested without spoken voice
using other social cues such as name tags or avatars [e.g., 2,3,5,9,16,24,32].
Moreover, these studies suggest that gender-specic cues in the chatbots design can
have both positive [e.g., 5,24] and negative outcomes [e.g., 2,9].
Therefore, we argue that there is a need to analyze how chatbots in contrast to
voice-based CAs are gendered (i.e., through gender-specic cues in their design) and
whether there is evidence of a potential gender bias in the design of chatbots. To the
best of our knowledge, an empirical analysis of gender-specic cues in the design of
chatbots in the wild has not been conducted so far. To address this gap and to com-
plement the ndings of the UNESCO report, we investigate the research question of
how gender-specic cues are implemented in the design of chatbots.
To address this question, we analyzed the design of 1,375 chatbots listed on the
platform chatbots.org. In our analysis, we focused on three cues that can indicate a
specic gender, namely the chatbots name, avatar, and description. In the following,
we refer to these cues as gender-specic cues. Our ndings suggest that there is a
gender bias in the design of chatbots. More specically, we nd evidence that there is a
trend towards female names, female-looking avatars, and descriptions including female
pronouns, particularly in domains such as customer service, sales, and brand repre-
sentation. Overall, our work contributes to the emerging eld of designing chatbots for
social good [20] by highlighting a gender bias in the design of chatbots and thus,
complementing the ndings of the recent UNESCO report on the design of voice-based
CAs. Subsequently, we derive propositions to provide a starting point for future dis-
cussion and research in order to mitigate this gender bias and pave the way towards a
more gender-equal design of chatbots.
80 J. Feine et al.
2 Related Work
2.1 Gender-Specic Cues of Conversational Agents
CAs are software-based systems designed to interact with humans using natural lan-
guage [14]. This means, users interact with CAs via voice-based or text-based inter-
faces in a similar way as they usually interact with other human beings. Research on
CAs and in particular text-based CAs (i.e., chatbots) has been around for several
decades [e.g., 42]. However, the hype around this technology did not start until 2016
[10]. Due to the major adoption of mobile-messaging platforms (e.g., Facebook
Messenger) and the advances in the eld of articial intelligence (AI) [10], chatbots
became one of the most hyped technologies in recent years in research and practice [3].
Extant research in the context of CAs builds on the Computers-Are-Social-Actors
(CASA) paradigm [33]. The CASA paradigm states that human users perceive com-
puters as social actors and treat them as relevant social entities [32]. Therefore, humans
respond similar to computers as they usually react to other human beings (e.g., say
thank you to a computer). These reactions particularly occur when a computer exhibits
social cues that are similar to cues usually expressed by humans during interpersonal
communication [16]. Since CAs communicate via natural language (i.e., a central
human capability), social reactions towards CAs almost always happen [16]. For
example, humans apply gender stereotypes towards CAs whenever they display
specic social cues such as a male or female name, voice, or avatar (see Table 1for an
overview [18]). In addition, not only rather obvious social cues, such as the avatar,
voice, or name, indicate a belonging to a specic gender, but even movements of an
animated avatar are sufcient to do so [40]. Thus, it appears that the tendency to
gender stereotype is deeply ingrained in human psychology, extending even to
machines[34].
Table 1. Exemplary studies investigating the impact of a CAs gender-specic cues.
Type
of CA
Investigated
Cue
Investigated
gender
User reaction towards gender-specic cue Reference
Voice-
based
Voice Female,
male
Gender impacts competence and perceived
friendliness of CA
[34]
Voice-
based
Avatar Female,
male
A specic gender was not preferred [13]
Voice-
based
Avatar,
voice
Female,
male
Gender impacts the comprehension scores
and impression ratings
[27]
Text-
based
Avatar Female,
male
Gender inuences the impact of excuses to
reduce user frustration
[24]
Text-
based
Avatar Ambiguous,
female, male
Gender impacts comfort, condence, and
enjoyment. Users did not prefer gender
ambiguous CAs
[35]
Voice-
based
Avatar,
voice
Female,
male
Gender impacts perceived power, trust,
expertise, and likability
[37]
(continued)
Gender Bias in Chatbot Design 81
2.2 Gender Bias in the Design of Voice-Based Conversational Agents
Since there is limited research on specic design guidelines for CAs [21,29], major
technology companies actively shape how CAs are designed [21]. However, the design
of the major voice-based CAs (e.g., Cortana, Alexa, Google Assistant, Siri) creates
considerable concerns whether the leadership position of technology companies in the
design of CA is desirable [43]. For example, if users directed sexual insults towards
Siri, she used to answer Id blush if I could(till April 2019) and now answers, I
dont know how to respond to that(since April 2019) [43].
Gender manifestations in the design of CAs also reinforce gender manifestations in
the user perception of CAs. This can have severe implications for everyday interper-
sonal interactions. For example, the fact that most of the female voice-based CAs act as
personal assistants leads to the general user expectation that these types of CAs should
be female [43]. Moreover, it creates expectations and reinforces assumptions that
women should provide simple, direct and unsophisticated answers to basic questions
[43 p., 115]. Therefore, such a development reinforces traditional gender stereotypes.
This is in particular harmful, since many children interact with voice-based CAs and
gender stereotypes are primarily instilled at a very young age [9].
Similarly, the active interventions of chatbot engineers into human affairs (e.g.,
establishing a gender bias in the design of chatbots) raises ethical considerations.
Several institutions are warning to avoid the gender-specic development of (interac-
tive) systems. For example, the UNESCO report [5] proposes several recommendations
to prevent digital assistants from perpetuating gender biases. Recently, the European
Unions High-Level Expert Group on AI dened the guidelines for trustworthy AI and
also highlights the importance of equality, non-discrimination, and solidarity [15].
Myers and Venable [31] propose ethical guidelines for the design of socio-technical
system and also emphasize the importance of empowerment and emancipation for all.
Moreover, several research associations (e.g., AIS, ACM) provide ethical guidelines to
ensure ethical practice by emphasizing the importance of designing for an gender-equal
society [39].
Table 1. (continued)
Type
of CA
Investigated
Cue
Investigated
gender
User reaction towards gender-specic cue Reference
Text-
based
Name,
Avatar
Female,
male,
robotic
Gender impacts the attribution of negative
stereotypes
[9]
Voice-
based
Avatar Female,
male
Gender impacts learning performance
and learning effort
[26]
Text-
based
Avatar Female,
male
Gender impacts learning performance [23]
Text-
based
Avatar Female,
male
Gender impacts the belief in the credibility
of advice and competence of agent
[5]
82 J. Feine et al.
3 Method
To answer our research question, we analyzed three cues in the design of a broad
sample of chatbots. Currently, there are several online platforms that list chatbots, but
there is no central repository. Therefore, we decided to rely on the data provided by
chatbots.org. Chatbots.org is a large online community with 8000 members. Members
can add their chatbots to the repository and provide additional information (e.g., name,
avatar, language, description, application purpose). We selected chatbots.org because it
is one of the longest running chatbot directory services (established 2008) [6] and has
been used in research before [e.g., 25].
For our analysis, we retrieved the data of all chatbots listed on chatbots.org on June
28, 2019 by using a web crawler. This resulted in a data sample consisting of 1,375
chatbots including their name, avatar, description, and other meta-information such as
the application domain (i.e., chatbots.org assigns twelve, not mutually exclusive
application domains to the listed chatbots).
In our analysis, we focused on three cues: the chatbots (1) name, (2) avatar, and
(3) description. We selected these cues since several studies revealed that the gender of
the (1) name and the (2) avatar of a chatbot trigger stereotypical gender responses [e.g.,
5,9] and (3) that written text can convey gender attributes and personality traits [4].
Given our large sample size, we decided to automatically extract and analyze the
gender-specic design (female, male, none) of these three cues using available tools
and services. In addition, to validate our automated approach, we randomly selected
and manually coded 100 chatbots.
Our automated gender analysis approach is illustrated in Fig. 1. First, to investigate
the gender of the chatbotsnames, we used two online services that identify the gender
of a given name, namely www.gender-api.com (includes the gender of over two million
names) and the node.js package gender-detection[36] (includes the gender of over
40,000 names). Only if both services recognized the same gender of one of the 1,375
chatbot names, we included it in the analyses. Second, to investigate the gender of an
avatar, we used Microsoft Azures Face API [30] which is able to detect, identify,
analyze, organize, and tag faces in photos and also to extract the gender of a face.
Investigated Cues
Gender Analysis
Methods
1,375 chatb ots listed
on chatbots.org
Avatar Descrip tionName
Micro so ft Azur e‘s f ace
rec o gn itio n AP I
Text mining o f gender
specific pronouns
gender-api.c om and node. js
package “ gender-d etection”
Investigated
Chatbot Sample
Fig. 1. Automated gender analysis approach to investigate gender-specic cues in the design of
chatbots.
Gender Bias in Chatbot Design 83
Therefore, we used this API to analyze 1373 chatbot avatar pictures that we down-
loaded from chatbots.org (two chatbots did not have a picture). Finally, to analyze the
chatbots description, we text-mined the description of all retrieved chatbots to identify
gender-specic pronouns that refer to one of the two genders (female: she,her,
hers; male: he,him,his)[4]. After excluding ambiguous descriptions (i.e.,
descriptions including both female and male pronouns), we assigned a gender to the
description of a chatbot. Table 2shows the results of the automated gender analysis
approach for three examples.
To investigate the reliability of our automated gender analysis approach, we
investigated whether there are conicting results between the three methods (e.g., a
chatbot has a male name and a female avatar). In total, we identied only 15 conicts in
our result set. Subsequently, we analyzed these conicts in more detail and manually
coded all conicting gender-specic cues. Overall, seven of these conicts were caused
by a wrong gender assignment to an avatar of a chatbot. After analyzing these wrong
assignments, we identied that Microsofts face recognition API potentially has
problems to assign the correct gender to cartoon avatars with a low resolution. Another
ve conicts were caused by the text mining approach. In ve cases, all pronouns in
the chatbots descriptions referred to another person (e.g., the chatbot engineer). Thus,
the pronouns did not refer to the chatbots itself. Finally, two chatbots names were
labeled wrong since the names (i.e., Nima, Charlie) are not clearly gendered and thus,
could have been assigned to both genders.
Table 2. Exemplary results of the automated gender analysis approach.
Name
(Company) Avatar Excerpt of Description (1) Name (2) Avatar (3) De-
scription
SOphiA
(BASF)
SOphiA is an Intranet Interactive
Assistant used internally by BASF
for its worldwide operations. She
answers questions about […].
Female Female Female
Frank
(Verizon)
Frank answers all of your Verizon
customer service support questions. Male Male None
BB
(KLM)
[…]. BB has her own professional,
helpful and friendly character, but
be warned; she can also be a bit
cheeky from time to time. […]
None None Female
Table 3. Comparison of automatic gender analysis approach and manual coding for a
subsample of 100 randomly selected chatbots.
Cues Number of conicts between automated
and manual coding
Number of not recognized
gender-specic cues
Name 0 25 (15 female: 10 male)
Avatar 0 20 (17 female: 3 male)
Description 0 0
84 J. Feine et al.
To further validate the reliability of the automated gender analysis approach, we
retrieved a random sample of 100 chatbots from the total sample of 1,375 chatbots. The
rst and second author manually coded the gender of the name, avatar, and description
of these chatbots. There were no disagreements between both coders. Subsequently, we
compared the results of the manual coding with our automated approach. The com-
parison showed that there were no conicts between the genders that were identied.
However, as illustrated in Table 3, the manual coding approach resulted in the iden-
tication of more gender-specic names and avatars. Most names that were not rec-
ognized as having a gender were female. Similarity, most of the avatars that were not
recognized were female.
4 Results
In the following, we present the results of our automated analysis of three cues (i.e.,
name, avatar, and description) in our sample of 1,375 chatbots. First, we provide an
overview of the total amount of gendered chatbots before reporting the gender distri-
bution (i.e., female vs. male) of the gender-specic cues, and their distribution
according to the chatbots application domain.
In total, we identied the gender of 620 chatbot names (45.09% of all investigated
chatbots), 347 chatbot avatars (25.24%), and 497 chatbot descriptions (36.15%) using
our automated approach. As illustrated in Fig. 2, there are some overlaps between the
cues. Overall, 501 (36.44%) of the chatbots did not have one gender-specic cue. In
addition, we identied 874 chatbots (63.56%) with at least one gender-specic cue (i.e.,
No Gender: 501 chatbots
36.44%
Only Name Gender: 199
chatbots
14.47%
Only Avatar Gender:
65 chatbots
4.72%
Only D e s cript ion G ende r:
141 chatbots
10.25%
Name & Avatar Gender:
113 chatbots
8.22%
Name & Description
Gender: 187 chatbots…
Avatar & Description
Gender: 48 chatbots
3.50%
Name & Avatar & De scription
Gender: 121 chatbots
8.80%
No gend ered cu e: 501 (36.44%)
One gen dered cu e: 405 (29.44% )
Two gen dere d cues : 348 (25.32%)
Thre e g end ered cu es: 121 (08.80%)
Fig. 2. Distribution of gender-specic names, avatars, and descriptions in the investigated
chatbot sample.
Gender Bias in Chatbot Design 85
either a gendered name, avatar, or description). Moreover, 469 chatbots (34.11%) had
at least two gender-specic cues, and 121 chatbots (8.80%) had all three gender-
specic cues (i.e., a gendered name, avatar, and description). Taken together, the results
suggest that the majority of chatbots listed on chatbots.org are gendered in their design.
Next, we identied whether the gender-specic cues are female or male. As shown
in Fig. 3, the large majority of gender- specic names were female (76.94%). The
analyses of avatars and descriptions revealed similar results: 77.56% of the avatars
were classied as female and 67.40% of the descriptions were classied as female.
These results strongly suggest that most chatbots are designed to be female.
Our analysis of gendered chatbots and their application domains revealed that
48.90% of them belong to only three application domains, namely branded conver-
sations, customer service, and sales (see Table 4). Additionally, most domains (8) were
clearly dominated by female names and only three domains by male names. The same
patterns emerged in the analyses of avatars (i.e., all but one domain were dominated by
female avatars) and descriptions (i.e., only four categories were dominated by male
descriptions). Taken together, we conclude that the gender bias is particularly evident
in the design of chatbots for specic application domains such as branded conversa-
tions, customer service, and sales.
Fig. 3. Gender-specic distribution of investigated cues.
86 J. Feine et al.
Table 4. Chatbot application domains listed on chatbots.org and their gender-specic design
(note: application domains are not mutually exclusive).
Application
domain
Description of application
domain as listed on Chatbots.
org
All
chatbots
Gendered
names
Gendered
avatars
Gendered
descriptions
Animals &
aliens
Speaking, listening and
responding virtual animals,
cartoonlike characters or
creatures from space
20 Female: 1
Male: 2
Female: 0
Male: 0
Female: 4
Male: 11
Branded
conversations
Dialogues on behalf of an
organization, on a product or
service
511 Female:
257
Male: 54
Female:
137
Male: 32
Female:
162
Male: 38
Campaign Designed for a limited
timeserving a campaign
objective
61 Female: 9
Male: 11
Female:
13
Male: 6
Female: 4
Male: 8
Customer
service
To answer questions about
delivered goods or services
532 Female:
251
Male: 55
Female:
137
Male: 22
Female:
164
Male: 33
Knowledge
management
To acquire information from
employees through natural
language interaction
63 Female:
30
Male: 3
Female:
13
Male: 1
Female: 16
Male: 4
Market
research
Conducting surveys with
consumers through automated
chat
16 Female: 6
Male: 0
Female: 3
Male: 0
Female: 6
Male: 0
Sales A conversion of a dialogue
focused on closing the deal
236 Female:
106
Male: 16
Female:
61
Male: 9
Female: 81
Male: 10
Clone A virtual version of a real
human being, whether still
alive or a historic person
40 Female: 3
Male: 14
Female: 5
Male: 20
Female: 2
Male: 11
E-Learning Human like characters in
virtual reality and augmented
reality with a scripted role
21 Female: 4
Male: 0
Female: 2
Male: 1
Female: 7
Male: 6
Gaming Conversational characters in
games or virtual worlds
14 Female: 5
Male: 2
Female: 0
Male: 1
Female: 5
Male: 5
Proof of
concept
Demonstrational versions
created by professional
developers on their own
websites
152 Female:
52
Male: 18
Female:
28
Male: 6
Female: 45
Male: 28
Robot toy Physical robotic gadgets with
natural language processing
capabilities
1 Female: 0
Male: 0
Female: 0
Male: 0
Female: 0
Male: 1
Gender Bias in Chatbot Design 87
5 Discussion
In this paper, we show that gender-specic cues are commonly used in the design of
chatbots in the wild and that many chatbots are explicitly or implicitly designed to
convey a specic gender. This nding ranges from names and avatars to the textual
descriptions used to introduce them to their users. More specically, most of the
chatbots have female names, female-looking avatars, and are described as female
chatbots. Thus, we found evidence that there is a tendency to prefer one gender (i.e.,
female) over another (i.e., male). Therefore, we conclude that there is a gender bias in
the design of chatbots. The gender bias is particularly evident in three domains (i.e.,
customer service, branded conversation, and sales).
Our ndings do not only mirror the results of the UNESCO report [43] on gender
bias in voice-based CAs, but also support an observation already made in 2006. In their
analysis of genders stereotypes implemented in CAs, De Angeli and Brahnam [2]
conclude that virtual assistants on corporate websites are often embodied by seductive
and nice looking young girls (p. 5). Considering the majority of chatbots currently
used in customer service or marketing, one could argue that not much has changed
since then. Although recent studies have raised concerns about ethical issues of gender
stereotyping in chatbot design [e.g., 28], there are no guidelines for a gender-equal
design of chatbots that could support chatbot engineers to diminish gender stereotypes
(at least) in the context of text-based CAs. Since gender-specic cues are often per-
ceived even before interacting with the chatbot, they have a large impact on how users
interact with them [9]. Therefore, discussions between researchers, practitioners, and
users will be highly important to answer relevant questions (e.g., Should a chatbot
have a specic gender?,Is it even possible to avoid gender attributions?). To
provide a starting point for discussions and suggest avenues for future research, we
formulate four propositions (P) that could help to mitigate the gender bias and pave the
way towards a more gender-equal design of chatbots.
P1: Diverse Composition of Chatbot Development Teams: The technology sector,
their programmers, and also chatbot engineers are often dominated by males (i.e.,
brogramming)[19]. Without criticizing the individual chatbot engineer, decision
makers could foster a more gender equal distribution in teams who develop socio-
technical systems that actively intervene in human affairs, such as chatbots. This could
reduce potential gender biases, since women generally tend to produce less gender-
biased language than men [4]. A more diverse team composition is also in line with the
ACM Code of Ethics and Professional Conductwhich states that computing pro-
fessionals should foster fair participation of all peoplealso based on their gender
identity[1]. Moreover, chatbot design teams should not solely consist of engineers but
should further include a diverse composition of people from different domains, such as
from linguistics and psychology.
P2: Leverage Tool-Support for Identifying Gender Biases in Chatbot Design:
Comprehensive tool support could help chatbot engineers to avoid potential gender
stereotypes in their development. Since gender stereotypes are often processed (and
88 J. Feine et al.
also implemented) in a unconscious manner [8], active tool support could help chatbot
engineers to avoid their mindless implementation. A similar approach has been pro-
posed in the context of general software evaluation. For example, the method Gen-
derMag[11] uses personas and cognitive walkthroughs in order to identify gender
inclusiveness issues in software. Therefore, such an approach could also help chatbot
engineers. While more effort is needed to develop tools that automatically evaluate the
gender inclusiveness of the design of chatbots, rst warning mechanism seem to be
easy to implement. For example, chatbot engineers could use the methods described in
this paper, namely gender analysis of chatbot names, avatar analysis using face
recognition, and text mining of descriptions. Additionally, chatbot conguration tools
could support chatbot engineers in making gender-equal design decisions [e.g., 17,18].
P3: Avoid Female-by-DefaultChatbot Designs: Overall, it does not appear nec-
essary to give a chatbot a default (female) gender. However, it is currently not clear
whether developing non-gendered chatbots or challenging human perceptions of
chatbot gender is the solution. Thus, chatbot engineers and the research community are
still far from resolving those issues, and the community should be open to discussing
them. Nevertheless, chatbot engineers need to actively implement mechanisms to
respond to unsavory user queries in order to avoid the manifestations of gender
stereotypes in the use of chatbots [9]. For example, Apples Siri is not encouraging
gender-based insults anymore (e.g., Id blush if I could). Other CAs do not pretend to
have a gender (e.g., if users ask Cortana, what is your gender?, Cortana automati-
cally replies, technically, Im a cloud of innitesimal data computation[43]. How-
ever, further research is needed to investigate user-centered designs and mechanisms to
mitigate and discourage negative stereotyping in the use of chatbots.
P4: Promote Ethical Considerations in Organizations: Although, gender equality is
one of the UN sustainability goals [39], gender-specic cues in the design of CAs are
rarely attracting the attention of governments and international organizations [43].
Therefore, decision makers and engineers need to take the rst step and challenge each
chatbot design towards potential gender stereotypes and other ethical considerations.
By actively promoting such considerations, chatbot development teams and other
people engaged in the development process will prot from an increased awareness in
order to build more gender-equal societies. Such endeavors could further complement
the ongoing discussions about gender-equal designs of algorithmic decision systems
and other types of articial intelligence [e.g., 12]. Finally, such organizational driven
approaches could complement the work of regulators to promote a more gender-equal
chatbot design.
5.1 Limitations and Future Research
There are limitations of this study that need to be addressed. First, our analysis is based
on a limited sample of chatbots. Although we did not differentiate between commercial
and research-based chatbots and did not check if they are still online, we argue that our
sample provides a sufcient base to draw conclusions about gender-specic cues in the
Gender Bias in Chatbot Design 89
design of chatbots. Future research could investigate gender-specic cues of different
chatbots using other samples and data sources such as BotList.co. This would help to
create a broader overview of the gender bias and would enhance our understanding of
the current design of chatbots.
Second, our automated approach for identifying the gender of the chatbots name,
avatar, and description might be susceptible to false positives and false negatives. To
address this limitation, we validated our approach by manually analyzing a subsample
of 100 chatbots. Because we did not identify any false positive result, we argue that
gender-specic cues identied by the approach are quite accurate. However, the
manual analysis also revealed that our automated approach did not identify all gender-
specic cues and indicated a few conicts between the three methods. For example,
Azures face recognition API struggled with extracting a gender from low-resolution
cartoon avatars and some pronouns in the description did not refer to the chatbot.
Therefore, we can only interpret the results of the automated gender analysis approach
as a conservative predictor for the amount of gender-specic cues in chatbot sample.
Thus, the true value of gendered chatbots might be much higher. Despite this limita-
tion, we believe that our ndings still hold because according to our manual analysis
most of the not recognized gender-specic cues where female.
Third, while our analysis included three important cues, several other cues in the
design of chatbots could be considered that may convey a gender-specic attribution.
Therefore, future research could extend our analysis to other relevant gender-specic
cues [16].
6 Conclusion
In this study, we examined the gender-specic design of three cues in the design of
1,375 chatbots using an automated gender analysis approach. Our results provide
evidence that there is a gender bias in the design of chatbots because most chatbots
were clearly gendered as female (i.e., in terms of their name, avatar, or description).
This bias is particularly evident in three application domains (i.e., branded conversa-
tions, customer service, and sales). Therefore, our study complements the ndings of a
recent UNESCO report that identied a gender bias in the design of voice-based CAs
and provides propositions as a starting point for future discussions and research.
References
1. ACM: Code of Ethics and Professional Conduct. https://www.acm.org/code-of-ethics
(2019). Accessed 26 July 2019
2. de Angeli, A., Brahnam, S.: Sex Stereotypes and Conversational Agents (2006)
3. Araujo, T.: Living up to the chatbot hype: the inuence of anthropomorphic design cues and
communicative agency framing on conversational agent and company perceptions. Comput.
Hum. Behav. 85, 183189 (2018). https://doi.org/10.1016/j.chb.2018.03.051
4. Artz, N., Munger, J., Purdy, W.: Gender issues in advertising language. Women Lang. 22(2),
20 (1999)
90 J. Feine et al.
5. Beldad, A., Hegner, S., Hoppen, J.: The effect of virtual sales agent (VSA) gender product
gender congruence on product advice credibility, trust in VSA and online vendor, and
purchase intention. Comput. Hum. Behav. 60,6272 (2016). https://doi.org/10.1016/j.chb.
2016.02.046
6. Bhagyashree, R.: A chatbot toolkit for developers: design, develop, and manage
conversational UI (2019). https://hub.packtpub.com/chatbot-toolkit-developers-design-
develop-manage-conversational-ui/. Accessed 22 July 2019
7. Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer
relationships. ACM Trans. Comput.-Hum. Interact. 12(2), 293327 (2005). https://doi.org/
10.1145/1067860.1067867
8. Bohnet, I.: What Works. Harvard University Press (2016)
9. Brahnam, S., de Angeli, A.: Gender affordances of conversational agents. Interact. Comput.
24(3), 139153 (2012). https://doi.org/10.1016/j.intcom.2012.05.001
10. Brandtzaeg, P.B., Følstad, A.: Chatbots: changing user needs and motivations. Interactions
25(5), 3843 (2018). https://doi.org/10.1145/3236669
11. Burnett, M., et al.: GenderMag: a method for evaluating softwares gender inclusiveness.
Interact. Comput. 28(6), 760787 (2016). https://doi.org/10.1093/iwc/iwv046
12. Council of Europe: Discrimination, articial intelligence, and algorithmic decision-making
(2018). https://rm.coe.int/discrimination-articial-intelligence-and-algorithmic-decision-
making/1680925d73
13. Cowell, A.J., Stanney, K.M.: Manipulation of non-verbal interaction style and demographic
embodiment to increase anthropomorphic computer character credibility. Int. J. Hum.-
Comput. Stud. 62(2), 281306 (2005). https://doi.org/10.1016/j.ijhcs.2004.11.008
14. Dale, R.: The return of the chatbots. Nat. Lang. Eng. 22(5), 811817 (2016). https://doi.org/
10.1017/S1351324916000243
15. EU: Ethics Guidelines for Trustworthy AI (2019). https://ec.europa.eu/futurium/en/ai-
alliance-consultation. Accessed 30 July 2019
16. Feine, J., Gnewuch, U., Morana, S., Maedche, A.: A taxonomy of social cues for
conversational agents. Int. J. Hum.-Comput. Stud. 132, 138161 (2019). https://doi.org/10.
1016/j.ijhcs.2019.07.009
17. Feine, J., Morana, S., Maedche, A.: Designing a chatbot social cue conguration system. In:
Proceedings of the 40th International Conference on Information Systems (ICIS). AISel,
Munich (2019)
18. Feine, J., Morana, S., Maedche, A.: Leveraging machine-executable descriptive knowledge
in design science research the case of designing socially-adaptive chatbots. In: Tulu, B.,
Djamasbi, S., Leroy, G. (eds.) DESRIST 2019. LNCS, vol. 11491, pp. 7691. Springer,
Cham (2019). https://doi.org/10.1007/978-3-030-19504-5_6
19. Følstad, A., Brandtzæg, P.B.: Chatbots and the new world of HCI. Interactions 24(4), 3842
(2017). https://doi.org/10.1145/3085558
20. Følstad, A., Brandtzaeg, P.B., Feltwell, T., Law, E.L.-C., Tscheligi, M., Luger, E.A.: SIG:
chatbots for social good. In: Extended Abstracts of the 2018 CHI Conference on Human
Factors in Computing Systems, SIG06:1SIG06:4. ACM, New York (2018). https://doi.org/
10.1145/3170427.3185372
21. Følstad, A., Skjuve, M., Brandtzaeg, P.: Different chatbots for different purposes: towards a
typology of chatbots to understand interaction design, pp. 145156 (2019)
22. Gnewuch, U., Morana, S., Maedche, A.: Towards designing cooperative and social
conversational agents for customer service. In: Proceedings of the 38th International
Conference on Information Systems (ICIS). AISel, Seoul (2017)
Gender Bias in Chatbot Design 91
23. Hayashi, Y.: Lexical network analysis on an online explanation task. Effects of affect and
embodiment of a pedagogical agent. IEICE Trans. Inf. Syst. 99(6), 14551461 (2016).
https://doi.org/10.1587/transinf.2015CBP0005
24. Hone, K.: Empathic agents to reduce user frustration. The effects of varying agent
characteristics. Interact. Comput. 18(2), 227245 (2006). https://doi.org/10.1016/j.intcom.
2005.05.003
25. Johannsen, F., Leist, S., Konadl, D., Basche, M., de Hesselle, B.: Comparison of commercial
chatbot solutions for supporting customer interaction. In: Proceedings of the 26th European
Conference on Information Systems (ECIS), Portsmouth, United Kingdom, 2328 June
2018
26. Kraemer, N.C., Karacora, B., Lucas, G., Dehghani, M., Ruether, G., Gratch, J.: Closing the
gender gap in STEM with friendly male instructors? On the effects of rapport behavior and
gender of a virtual agent in an instructional interaction. Comput. Educ. 99,113 (2016).
https://doi.org/10.1016/j.compedu.2016.04.002
27. Louwerse, M.M., Graesser, A.C., Lu, S.L., Mitchell, H.H.: Social cues in animated
conversational agents. Appl. Cogn. Psychol. 19(6), 693704 (2005). https://doi.org/10.1002/
acp.1117
28. McDonnell, M., Baxter, D.: Chatbots and gender stereotyping. Interact. Comput. 31(2), 116
121 (2019). https://doi.org/10.1093/iwc/iwz007
29. McTear, M.F.: The rise of the conversational interface: a new kid on the block? In: Quesada,
J.F., Martín Mateos, F.J., López-Soto, T. (eds.) FETLT 2016. LNCS (LNAI), vol. 10341,
pp. 3849. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69365-1_3
30. Microsoft: Face recognition API (2019). https://azure.microsoft.com/en-us/services/
cognitive-services/face/. Accessed 22 July 2019
31. Myers, M.D., Venable, J.R.: A set of ethical principles for design science research in
information systems. Inf. Manag. 51(6), 801809 (2014). https://doi.org/10.1016/j.im.2014.
01.002
32. Nass, C., Moon, Y.: Machines and mindlessness social responses to computers. J. Soc.
Issues 56(1), 81103 (2000). https://doi.org/10.1111/0022-4537.00153
33. Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems, pp. 7278. ACM, New York
(1994). https://doi.org/10.1145/191666.191703
34. Nass, C., Moon, Y., Green, N.: Are machines gender neutral? Gender-stereotypic responses
to computers with voices. J. Appl. Soc. Pyschol. 27(10), 864876 (1997). https://doi.org/10.
1111/j.1559-1816.1997.tb00275.x
35. Niculescu, A., Hofs, D., van Dijk, B., Nijholt, A.: How the agents gender inuence users
evaluation of a QA system. In: International Conference on User Science and Engineering (i-
USEr) (2010)
36. npmjs: Gender-detection (2019). https://www.npmjs.com/package/gender-detection. Acces-
sed 22 July 2019
37. Nunamaker, J.E., Derrick, D.C., Elkins, A.C., Burgoon, J.K., Patton, M.W.: Embodied
conversational agent-based kiosk for automated interviewing. J. Manag. Inf. Syst. 28(1), 17
48 (2011). https://doi.org/10.2753/mis0742-1222280102
38. Rosenwald, M.S.: How millions of kids are being shaped by know-it-all voice assistants
(2019). https://www.washingtonpost.com/local/how-millions-of-kids-are-being-shaped-by-
know-it-all-voice-assistants/2017/03/01/c0a644c4-ef1c-11e6-b4ff-ac2cf509efe5_story.html?
noredirect=on&utm_term=.7d67d631bd52. Accessed 16 July 2019
39. United Nations: Sustainability development goals. Goal 5: gender equality (2015). https://
www.sdgfund.org/goal-5-gender-equality. Accessed 30 Oct 2019
92 J. Feine et al.
40. Vala, M., Blanco, G., Paiva, A.: Providing gender to embodied conversational agents. In:
Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, Kristinn R. (eds.) IVA 2011. LNCS
(LNAI), vol. 6895, pp. 148154. Springer, Heidelberg (2011). https://doi.org/10.1007/978-
3-642-23974-8_16
41. Verhagen, T., van Nes, J., Feldberg, F., van Dolen, W.: Virtual customer service agents.
Using social presence and personalization to shape online service encounters. J. Comput.-
Mediat. Commun. 19(3), 529545 (2014). https://doi.org/10.1111/jcc4.12066
42. Weizenbaum, J.: ELIZA - a computer program for the study of natural language
communication between man and machine. Commun. ACM 9(1), 3645 (1966)
43. West, M., Kraut, R., Chew, H.E.: Id blush if I could: closing gender divides in digital skills
through education (2019). https://unesdoc.unesco.org/ark:/48223/pf0000367416
Gender Bias in Chatbot Design 93
... The design decision is based on findings of previous studies revealing that female representations might be more beneficial in supportive relationship contexts (e.g., [25], [60]). In addition, there is a general bias with regard to the use of female representations in the research field of CAs [15], [61]. ...
... This bias is particularly evident in customer service and sales. The chatbot gender is also influenced by the sector: for instance, the technology sector has a great number of men, so chatbots are often male [12]. ...
Conference Paper
Full-text available
The use of Artificial Intelligence (AI) among companies is expected to be pervasive in the next few years in order to support decision making, increase productivity and efficiency and define User-Centred experiences for both customers and employees. This scenario is affecting the context of Customer Management Processes, such as marketing, sales and after sales activities including front line and back office tasks where companies are experiencing an increase of volumes of available data and an unexpected amount of multichan-nel interactions. This caused the need to invest in AI-based solutions such as Virtual Assistants, Chatbots and Vocalbots, Robotic Process Automation and analytics tools. In early 2021, the CMMC Club (Customer Management Multi-media Competence Club, born in 1997 at the scope to promote the benchmar-king of performance and experiences among Companies with the mission to improve the relationship with their Customers through multimedia channels) and BSD (Italian company focused on research and Interaction Design) launched a survey on AI in Customer Management Processes. The goal was to understand how AI is used to create value in Customer Management processes, exploring the current and future trends and the most relevant issues in AI adoption. Results showed that companies are focusing not only on technological issues but also on the relationship between human beings and organizations, facing interaction design issues and User Experience topics.
... As technologies advance to highly interactive user interface with realistic digital representations, it is unclear how their connection to certain contextual information, such as a background story, will affect user appeal and engagement [16,[49][50][51]. Such technological advantages tend to serve privileged populations better and can implicitly embody prejudices against those who are vulnerable and marginalized [52]. ...
Article
Background Leveraging artificial intelligence (AI)–driven apps for health education and promotion can help in the accomplishment of several United Nations sustainable development goals. SnehAI, developed by the Population Foundation of India, is the first Hinglish (Hindi + English) AI chatbot, deliberately designed for social and behavioral changes in India. It provides a private, nonjudgmental, and safe space to spur conversations about taboo topics (such as safe sex and family planning) and offers accurate, relatable, and trustworthy information and resources. Objective This study aims to use the Gibson theory of affordances to examine SnehAI and offer scholarly guidance on how AI chatbots can be used to educate adolescents and young adults, promote sexual and reproductive health, and advocate for the health entitlements of women and girls in India. Methods We adopted an instrumental case study approach that allowed us to explore SnehAI from the perspectives of technology design, program implementation, and user engagement. We also used a mix of qualitative insights and quantitative analytics data to triangulate our findings. Results SnehAI demonstrated strong evidence across fifteen functional affordances: accessibility, multimodality, nonlinearity, compellability, queriosity, editability, visibility, interactivity, customizability, trackability, scalability, glocalizability, inclusivity, connectivity, and actionability. SnehAI also effectively engaged its users, especially young men, with 8.2 million messages exchanged across a 5-month period. Almost half of the incoming user messages were texts of deeply personal questions and concerns about sexual and reproductive health, as well as allied topics. Overall, SnehAI successfully presented itself as a trusted friend and mentor; the curated content was both entertaining and educational, and the natural language processing system worked effectively to personalize the chatbot response and optimize user experience. Conclusions SnehAI represents an innovative, engaging, and educational intervention that enables vulnerable and hard-to-reach population groups to talk and learn about sensitive and important issues. SnehAI is a powerful testimonial of the vital potential that lies in AI technologies for social good.
Chapter
Anthropomorphism is defined as a tendency to apply human knowledge to a nonhuman target. Under this process, individuals perceive social characteristics from human-like cues in a nonhuman target, make sense of the situation in which they interact with the target, and behave in a way appropriate to the inferred situation. Acknowledging that anthropomorphism is the key mechanism in users' responses to human–AI interaction, this chapter reviewed literature on user experience in interacting with AI agents and reinterpreted their findings through the theoretical lens of anthropomorphism. The underlying mechanism of user experience reported in the literature was articulated, and the determinants of anthropomorphism were identified. Positive and negative consequences of anthropomorphism in the context of using a recommendation system were discussed. Potential solutions for the negative outcomes of anthropomorphism were also presented.
Article
Purpose AI-driven product recommendation chatbots have markedly reduced operating costs and increased sales for marketers. However, previous literature has paid little attention to the effects of the personality of e-commerce chatbots. This study aimed to examine the ways that the interplay between the chatbot's and the user's personality can increase favorable product attitudes and future intentions to use the chatbot. Based on prior literature, we specifically focused on the degree of extroversion of both chatbot and user. Design/methodology/approach A total of 291 individuals participated in this study. Two different versions of chatbot were created for this study (i.e. extroversion: high vs. low). Participants self-reported their degree of extroversion. The PROCESS macro Model 1 and Model 7 with the Johnson–Neyman technique were employed to test the hypotheses. Findings The results showed that the high extroversion chatbot elicited greater user satisfactions and perceptions of chatbot friendliness among users with a high level of extroversion. On the contrary, the low extroversion chatbot resulted in greater user satisfactions and perceived chatbot friendliness among users with a low level of extroversion. This study further found that user satisfactions and perceived chatbot friendliness mediated the effects of the chatbot on greater intentions to use the chatbot and more favorable product attitudes. Originality/value By showing the effects of matching the personality of the chatbot and user, this study revealed that similarity-attraction effects also apply to human–chatbot interaction in e-commerce. Future studies would benefit by investigating the similarity-attraction effects in different characteristics, such as appearance, opinion and preference. This study also provides useful information for e-commerce marketers and chatbot UX/UI designers.
Article
We survey SoTA open-domain conversational AI models with the objective of presenting the prevailing challenges that still exist to spur future research. In addition, we provide statistics on the gender of conversational AI in order to guide the ethics discussion surrounding the issue. Open-domain conversational AI models are known to have several challenges, including bland, repetitive responses and performance degradation when prompted with figurative language, among others. First, we provide some background by discussing some topics of interest in conversational AI. We then discuss the method applied to the two investigations carried out that make up this study. The first investigation involves a search for recent SoTA open-domain conversational AI models, while the second involves the search for 100 conversational AI to assess their gender. Results of the survey show that progress has been made with recent SoTA conversational AI, but there are still persistent challenges that need to be solved, and the female gender is more common than the male for conversational AI. One main takeaway is that hybrid models of conversational AI offer more advantages than any single architecture. The key contributions of this survey are (1) the identification of prevailing challenges in SoTA open-domain conversational AI, (2) the rarely held discussion on open-domain conversational AI for low-resource languages, and (3) the discussion about the ethics surrounding the gender of conversational AI.
Article
Full-text available
Organizations’ hiring processes are increasingly shaped by various digital tools and e-recruitment systems. However, there is little understanding of the recruiters’ needs for and expectations towards new systems. This paper investigates recruitment chatbots as an emergent form of e-recruitment, offering a low-threshold channel for recruiter-applicant interaction. The rapid spread of chatbots and the casual nature of their user interfaces raise questions about the perceived benefits, risks, and suitable roles in this sensitive application area. To this end, we conducted 13 semi-structured interviews, including 11 interviews with people who are utilizing recruitment chatbots and two people from companies that are developing recruitment chatbots. The findings provide a qualitative account of their expectations and motivations, early experiences, and perceived opportunities regarding the current and future use of chatbots in recruitment. While chatbots answer the need for attracting new candidates, they have also introduced new challenges and work tasks for the recruiters. The paper offers considerations that can help to redesign recruitment bots from the recruiter’s viewpoint.
Chapter
Natural language processing (NLP) is increasingly applied to a broad range of sensitive tasks, such as human resources, biomedicine, and healthcare. Accordingly, a growing body of research is investigating the impact of sex and gender bias in the models and the data on which such models are trained. As NLP systems become more pervasive in our societies, the vulnerability to sex and gender bias may cause the perpetuation of prejudice and discriminatory decisions. To address this challenge, a widespread awareness of bias needs to be created in the NLP community and more robust learning algorithms and fair solutions are required for the development and evaluation of NLP methods. In this chapter, we survey the state-of-the-art NLP models and some popular applications to biomedicine and health, with special emphasis on chatbots for mental health. Moreover, we discuss sources and implications of bias in this area and show examples of notable debiasing methods.
Chapter
The design cue gendered anthropomorphism plays an important role in tailoring chatbots towards customers’ design preferences. Yet, it still remains unexplored if this design cue affects the determinants of the behavioral intention towards using chatbots as moderating variable. Therefore, this paper investigates the moderating influence of gendered anthropomorphic design cues on the key determinants of the behavioral intention that are defined by the Unified Theory of Acceptance and Use of Technology (UTAUT). For this purpose, three chatbot design groups have been created and compared: a female and a male designed chatbot with gendered anthropomorphic design cues, as well as one chatbot without gendered anthropomorphic design cues. A sample of 209 participants is randomly assigned to each design group. The PLS-SEM multigroup analysis shows gendered anthropomorphic chatbot design cues yield significant positive impact of the performance expectancy towards the behavioral intention to use chatbots. This was not the case for the design group without gendered anthropomorphic design cues. Also, in contrast to the male anthropomorphic chatbot design, the female anthropomorphic chatbot design yields significant positive impact of social influence on the behavioral intention.
Conference Paper
Full-text available
Social cues (e.g., gender, age) are important design features of chatbots. However, choosing a social cue design is challenging. Although much research has empirically investigated social cues, chatbot engineers have difficulties to access this knowledge. Descriptive knowledge is usually embedded in research articles and difficult to apply as prescriptive knowledge. To address this challenge, we propose a chatbot social cue configuration system that supports chatbot engineers to access descriptive knowledge in order to make justified social cue design decisions (i.e., grounded in empirical research). We derive two design principles that describe how to extract and transform descriptive knowledge into a prescriptive and machine-executable representation. In addition, we evaluate the prototypical instantiations in an exploratory focus group and at two practitioner symposia. Our research addresses a contemporary problem and contributes with a generalizable concept to support researchers as well as practitioners to leverage existing descriptive knowledge in the design of artifacts.
Article
Full-text available
Conversational agents (CAs) are software-based systems designed to interact with humans using natural language and have attracted considerable research interest in recent years. Following the Computers Are Social Actors paradigm, many studies have shown that humans react socially to CAs when they display social cues such as small talk, gender, age, gestures, or facial expressions. However, research on social cues for CAs is scattered across different fields, often using their specific terminology, which makes it challenging to identify, classify, and accumulate existing knowledge. To address this problem, we conducted a systematic literature review to identify an initial set of social cues of CAs from existing research. Building on classifications from interpersonal communication theory, we developed a taxonomy that classifies the identified social cues into four major categories (i.e., verbal, visual, auditory, invisible) and ten subcategories. Subsequently, we evaluated the mapping between the identified social cues and the categories using a card sorting approach in order to verify that the taxonomy is natural, simple, and parsimonious. Finally, we demonstrate the usefulness of the taxonomy by classifying a broader and more generic set of social cues of CAs from existing research and practice. Our main contribution is a comprehensive taxonomy of social cues for CAs. For researchers, the taxonomy helps to systematically classify research about social cues into one of the taxonomy's categories and corresponding subcategories. Therefore, it builds a bridge between different research fields and provides a starting point for interdisciplinary research and knowledge accumulation. For practitioners, the taxonomy provides a systematic overview of relevant categories of social cues in order to identify, implement, and test their effects in the design of a CA.
Chapter
Full-text available
In Design Science Research (DSR) it is important to build on descriptive (Ω) and prescriptive (Λ) state-of-the-art knowledge in order to provide a solid grounding. However, existing knowledge is typically made available via scientific publications. This leads to two challenges: first, scholars have to manually extract relevant knowledge pieces from the data-wise unstructured textual nature of scientific publications. Second, different research results can interact and exclude each other, which makes an aggregation, combination, and application of extracted knowledge pieces quite complex. In this paper, we present how we addressed both issues in a DSR project that focuses on the design of socially-adaptive chatbots. Therefore, we outline a two-step approach to transform phenomena and relationships described in the Ω-knowledge base in a machine-executable form using ontologies and a knowledge base. Following this new approach, we can design a system that is able to aggregate and combine existing Ω-knowledge in the field of chatbots. Hence, our work contributes to DSR methodology by suggesting a new approach for theory-guided DSR projects that facilitates the application and sharing of state-of-the-art Ω-knowledge.
Chapter
Full-text available
Chatbots are emerging as interactive systems. However, we lack knowledge on how to classify chatbots and how such classification can be brought to bear in analysis of chatbot interaction design. In this workshop paper, we propose a typology of chatbots to support such classification and analysis. The typology dimensions address key characteristics that differentiate current chatbots: the duration of the user’s relation with the chatbot (short-term and long-term), and the locus of control for user’s interaction with the chatbot (user-driven and chatbot-driven). To explore the usefulness of the typology, we present four example chatbot purposes for which the typology may support analysis of high-level chatbot interaction design. Furthermore, we analyse a sample of 57 chatbots according to the typology dimensions. The relevance and application of the typology for developers and service providers are discussed.
Conference Paper
Full-text available
Chatbots are emerging as an increasingly important area for the HCI community, as they provide a novel means for users to interact with service providers. Due to their conversational character, chatbots are potentially effective tools for engaging with customers, and are often developed with commercial interests at the core. However, chatbots also represent opportunities for positive social impact. Chatbots can make needed services more accessible, available, and affordable. They can strengthen users' autonomy, competence, and (possibly counter-intuitively) social relatedness. In this SIG we address the possible social benefits of chatbots and conversational user interfaces. We will bring together the existing, but disparate, community of researchers and practitioners within the CHI community and broader fields who have an interest in chatbots. We aim to discuss the potential for chatbots to move beyond their assumed role as channels for commercial service providers, explore how they may be used for social good, and how the HCI community may contribute to realize this.
Article
Chatbots are very much an emerging technology, and there is still much to learn about how conversational user interfaces will affect the way in which humans communicate not only with computers but also with one another. Further studies on anthropomorphic agents and the projection of human characteristics onto a system are required to further develop this area. Gender stereotypes operate a profound effect on human behaviour. The application of gender to a conversational agent brings along with it the projection of user biases and preconceptions. These feelings and perceptions about an agent can be used to develop mental models of a system. Users can be inclined to measure the success of a system based on their biases and emotional connections with the agent rather than that of the system’s performance. There have been many studies that show how gender affects human perceptions of a conversational agent. However, there is limited research on the effect of gender when applied to a chatbot system. This chapter presents early results from a research study which indicate that chatbot gender does have an effect on users overall satisfaction and gender-stereotypical perception. Subsequent studies could focus on examining the ethical implications of the results and further expanding the research by increasing the sample size to validate statistical significance, as well as recruiting a more diverse sample size from various backgrounds and experiences. RESEARCH HIGHLIGHTS Many studies have indicated how gender affects human perceptions of a conversational agent. However, there is limited research on the effect of gender when applied to a chatbot system. This research study presents early results which indicate that chatbot gender does have an effect on users overall satisfaction and gender-stereotypical perception. Users are more likely to apply gender stereotypes when a chatbot system operates within a gender-stereotypical subject domain, such as mechanics, and when the chatbot gender does not conform to gender stereotypes. This study raise ethical issues. Should we exploit this result and perpetuate the bias and stereotyping? Should we really have a male chatbot for technical advice bots? Is this perpetuating stereotyping, the dilemma being that a male version would elicit more trust?
Article
We have created an automated kiosk that uses embodied intelligent agents to interview individuals and detect changes in arousal, behavior, and cognitive ef- fort by using psychophysiological information systems. In this paper, we describe the system and propose a unique class of intelligent agents, which are described as Special Purpose Embodied Conversational Intelligence with Environmental Sensors (SPECIES). SPECIES agents use heterogeneous sensors to detect human physiology and behavior during interactions, and they affect their environment by influencing hu- man behavior using various embodied states (i.e., gender and demeanor), messages, and recommendations. Based on the SPECIES paradigm, we present three studies that evaluate different portions of the model, and these studies are used as founda- tional research for the development of the automated kiosk. the first study evaluates human–computer interaction and how SPECIES agents can change perceptions of information systems by varying appearance and demeanor. Instantiations that had the agents embodied as males were perceived as more powerful, while female embodied agents were perceived as more likable. Similarly, smiling agents were perceived as more likable than neutral demeanor agents. the second study demonstrated that a single sensor measuring vocal pitch provides SPECIES with environmental awareness of human stress and deception. the final study ties the first two studies together and demonstrates an avatar-based kiosk that asks questions and measures the responses using vocalic measurements.
Article
Disembodied conversational agents in the form of chatbots are increasingly becoming a reality on social media and messaging applications, and are a particularly pressing topic for service encounters with companies. Adopting an experimental design with actual chatbots powered with current technology, this study explores the extent to which human-like cues such as language style and name, and the framing used to introduce the chatbot to the consumer can influence perceptions about social presence as well as mindful and mindless anthropomorphism. Moreover, this study investigates the relevance of anthropomorphism and social presence to important company-related outcomes, such as attitudes, satisfaction and the emotional connection that consumers feel with the company after interacting with the chatbot.
Conference Paper
The conversational interface has become a hot topic in the past year or so, providing the primary means of interaction with chatbots, messaging apps, and virtual personal assistants. Major tech companies have been making huge investments in the supporting technologies of artificial intelligence, such as deep learning and natural language processing, with the aim of creating systems that will enable users of smartphones and other devices to obtain information and access services in a natural, conversational way. Yet the vision of the conversational interface is not new, and indeed there is a history of research in dialogue systems, voice user interfaces, embodied conversational agents, and chatbots that goes back more than fifty years. This chapter explores what has changed to make the conversational interface particularly relevant today, examines some key issues from earlier work that could inform the next generation of conversational systems, and highlights some challenges for future work.