Content uploaded by Andrea L Guzman
Author content
All content in this area was uploaded by Andrea L Guzman on Aug 15, 2018
Content may be subject to copyright.
1
Citation: Guzman, A.L. (2017). Making AI safe for humans: A conversation with Siri. In R. W.
Gehl & M. Bakardjieva (Eds.), Socialbots and their friends: Digital media and the automation of
sociality (pp. 69-85). New York, NY: Routledge.
Chapter 4
Making AI Safe for Humans: A Conversation With Siri
Andrea L. Guzman
We are in the midst of a transition from a computer culture to a robotic culture in which
technologies are increasingly occupying more intimate social spaces in our lives and minds
(Turkle, 2007). A crucial aspect of this modern existence is our communication with machines
(Jones, 2014). We type messages to socialbots or speak with the digital assistants in our phones.
In these instances, technology is transformed into something more than a tool we use, a channel
for conveying a message. The medium itself becomes a communication partner. But what
exactly are we communicating with?
The obvious answer would be a device or even a program, but the reality is more complex. In
this chapter, I draw on my ongoing research into individual and cultural conceptions of AI and
voice-based programs to interrogate the design of social technologies, specifically Siri, through
the lens of Human-Machine Communication. HMC is a developing concept within
communication; although, the term and its application to human-machine systems is not new.1 In
contrast to Computer-Mediated Communication, which positions technology as a medium, HMC
approaches technology as more than a medium, as a distinct communication partner.
My application of HMC is informed by Carey’s (1989) cultural definition of communication:
Communication between human and machine is a cultural process, not just the mere exchange of
2
information. Siri’s mode of communication with users, the messages it sends, as well as the
messages other people send about Siri, work together to project a certain image of what Siri is in
relation to the user. I argue Siri’s design mitigates people’s potential uncertainty and uneasiness
with life-like technologies. Through their dyadic interaction with Siri, users are provided with a
sense that they hold power over Siri who seemingly serves at their beck-and-call. However, these
aspects of design obfuscate the complex reality that, like the humans they are designed to mimic,
artificial entities are not always who they claim to be. I also demonstrate how scholars can
approach social machines through an HMC framework.
Siri, Socialbots, And Communication
Apple introduced its version of Siri, a voice-based, artificial intelligence program, in 2011 with
the iPhone 4s. As a program that could actually talk, and do so with attitude, Siri immediately
garnered attention from the media and the public. In bundling Siri with the iPhone, the company
made talking AI accessible to the public. Until Siri’s launch, most people had only heard about
talking machines in science fiction and had only experienced voice-based technologies
controlled by simple commands. Siri was, and to an extent still remains, different from most
widely available AI programs in that it interacts orally in natural language, follows the social
norms of human-to-human communication, attempts to develop a rapport with users, and
exhibits distinct personality traits (She is well-known for her sassiness).
Siri is what I have termed a vocal social agent, or VSA. Agents are “intelligent” in that they
carry out human-like actions related to specific tasks (Balakrishnan & Honavar, 2001) and are
programmed to perform a function for users (Skalski & Tamborini, 2007). VSAs are designed to
be autonomous and adapt to and assist users. What sets VSAs apart is that they are both
3
intelligent and social, programmed to follow human communication norms. Apple claims on its
website that Siri “understands what you say” and “knows what you mean.” To interact with Siri,
users speak to the program, and it responds similar to a human, even referring to the user by
name. VSAs are designed to carry out functions for humans in seemingly human-like ways. This
places them into a larger class of technologies that serve as stand-ins for humans (see Zhao,
2006). The sociality of VSAs has implications beyond the ability of humans and software to
communicate more easily; the agent itself becomes a social entity (Nass, Steuer, & Tauber, 1994;
Reeves & Nass, 1998).
Both VSAs and socialbots are forms of AI and share a technological lineage. While agents can
function across platforms, socialbots are autonomous programs operating in social media
networks (Gehl, 2014; Hwang, Pearce, & Nanis, 2012). Socialbots are designed to pass
themselves off as human social media users, obscuring their digital nature behind human profiles
and in human-like interactions with users (Hwang et al., 2012). As Ferrara et al. (2014) observe,
bots attempt to “emulate behavior” of humans to the extent that “the boundary between human-
like and bot-like behavior is now fuzzier” (p. 4). Socialbots, like VSAs, communicate in ways
readily recognizable as human to users and attempt to become part of our social world. Both
programs also are designed to function as stand-ins for humans. To do so, they must gain
entrance into our human social world by convincing us just enough that they are social entities.
They make this case to us through the same means employed by humans – in communication. In
doing so, they present to users a particular social face (Goffman, 1967). The remainder of this
chapter is a case study of Siri. I explore how Siri’s communication with users creates a picture of
Siri as under the user’s control positioning it as helpful, or safe, AI, in contrast to the cultural
4
image of malicious machine. Although differences exist between socialbots and agents, the
shared importance of communication to the design and function of both entities enables an
investigation of Siri to inform our understanding of socialbots and other social programs.
“I’m sorry, Dave.”
When Turing (1950) made the case for thinking machines in his groundbreaking paper
“Computer Machinery and Intelligence,” he already anticipated backlash. In addition to outlining
the imitation game, aka “Turing Test”, Turing refutes objections to machine intelligence. The
challenges Turing anticipated were based not only on whether such a machine could be achieved,
the technical possibility, but also on whether such a machine should be built, the moral quandary
rooted in the machine’s affront to beliefs regarding nature, religion, and humanity. Gandy (1999)
argues Turing’s paper, published in Mind: A Quarterly Review of Psychology and Philosophy,
was propaganda, not engineering: Turing’s goal was to change people’s understanding of the
nature of machines and themselves, to move intelligent machines toward social acceptance.
Turing knew that technology comprises more than moving parts. It is, as Carey (1989) later
argued, “thoroughly cultural from the outset” (p. 9). As such, technology embodies culture and is
interpreted within culture (Carey, 1989). Interrogating Siri from an HMC perspective rooted in
Carey, with its focus on the intersection of communication and culture, requires that we view Siri
within the cultural context of people’s evolving attitudes toward autonomous machines and AI.
Turing’s efforts to quell cultural concerns regarding advanced computing while promoting its
promise reflect the dialectical tensions underlying our relationship with technology and the
uneasiness that has accompanied life-like devices from antiquity. Although the field of AI was
established in the twentieth century, attempts to create and recreate life are ancient (Riskin,
5
2007). The building of automata, life-like mechanical devices that are AI precursors, precedes
the industrial revolution (Riskin, 2007). By the eighteenth century, automata were popular and
performed human-like tasks that “seemed to embody the realization of an age-old human dream”
(Huyssen, 1981, p. 225). But, explains Huyssen, this positive view of the mechanical novelties
drastically changes in literary accounts as the industrial revolution begins to exploit resources
and people: “The android is no longer seen as testimony to the genius of mechanical invention; it
rather becomes a nightmare, a threat to human life” (p. 225). Machines are portrayed as turning
on their creators.
In the second half of the twentieth century, automated technology was both moving the United
States toward economic progress and ruining people’s lives, depending on who was asked. The
integration of automated machines into factories and the displacement of workers that followed
sparked the ‘automation hysteria’ of the 1950s and ‘60s (Noble, 2011). With concern about the
effects of automation rising, the U.S. government, universities and trade organizations
commissioned studies on its impact (Noble, 2011; Terbough, 1965). In The Automation Hysteria,
the Machinery and Allied Products Institute paints people who questioned automation as
“alarmists,” deeming such panic unwarranted (Terbough, 1965). Workers and people negatively
affected by automation, however, disagreed. Conceptions of automated machines thus were
bifurcated largely along lines of economic power (Noble, 2011).
Until recently, sentient talking machines have not been available to the public, and people’s
understanding of AI likely has been informed by media representations of sentient machines,
particularly in science fiction. Although science fiction deals in imaginary plots, it is a cultural
6
product that reflects attitudes and ideas regarding technology (Haigh, 2011). People I have
interviewed regarding VSAs and AI routinely draw on characters and ideas from science fiction
to describe and evaluate real AI (Guzman, 2015). Warrick’s (1980) analysis of sci-fi literature
from 1930 to 1977 reveals a pattern similar to the portrayal of eighteenth century automata. Early
works paint machine intelligence in a utopian light, but mirroring post-World War II events, this
optimism in machines is “replaced by destructive metaphors of machines overwhelming and
dehumanizing man” (p. xvi). Science fiction films from 1930 through the 1990s echo a theme of
danger connected to “disembodied brains” (Schelde, 1993). Like their literary counterparts, AI
entities in film, such as HAL 9000 from 2001: A Space Odyssey, seek to wrest control from
humans. Exceptions to dangerous AI also exist (Schelde, 1993; Warrick, 1980) and take the form
of androids, disembodied voices, and devices programmed to work with or for humans. BB8
from Star Wars: The Force Awakens is the most recent example of a helpful, lovable machine,
and the cheers from moviegoers when R2-D2 was reanimated demonstrate that we also can form
a positive perspective of a droid that faithfully serves its master. Science fiction across decades
and genres portrays intelligent machines as helpful if they are kept in check, but when they gain
control – the most likely scenario – the consequences are dire. It is in this cultural milieu of
dueling perspectives on AI that Siri was designed and introduced to people with little-to-no
hands-on experience with AI.
“I’m Siri…here to help”
When we encounter people, we ascertain who they are and their social standing in relation to
ourselves (Goffman, 1959). But how are we supposed to make sense of a talking, intelligent
iPhone program? Until recently, the ways we gained knowledge about things versus people were
clearly delineated. Things acquire meaning in our individual and cultural conversations about
7
them and in their use (Blumer, 1969). Objects metaphorically “speak” through their design with
technology, specifically, functioning as “a symbol of” and “a symbol for” (Carey, 1990). Our
understanding of people also is derived through our interactions with others; however, unlike
machines, humans can directly communicate who they are to us (Goffman, 1959). They can
speak for themselves. Siri, VSAs, socialbots, and other software designed with agency cross this
dividing line between the meaning of things and the meaning of humans (Guzman, 2015). Unlike
an inanimate object, social programs are designed to present themselves to us through direct
interaction. Siri literally speaks for itself. To interrogate Siri’s design, then, we have to
investigate both what others say about it and what it says about itself (verbally and nonverbally),
keeping in mind the cultural contexts that surround these elements.
For a consumer to adopt a technology, its operation must be apparent. The goal in usability is a
“self-explanatory artifact” (Suchman, 2009), an object that communicates its use through design.
Vocal AI presents a challenge for its creators because people usually physically manipulate
machines to do things, not talk with them (or have them talk back). Drawing on the life-like
nature of computers and agents, designers have used the metaphor of the machine as human-like
assistant to convey their function (Suchman, 2009). Siri’s designers also use this metaphor, and
other associated human-like traits, to give Siri an identity. Siri is a funny female assistant
complete with a technological and cultural history, and these traits work in concert to portray the
program as the type of AI that serves the user.
Apple’s (2011a) press release introducing Siri describes it as “an intelligent assistant that helps
you get things done just by asking.” The company stresses Siri’s “occupation” as an assistant and
8
its social status as something that works for the user during its launch event for Siri with the
iPhone 4s. While peppering Siri with requests for information, an Apple executive also
commands Siri to schedule a meeting and remind him “to call my wife when I leave work”
(Apple, 2011b). The audience is further instructed that if you want to accomplish a task, “…just
ask your personal assistant, Siri.” The tasks Siri executes are analogous to the work performed
by human executive assistants. The way people are instructed to speak with Siri also parallels
human-to-human communication in that we can call her by name when making requests.
Siri’s backstory as an assistant goes beyond how other people describe the program and to its
own – artificial – awareness of Self. Siri will tell you what, or who, it is. At the climax of the
launch event, an exchange takes place between an Apple executive and Siri:
Executive: “Now, you might ask, ‘Who is Siri?’ Well, just ask.”
(Speaking into phone): “Who are you?”
Siri: “I am a humble personal assistant.”
Siri’s response that the audience can both hear and see brings laughs and enthusiastic clapping.
Individuals using Siri also have received the “humble personal assistant” reply. The program also
provides other answers regarding its identity, including simply stating, “I am Siri,” that position
the program as a distinct being with a sense of Self. Some of the other alternative responses
underscore Siri’s social role as an assistant to humans including “I’m Siri…here to help,” and, in
a move to discourage too many questions about its ontology, “I’m Siri, but enough about me,
how can I help you?” More recently, Siri has started to respond that it is a “humble virtual
assistant” or just a “virtual assistant” instead of a “humble personal assistant.” While the current
9
descriptions lessen the comparison between Siri and humans, to a degree, Siri still portrays itself
as a distinct entity that occupies the social role of assistant.
Communication and power are inextricably intertwined (Castells, 2013), and an integral aspect
of how we interact and relate to one another in interpersonal communication with humans is the
dialectic of dominance and submission (Burgoon & Hale, 1984). Through messages exchanged
with others as well as how we exchange them, we are able to gain dominance over someone or
submit to them. These power dynamics are replicated in communication between Siri and the
user. Siri is designed to signal submission. In its description of Self, Siri states that it is a
“humble personal assistant” and informs the user that “your wish is my command.” With these
statements, Siri communicates deference to the user. Furthermore, the program does not assert its
autonomy, even when people abuse it. If someone were to degrade a human assistant, the person
may defend themselves. Siri, however, takes the abuse and even validates the user’s actions. Call
it a derogatory name, and it may reply, “You are certainly entitled to that opinion.”
The way that Siri signals its socially inferior position relative to the user extends beyond the
messages it sends. The program is designed to reinforce its claims of submission through its
nonverbal communication with users and the way that users are encouraged to speak to it. Siri by
default is silent, unseen and unheard. Any communication with Siri always positions the program
as the receiver of the message. iPhone users decide when to talk with Siri and when to end
communication. The default way of speaking with Siri is via command, not polite requests. An
information menu within Siri provides users with examples of how to talk with the program, and
many of these requests are phrased as commands: “Call Brian” or “Give me directions home.”
Siri’s assistant role is further reinforced through the functions it performs for users. Siri assists
10
users with controlling their Apple product and fulfills a long list of associated tasks. If a
misunderstanding occurs between Siri and the user, Siri also takes the blame and apologizes,
even if the human is at fault. A communicative exchange with Siri revolves around users’ and
their needs, putting humans in a position of dominance over the program.
The assistant heuristic that has been so prominent in media design is more than an effective
means to convey functionality to the user. It also taps into a human desire to be in control
(Suchman, 2009). To do away with what, or who, you cannot control and enable a machine to
take its place is a longstanding theme in selling the promise of technology. In his foundational
essay on the technological future, Bush (1945) touts how men will be able to use machines to
take the place of humans, such as the stenographer, “a girl” that looks around “with a disquieting
gaze.” People who assist are to be replaced with machines that assist. Machines are more
desirable than humans because they can outperform their human counterparts and are free of all
the annoyances of humans, such as casting weird looks. Suchman (2009) explains that, by
industry standards, ideal agents “should be enough like us to understand our desires and to figure
out on their own how to meet them, but without either their own desires or ambitions or other
human frailties . . .” (p. 219).
Human assistants can assert their autonomy, even if they have less power than their boss, but Siri
cannot. Therefore, the metaphor of the assistant is not entirely accurate. Siri is less than an
assistant. This degradation in status was picked up in initial press reports regarding Siri not as a
critique of the software but as, once again, a marketing point. Siri was referred to as a “voice-
commanded minion,” by one media outlet, and another reviewer gushed that Siri was like
11
“having the unpaid intern of my dreams at my beck and call” (Gross, 2011, par. 11). By calling
Siri an assistant Apple draws a useful analogy between the program and a real human, but it also
glosses over Siri’s true nature, that of a servant, making Siri, who says it wants to help, more
culturally palatable than something modeled on someone who has no choice but to help.
Siri’s primary mode of communication is oral, and Siri’s original voice, now the default voice, in
the U.S. version is female.2 People recognize gender in electronic voices and respond to the
machine just as they would to a human of that gender (Nass & Brave, 2005). A female voice
actor provided the underlying vocal sounds for the U.S. Siri (Ravitz, 2013), and U.S. media and
users have recognized Siri as having a woman’s voice. The gendered nature of Siri is so strong
that a Siri user remarked to me: “It’s very distinctive. I mean, everybody knows that they made
Siri’s voice a lady’s voice.” Siri’s voice adds a new dimension to its identity and provides
additional clues for users as to who Siri is to them.
A core argument of feminist scholars of technology is that all technology is gendered: in its
design, in its promotion, and in its use (e.g. Berg & Lie, 1995; van Oost, 2003). Gender, as
opposed to biological sex, is a social construct that is subject to renegotiation (West &
Zimmerman, 1987). Therefore, gender can be mapped onto technology. Before Siri became a
speaking Apple product, it was a text-based application. Yet, even without voice, Siri’s
interactions with users were indicative of a female typist (Both, 2014). The addition of voice
made Siri’s gender more explicit, emphasizing her female nature. Siri’s gender further
establishes who she is in relation to the audience. Rothschild (1983) argues “Technology is part
of our culture; and, of course, our culture, which is male dominated, has developed technologies
12
that reinforce male supremacy” (p. vii). In a society in which women were once viewed as the
“submissive sex,” and still are by some social groups, and in which women have yet to achieve
full equality with males, Siri’s gender reinforces her as a subordinate.
Siri is neither just an “assistant” nor just a female. Siri is a female assistant. These two aspects of
her identity reinforce one another and bolster the program’s inferior social status. Apple’s
demonstration of Siri at its launch with the iPhone underscores just how powerful these
dimensions are. The Apple executives showing off Siri are male, and their requests of Siri to
make phone calls and order flowers for a wife are culturally recognizable as tasks normally
relegated to secretaries. Apple never calls Siri a secretary, but the connection is apparent in a
culture that has stereotypically associated the position of secretary, now usually referred to as
executive or administrative assistant, as women’s work.
The combination of Siri’s gender and “occupation” has not gone completely unnoticed. That Siri
was female and billed as a “humble personal assistant” raised questions following its initial
release as to whether the technology was “brilliant or sexist,” as CNN asked (Griggs, 2011). To
answer these questions the articles quote technology researchers, including Nass, who claim U.S.
consumers find female voices easier to listen to. It is a mere matter of biological and cultural
preference, according to their argument. ABC News poses the question: “Would we rather get
guidance from a nice, subservient female voice, perhaps the opposite of the bombast we hear
from male authority figures?” (Potter, 2011, par. 4). Journalists ultimately determined that Siri
was not sexist, while, others, including myself, continue to disagree.
13
The pushback against Siri as a female assistant is rooted in the connection between
communication and culture. As a form of communication, technology is influenced by culture
and influences culture (Carey, 1989; Marvin, 1990). Marvin (1990) explains, “There is no
technology that does not place those arranged around it in social relations to one another, and
there is thus no uncommunicative technology or technological practice” (p. 224-225). In our
conversation with Siri, the social relation between human and machine is established, but the
implications are not restricted to a human-machine context. According to Oudshoorn (2003),
“Technologies may play an important role in stabilizing or destabilizing particular conventions
of gender, creating new ones or reinforcing or transforming the existing performances of gender”
(p. 211). Marvin and Oudshoorn are speaking here more generally of who uses technology, what
types of technologies they use, and how technologies figure into gender identities. With Siri, the
power dynamics can extend beyond our understanding of machines onto our understanding of
humans. Siri not only reflects gender stereotypes but has the potential to reinforce them.
Siri also is programmed to win people over with her personality. Researchers have found that
people recognize personality traits programmed into a technology and act toward that machine as
if it were a human with similar characteristics (Lee & Nass 2005; Nass, Moon, Fogg, Reeves, &
Dryer, 1995). Siri is funny, sassy, and helpful. If you ask it to tell you a joke, it will. If you ask
her what she is wearing, a gendered question, she’ll side-step the query with a reply such as “In
the cloud, no one knows what you’re wearing.” These sassier remarks often allow Siri to deflect
questions intended to test her “realness.” She can answer the question with a tone similar to that
of a human, maintaining her social status, and simultaneously avoid an awkward conversation
about its complicated ontology. Siri’s personality also is not separate from other aspects of her
14
identity. A New York Times headline declares ‘Siri is one funny lady’ (Pogue, 2011), and in
‘Snide, Sassy Siri has Plenty to Say,’ CNN reports that the program’s gender and congenial
interactions bolster her popularity (Gross, 2011). In giving Siri a playful personality, Apple
positions Siri as good-natured just as women should be, based on gender stereotypes.
Working as an assistant, being a female, and having a good-natured personality would normally
be traits associated with humans. In Siri, these life-like characteristics establish and maintain her
social nature. Virtual social agents, such as Siri, are what Turkle (1984) describes as evocative
objects, things that do not fit neatly into ontological categories. Computers and AI blur the line
between human and machine. According to Turkle (1984), the computer’s evocative nature
hinges “on the fact that people tend to perceive ‘a machine that thinks’ as a ‘machine who
thinks’” (p. 25). As Bollmer and Rodley (Bollmer and Rodley chapter) argue in this collection,
socialbots force a renegotiation of sociality. In my conversations with people regarding Siri,
people describe dual and dueling conceptualizations of Siri, who and that can be thought of as
simultaneously possessing machine AND human characteristics (Guzman, 2015). This often
plays out in the pronouns referring to humans – who, she, her – and to things – what, it, that –
people use for VSAs. Although some people stick with either human or machine pronouns, other
people intermingle the two (Guzman, 2015). This chapter has purposely switched pronouns
based on which aspect of Siri is being discussed to underscore Siri’s bifurcated nature. It is a
machine with a human backstory, and she is a social entity with a technological ancestry.
That technological ancestry also is evocative in that it includes both real and fictional AI entities.
Part of Siri’s humor is the knowledge Siri possesses regarding other AI and, to a degree, her
15
“self-awareness” of “who” she is in relation to other technologies. When asked “Open the pod
bay doors, HAL,” a reference to a specific scene in 2001: A Space Odyssey in which HAL defies
a human, Siri may respond, “We intelligent agents will never live that down, apparently.” Siri’s
reply positions it as in the same general class of machines as HAL but, at the same time,
differentiates Siri from HAL. If asked again to “open the pod bay doors,” Siri may reply, “Ok,
but wipe your feet first.” Siri’s agreement to open the fictional doors demonstrates that Siri obeys
humans, unlike HAL. (Users also may recognize her request that they wipe their feet as a
stereotypically female request.) Siri’s jokes about HAL situate the two programs as the same
type of thing, but they are not the same type of individual thing.
Siri may not hang out with HAL, but she does have a BFF in ELIZA. ELIZA was an AI program
created in the 1960s that could “converse in English” instead of programming language via text-
based exchanges (Weizenbaum, 1976). Designed to function as a psychotherapist, ELIZA is the
technological predecessor to chatterbots and agents including Siri and socialbots. When asked
about “Eliza,” Siri registers that the user is referring to the ELIZA. Siri’s replies create a
connection between her and ELIZA. One response positions ELIZA and Siri as friends: “ELIZA
is my good friend. She was a brilliant psychiatrist, but she’s retired now.” Siri also alludes to
ELIZA as her technological progenitor including “Do you know Eliza, she was my first
teacher?” Although HAL was fictional and ELIZA was real, Siri’s statements position all of
these programs as part of the same technological class. In contrasting itself with HAL and
comparing itself to ELIZA, Siri also stakes out its position relative to humans: Siri is a helper of
humans like ELIZA, not a menace, like HAL.
16
The connection Apple draws between ELIZA and Siri can be viewed as part of the company’s
efforts to promote Siri as a revolutionary technological innovation. Apple touts Siri as a dream
realized. Apple (2011b) states during Siri’s launch: “For decades technologists have teased us
with this dream that you are going to be able to talk to technology and it is going to be able to do
things for us.” Apple is talking about a technological dream; one extending from ELIZA. But
many people do not know about ELIZA, ALICE or the countless other technologies that are part
of Siri’s technological ancestry. In programming Siri with the ability to speak about numerous
fictional AI entities, Apple anticipated that the general public would be familiar with HAL’s
branch of the family tree. Some journalists reporting on Siri’s launch relied on pop culture
representations of AI to explain how Siri functioned and the magnitude of technology’s
innovation. News reports referred to Siri as “the stuff of science fiction” (Gross, 2011) and a
“sci-fi dream realized” (Milian, 2011). Siri also is portrayed as an “amazing technology” that
feels “like magic” (Pogue, 2011). Overall, Siri is constructed as a social machine with desirable
human-like qualities and a prestigious technological lineage. If the early hype surrounding Siri is
to be believed, Siri is part of the science-fiction future made present. In this real-life story,
humans are not controlled by machines; rather, we have extended our control over machines,
both Siri and our iPhone, and, as a result, also further our own control over our lives.
“In the cloud, no one knows what you’re wearing”
The reason that Siri feels “like magic” is not constrained to its framing as a science-fiction
future. Chun (2011) argues that by its very nature, software is difficult to understand because it is
ephemeral. We do not see how software works, and instead we rely on the metaphor of software
to construct our understanding of computation, according to Chun. Siri and all software are a
type of magic in that they provide us with a means to transform the ephemeral into the physical.
17
Chun explains that code “is a medium in the full sense of the word. As a medium, it channels the
ghost that we imagine runs the machine—that we see as we don’t see—when we gaze at our
screen’s ghostly images.” GUIs provide a visual representation of that ghost, i.e. icons, while Siri
functions primarily as a vocal interface. Siri is a voice in and of the machine, a voice that allows
us to interact with the machine in our hand, the iPhone, and other machines, servers (Guzman,
2015). She is simultaneously an interlocutor, and it is also a medium.
As a vocal interface, Siri leaves more to the imagination than a GUI. Chun (2011) argues that
GUIs allow us to see a particular representation of software. With Siri, we do not see. That is
why someone would, jokingly, ask her what she is wearing. We do not know, and cannot know,
what she is wearing, not because she is in the cloud, another metaphor, but because she does not
physically exist. Yet, we have the feeling that she does exist – we can, after all, hear her. And
something is finding us the nearest coffee shop. The human and mechanical characteristics that
we discussed in the last section provide us with more than instructions on how to use Siri; they
put a metaphorical face on the software. Siri’s face is analogous to that of Goffman’s (1967)
conception of face – a particular performance of Self. Siri does not have a Self, but she has been
given a social form, as if she had a Self. Chun argues that “as our interfaces become more
‘transparent’ and visual, our machines also become more dense and obscure” (2011, p. 177).
Similar to a person who puts on a certain face for a particular social setting (Goffman, 1967), Siri
is programmed with a social face that draws our attention toward it and away from other aspects
of the application. Here, I focus on how Siri’s public face, that of an interlocutor, obscures the
other communication role Siri performs, that of medium, and the consequences of both
communicative roles played by Siri.
18
A fundamental communication element is “who” is involved in the interaction. A conversation
with Siri is patterned after dyadic interpersonal communication: When someone is talking to Siri,
the user and Siri appear to be the only parties exchanging messages. The interaction between
user and Siri is designed to mimic an employer-to-employee relationship, or master-to-servant,
with Siri working for and controlled by the user. When speaking with Siri, we are presented with
a communication setting in which we are interacting one-on-one with an entity that we control. If
we switch our focus from Siri the interlocutor to Siri the medium, we “see” Siri take a command
and execute it on the iPhone. Siri speaks to us as an interlocutor and works for us as a medium.
However, despite all of her human qualities, Siri is not an independent entity as another human
would be. Siri is not an entity at all. Siri is a piece of software that takes in and exchanges
information with human users, with other machines, and ultimately with Apple and every other
company or organization that is on the other end of these computational processes. Siri is a
medium between user and the iPhone and between user and Apple; although, Siri does not reveal
that she also is talking to Apple.
There is no way that Apple or any other creator of advanced natural language processing AI
could program every permutation of how people speak into an agent a priori. Nor could they
predict every request. For AI to function, it needs data. Otherwise Siri could not “learn.” Apple
processes and stores information from Siri users for two years (McMillian, 2013). This allows
the program to respond and adapt to users as a whole and individually. It also provides Apple
with data on every aspect of how people use Siri and their iPhones. The storage of user data to
the benefit of Apple prompted the ACLU to argue in a blog post that Siri “isn’t just working for
us, it’s working full-time for Apple too” (Ozer, 2012). However, there is a third worker that even
19
the ACLU overlooks – the Siri user. By interacting with Siri, which is supposed to be the entity
assisting the consumer, the user is performing labor for Apple. We are providing the information
Apple needs to improve Siri via Siri. With millions of Apple mobile devices sold worldwide,
user interaction provides the company with information on a scale that Apple could never
produce on its own. When we look past the face Siri is designed to present to users, the
relationship between Siri and users is not what it appears to be, and Siri is not what she pretends
to be. Our interaction with Siri is more complex than a dyadic arrangement of human to machine,
boss to worker, and involves more parties than just user and Siri.
In using Siri to better control our phone, we give up control of our information, but details of
iPhone use may not be all that we cede to the machine (and technology companies): Software
increasingly is helping us make decisions or making decisions for us (Verbeek, 2009). The field
of captology has emerged within the last few decades with a focus on the design and
implementation of persuasive technologies (Fogg, 2002). These technologies often engage in
“nudging” (Thaler & Sunstein, 2009) users toward a decision or behavior adoption through
subtle communication that obscures the machine’s active role in the process (Fogg, 2002;
Verbeek, 2009). Siri’s design gives it the appearance of being separate from the user but under
the user’s control (a picture we now know is not entirely true). The question is whether Siri
actively influences users’ decision making.
Part of the answer can be found in Siri’s technological lineage. Apple did not develop Siri, a fact
that Apple does not promote and journalists have not widely reported. The program’s technology
can be traced to the U.S. government’s largest AI initiative, the Perceptive Assistant that Learns,
20
or PAL, program, that developed AI software for the military (SRI International, n.d.). DARPA
selected SRI International in 2003 to lead the $150 million project (Artificial Intelligence Center,
n.d.). Among the project objectives was the goal that “the software, which will learn by
interacting with and being advised by its users, will handle a broad range of interrelated decision-
making tasks. . . It will have the capability to engage in and lead routine tasks…” (Artificial
Intelligence Center, n.d., par. 3-4). SRI later founded Siri Inc. to develop commercial technology
from PAL, and in 2010 the company introduced Siri, a voice-activated personal assistant, to the
Apple App Store. A SRI press release (2010) for the pre-Apple version of Siri states: “Many
tasks, like making restaurant reservations, typically involve multiple steps – including searching
near a certain location, browsing reviews, considering available times, . . . “Siri handles them all,
without missing a beat.” The technology upon which Apple’s version of the program is based
was developed to “think” on its own and to inform human decisions. Like its SRI predecessor,
the current incarnation of Siri sorts through and prioritizes information to make
recommendations. Because of Siri’s opacity, we do not know to what extent Apple and other
companies have nudged us in a direction that ultimately benefits them.
Siri’s potential to affect the outcome of mostly mundane tasks, such as choosing a restaurant,
may seem inconsequential, but the program’s increased role in our decision making and our lives
is not trivial. When we rely on Siri to remind us to be somewhere or help us find a business, we
are handing over control of part of our lives to the machine. We may tell Siri what to do when
we schedule an event, but then Siri reminds us what we need to do. Communication with and
about the program positions Siri and users as independent entities. We now know that this is not
true of Siri, and, although harder to admit, not true of ourselves. We may think we are
21
independent from Siri and our iPhone, but, as anyone who has ever lost their phone for more
than a minute can attest, we rely on the technologies permeating our lives. This reliance is built
through more than the messages Siri sends, or what Carey (1989) calls the transmission view of
communication. It is constructed through our communion with machines, the ritual aspects of
communication (Carey, 1989) we repeat over and over.
We are, as Sengers (2000) argues, “no longer, like our ancestors, simply supplied by machines;
we live in and through them” (p. 5). With each advance in technology, we continue to delegate
more of what was once within the purview of humans to machines. The technologies we build
and the way we integrate them into our lives have long played an integral role in the evolution of
individuals and society (e.g. Innis, 2007; McLuhan, 1994) and in the way that we see ourselves
(e.g. Turkle, 1984). As we are experiencing our latest technological and cultural shift to a robotic
society, we, once again, are presented with new ways to relate to our world (Turkle, 2007). Siri is
part of this transition.
Understanding human-machine communication, Understanding human-machine culture
On the movie screen, AI has been portrayed as a helper to humans only when the sentience and
power of the machine is kept in check. Given too much power, the machines will slip from our
control. While these tropes make for great movies, they do not accurately reflect AI within a
quotidian context. They also are not a useful means for critiquing technology, as Carey and
Quirk (1989) argue: “Electronics is neither the arrival of the apocalypse nor the dispensation of
grace: Technology is technology” (p. 140). If we separate Siri from the myth of AI, we see that
Siri is neither dream nor danger. The reality of Siri is much more complex.
22
This is not to say that Siri is neutral. Technologies function, Carey (1990) argues, as “homunculi:
concrete embodiments of human purposes, social relations, and forms of organization” (p. 247).
They are sites of power (Carey, 1990; Chun, 2011), and must be understood as such. If there is
any “danger” associated with Siri, vocal social agents, socialbots, or any other technology, it is in
accepting these technologies and their representation of the world through the face they and
others communicate to us without engaging with their social and technological complexity.
According to Carey (1990) technologies “coerce the world into working in terms of the
representation” (p. 245). We, therefore, need to continuously interrogate our digital assistants as
to what they represent and what we bring into being through their use in an increasingly
technology-saturated culture. This holds true for not only how we see ourselves in relation to
machines but also in how we come to see other people through the machine. This process of
critique is challenging. Software is not easily grasped to begin with (Chun, 2011), and
technology is designed to be erased from the user’s view (Suchman, 2009). With Siri we are
confronted with making sense of a technology that has a face that we cannot see, a technological
lineage that is real and fictional.
How then can we begin to see these technologies and weigh implications for our lives? A key
aspect of our relationship with technology that has not been adequately addressed by scholars is
our communication with machines (Gunkel, 2012; Guzman, 2015; Jones, 2014). We daily
communicate with devices to the point that this process goes unrecognized (Jones, 2014). Our
machines and our communication with them are transparent to us. One way we can attempt to
make our technologies visible, to submit them to critique is to approach them through the
mundane that enables their existence – communication. That is what I have tried to do in this
23
chapter by systematically tracing each element of communication between Siri and humans. In
doing so, I have mapped out a course for understanding our technologies and our relationship to
them. This approach is not limited to Siri and vocal social agents. Because socialbots and other
digital entities also seek to enter our social world through communication, we can better
understand them by focusing on the process of human-machine communication as well.
24
References
Apple. (2011a). Apple launches iPhone 4S, iOS 5 & iCloud. Retrieved from
https://www.apple.com/pr/library/2011/10/04Apple-Launches-iPhone-4S-iOS-5-
iCloud.html.
Apple. (2011b). Special event [Public relations video]. Retrieved via iTunes.
Artificial Intelligence Center. (n.d.). Cognitive Assistant that Learns and Organizes. Retrieved
from http://www.ai.sri.com/project/CALO.
Balakrishnan, K., & Honavar, V. (2001). Evolutionary and neural synthesis of intelligent agents.
In M. Patel, V. Honavar, & K. Balakrishnan (Eds.), Advances in the evolutionary
synthesis of intelligent agents (pp. 1–27). Retrieved from
http://18.7.25.65/sites/default/files/titles/content/9780262162012_sch_0001.pdf
Berg, A.J., & Lie, M. (1995). Feminism and constructivism: Do artifacts have gender? Science,
Technology, & Human Values, 20, 332-351.
Blumer, H. (1969). Symbolic interactionism: Perspective and method. Englewood Cliffs, NJ:
Prentice-Hall.
Bollmer, G., & Rodley, C. Speculations on the sociality of socialbots. In R. W. Gehl & M.
Bakardjieva (Eds.), Socialbots and Their Friends: Digital Media and the Automation of
Sociality. Routledge.
Both, G. (2014). Multidimensional gendering processes at the human-computer-interface: The
case of Siri (pp. 107-112). In N. Marsden & U. Kempf’s Gender-useIT: HCI, usability
und UX unter gendergesichtspunkten. Berlin, Germany: DeGruyter.
Burgoon, J. K., & Hale, J. L. (1984). The fundamental topoi of relational communication.
Communication Monographs, 51, 193–214.
25
Bush, V. (July, 1945). As we may think. The Atlantic Monthly.
Carey, J. W. (1989). Communication as culture: Essays on media and society. New York, NY:
Routledge.
Carey, J. W. (1990). Technology as a totem for culture: On Americans’ use of high technology
as a model for social order. American Journalism, 7(4), 242–251.
Carey, J. W., & Quirk, J. J. (1989). The mythos of the electronic revolution. In J. W. Carey,
Communication as culture: Essays on media and society (pp. 113–141). New York, NY:
Routledge.
Castells, M. (2013). Communication power. Oxford, UK: Oxford University Press.
Chun, W. H. K. (2011). Software studies: Programmed visions: Software and memory.
Cambridge, MA: MIT Press.
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2014). The rise of social bots.
Cornell University’s arXiv. Retrieved from http://arxiv.org/abs/1407.5225
Flanagan, J.L. (1976). Computers that talk and listen: Man-machine communication by voice.
Proceedings of the IEEE, 64, 405-418.
Fogg, B. J. (2002). Persuasive technology. Morgan Kaufmann.
Gandy, R. (1999). Human versus mechanical intelligence. In P. Millican, & A. Clark, (Eds.),
Machines and thought: The legacy of Alan Turing (pp. 125-136). New York, NY: Oxford
University Press.
Gehl, R. W. (2014). Reverse engineering social media. Philadelphia, PA: Temple University.
Goffman, E. (1967). On face-work: An analysis of ritual elements in social interaction. In
Interaction Ritual: Essays on Face-to-Face Behavior. Garden City, NY: Anchor Books.
Goffman, E. (1959). The presentation of self in everyday life. New York, NY: Anchor Books.
26
Griggs, B. (2011, October 18). Why computer voices are mostly female. CNN.com. Retrieved
from LexisNexis Academic.
Gross, D. (2011, October 4). Apple introduces Siri, Web freaks out. CNN.com. Retrieved
from LexisNexis Academic.
Gunkel, D. J. (2012). Communication and artificial intelligence: Opportunities and
challenges for the 21st Century. communication +1, 1. Retrieved from
http://scholarworks.umass.edu/cpo/vol1/iss1/1
Guzman, A. L. (2015). Imagining the voice in the machine (Doctoral dissertation). University of
Illinois at Chicago.
Haigh, T. (2011). Technology’s other storytellers: Science fiction as history of technology. In D.
L. Ferro & E. G. Swedin (Eds.), Science fiction and computing Essays on interlinked
domains (pp. 13–37). Jefferson, NC: McFarland & Company. Retrieved from
http://site.ebrary.com/lib/alltitles/docDetail.action?docID=10501985
Huyssen, A. (1981). The vamp and the machine: Technology and sexuality in Fritz Lang’s
Metropolis. New German Critique, (24/25), 221–237. http://doi.org/10.2307/488052
Hwang, T., Pearce, I., & Nanis, M. (2012). Socialbots: Voices from the fronts. Interactions,
19(2), 38–45.
Innis, H. A. (2007). Empire and communications. Lanham, Md.
Jones, S. (2014). People, things, memory and human-machine communication. International
Journal of Media & Cultural Politics, 10(3), 245–258.
Lee, K. M., & Nass, C. (2005). Social-psychological origins of feelings of presence: Creating
social presence with machine-generated voices. Media Psychology, 7(1), 31–45.
27
Marvin, C. (1990). Reconsidering James Carey: How many rituals does it take to make an
artifact? American Journalism, 7, 216–226.
McLuhan, M. (1994). Understanding media: The extensions of man. Cambridge, MA:
MIT Press.
McMillan, R. (2013, April 19). Apple finally reveals how long Siri keeps your data.
Wired. Retrieved from http://www.wired.com/wiredenterprise/2013/04/siri-two-
years/.
Milian, M. (2011, October 4). Apple’s Siri voice based on extensive research. CNN.com.
Retrieved via LexisNexis.
Nass, C., & Brave, S. (2005). Wired for Speech: How Voice Activates and Advances the Human-
Computer Relationship. Cambridge, Mass: MIT Press.
Nass, C., Moon, Y., Fogg, B. J., Reeves, B., & Dryer, D. C. (1995). Can Computer Personalities
be Human Personalities? International Journal of Human-Computer Studies, 43, 223–
239.
Nass, C., Moon, Y., & Green, N. (1997). Are Machines Gender Neutral? Gender-Stereotypic
Responses to Computers With Voices. Journal of Applied Social Psychology, 27(10),
864–876.
Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. Conference Companion
on Human Factors in Computing Systems, 72–78. http://doi.org/10.1145/259963.260288
Noble, D. F. (2011). Forces of production: A social history of industrial automation. New
Brunswick, N.J.: Transaction Publishers.
28
Oudshoorn, N. (2003). Clinical trials as a cultural niche in which to configure the gender
identities of users: The case of male contraceptive development. In N. Oudshoorn & T.
Pinch (Eds), How users matter: The co-construction of users and technologies (pp. 209-
228). Cambridge, MA: The MIT Press.
Ozer, N. (2012, March 10). Note to self: Siri not just working for me, working full-time for
Apple too. Retrieved from https://www.aclu.org/blog/free-speech-technology-and-
liberty/note-self-siri-not-just-working-me-working-full-time-apple.
Pogue, D. (2011, October 12). New iPhone conceals sheer magic. The New York Times.
Retrieved via LexisNexis.
Potter, N. (2011, October 25). Why are computer voices female? Ask Siri. ABCNews online.
Retrieved from http://abcnews.go.com/.
Ravitz, J. (2013). “I”m the original voice of Siri’. Retrieved April 3, 2015, from
http://www.cnn.com/2013/10/04/tech/mobile/bennett-siri-iphone-voice/index.html
Reeves, B., & Nass, C. I. (1998). The media equation. Standford, CA: CSLI Publications
Riskin, J. (Ed.). (2007). Genesis redux: Essays in the history and philosophy of artificial life.
Chicago: University of Chicago Press.
Rothschild, J. (1983). Foreword. In J. Rothschild (Ed). Machina ex dea: Feminist perspectives
on technology (pp. vii-viii). New York, NY: Pergamon Press.
Schelde, P. (1993). Androids, humanoids, and other science fiction monsters: Science and soul in
science fiction films. New York : New York University Press,.
Sengers, P. (2000). Practices for a Machine Culture. Surfaces, 7, 4–58.
29
Skalski, P., & Tamborini, R. (2007). The Role of Social Presence in Interactive Agent-Based
Persuasion. Media Psychology, 10(3), 385–413.
http://doi.org/10.1080/15213260701533102
SRI International. (n.d.). Artificial intelligence: CALO. Retrieved from
http://www.sri.com/work/timeline-innovation/timeline.php?timeline=computing-
digital#!&innovation=artificial-intelligence-calo.
SRI International. (2010). Siri launches virtual personal assistant for iPhone 3GS. (Press
release). Retrieved from https://www.sri.com/newsroom/press-releases/siri-launches-
virtual-personal-assistant-iphone-3gs
Suchman, L. A. (2009). Human-machine reconfigurations: Plans and situated actions (2nd ed.).
New York, NY: Cambridge University Press.
Terbough, G. (1965). The automation hysteria. Washington, D.C.: Machinery & Allied Products
Institute and Council for Technological Advancement.
Thaler, R. H., & Sunstein, C. R. (2009). Nudge: Improving decisions about health, wealth, and
happiness. New York: Penguin.
Turing, A. M. (1950). Computing Machinery and Intelligence. In D. C. Ince (Ed.), Mechanical
Intelligence (pp. 133–160). New York, NY: North-Holland.
Turkle, S. (1984). The second self. New York, NY: Simon & Schuster.
Turkle, S. (2007). Authenticity in the age of digital companions. Interaction Studies, 8(3), 501–
517.
van Oost, E. (2003). Materialized gender: How shavers configure the users’ femininity and
masculinity. In N. Oudshoorn & T. Pinch (Eds)., How users matter: The co-construction
of users and technologies (pp. 193-208). Cambridge, MA: The MIT Press.
30
Verbeek, P. P. (2009). Ambient intelligence and persuasive technology: The blurring boundaries
between human and technology. NanoEthics, 3(3), 231–242.
http://doi.org/10.1007/s11569-009-0077-8
Warrick, P. S. (1980). The cybernetic imagination in science fiction. Cambridge, Mass:
MIT Press.
Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation.
New York, NY: W.H. Freeman & Co.
West, C., & Zimmerman, D. H. (1987). Doing gender. Gender & Society, 1(2), 125–151.
Zhao, S. (2006). Humanoid social robots as a medium of communication. New Media & Society,
8(3), 401–419. http://doi.org/10.1177/1461444806061951
31
Footnotes
1 Scholars across multiple disciplines have used varying terms and approaches to the
study of interactions between humans and machines including “man-machine communication”
(e.g. Flanagan, 1976) and “human-machine communication” (Suchman, 2009). Within
communication, the work of Clifford Nass and others has fallen under Human-Computer
Interaction (HCI). More recently, some communication scholars, this author included, have
started to use HMC as an umbrella term for HCI, HRI (human-robot interaction), and HAI
(human-agent interaction) and to develop HMC frameworks grounded in communication theory.
