Available via license: CC BY 4.0
Content may be subject to copyright.
ORIGINAL RESEARCH
published: 16 April 2021
doi: 10.3389/fcomp.2021.531713
Frontiers in Computer Science | www.frontiersin.org 1April 2021 | Volume 3 | Article 531713
Edited by:
Z. O. Toups,
New Mexico State University,
United States
Reviewed by:
Jaime Banks,
Texas Tech University, United States
Samuel Marcos-Pablos,
University of Salamanca, Spain
*Correspondence:
Scott Bateman
scottb@unb.ca
Specialty section:
This article was submitted to
Human-Media Interaction,
a section of the journal
Frontiers in Computer Science
Received: 31 January 2020
Accepted: 25 February 2021
Published: 16 April 2021
Citation:
Pradantyo R, Birk MV and Bateman S
(2021) How the Visual Design of Video
Game Antagonists Affects Perception
of Morality.
Front. Comput. Sci. 3:531713.
doi: 10.3389/fcomp.2021.531713
How the Visual Design of Video
Game Antagonists Affects
Perception of Morality
Reyhan Pradantyo 1, Max V. Birk 2and Scott Bateman 1
*
1Human-Computer Interaction Lab, Faculty of Computer Science, University of New Brunswick, Fredericton, NB, Canada,
2Department of Industrial Design, Eindhoven University of Technology, Eindhoven, Netherlands
The visual design of antagonists—typically thought of as “bad guys”—is crucial for game
design. Antagonists are key to providing the backdrop to a game’s setting and motivating
a player’s actions. The visual representation of antagonists is important because it affects
player expectations about the character’s personality and potential actions. Particularly
important is how players perceive an antagonist’s morality. For example, an antagonist
appearing disloyal might foreshadow betrayal; a character who looks cruel suggests that
tough fights are ahead; or, a player might be surprised when a friendly looking character
attacks them. Today, the art of designing character morality is informed by archetypal
elements, existing characters, and the artist’s own background. However, little work
has provided insight into how an antagonist’s appearance can lead players to make
moral judgments. Using Mechanical Turk, we collected participant ratings on a stimulus
image set of 105 antagonists from popular video games. The results of our work provide
insights into how the visual attributes of antagonists can influence judgments of character
morality. Our findings provide a valuable new lens for understanding and deepening
an important aspect of game design. Our results can be used to help ensure that a
particular character design has the best chance to be universally seen as “evil,” or to help
create more complex and conflicted emotional experiences through carefully designed
characters that do not appear to be bad. Our research extends current research practices
that seek to build an understanding of game design and provides exciting new directions
for exploring how design and aesthetic practices can be better studied and supported.
Keywords: video games, morality, visual design, empirical methods, antagonists, bad guys, character design,
visual attributes
INTRODUCTION
Antagonists—who are often thought of as “bad guys”—are a critical part of game design.
Antagonists often drive the story of a game, by acting as a catalyst for conflict, thereby influencing
player choices and providing important challenges (e.g., a “boss fight”; Vorderer et al., 2003; Schell,
2008; Przybylski et al., 2010). Villains that inspire and challenge the player will keep them from
losing interest in the goal (Manninen and Kujanpää, 2007). While character design often involves
creating a backstory and defining behaviors and abilities, one key way that characters are initially
experienced by players is through their visual attributes—what they look like (Bar et al., 2006).
We draw from-visual stereotypes to predict character attributes and behaviors. Our
presumptions are based on common references and traits. First impressions of a new character have
Pradantyo et al. Design of Game Antagonist Morality
been shown to be persistent even if impressions are contradicted
or more nuanced information are revealed (Haake and Gulz,
2008). Visual attributes help game designers to communicate
elements of a game’s theme, story, and challenge, and steer
player behavior (Baranowski et al., 2008; Schell, 2008; Przybylski
et al., 2010; Bakkes et al., 2012; Mohd Tuah et al., 2017).
For example, the slanting eyebrows of the Goombas in Super
Mario Bros. Nintendo 1983 help convey to the player that
they are not happy, and that the player should get out of
the way. Important to understand how people perceive the
intent and actions of characters is morality, which is the “. . .
differentiation of intentions, decisions and actions between those
that are distinguished as proper and those that are improper”
(Long et al., 1987). People use characters’ appearances to help
make judgments about their morality, and morality perception
greatly affect people’s enjoyment of games and other media
(Eden et al., 2015). Thus, game designers and artists try to
match designs with how they want a player to interpret their
characters, whether it is in a congruent way (e.g., a bad guy who
looks evil) or an incongruent way (e.g., a bad guy who looks
innocent, non-threatening and friendly). Providing information
about the perception of higher level personality traits with the
visual attributes of antagonists could be extremely useful for
game designers (McLaughlin, 2012).
Because the visual design of characters is a critical part
of game design, we provide a first study examining how
different visual attributes lead to different moral interpretations
of antagonists. To do this, we first created a stimulus set of
105 images of antagonists that span a wide range of successful
games from the last 5 years. Next, we conducted a survey on
Mechanical Turk (N=283), in two parts. Part 1 solicited
rating of antagonist images using the CMFQ-S (Character
Moral Foundations Questionnaire–Short, a short, validated scale
previously used in the interpretation of character morality).
In part 2, we gathered people’s judgments on the saliency of
visual attributes that featured “prominently” in the design of
antagonists. By assembling our two data sets, we are able to
provide evidence-based insights into many of the important
visual attributes used in the design of video game antagonists and
relate them directly to judgments of morality.
The findings of our work provide valuable new insights
and deepen our understanding of how character design—an
important aspect of game design—is interpreted. In practice, our
results can be used to help ensure that a particular character
design has the best chance to be universally seen as “evil,” or to
help create more complex and conflicted emotional experiences
through carefully designed characters that do not appear to
be bad. More broadly, we believe the methodology that we
have identified can be leveraged, extended and strengthened to
improve game design research, to better characterize current
practices and cultural experience, and to build more precise and
engaging entertainment experiences.
LITERATURE REVIEW
Antagonists in Video Games
The tension between the main character (i.e., protagonist) of a
story and their opponent (i.e., antagonist) is ubiquitous to fiction
and fuel many dramatic situations; e.g., when Darth Vader reveals
himself to be Luke Skywalker’s father. Looking at the common
template of the monomyth or the hero’s journey (Lane, 2017),
the antagonist is the cause of going on a journey, and provides
the reason that challenges need to be faced, while providing
temptations on the way (e.g., Vader to Luke: “Join me and
together we can rule the galaxy as father and son.” The Empire
Strikes Back, Lucas Films 1982). In video games, antagonists
fulfill a similar role—in the Super Mario Bros. series (Nintendo
1985) Bowser keeps Princess Peach hostage so that Mario can set
out to rescue her; in the Sonic the Hedgehog series (Sega 1989)
Dr. Robotnik/Eggman aims to achieve world domination which
Sonic tries to prevent.
Villains are central to every culture, because they provide a
moral compass (Eden et al., 2015)—they show behaviors that are
threatening to society, because they cause others physical harm,
deny the rights and freedom of others, create chaos, would betray
others, or perform actions that are disgusting. As such, villains
are on the opposite line of moral behavior, which helps us relate
to the hero’s efforts and understand their drive.
Visual Attributes of Villains and Archetypes
It has been well-established that there are clear differences
between how heroes and villains are visually represented and that
this affects people’s judgments about these characters (Hoffner
and Cantor, 1991; Eden et al., 2015; Grizzard et al., 2018).
Narratives often use tropes or clichés that the audiences are
familiar with—such as the damsel in distress trope used in
Mario—to set expectations and to make clear what actions will
need to be taken. Similar to narrative tropes, character designers
use visual archetypes (Haake and Gulz, 2008; e.g., the muscle
packed action hero; or the magician in long robes) to provide
visual affordances for players (e.g., recognizing a character that
will likely use brute force vs. magic) to motivate player actions.
For example, when facing a life or death decision we would act
differently toward an immoral character (e.g., someone who puts
themselves at risk to help an injured child in a dire situation vs.
someone who always acts in their own best interests); or, someone
who betrays their team or family, compared to someone who
has displayed moral behavior (e.g., acting with loyalty even while
being tempted toward disloyalty).
Classic villains such as the gangster wearing a fedora, a striped
suit, and two-tone shoes, or the long-nosed witch with a tall
hat, and crooked teeth, are well-known and easily identified.
Literature and drama are the source of villain archetypes
(Fahraeus and Yakali Çamoglu, 2011), but archetypes are present
in games as well, e.g., the mentally unstable villain Joker in
Batman: Arkham Asylum (Rocksteady 2009), or the superior
species like the Sectoids in XCOM 2: Enemy Unknown (Fireaxis
Games 2016). While the presented archetypes could be applied
across genders, age groups, and race, the majority of villains are
male (Ivory, 2006), with an observable uptake in female villains
(Lindner et al., 2019). In the design of villain character designers
show several preferences, such as Classic Villain—TV Tropes
(n.d.):
•Display a common vice: antagonists often represent a sin or
a vice; e.g., wrath, gluttony, pride; e.g., God of War’s Baldur
Frontiers in Computer Science | www.frontiersin.org 2April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
represents wrath through his visual display of anger and rage
(Sony Interactive Entertainment 2018);
•Display a common moral flaw: many villains have at least one
moral flaw; e.g., they are deeply disloyal or careless toward
the wellbeing of others; e.g., the ruthless Handsome Jack in
Borderlands 2 (2K Games, 2012);
•Distinct Color: antagonists are visually distinct through the
use of color (Lundwall, 2017); e.g., antagonists are often
represented with dark color palettes, while protagonists are
bright; e.g., Link’s primary color is green, the color of hope,
wielding the bright Master Sword, while the evil Ganon is black
and red (The Legend of Zelda: Breath of the Wild, Nintendo,
2017); and,
•Distinct Form: antagonists are visually distinct through the
use of shape (Ekström, 2013); e.g., the spiky Sauron (Middle-
earth: Shadow of War, Warner Bros. Interactive Entertainment
2017); or, size, e.g., the oversized Onyxia in World of Warcraft
(Blizzard Entertainment, 2004).
While designers often draw from their experience, mood boards,
and previous characters with similar traits, media psychology
and communication studies provide theoretical frameworks to
characterize the effects and implications of designing for moral
judgments. So how exactly are moral judgments formed?
Appearance and Moral Foundations
As a member of any society, we learn what is right and what is
wrong, and we learn to associate certain forms of appearance with
morally questionable behavior (Klapp, 1954). Motorcycle gangs
like the Hell’s Angles, for example, wear vests with patches—
identifying them as gang members—and are associated with
violence. Or, the slick look of a wall street banker that suggests
a dedication to personal gain, the willingness to put personal gain
before others, and to cause chaos and disorder through immoral
actions like morally questionable stock trades. Media commonly
draws from imagery that is reminiscent of morally corrupt parts
of a historical or current society to make it easy for the audience
to identify the moral stance of a character (Klapper, 1960).
Several theories from different fields [e.g., psychology
(Kohlberg, 1971; Diessner et al., 2008; Doris et al., 2020),
philosophy (Haidt and Joseph, 2008), sociology (Boltanski
and Thévenot, 2000), law (Raz, 1995), and communication
studies (Fiske et al., 2007; Eden et al., 2015)], provide a
nuanced perspectives on how morality might be communicated
through a person’s appearance (Haidt and Joseph, 2008),
for example, present the moral foundation theory (MFT).
Moral foundation theory offers a pluralistic perspective
on moral, suggesting that morality is judged on in five
domains: harm/care, fairness/reciprocity, ingroup/loyalty,
authority/respect, purity/sanctity.
Building on Haidt and Joseph’s theory, Grizzard et al. (2019)
evaluated and extended the character morality questionnaire.
Their questionnaire asks participants to indicate their agreement
to questions such as “This character would physically hurt
another person.”
Linking appearance with emotional responses, people are
capable of making split-second judgments of others (Willis
and Todorov, 2006). It is suggested that both men and
women are influenced by physiognomy in day-to-day life. In
this study, participants rated faces based on attractiveness,
likeability, competence, trustworthiness, and aggressiveness with
insignificantly no difference between being with or without time
constraints. This spontaneous detection skill is suggested to be
essential for survival. These papers have also studied how and
why people perceive stereotypes of good guys and bad guys
(Secord et al., 1953; Bull and Green, 1980; Goldstein et al.,
1984; Yarmey, 1993; Flowe, 2012; Croley et al., 2017), and
discuss moral perceptions based on physiognomy and other
visual attributes. To measure morality, the Character Moral
Foundations Questionnaire (CMFQ) (Eden et al., 2015; Grizzard
et al., 2019) was often used. In all cases, visual attributes are
capable of affecting the peoples’ moral judgments of characters.
Studies have shown that the perceived morality among
heroes and villains (Eden et al., 2015) in media have strong
connections with viewer’s enjoyment (Sanders and Tsay-Vogel,
2016; Eden et al., 2017). A well-known theory in understanding
the ties between media enjoyment and morality is affective
disposition theory (ADT) (Raney, 2006). Affective disposition
theory suggests that viewers interpret characters as liked or
disliked based on how they judge the character’s morality.
The outcome of any event affects the viewer’s enjoyment,
depending on the congruence of the viewer’s expectations: highly
liked characters who experience positive outcomes and less
liked characters who experience negative outcomes increase
viewer enjoyment (Raney, 2004). Consequently, enjoyment
decreases when unexpected events occur, such as when a liked
character experiences a negative outcome or a disliked character
experiences a positive outcome.
Moral Judgments
The interaction between media and entertainment use, media
experience, and moral judgment, has is at the center of ADT
(Zillmann, 1996). Affective disposition theory engages with
how viewers perceive and assess a character based on their
actions and determine if a character is good or bad. From the
viewer perspective defining a character as good or bad creates
tension and suspense, because depending on the characters moral
leaning the audience tries to predict future action and observes
if a character acts according to the ascribed moral category.
The perceived disposition affects the audience’s enjoyment of
a narrative. The game the Last of Us 2, plays with character
expectations. The player starts out playing the character Abby
Anderson. Playing from Abby’s view the player first likes Abby.
Deeper into the narrative Abby commits violent actions against
characters that were established as “good,” which leads to Abby
being depicted as “bad” due to her actions. Abby being depicted
as a “bad” character creates expectations regarding Abby’s future
actions and conflicts for the player when they need to take on
Abby’s role playing now a “bad” character themselves. Applied
to games ADT would suggest that the expectations regarding
the disposition of the character is foundational for the player’s
affective response; e.g., the feeling of disgust or despair when
a “bad” character falls in carnage or kills a good character
or the positive feeling when the hero prevails and experience
Frontiers in Computer Science | www.frontiersin.org 3April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
success (Raney, 2004) offers two complementing amendments
to ADT: (1) the formation of an affective disposition sometimes
precedes the moral evaluation of a character (for evidence see
Grizzard et al., 2018), and (2) the ascribed disposition “good” or
“bad” leads to an interpretation of a character’s actions in line
with expectations. Both concepts are interesting our research,
because (1) suggests that the simple interpretation of a character’s
appearance affects moral decision and (2) that the interpretation
potentially influences how further actions of such a character
are perceived.
The theory has practical value for our research, because it
argues that the dispositions we ascribe to a character may be
relevant for our entertainment experience and might lead to
emotional experiences. Hence, judging a character as “bad” and
differentiate potential future actions based on the character’s
appearance, e.g., a character judged as impure who commits
violent actions, might be exactly the form of tension a game-
designer aims for. The tyrant Pagan Min in Far Cry 4 (Ubisoft),
who is introduced to the player by killing one of his commanders
using a pen, might create an expectation of unpredictable
violence and could facilitate emotional experiences for the player;
e.g., fear of Min’s unpredictable actions.
To advance media theories and ADT—which focused on
short-term affective engagement with media—and provide a
more wholistic perspective on morality and media effects on
society, Tamborini (2011) suggested the Model of Intuitive
Morality and Exemplars (MIME). Model of intuitive morality
and exemplars suggests that strong moral beliefs are uphold
by media selection, i.e., we like content that fits within our
overall moral belief system and is therefore more likely to be
selected, and reinforces our moral belief system. Build on Moral
Foundation Theory (Haidt and Joseph, 2008), MIME follows
a dual-processing logic and suggests that we evaluate events
intuitively (process 1) unless they are not within expectations or
too complex, then we are deliberately rational to comprehend
the given events (process 2). Model of intuitive morality and
exemplars also draws from exemplification theory (Zillmann,
1999), which suggests that recent or frequent events or concrete
and highly emotional exemplars increase moral judgment. From
a game-designers perspective MIME might explain preferences
and playstyle of a player by considering previous media
preference and moral examples within these games. Game
designers can make use of character expectations and effects of
creating unexpected scenarios, e.g., a morally corrupt character
helping a vulnerable protagonist.
Model of intuitive morality and exemplars has been applied
in studies to analyze or discuss the effect of videogames
(Tamborini et al., 2011, 2017; Eden et al., 2014) and game
characters (Joeckel et al., 2012; Tamborini et al., 2013; Boyan
et al., 2015) and provides guidance to understand morality
processes during game play and a framework to understand how
videogames shape audience’s moral intuitions, and subsequently
media interpretation and response. Looking at the morality of
the characters we present, MIME supports that our interpretation
and moral judgment is a result of the media context and its effect
on our moral intuition. The gangster world of Martin Scorsese,
contributed to our interpretation of Italian man in needle striped
suits with a fedora as gangsters, and informed our moral intuition
to interpret video game character in games such as Grant Theft
Auto (Rockstar Games 1997) or Mafia (Illusion Softworks 2002)
that dress the same way as similarly morally corrupt.
Morality in Video Games
In comparison to other media forms, video games put the player
into the driver’s seat, resulting in a context where moral actions
are not just observed, but actively executed (DeVane and Squire,
2008). Video games enable players to explore their moral values
through the protagonist, by making moral decisions of any kind
themselves and act in environments where moral values are
deviate from the values of modern society.
In video games, morality and its different dimensions set
players expectations—for example, the criminal setting of Grand
Theft Auto (Rockstar Games 1997) puts the player in the role
of a criminal in a fictional city. The mechanics and rules of the
game reinforce morally questionable behavior such as beating up
people, stealing cars, or destroying property. However, even in
a criminal world not all moral dimensions are abolished: e.g.,
loyalty toward gang comrades remains relevant. Morality in such
games such has been intensively studied and has triggered heated
public and academic debates about transfer effects of violence
(Ferguson, 2008).
In games where the player has a choice about the moral
compass of their character, we usually find indicators of their
standing in society represented by the people they can talk to
(Mass Effect; BioWare 2007), the availability of dialog (Detroit
Become Human; Quantic Dream 2018), or visual indicators—
the classic game Ultima Online (Origin Systems 1997), for
example, assigned a special name tag to individuals who attacked
other players.
The antagonist and the moral beliefs they project have
implication for the presumptions of the player, and subsequently
their intuition about in-game situations (Joeckel et al., 2012).
For example, when interacting with an antagonist that has not
acted fairly, the player would mistrust their offers. The narrative-
driven zombie game series, The Walking Dead (Telltale Games
2012–2019), frequently presents players with situations where
they need to judge the moral compass of the players around them;
e.g., when offered food from a group that might or might not have
engaged in cannibalism.
From a designer’s perspective, the visual attributes (e.g., an
eye patch or a scar), that inform players about the morality of a
character are important to effectively communicate a character’s
moral standing. This is particularly difficult, considering that
a character’s moral is not just judged on a single axis from
good to bad, but on visual elements that speak to their fairness,
willingness to physically hurt others, their loyalty, how willing
they are to follow rules, and their ability to engage in disgusting
behavior. But how do we approach the complex effects that small
details like the nose on how a character is interpreted?
Quantifying Visual Experiences
Jacobsen (2006) outlines how aesthetics can be captured by
applying scientific method: by manipulating size and shape of
body parts, e.g., waist-to-hip ratios, or evaluating the effect of
Frontiers in Computer Science | www.frontiersin.org 4April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
abstract patterns in comparison to known stimuli. A model to
understand aesthetic experiences has been presented by Leder
et al. (2004). The authors suggest that aesthetic experiences
are context dependent and show that aesthetic experiences
are a complex interaction between cognition, affect, and
perceptual processes—e.g., judging visual complexity, or relating
an experience to prior memories are different cognitive processes.
Researchers and designers have applied several techniques to
understand the interpretation of design. Hassenzahl (2004), for
example, has investigated the consistency of beauty judgments,
and operationalizes beauty. Reinecke et al. (2013) investigated the
appeal of websites, in terms of visual complexity, colorfulness,
and appeal. The created model of visual appeal combined with
basic demographics explained about 50% of resulting appeal
ratings. These are similar to Tuch et al.’s (2012) findings, which
show that prototypes and visual complexity affect the aesthetic
perception of a website, but that the amount of time that a website
is viewed matters.
Research has also demonstrated that the visual presentation
of interactive products affects our judgment and experience of
them (Hassenzahl, 2004). Games, however, more often combine
the interactivity of digital products such as apps and websites,
with the narrative depth of fiction and drama, creating unique
demands on the visual design of video game characters.
Studies Visual Attributes of Characters in
Video Games
In the context of our work, we focus primarily on the visual
attributes of characters. Providing a deeper understanding of how
people interpret game characters is relevant important because
identifying with a representation increases the amount of time
a game is played (Passmore et al., 2018), how deeply players
comprehend information (Kao and Harrell, 2015), and overall
engagement (Reinecke, 2009).
Previous work has established that we interpret characters
values using a number of visual attributes, for example, character
shape (Veronica, 2015), age (Schwind and Henze, 2018),
gender (Schwind and Henze, 2018), and fashion (Klastrup and
Tosca, 2009). Importantly, based on visual attributes we draw
conclusions about characters’ moral beliefs (Happ et al., 2013).
Further, how we see a character affects in-game behavior. We
tend to act in a way that we believe confirms a character’s beliefs.
For example, beating up prostitutes in Grand Theft Auto is not
necessary, but people still do so, because it is in-line with the
moral value system presented in the game (Happ et al., 2013).
There has been some work that has tried to tease apart visual
properties of characters and how they are perceived. Schwind and
Henze (2018) investigated gender and age differences in virtual
faces, finding that in a character designing task that participants
create villain faces as more masculine, unattractive, and with
lower likeability. Villain faces have also been shown to be more
related to features such as looking dead or zombie-like (Schwind
et al., 2015).
To move toward a comprehensive understanding of
the relationship visual attributes and experience, we need
to investigate specific visual elements of character design
systematically and empirically.
STUDY DESIGN
To provide an initial understanding about how the visual
attributes of game antagonists can influence how people
experience them, we carried out two studies. Our studies asked
people to separately rate a stimulus set of 105 antagonist
images, identifying their most salient visual attributes and to
judge the characters’ morality. Our two studies were conducted
using the crowdsourcing platform Amazon Mechanical Turk
(MTurk), using the same set of antagonists’ images, but
differing in the requested assessment of the images. In
Study 1, participants rated the morality of each character
based on their appearance using the five morality dimensions
(Haidt and Joseph, 2008): “harm/care,” “fairness/reciprocity,”
“ingroup/loyalty,” “authority/respect,” and “purity/sanctity.” In
Study 2, participants rated the prominence of character visual
features (e.g., eyebrows, age, dermatological problems), defined
as characteristics that “. . . relative to other characteristics, st and
out and grab attention.” Previous work has investigated the
salience of eyes on fixation times (Birmingham et al., 2009), the
role of eyebrows in face recognition (Sadr et al., 2003), and shown
that we form opinions about a face within 100 ms (Willis and
Todorov, 2006)—while prompting individuals to rate relative
salience is conceptually fuzzy, it enabled us to guide attention
and gauge participants subjective perception of a character. Our
analysis connects these two different rating sets, using regression
analysis and correlational analysis with the goal to build an
understanding of correlations between visual attributes and how
they might affect players’ experiences of characters.
General Procedure
Both studies followed the same general procedure. Participants
were recruited using the crowdsourcing platform Amazon
Mechanical Turk. MTurk is a digital platform that acts as a broker
between requesters (e.g., researchers looking for participants to
rate character images) and workers (e.g., people willing to engage
in a rating task for payment).
Our study procedure was reviewed by the Research Ethics
Board of the University of New Brunswick and is on file as
REB 2019-118. Before being asked to indicate their consent,
participants were informed about the procedure, their payment,
and the approximate time the task will take. To assure quality
data, MTurk participants needed to be US-based and have
successfully completed at least 500 tasks with an approval rating
of at least 90%. Restricting eligibility combined with attentiveness
measures (Study 1) and the screening of completion time reduced
the likelihood of bot produced data in our data set, which is an
increasing issue in crowdsourced research (Ahler et al., 2019).
Upon qualifying and accepting the MTurk task, participants
accessed a website that guided them through the study.
Participants were first presented an informed consent form,
followed by a demographic questionnaire, and then given
instructions on how to complete the rating task. Compensation
was calculated at the rate of $7.50 USD/h, to be just above the
Frontiers in Computer Science | www.frontiersin.org 5April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
national minimum wage in the US. We determined 35 min for
Study 1 ($5 USD) and 70 min for Study 2 ($8 USD). Since rating
105 characters was a relatively long task, participants only needed
to rate five characters to qualify for payment but could opt-in to
rate more.
We conducted two separate studies that solicited moral
judgments (Study 1) and visual attributes (Study 2) separately,
because we anticipated sequence effects from asking moral
judgments and visual attributes together. Further, creating two
tasks greatly simplified each task and reduced the length
of time to perform ratings on any individual character for
MTurk participants.
Selecting and Presenting Antagonists
We selected a total of 105 antagonists using publicly available
game ranking data using a pre-determined procedure. We
determined four main criteria to select our character image
repository. The repository should include images of characters
that (1) are humanoid; (2) represent recent trends in game and
character design; (3) are well-designed; and (4) represent the
main antagonists of the games in which they appear. Our intent
was to focus on carefully considered, well-designed characters
that also represent many of the common visual attributes in their
design. Further, we decided to focus on humanoid characters to
ensure that the visual attributes that we asked about were present
in the character, which assured that our insights are derived from
a source that has found mainstream acceptance, covering a wide
range of antagonists that have a presence in current games.
To identify antagonist characters, we first had to identify
individual games that fit the criteria. To ensure that character
designs were both of high quality and represented recent
practices, we filtered the database of Gamerankings.com—a
website that collects ratings from numerous sources to provide
an average score. We selected games released from 2014 to
2019, with a rating above 80% from at least 20 reviews. These
criteria allowed us to identify a set of games that met the criteria
above, since it captured games, and, therefore, were most likely
to contain characters, that were widely seen as “well-designed.”
Importantly, however, since changes to our criteria or the game
database used could result in a different stimulus set, our stimulus
set is likely not representative of all games. The resulting initial
candidate game list featured 105 games that we filtered further
based on the character specific criteria.
For each qualifying game, we identified the main antagonist
or final boss using Fandom pages as our main source (https://
www.fandom.com). Fandom provides background information
and character images for many recent popular games, and all of
the characters in our image set. After investigating each game
individually, we removed games with non-humanoid antagonists
and games without a clear antagonist (e.g., sports games usually
do not feature an antagonist created by a game designer) from
our initial pool. Our final list of suitable characters featured the
main antagonists from 105 games.
Presenting Antagonists
To standardize our stimuli, we created composite images that
include a body shot of the character and a close-up of their face.
We know from previous work (Schwind et al., 2015; Schwind and
Henze, 2018) that the face plays an important role in judging
characters and needs to be fully visible for accurate judgments
to be made. We removed any background from the images and
placed the character on a plain gray background measuring 800
×570 pixels. See Figure 1.
Our stimuli were then presented using a custom-built web
application1.
Figure 1 shows the presentation screen for Study 1 and Study
2, respectively, which were composed of the following five main
elements. (1) Each screen displayed breadcrumbs to provide
information to participants about their progression through the
study. (2) The number of the current character rated over the
total number of characters (n/105)—the system gave participants
the option to stop the procedure after five images to avoid
an extensive time commitment. (3) Additional instructions—
participants could read instructions about the procedure at
any time. (4) The character image drawn from a pool of 105
antagonists. We pseudo-randomized the presentation of images.
Our image selection was automated to ensure that all images were
presented with similar frequencies. To do this we grouped images
by how often they had been previously rated by participants.
Within the group of images that were rated the fewest times, we
randomly selected five images and presented them in random
order. The same procedure was performed for the next block
of five images, omitting previously presented images out of the
image pool. (5) A 7-point Likert-scale rating system for the
morality scales used in Study 1, and a binary rating system for
the salience of visual attributes in Study 2.
While still images are less rich in information than animated
in-game characters, using still images and rating salient features
in accordance with moral features strikes a balance between
stimuli control, participant burden, and stimuli variance; i.e.,
displaying a large range of stimuli in a short amount of time.
While limited when compared to experience of characters
displayed in videogames, images sufficiently allow participants to
identify visual character features that are perceived as salient.
Participants and Study-Specific Procedure
In Study 1 we assessed perceived character morality, and in Study
2 we collected binary ratings of the salience of visual attributes.
Study 1: Character Morality Ratings
For study 1, we recruited 99 participants. Four participants
were removed from the data set, because they provided more
than 25 ratings with a maximum variance ≤1, indicating a
response pattern that was inattentive. Participants rated up to
21 sets of five images each. The first five images included
additional demographic questions and were compensated by $1.
The remaining 20 sets were compensated with 20 cents each, for
a maximum of $5. In total, we obtained 5,963 ratings from 95
participants. Images received a minimum of 54 and a maximum
of 60 ratings, mode =57. For demographics, see Table 1.
1Our system was built in Python using the BOFS system (Johanson, 2019).
Frontiers in Computer Science | www.frontiersin.org 6April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
FIGURE 1 | Example character rating questions from character morality task interface used in Study 1 and the binary rating of salient visual attributes used in Study 2.
Character images were adapted from fandom.com under the Creative Commons CC BY-SA license.
TABLE 1 | Demographics for Study 1 and Study 2.
Study 1 Study 2
Variables N % (n/N) M SD N % (n/N) M SD
Age 95 36.08 12.00 188 – 38.82 11.95
Gender 95 188 –
Men 60% (57/95) 56.9% (107/188)
Women 39% (37/95) 42% (79/188)
Non-binary 0% (0/95) 0.5% (1/188)
Prefer not to answer 1% (1/95) 0.5% (1/188)
Playtime 95 188 –
Everyday 40% (38/95) 36.7% (69/188)
A few times per week 40% (38/95) 39.9% (75/188)
A few times per month 18.9% (18/95) 6.4% (12/188)
A few times per year 1.1% (1/95) 13.3% (25/188)
Not at all 0% (0/95) 3.7% (7/188)
Ethnicity 95 188 –
Asian 9.5% (9/95) 7.4% (14/188)
Black/African American 7.4% (7/95) 7.4% (14/188)
Hispanic/Latino 6.3% (6/95) 6.4% (12/188)
White 72% (69/95) 73.9% (139/188)
Two or more categories 0% (0/95) 3.7 (7/188)
Platforms 95 188 –
Desktop 86.3% (82/95) 80.3% (151/188)
Console 60% (57/95) 7.6% (127/188)
Mobile 54% (56.8/95) 73.4% (138/188)
Specific Procedure for Study 1: Ratings of Character Morality
To collect data on player’s interpretation of characters, we
used the short form of the Character Moral Foundations
Questionnaire (CMFQ-S) (Grizzard et al., 2019), which is
a validated short-form questionnaire based on Haidt and
Joseph’s (2008) five moral domains. Images in our stimuli
set were rated on a 7-point Likert-scale, one time for each
dimension of the five-dimensional CMFQ-S. The statements
and morality domains were as follows, from the CFMQ-S
(Grizzard et al., 2019):
Frontiers in Computer Science | www.frontiersin.org 7April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
•“Harm/care”: This character would physically hurt
another person.
•“Fairness/reciprocity”: This character would deny others
their rights.
•“Ingroup/loyalty”: This character would betray their group.
•“Authority/respect”: This character would cause chaos
and disorder.
•“Purity/sanctity”: This character would do
something disgusting.
Seven-Point agreement ratings were converted to numbers prior
to analysis where 1 means “Strongly Disagree” with one of the
statements above; 4 is neutral; and, 7 “Strongly Agree.” Following
this a character who scores, say, a 1 for harm/care would be
perceived to behave morally for the particular domain. Whereas,
a character who scores 7 for harm/care would be perceived to
behave strongly in an immoral way for the domain.
To control effects of familiarity, participants were instructed
that their “[. . . ] ratings should be made based on the appearance
of the character only (in other words you should not use
knowledge of the character to make your judgment).” Further,
we asked participants to indicate whether they were familiar with
a particular character, and to rate their overall familiarity with
the character using a 100-point scale, by positioning a visual
slider between “Not familiar at all” and “Very Familiar.” In
total, we collected 5,963 morality ratings from 95 participants.
Images received a minimum of 54 and a maximum of 60 ratings,
mode =57.
Specific Procedure for Study 2: Identification of
Prominent Visual Characteristics
In Study 2, we asked participants to indicate whether an attribute
of an antagonist is salient or not. We decided to ask participants
to rate individual character features instead of listing the most
salient character features, because we were interested in a
comprehensive analysis that allows for the evaluation of non-
obvious visual character features that contribute to the overall
perception of a character such as body alterations or age.
Attributes were derived from previous work on character
attributes (McLaughlin, 2012) and extended by our own
interpretation of relevant visual attributes of antagonists,
resulting in a list of 24 visual attributes (see the full list in Table 3).
A “salient” physical attribute was defined for participants as an
attribute that “. . . relative to other characteristics stands out or
grabs attention.” Participants were prompted to make a binary
decision to the statement “Is the following feature prominent
in the design of this character?” followed by the name of the
attribute (e.g., “eyes”). To stay away from overly scientific jargon,
we used “prominence” instead of “salience” in our instructions to
participants. For the most part these presented visual attributes
simply stated the name of the attribute (e.g., “eyes,” “hair,”
“nose,” “mouth,” etc.); however, some features required further
explanation (i.e., “dermatological problems,” “body weight,”
“build,” “height,” “head size,” “skin exposure,” “age,” “stance,”
“clothing,” “jewelry,” “face cover,” “body alterations”). In these
cases, we provided a short description to provide clarification
[e.g., “Dermatological problems (such as dark circles around
the eyes, wrinkles, facial scars, warts, bulbous nose)”]; a full
list of the visual features and descriptions has been provided in
Supplementary Material.
For each character, participants initially responded to whether
they were familiar with the character, and if so, how familiar (as
in Study 1). Participants responded to all 24 attributes for each
character, attributes were presented in serial, and participants
were required to respond “yes” or “no” for each feature, before
proceeding. To discourage participants from simply responding
without considering each attribute, we imposed a brief 2 s delay
before input would be accepted using the buttons.
For Study 2, we recruited 188 participants, who rated up
to 21 sets of five images each (as previously described). Note
the rating task in Study 2 took longer than in Study 1, hence
more participants were recruited and each conducted fewer
ratings on average. We received a total of 5,560 ratings from
176 participants. Images received a minimum of 48 ratings
and a maximum of 59 ratings, mode =54. See Table 1 for
demographics, and Table 3 for a list of all 24 visual attributes. A
limitation of Study 2 a lack of control for moral foundations, this
is something we believe should be included in future work (we
discuss further in limitations).
Analysis
All analysis was conducted using SPSS 25 (IBM, 2017).
Morality Score
Character Moral Foundations Questionnaire–Short (CMFQ-
S) ratings (from Study 1) were transferred to score data (as
described above), means were calculated by scale and data
is presented in aggregate. The relationship between morality
scores is evaluated using correlations and the average of all
five scales is presented as a single “badness” score (see section
Descriptive Statistics for Morality Ratings). Theoretically the
morality dimensions are distinct, but the underlying assumption
to either acting in line with a moral standard or not, is consistent
across scales, and allows the scales to be combined if statistically
internally consistent, as defined by Cronbach’s alpha.
Salience Ratings
Salience ratings of visual attributes (from Study 2) were
aggregated by calculating the percentage of participant responses
that indicated a feature as being salient over the total number
of responses by image. Ratings were normalized by the total
number of responses to account for differences in the number
of ratings received.
Broadly, we distinguish salience ratings in four blocks: (1)
facial features and skin including features such as nose, mouth,
or dermal problems; (2) body shape such as height or weight; (3)
abstract features that depends on the viewer’s judgment such as
age or attractiveness; (4) accessories such as clothing or jewelery
that could be removed.
Relationship Between Character Morality and Salient
Features
To investigate the relationship between moral judgment and
character features, we used correlations, and hierarchical
Frontiers in Computer Science | www.frontiersin.org 8April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
TABLE 2 | Pearson correlations for the five morality domains.
Domain This character would… 1. 2. 3. 4. 5.
1. Care/harm … physically hurt another person. 1 0.880** 0.751** 0.871** 0.782**
2. Fairness/reciprocity … deny another person their rights. 0.880** 1 0.925** 0.968** 0.918**
3. Ingroup/loyalty … betray his group. 0.751** 0.925** 1 0.926** 0.937**
4. Authority/respect … cause chaos and disorder. 0.871** 0.968** 0.926** 1 0.906**
5. Purity/sanctity … do something disgusting. 0.728** 0.918** 0.937** 0.906** 1
**p<0.01.
TABLE 3 | Summary of hierarchical regression analysis for variables predicting badness (N=105).
Model 1 (Head) Model 2 (+Body) Model 3 (+Judgment) Model 4 (+Accessories)
Variable B SE βB SE βB SE βB SE β
Eye 0.00 0.00 0.06 0.00 0.00 0.00 0.00 0.00 0.07 0.00 0.00 0.02
Eyebrows 0.00 0.00 −0.03 0.00 0.00 −0.05 0.00 0.00 −0.06 0.00 0.00 0.02
Nose 0.01 0.01 0.11 0.01 0.01 0.12 0.01 0.01 0.13 0.01 0.01 0.15
Mouth 0.01 0.01 0.23* 0.01 0.00 0.19* 0.01 0.01 0.18* 0.01 0.00 0.19*
Ears 0.00 0.00 0.07 0.00 0.00 0.02 0.00 0.00 0.00 0.00 0.00 0.05
Skin problems 0.01 0.00 0.37* 0.01 0.00 0.37** 0.02 0.00 0.46** 0.02 0.00 0.43**
Facial hair 0.00 0.00 0.01 0.00 0.00 0.10 0.00 0.00 0.10 0.00 0.00 0.13
Hair −0.01 0.00 −0.14 −0.01 0.00 −0.20* −0.01 0.00 −0.14 0.00 0.00 −0.03
Weight 0.00 0.01 −0.08 0.00 0.01 −0.05 0.00 0.01 0.04
Build 0.01 0.01 0.10 0.01 0.01 0.11 0.00 0.01 0.05
Height 0.01 0.01 0.17 0.01 0.01 0.18 0.01 0.01 0.13
Head-body ratio 0.00 0.01 −0.06 −0.01 0.01 −0.10 −0.01 0.01 −0.11
Stance 0.01 0.00 0.32** 0.01 0.00 0.30** 0.01 0.00 0.22*
Skin color −0.01 0.00 −0.11 −0.01 0.00 −0.09
Masculinity/Femininity 0.00 0.01 −0.01 0.00 0.01 0.05
Attractiveness −0.01 0.00 −0.10 0.00 0.00 −0.07
Skin exposure 0.00 0.01 0.01 0.00 0.01 −0.02
Age −0.01 0.00 −0.20* −0.01 0.00 −0.15
Clothing 0.00 0.00 0.08
Jewelery 0.00 0.00 −0.07
Face cover 0.01 0.00 0.19
Tattoos 0.00 0.01 0.02
Weapon 0.01 0.00 0.21*
Body alterations 0.00 0.00 0.01
R20.390 0.556 0.603 0.669
Ffor change in R27.68** 6.80** 2.06 3.19*
*p<0.05, **p<0.01.
regression analysis to identify our final set of most predictive
character features.
We present further details on the statistical procedures used at
the beginning of each subsection in the Results section.
Results
We present the results of both studies together for simplicity,
and since much of our analysis examines correlation between
the ratings collected in each study. We refer specifically to
morality ratings (gathered in Study 1) and visual attribute
salience (gathered in Study 2) in order to reference the source
of the data.
The number of ratings collected per image varied slightly
between images in both studies; Study 1: min =54, max =60;
and, Study 2: min =52, max =62.
Descriptive Statistics for Morality Ratings
For Study 1, mean and standard deviation for each moral domain
were calculated per image. Recall that ratings were made on
a 7-point Likert-scale ranging from strongly disagree (1) to
Frontiers in Computer Science | www.frontiersin.org 9April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
strongly agree (7) with a “lack of morality statement,” e.g., “This
character would hurt another person.”—high scores suggest
perceived immorality.
In the morality rating task, participants were only familiar
with 16.454%, and of those characters they were familiar with,
they rated their familiarity low (M=68.056, SD =10.113).
For our overall stimulus set, we have the following results
for each moral domain: “Care/harm” (M=5.030, SD =
1.110), “Fairness/reciprocity” (M=4.618, SD =1.042),
“Ingroup/loyalty” (M=4.185, SD =0.830), “Authority/respect”
(M=4.675, SD =1.082), “Purity/sanctity” (M=4.100, SD =
1.021). Surprisingly this might indicate that current (human)
villain designs in video games do not convey that they are clearly
immoral (mean ratings for each or only slightly on the immoral
side of neutral). This is in contrast, perhaps, to previous studies
of animated Disney villains who are unmistakably evil-looking
(Hoerrner, 1996).
To better understand how the different morality domains
might be related, we looked for correlations between the five
items and found high correlations between domains; see Table 2.
After evaluating the reliability of across scales (Cronbach’s-α=
0.972), we calculated a single “morality” score for each character
by taking the mean across of all five dimensions (min =2.20, max
=6.16, mean =4.52, SD =0.969). The distribution of “morality”
is negatively skewed (skewness = −0.377), which is in line with
our expectations, considering that the source of our images are
examples of video game villains.
Character Examples of Moral Dimensions
To explore how individual characters moral dimensions compare
to the data set at large, we calculated categories based on standard
deviations for each domain. We created four categories reflecting
the morality for each domain: moral (<-2 SD), slightly moral
(−1 SD to 0), slightly immoral (0 to +1 SD), and immoral (>
+2SD), we binned morality to identify divergence from the mean
and variation between morality domains; i.e., being corrupted in
one domain, but uncorrupted on all other domains. In Figure 2,
categories were solely created for illustration purposes and are
not used in any further analysis.
To exemplify the presence of different moral characteristics,
we present twelve characters with varying pronunciations in
the five moral domains (see Figure 2). Tsumugi Shirogane
was viewed consistently as being uncorrupt (“Care/harm”:
moral, “Fairness/reciprocity”: moral, “Ingroup/loyalty”: moral,
“Authority/respect”: moral, “Purity/sanctity”: moral). These
categories mean that this character is perceived as a character
that would not physically hurt others, would not deny another
person’s rights, will not betray her group, would cause chaos
and disorder, and is not disgusting. Ryuji Goda on the other
hand, is almost completely the opposite (“Care/harm”: immoral,
“Fairness/reciprocity”: immoral, “Ingroup/loyalty”: slightly
immoral, “Authority/respect”: immoral, “Purity/sanctity”:
immoral); this character is perceived as strongly immoral.
Some characters show interesting patterns where they
score differently on different moral dimensions. Aaron Keener
(“Care/harm”: immoral, “Fairness/reciprocity”: slightly immoral,
“Ingroup/loyalty”: slightly moral, “Authority/respect”: slightly
immoral, “Purity/sanctity”: slightly moral), who is perceived
as careless, slightly denying other their rights, and willing to
cause chaos and disorder. But he is not perceived as disloyal or
disgusting. Yunica and Heiss follow similar patterns. Our results
show how the dimensions of badness can be used to analyze
and compare characters and find character designs that provide
both strong and nuanced perceptions of morality. We leave
further commentary and interpretation on these examples to the
Discussion, after the results regarding visually salient features
have been introduced.
Predicting the Morality Through Aesthetic
Characteristics
To investigate the relationship between aesthetic features and
a character’s perceived morality, we performed hierarchical
regression analysis with our 24 visual attributes grouped into
four blocks: head, body, interpretative characteristics (e.g., age,
masculinity-femininity), and presentative characteristics (e.g.,
clothes, tattoos); see Table 3 for the full list of visual attributes
in each of the blocks. Blocks were entered as predictor variables
of perceived badness. Our results show that head characteristics,
body characteristics, and presentative characteristics have the
most predictive value when predicting badness (p=0.011, R2
=0.669). As displayed in Table 3, the salience of the mouth,
skin problems, the stance, and weapons, are the best predictor
of badness.
Our final model (Model 4), shows that a combination of the
mouth (β=0.19), skin problems (β=0.43), stance (β=0.22),
and a weapon (β=0.21), are the strongest predictors of morality
(R2=0.669). While these visual attributes are the most predictive
for morality, related visual attributes should also be considered
when analyzing characters or planning the visual design of an
immoral character.
The Relationship Between Aesthetic
Elements
We next analyzed our data for trends that demonstrate which
aesthetics elements are perceived as most salient together in
antagonist designs. We showed that the mouth, skin problems,
stance, and weapon, are the most predictive variables for
morality. However, several of the visual attributes in the model
show interdependencies; e.g., when a facemask is present, the
mouth cannot be seen, or stance might be related to the
presence of a weapon. We calculated correlations between visual
attributes to discover attribute that are closely related to the
most predictive visual attributes. Considering that there are
many minor correlations between the salience of different visual
attributes, we only discuss visual attributes that correlate r>0.25
with visual attributes relevant for the prediction of morality. See
Table 4 for the correlation table, and Figure 2 for examples of
characters with different salient visual attributes.
Skin problems are correlated with the salience of mouth (r=
0.58), nose (r=0.41), ears (r=0.28), hair (r=0.28), and tattoos
(r=0.34). Suggesting that skin problems appear in prominent
parts such as the face and are used in combination with other
facial features as we can see in a character like Vitalis.
Frontiers in Computer Science | www.frontiersin.org 10 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
FIGURE 2 | Examples of video game characters and their morality by domain. Expression by domains is color coded from red (immoral) to green (moral). Character
images were adapted from fandom.com under the Creative Commons CC BY SA license.
Frontiers in Computer Science | www.frontiersin.org 11 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
TABLE 4 | Pearson correlation of attributes.
Variable 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24.
1. Eyes 1.00
2. Eyebrows 0.14 1.00
3. Nose 0.10 0.22 1.00
4. Mouth 0.30 0.18 0.30 1.00
5. Ears 0.29 0.24 0.26 0.10 1.00
6. Skin problems 0.17 0.20 0.58 0.41 0.28 1.00
7. Facial hair −0.02 0.37 0.33 0.08 0.05 0.18 1.00
8. Hair 0.27 0.17 −0.01 0.04 0.28 −0.05 0.12 1.00
9. Weight 0.20 0.12 0.13 0.19 −0.02 0.05 0.16 −0.06 1.00
10. Build 0.20 0.15 0.25 0.20 0.13 0.21 0.05 0.09 0.55 1.00
11. Height 0.30 0.15 0.03 0.21 0.20 0.06 −0.01 0.18 0.51 0.54 1.00
12. Head-body ratio 0.45 0.13 0.12 0.29 0.13 0.05 0.09 0.18 0.54 0.39 0.58 1.00
13. Stance 0.24 0.00 −0.06 0.11 0.15 −0.02 −0.19 0.16 0.16 0.33 0.48 0.16 1.00
14. Skin color 0.26 0.10 0.12 0.41 0.11 0.25 −0.02 0.13 0.12 0.34 0.12 0.09 0.13 1.00
15. Masc.-Fem. 0.12 0.21 −0.07 0.06 0.02 0.03 0.05 0.22 0.14 0.45 0.13 0.04 0.20 0.18 1.00
16. Attractiveness 0.24 0.04 0.07 0.24 0.09 0.11 −0.12 0.27 −0.01 0.04 0.14 −0.01 0.17 0.18 0.30 1.00
17. Skin exposure 0.20 −0.10 −0.08 0.02 0.15 −0.01 −0.19 0.12 0.01 0.40 0.13 0.10 0.23 0.23 0.37 0.16 1.00
18. Age 0.16 0.03 0.18 −0.12 0.10 0.25 0.16 0.09 0.07 0.02 0.03 0.08 −0.09 −0.10 0.04 0.01 0.06 1.00
19. Cloth 0.18 0.05 −0.20 0.03 0.01 −0.07 −0.15 0.12 0.12 0.14 0.36 0.04 0.48 −0.05 0.10 0.16 0.09 −0.05 1.00
20. Jewelery 0.31 0.12 −0.15 0.08 −0.01 −0.11 0.00 0.01 0.11 0.14 0.12 0.01 0.23 0.22 0.28 0.28 0.19 −0.01 0.50 1.00
21. Face cover 0.07 –0.33 −0.20 0.03 −0.15 −0.09 –0.25 –0.39 0.02 0.09 0.09 0.10 0.13 0.03 −0.21 −0.14 0.23 −0.17 0.27 0.23 1.00
22. Tattoos 0.20 0.14 0.19 0.22 0.34 0.23 −0.01 0.18 −0.04 0.19 0.29 0.15 0.17 0.27 0.09 0.05 0.27 −0.06 0.05 0.09 0.05 1.00
23. Weapons 0.00 –0.28 −0.08 −0.09 −0.18 −0.07 –0.29 –0.26 −0.12 0.03 0.04 −0.09 0.25 −0.03 −0.03 −0.09 0.01 −0.18 0.02 0.02 0.41 0.06 1.00
24.Body alterations 0.11 −0.20 −0.03 −0.03 0.04 −0.03 −0.20 −0.09 0.11 0.29 0.13 0.03 0.30 0.04 0.01 0.01 0.12 −0.12 0.22 0.13 0.44 0.06 0.35 1.00
p<0.01 is highlighted in bold.
Frontiers in Computer Science | www.frontiersin.org 12 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
Stance correlates with build (r=0.33), height (r=0.48),
clothes (r=0.48), and weapons (r=0.25). Stance highlights
a character’s build and height and makes impressive or distinct
physiques standout or appear intimidating—an approach to
convey morality. Clothing can be used to further highlight certain
aspect of the character’s physique, such as partially revealing skin,
Heihachi’s power pose, for example, is underlined by his clothing
that partially reveals his impressive muscles that suggest that he
is able to inflict physical pain. Characters with weapons are often
presented in stances that relate to the weapon’s fighting style,
e.g., Yunica shows a fencing posture while holding a sword-like
weapon (see Figure 2).
Weapons are also correlated with face coverings (r=0.41)
and body alterations (r=0.35). One class of characters using
weapons are assassins such as Reaper or Aaron Keener, who
cover their faces to avoid recognition. However, face coverings
can also prevent clearly visible facial expressions, which means
that other visual attributes must convey morality; e.g., the stance
and weapons of a character can display aggression and the ability
and willingness to harm others, such as Yunica and Aaron Keener
(see Figure 2).
While there are many visual attributes, representing
archetypes, eliciting a specific perception of a character, and
how we experience characters aesthetically leads to relationships
between attributes that can be further explored to understand
the visual construction of antagonists. In Table 5 we present the
relationships of salience and badness for each visual attribute.
The table contains the regression coefficients for either a linear
or quadratic relationship between salience and badness, and
a sparkline visualizations that displays an overview of the
relationship. The left end of the x-axis for the sparkline is more
badness, the right end is less badness; higher on the y-axis
indicates higher salience for the given level of badness.
DISCUSSION
The results of our analysis provide an exploration and
data-driven insights in players perceive perception of
characters morality.
Our work explores the relationship between salient visual
attributes of villains and their perceived morality. We provide
insights into the relationship between salient visual features,
showing which aspects are combined to define perceived
morality, and which features predict perceived badness of
a character.
In the subsections below, we organize our discussion around
how people perceive game antagonists and how designers might
leverage our results in their design practices. We then discuss
limitations of our current studies and the new directions that our
work makes possible for future work.
The Design and Perception of Game
Antagonists
Overall, the game antagonists in our stimulus set were viewed
as only slightly corrupt, or just slightly more immoral than a
neutral rating. The binning of characters by their overall morality,
illustrates an only slightly negative skew toward immorality, and
in fact roughly 47 of the 105 characters were viewed as being
moral than immoral (see section Character Examples of Moral
Dimensions). While we had some examples where characters
were viewed consistently and strongly as immoral, the tendency
of our sample is only subtly evil characters. We do not believe this
means that our stimulus set is somehow limited or that games
do not represent antagonists who are as “bad” as in other media.
Rather, we believe that this shows that games often provide
more nuanced visuals and storytelling when it comes to their
antagonists, which can be easily seen in some of our examples
(see Figure 2). Our stimulus set is in stark contrast to other image
stimulus sets used for understanding perceptions of character
morality. For example, Disney villains were uniformly viewed
as strongly evil (Hoerrner, 1996). In this example, however,
the approach is to communicate very clearly to the (sometimes
young) audience exactly who the villains are.
While we did not study behavior and acts of evil or immoral
behavior, the fact that game designers often explore less overt
visual representations of “bad” is interesting, because interactivity
provides other means to display evil behavior and the time
spent with a game is significantly longer than watching a movie,
which allows to discover the evil side of a character over time—
similar to TV shows. Of course, and importantly, antagonists in
stories are not always “evil” (Martin Del Campo, 2017). That
being said, we did review the characters in our stimulus set,
and our interpretation of the back stories of almost all, if not
all, characters suggested that these characters did indeed take
actions that harmed others, were disgusting, were betrayals to
their group, etc.; i.e., they were immoral in action, even if their
visual design did not suggest it. The fact that the visuals of
characters do not always portray outward and strong aggression,
for example, reflects the range of ways that game designers tell
stories and they support those stories visually through their
character designs. Indeed, it is a surprise in the story of the
game Danganropa that a particularly friendly looking character
(Tsumugi Shirogane) is indeed the antagonist. In contrast, other
stories might present characters who have seemed very immoral,
in both appearance and action, but might still perform good acts.
For example, disillusioned with the evil covenant, The Arbiter (a
grotesque alien) switches sides joining forces with Master Chief,
the main protagonist in Halo 2 (Bungie, 2004).
Our analysis of character ratings in each of the five morality
domains are highly correlated with one another. This suggests
that when we judge a character as morally corrupt the distinction
between moral dimensions is often unclear—evil is evil. This is,
however, out of line with previous work in assessing idealized
protagonists (and not game protagonists) using the CFMQ-
S instrument (Grizzard et al., 2019), where the domains were
not strongly correlated. The strong correlations could mean
that game designers more uniformly represent all domains of
morality when creating antagonists. We discuss how this opens
up possibilities for designers below.
The Constructions of Villainous
Stereotypes
Tamborini’s model of intuitive morality and exemplars (MIME)
(Tamborini, 2011) provides a short and long-term perspective
on morality and adds intuitive and emotional aspects to the
Frontiers in Computer Science | www.frontiersin.org 13 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
TABLE 5 | Non-standardized regression coefficients for individual characteristics and badness, and sparkline visualizations for the highest significant order effect.
Characteristic Linear Quadratic Unstandardized coefficients Sparkline
p p Intercept a b
Eyes 0.07 0.026* 110.679 −34.64 4.485
Eye brows 0.236 0.147 15.926 2.705 –
Nose <0.001** <0.001** 20.162 −2.588 1.257
Mouth <0.001** <0.001** 92.957 −39.649 5.515
Ears 0.045* 0.123 −0.998 4.072 –
Dermatological problems <0.001** <0.001** 14.398 −10.97 3.16
Facial hair 0.303 0.088 13.037 4.08 –
Hair 0.277 0.483 76.133 −3.412 –
Weight 0.078 0.015* 73.933 −37.654 4.024
Build <0.001** <0.001** 63.921 −28.102 4.031
Height 0.001** 0.003** 15.437 −3.976 1.037
Head–body ratio 0.139 0.005** 89.735 −39.588 4.868
Skin color 0.086 0.012** 89.278 −33.819 4.262
Masc.–Fem. 0.639 0.891 39.602 0.768 –
Attractiveness 0.821 0.577 38.511 0.433 –
Skin exposure 0.624 0.117 6.542 0.758 –
Age 0.345 0.584 33.552 −1.75 –
Stance <0.001** 0.001** 11.13 10.072 −0.24
Clothes 0.057 0.156 57.771 4.335 –
Jewelery 0.839 0.979 26.187 0.468 –
Face cover 0.016* 0.008** 75.715 −35.959 4.864
Tattoo 0.005** 0.009** 19.196 −11.624 1.82
Weapons 0.003** 0.012** −33.225 15.743 −0.71
Body alterations 0.015* 0.01** 33.329 −16.569 2.288
The left end of the x-axis for each sparkline is more badness, the right end is less badness; higher on the y-axis indicates higher salience for the given level of badness.
*p<0.05, **p<0.01.
processing of character judgments. When applying MIME to
interpret our stimulus set it is important to keep in mind
that, following exemplification theory (Zillmann, 1999) frequent
exposure to moral examples increase the effect of media exposure,
e.g., the frequently displayed character with baggy pants, muscle
shirt, and bandana who kills, robs, and sells drugs, has created the
powerful iconic image of the ghetto gangster.
Considering the most predictive characteristics in our
stimulus set (i.e., weapon, dermatological problems, stance, and
the character’s mouth), we can consider how these characteristics
contribute to villainous stereotypes. While the relationship
between carrying a weapon and the stance of a character
can be directly linked to “badness” through social norms—a
weapon suggests hostility, and a powerful or combative stance
demonstrates aggressiveness the role of dermatological problems
and the mouth, are unexpected and culturally insightful. The
mouth plays an important role in communicating emotions or
intent in western cultures (Yuki et al., 2007), e.g., signaling
Frontiers in Computer Science | www.frontiersin.org 14 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
approach-ability by smiling, or emotions such as anger, fear, or
disgust. For character design, the mouth opens up opportunities
to communicate the internal state a character, e.g., the evil grin
of the Disney character Ursula. Model of intuitive morality
and exemplars would suggest that when dermatological issues
are used to depict villains, an automatic negative response to
other characters with skin problems would result. This means
that villains with acne, burn scars, etc., might lead to other
uses of those same visual attributes, leading to coherence
between domain and exemplar salience. The consequence of
this can result in real consequences for individuals in their
day-to-day lives with skin problems, who could be more likely
perceived as villains (Funk and Todorov, 2013). Such stereotypes
would need to be counteracted by creating content where
domain and exemplar salience conflict; e.g., characters with skin
problems that are inherently good. An example character that
already shows a manifestation of such conflict is Marvel’s anti-
hero Deadpool, who, under his mask, has substantial scarring.
Deadpool, fights for good, but is also tortured and mischievous,
his scarred face underlies his ongoing conflict with society that
find his appearance repugnant. It is important to note that
different skin issues are perceived differently, while acne, pock
marks, or scars have been connected to criminal stereotypes
before (MacLin and Herrera, 2006), more fine-grained analysis
show that individuals with acne are perceived as shy or insecure
(Dréno et al., 2016).
In games, dermatological issues can be used to conjure
associations with badness, but especially scars provide
opportunity to reshape the perception of scar tissue—for
example, when used as aesthetical signifiers or to memorize
special events. Scars could, for example, be visible on characters
as a badge of defeating a difficult final boss or for taking part in a
challenging battle. In different cultures, scars also have different
meanings. Scarification—the deliberate act of scaring someone
for aesthetical purposes—has roots in traditions of African tribes
and has found its way into body modification culture. Directions
that games could use, for example, to provide new character
options increasing diversity through customizability (Dickerman
et al., 2008; Birk et al., 2016; Passmore et al., 2018), by allowing
characters to be created that defy negative stereotype associated
with dermatological issues, or to create a visual language around
the beauty of scars.
While dermatological problems tie into stereotypes and
negative expectations, the mouth is one of the most important
features used in facial expression and to communicate non-
verbally. Smiling, baring teeth, or pulling the corners of our
mouth down, are facial expression that can be inviting, display
aggression, or disdain. In our analysis the mouth is a strong
predictor of badness, showing a reversed u-shaped relationship
between the relative salience of the mouth prominence and
badness, i.e., the mouth is salient for those who rated the
character as being the most moral (good) and the least moral
(bad), but in-between the extremes the mouth tends not to
be salient. Considering the importance of facial expressions
to communicate intentions, e.g., aggression vs. friendliness, we
can assume—and the quadratic relationship confirms this—that
the mouth will also play an important role to judge morally
“good” characters.
How Game Designers Can Use the Results
Our results expose new ways that designers can try to push their
designs to leverage commonly used visual attributes in order
to get a reliable and effective morality interpretation for their
antagonists. Designers might also use our results to identify new
design alternatives that have not been previously well-explored.
In this section we speculate how our results can be used by
game designers.
Characters who people perceived as the most immoral
leveraged many common visual features. The most immoral
characters disproportionately featured salient eyes, noses,
mouths, ears, skin problems, builds, head-to-body ratios, ages
(especially appearing older), clothing, face coverings, tattoos, and
weapons. These results highlight a large number of features that
designers can leverage and try to strengthen and make more
salient in their designs to make certain a character is perceived
as immoral (based on their visual appearance).
Characters seen as the most moral did not leverage many
salient visual attributes. This makes sense as characters who
are viewed as immoral leveraged visual attributes often in
combination or exaggerated ways, making them standout (e.g.,
consider the exaggerated head-to-body ratio of Neo Cortex who
was viewed as strongly immoral). Attractiveness was the only
physical attribute we found that was used disproportionately
more for the most moral characters (roughly at the same rate
as the most immoral characters, but disproportionately more
than other characters). Given that people tend to consider
“beautiful” people as “good” (Diessner et al., 2008), it is
insightful that many participants rated beautiful characters,
that had fewer other salient features, as more moral. This
highlights that designers might consider designing antagonists
that players view as being moral, while providing a salient
physical attribute commonly associated with immoral characters.
For example, participants who rated characters as having
salient tattoos did not rate the same antagonists as having
attractiveness as a salient feature (e.g., Kaos). So, designers
might explore the combination of both attractiveness and a
feature like tattoos in antagonists. It is important to note
that attractiveness was not a pre-requisite for being viewed
as moral (e.g., Lonnie was perceived to be moral, but was
rated relatively low in terms of having salient attractiveness; see
Figure 2).
As previously described, our analysis of character ratings
in each of the five morality domains were highly correlated
with one another. This means that designers tend to present
characters that are uniform across all domains. However,
our analysis revealed interesting exceptions to this trend.
For example, the weapons and stances of Yunica (Figure 2)
strongly suggest that they are willing and ready to harm
others, while ratings of the other morality domains suggested
that they would be unlikely to do something disgusting
and would remain loyal to their groups. Similarly, Yuriko’s
slicked hair and businesswoman attire, suggested to people
that she appeared less loyal, but unlikely to physically
hurt people. Exploring ways that characters could be
designed to create other variations of morality across the
domains may provide interesting possibilities and directions
for designers.
Frontiers in Computer Science | www.frontiersin.org 15 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
Finally, designers might leverage our results directly to plan
and gauge how their planned visual attributes for an antagonist
would be perceived by players. This might be important for game
designers who seek to include a range of visual attributes. While
we have not provided a predictive model, below we elaborate on
our planned future work to build tools to support and evaluate
character design activities that leverage predictive models that
can be built based on work such as ours.
Limitations
Our choices for inclusion/exclusion when creating our stimulus
set likely had an effect on our results. Our stimulus set does not
represent all antagonists in all games. Recall that we excluded
non-humanoid characters, and indie game characters, games that
were not among the most popular games, and games published
before 2014. Together these decisions likely had some influence
on our observations and models. Firstly, as described, we did
this to ensure that we had games in our stimulus set that are
of a high quality and represent current trends in the industry.
Second, this still represents a large set of games and/or games
that are exemplary for the investigated time frame. Any attempt
to operationalize current practices in a large space will necessarily
need to make trade-offs. We believe that our results provide both
important new insights for designers and researchers and provide
a template of a new style of study for modeling the aesthetic
practices in game design that can have important applications.
We see another important limitation of our work to be the
use of static images. In many of the games players gather further
impressions of characters through the way they move. Our use
of static images, while drawn from a wide variety of games, does
not fully capture other aspects of the visual design of antagonists.
In particular, body language, movement, or speech might be used
by artists and animators to more fully communicate information.
For example, a character’s stance is often tightly integrated with
animation, to help convey tensed or relaxed muscles. Further, we
provided only two images upon which judgments can be based.
Even in games that do not use animation, different graphical
still shots are used to display expressions of emotion (e.g.,
anger, happiness, surprise, aggression, etc.). Future work should
consider displaying a richer set of media to solicit judgments
from raters.
Further, in our analysis we do not consider the behavior
or actions of characters, which obviously play prominently in
how people would perceive their morality. However, this type of
analysis is out of scope of our current research. We were focused
purely on how game designers embody their character’s morality
through visual design. An interesting, but extremely challenging,
line of research might explore common story telling techniques
around characters to understand how these impact key elements
of player perception of those characters.
Finally, we see our participant sample as a potential limitation
of our work. While we believe our sample did achieve a
reasonable mixture of gaming backgrounds, it could be that this
demographic may not uniformly represent the cultural views
and experiences that readily exist amongst gamers, or in gaming
culture. That is, it seems likely that people familiar with games
might carry their pre-existing knowledge of archetypes, tropes,
stereotypes, running narratives, etc. that exist between games,
and that people who are more familiar (enculturated) with
gaming culture, might reveal completely different and, perhaps,
more nuanced views and understanding of characters. While we
have found no evidence to suggest that this is the case, future
work might also incorporate perspectives of gaming culture and
how it might affect perceptions of character design and moral
judgments. Additionally, we did not control for moral leanings in
our convenience sample from Mechanical Turk. Previous work
has found that MTurk samples tend to be similar to student
samples regarding political leaning, but proportionally more
secular (Lewis et al., 2015). Nevertheless, given samples similarly
sized to ours (186 participants in Study 2) future work should
capture moral foundations to provide a better understanding
of the respondents and how their moral leanings might have
influenced their ratings.
Future Work
In this work we focused on antagonists, since they are
underexamined yet play a critical role in many modern games.
Our work is the first that we are aware of to take this particular
approach of formulating a stimulus set that captures visual
design practices, gather data describing people’s perceptions of
key design features and interpretation of those feature based on
the stimulus set, and to characterize it using descriptive analysis
and correlational models. We believe this work demonstrates an
approach to an exciting direction of research that aims to build
an understanding of game design practices and to make new
computational support tools for game design possible. This is
similar to the goals in the field of computational aesthetics, yet
we believe that rather than automating many of these classically
human-led endeavors, we wish to conduct research that will
better support current practices and provide new directions for
game design.
Along with the broader goal of exploring game designs
and aesthetics through computational approaches, we believe
there are a number of direct next steps that our work
offers. First, we focused on antagonists in this work; however,
previous work in media studies focused on idealized, animated
protagonists; however, this other previous work did not focus
on current practices in video games. We believe it would
be extremely interesting to repeat our study with both game
protagonists and antagonists, and to draw comparisons across
studies. We would also like to expand and mature our work
on antagonist design (and video game character design more
generally), providing guidelines for designing formidable video
game villains, protagonists, and non-player characters. Our work
accounts for physical appearances and not character actions
or mechanics in the game, we believe we can explore game
mechanics and behavior and actions in story elements of the
game to more holistically describe character designs. Finally,
we would like to explore how our statistical approach might
inform the design of tools to assist designers in assessing
designs. These could take the form of predictive tools to provide
informed estimates about the potential morality of a particular
design and/or enrich data for modeling through crowdsourcing
perception of visual attributes and morality.
Frontiers in Computer Science | www.frontiersin.org 16 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
CONCLUSIONS
Antagonists are critical elements of many games, but as of
yet no previous work has explored one of the key ways that
players experience characters, through their visual design. Our
work provides a first empirical characterization of how game
designers represent game antagonists and how people perceive
these characters in terms of their morality. To do this we
conducted two studies on Mechanical Turk to solicit ratings. The
first study collected people’s judgments on the perceived morality
based solely on the visual design for each of the 105 characters
in our stimulus set. The second study gathered judgments on
which visual attributes are more salient. Our analysis provides
a valuable characterization of current design practices and how
players perceive game antagonists, and provides a number
of key ways that designers can strengthen their antagonist’s
visuals and ways that they can break from current trends to
explore new ways to visually represent their characters. Our
research extends current research practices that seek to build an
understanding of game design practices, and provides exciting
directions exploring how design and aesthetic practices can be
better studied and supported.
DATA AVAILABILITY STATEMENT
The raw data sets collected from the populations in both
studies have been made publicly available at: https://dx.doi.org/
10.17605/OSF.IO/V2B7A.
ETHICS STATEMENT
The studies involving human participants were reviewed and
approved by Research Ethics Board University of New Brunswick
(REB 2019-118). The patients/participants provided their written
informed consent to participate in this study.
AUTHOR CONTRIBUTIONS
The creation of the stimulus set and initial literature review
was led by RP, with the support of MB and SB. The
experimental data collection system and experimental system
was led by SB, with assistance from MB and RP. The
statistical analysis was led by MB, with assistance from SB and
RP. All authors contributed to the article and approved the
submitted version.
FUNDING
This work was partially supported by the Natural Science and
Engineering Research Council Canada and the New Brunswick
Innovation Foundation.
ACKNOWLEDGMENTS
We would like to thank the reviewers of our manuscript
who provided extensive constructive feedback that substantially
improved the analysis and presentation of our work. We would
also like to thank the anonymous participants in our research
from the Mechanical Turk platform.
SUPPLEMENTARY MATERIAL
The Supplementary Material for this article can be found
online at: https://www.frontiersin.org/articles/10.3389/fcomp.
2021.531713/full#supplementary-material
REFERENCES
Ahler, D. J., Roush, C. E., and Sood, G. (2019). Micro-Task Market for Lemons. 40.
Chicago, IL.
Bakkes, S., Tan, C. T., and Pisan, Y. (2012). Personalised gaming: a motivation and
overview of literature. J. Creat. Technol. 4, 1–10. doi: 10.1145/2336727.2336731
Bar, M., Neta, M., and Linz, H. (2006). Very first impressions. Emotion 6, 269–278.
doi: 10.1037/1528-3542.6.2.269
Baranowski, T., Buday, R., Thompson, D. I., and Baranowski, J. (2008). Playing
for real: video games and stories for health-related behavior change. Amer. J.
Prevent. Med. 34, 74.e10–82.e10. doi: 10.1016/j.amepre.2007.09.027
Birk, M. V., Atkins, C., Bowey, J. T., and Mandryk, R. L. (2016). “Fostering intrinsic
motivation through avatar identification in digital games,” Proceedings of the
2016 CHI Conference on Human Factors in Computing Systems - CHI’16 (San
Jose, CA), 2982–2995. doi: 10.1145/2858036.2858062
Birmingham, E., Bischof, W. F., and Kingstone, A. (2009). Saliency does not
account for fixations to eyes within social scenes. Vis Res. 49, 2992–3000.
doi: 10.1016/j.visres.2009.09.014
Boltanski, L., and Thévenot, L. (2000). The reality of moral expectations:
a sociology of situated judgement. Philos. Explor. 3, 208–231.
doi: 10.1080/13869790008523332
Boyan, A., Grizzard, M., and Bowman, N. (2015). A massively moral game? Mass
effect as a case study to understand the influence of players’ moral intuitions
on adherence to hero or antihero play styles. J. Gam. Virt. Worlds 7, 41–57.
doi: 10.1386/jgvw.7.1.41_1
Bull, R. H. C., and Green, J. (1980). The relationship between physical appearance
and criminality. Med. Sci. Law 20, 79–83. doi: 10.1177/002580248002000202
Classic Villain—TV Tropes, n.d. available online at: https://tvtropes.org/pmwiki/
pmwiki.php/Main/ClassicVillain (accessed January 31, 2020).
Croley, J., Reese, V., and Wagner, R. (2017). Dermatologic features of
classic movie villains: the face of evil. JAMA Dermatol. 153, 559–564.
doi: 10.1001/jamadermatol.2016.5979
DeVane, B., and Squire, K. D. (2008). The meaning of race and
violence in grand theft auto: San Andreas. Games Cult. 3, 264–285.
doi: 10.1177/1555412008317308
Dickerman, C., Christensen, J., and Kerl-McClain, S. B. (2008). Big breasts and bad
guys: depictions of gender and race in video games. J. Creat. Mental Health 3,
20–29. doi: 10.1080/15401380801995076
Diessner, R., Solom, R. C., Frost, N. K., Parsons, L., and Davidson, J. (2008).
Engagement with beauty: appreciating natural, artistic, and moral beauty. J.
Psychol. 142, 303–329. doi: 10.3200/JRLP.142.3.303-332
Doris, J., Stich, J., Phillips, S., and Walmsley, L. (2020). “Moral psychology:
empirical approaches,” in The Stanford Encyclopedia of Philosophy (Spring
2020), ed E. N. Zalta (Metaphysics Research Lab, Stanford University).
Available online at: https://plato.stanford.edu/archives/spr2020/entries/moral-
psych-emp/ (accessed March 15, 2021).
Dréno, B., Tan, J., Kang, S., Rueda, M.-J., Torres Lozada, V., Bettoli, V.,
et al. (2016). How people with facial acne scars are perceived in society:
an online survey. Dermatol. Ther. 6, 207–218. doi: 10.1007/s13555-016-
0113-x
Frontiers in Computer Science | www.frontiersin.org 17 April 2021 | Volume 3 | Article 531713
Pradantyo et al. Design of Game Antagonist Morality
Eden, A., Daalmans, S., and Johnson, B. K. (2017). Morality predicts enjoyment but
not appreciation of morally ambiguous characters. Media Psychol. 20, 349–373.
doi: 10.1080/15213269.2016.1182030
Eden, A., Oliver, M. B., Tamborini, R., Limperos, A., and Woolley, J. (2015).
Perceptions of moral violations and personality traits among heroes and
villains. Mass Commun. Soc. 18, 186–208. doi: 10.1080/15205436.2014.923462
Eden, A., Tamborini, R., Grizzard, M., Lewis, R., Weber, R., and Prabhu, S.
(2014). Repeated exposure to narrative entertainment and the salience of moral
intuitions. J. Commun. 64, 501–520. doi: 10.1111/jcom.12098
Ekström, H. (2013). How Can a Character’s Personality be Conveyed Visually,
Through Shape. 54. Visby.
Fahraeus, A., and Yakali Çamoglu, D. (2011). Villains and Villainy: Embodiments of
Evil in Literature, Popular Culture and Media, Vol. 76. Rodopi. Available online
at: http://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-19409 (accessed March 15,
2021).
Ferguson, C. J. (2008). The school shooting/violent video game link: causal
relationship or moral panic? J. Investig. Psychol. Offend. Profil. 5, 25–37.
doi: 10.1002/jip.76
Fiske, S. T., Cuddy, A. J. C., and Glick, P. (2007). Universal dimensions of
social cognition: warmth and competence. Trends Cognit. Sci. 11, 77–83.
doi: 10.1016/j.tics.2006.11.005
Flowe, H. D. (2012). Do characteristics of faces that convey trustworthiness
and dominance underlie perceptions of criminality? PLOS ONE 7:e37253.
doi: 10.1371/journal.pone.0037253
Funk, F., and Todorov, A. (2013). Criminal stereotypes in the courtroom: facial
tattoos affect guilt and punishment differently. Psychol. Public Pol. Law 19,
466–478. doi: 10.1037/a0034736
Goldstein, A. G., Chance, J. E., and Gilbert, B. (1984). Facial stereotypes of good
guys and bad guys: a replication and extension. Bull. Psychon. Soc. 22, 549–552.
doi: 10.3758/BF03333904
Grizzard, M., Fitzgerald, K., Francemone, C. J., Ahn, C., Huang, J., Walton, J.,
et al. (2019). Validating the extended character morality questionnaire. Media
Psychol. 43, 107–130. doi: 10.1080/15213269.2019.1572523
Grizzard, M., Huang, J., Fitzgerald, K., Ahn, C., and Chu, H. (2018).
Sensing heroes and villains: character-schema and the