Content uploaded by Renkai Ma
Author content
All content in this area was uploaded by Renkai Ma on Feb 08, 2023
Content may be subject to copyright.
Transparency, Fairness, and Coping: How Players Experience
Moderation in Multiplayer Online Games
Renkai Ma
The Pennsylvania State University
renkai@psu.edu
Yao Li
The University of Central Florida
yao.li@ucf.edu
Yubo Kou
The Pennsylvania State University
yubokou@psu.edu
ABSTRACT
Multiplayer online games seek to address toxic behaviors such as
trolling and grieng through behavior moderation, where penalties
such as chat restriction or account suspension are issued against
toxic players in the hope that punishments create a teachable mo-
ment for punished players to reect and improve future behavior.
While punishments impact player experience (PX) in profound
ways, little is known regarding how players experience behavior
moderation. In this study, we conducted a survey of 291 players
to understand their experiences with punishments in online multi-
player games. Through several statistical analyses, we found that
moderation explanation plays a critical role in improving players’
perceived transparency and fairness of moderation; and these per-
ceptions signicantly aect what players do after punishments.
We discuss moderation experience as an important facet of PX,
bridge the game and moderation literature, and provide design
implications for behavior moderation in multiplayer online games.
CCS CONCEPTS
•Human-centered computing;•Human computer interac-
tion;
KEYWORDS
Multiplayer online games, behavior moderation, toxicity, modera-
tion design
ACM Reference Format:
Renkai Ma, Yao Li, and Yubo Kou. 2023. Transparency, Fairness, and Coping:
How Players Experience Moderation in Multiplayer Online Games. In Pro-
ceedings of the 2023 CHI Conference on Human Factors in Computing Systems
(CHI ’23), April 23–28, 2023, Hamburg, Germany. ACM, New York, NY, USA,
21 pages. https://doi.org/10.1145/3544548.3581097
1 INTRODUCTION
Multiplayer online games face rampant toxic behaviors such as
trolling, harassment, and grieng i n t heir p layer communities
[
1
,
17
,
28
,
59
,
61
], and game researchers and practitioners have long
recognized the grim challenge of designing eective moderation
systems that discourage toxic behaviors while encourage cooper-
ative ones (e.g., [
31
,
64
,
76
,
90
]). Game platforms also routinely
This is preprint
CHI ’23, April 23–28, 2023, Hamburg, Germany
https://doi.org/10.1145/3544548.3581097
update their moderation policy and practice to cover new types of
toxicity [
70
,
71
]. Their present moderation systems usually adopt a
punitive model, where players, once convicted, will receive one or
more penalties, ranging from losing access to certain in-game priv-
ileges (such as in-game rewards) to permanent account suspension.
Game companies may issue penalties against individual players or
a group of players for unwanted behaviors (e.g., [
7
,
82
]). Numerous
celebrity players make headlines for receiving permanent bans for
their in-game toxicity (e.g., [3, 6, 30, 86]).
Given how commonplace it is for game companies’ moderation
systems to hand out penalties to players, understanding players’
moderation experience is of important value to multiplayer on-
line games for many reasons. First, although moderation plays
a central role in managing toxicity in multiplayer online games
[
76
], there is limited understanding regarding its eectiveness in
reforming player behavior if those punished players are to stay.
Second, receiving a moderation penalty intersects deeply with the
player experience (PX) in game if they are temporarily banned
from play. Third, a moderation penalty could often incur negative
emotions such as frustration and anger [
26
,
98
], thus intersecting
with players’ emotional experiences.
The punitive model of online moderation systems is not with-
out limitations when a penalty is likely the rst and only point of
contact between users and the moderation system. In other words,
the design of punishment matters. Prior moderation research, most
of which is done in the context of social media platforms such as
Reddit [
44
,
47
], Facebook [
73
,
91
,
98
], and Instagram [
26
,
34
], has
provided ample reections on this. For instance, users may not
understand why they are punished and have to gure that out on
their own [
47
,
56
]. Aected users need more fairness, accountabil-
ity, and transparency in punishment design in order to develop
better trust [
47
,
81
,
98
,
108
]. Rich empirical ndings from the recent
moderation literature suggest that how users experience punish-
ments matters to their compliance with behavioral standards as
well as their later conducts [
47
,
50
]. However, little attention has
been paid to how players experience moderation in multiplayer
online games. In this study, we use behavior moderation and pun-
ishment design interchangeably, where behavior moderation is
more conceptual and denotes a cluster of approaches to manage
player behavior, while punishment design is more operational and
represents specic moderation actions that players experience.
To approach this question, we conducted a survey in May 2022
to understand how players in multiplayer online games experi-
ence punishments from behavior moderation. Specically, the study
leverages the existing moderation literature (e.g., [
44
,
47
,
66
,
98
,
99
])
to focus on the perceived transparency and fairness of moderation
as well as the intended adoption of coping strategies for punish-
ments in the context of online gaming. We performed exploratory
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
factor analysis (EFA), conrmatory factor analysis (CFA), and struc-
tural equation modeling (SEM) on the valid survey data (N
=
290).
We found that while punishment notication and explanation sig-
nicantly increase players’ perceived transparency of moderation,
explanation provision plays a more critical role than explanation
and punishment types in increasing all notions of fairness percep-
tions. Also, as perceived fairness, especially retributive, restorative,
and procedural justice, plays more critical roles than perceived
transparency in motivating players’ intended adoption of coping
strategies for punishments, moderation explanation as one punish-
ment design became more important to help punished players cope
with punishments. We discuss how these ndings extend our under-
standing from the moderation literature that primarily focuses on
the social media context (e.g., [
44
,
47
,
51
,
67
,
98
]). We then discuss
the necessity of considering players’ moderation experience as part
of PX and derive practical implications for moderation design and
policymaking in game from our ndings.
We contribute to HCI and game research in four ways: First, we
contribute quantitative insights into players’ moderation experi-
ences. Second, we contribute survey items for assessing players’
punishment/moderation experiences with high validity and relia-
bility for future work that focuses on this topic. Third, we theorize
players’ moderation experiences in relation to player experience
by bridging the moderation literature and the player experience
literature. Lastly, we contribute concrete design implications for
moderation in multiplayer online games.
2
BACKGROUND: TRANSPARENCY, FAIRNESS,
AND COPING WITH MODERATION
PUNISHMENT
In line with rising ethical concerns about algorithmic systems (e.g.,
[
20
,
63
,
65
]), HCI researchers have recently paid attention to trans-
parency and fairness in users’ experiences with moderation sys-
tems (e.g., [
47
,
68
,
98
]). Transparency implies openness and com-
munication [
93
], allowing users to “uncover the true essence of
a system” [
14
]. A large body of prior work has seen moderation
notication and explanation as important design approaches for
users to understand moderation system’s decision-making (e.g.,
[
47
,
56
,
67
,
98
]). Moderation notications and explanations as plat-
forms’ transparency eorts thus are critical for users to assess mod-
eration transparency. Fairness can be dened on diverse ontological
bases. Moderation researchers have initiated various discussions
by leveraging diverse dimensions of fairness notions, such as pro-
cedural or restorative justice, to assess moderation fairness (e.g.,
[67, 85, 104]).
Investigating the perceived transparency of moderation systems
is a growing research interest. When Facebook failed to inform
users of content removal at the time of its issuance [
91
], users
questioned what content rules Facebook deemed they violated [
73
].
Researchers also uncovered that users complained about the incon-
sistent punishments that happened between them and others, and
thus the users requested further explanations (e.g., [
68
,
98
]). Prior
work also stressed the importance of disclosing sucient informa-
tion in moderation explanations [
47
], which could be educational to
punished users for behavior reform [
44
] and build up trust for plat-
forms [
91
]. Especially, as harmful content can be categorized based
on dierent severities [
84
], it becomes important for platforms to
make transparency eorts pertaining to the varying severities to
show how a moderation decision is made.
Beyond the moderated users’ perspectives, transparency is also
a design consideration stressed by human moderators who prac-
tice moderation in game-related contexts (e.g., live-streaming plat-
forms). Sometimes, it could be intuitive that human moderators
inform rule breakers what and why they are accused. For example,
Cai et al. found that volunteer moderators on the live streaming plat-
form, Twitch, actively communicate with rule violators to ensure
moderation practices and decisions are in an appropriate degree
of visibility to the public [
12
]. But oftentimes, it is challenging for
platforms to decide to disclose what degree of moderation trans-
parency. Jiang et al. found that on Discord, human moderators
usually encounter challenges in making all content rules explicit
and transparent because user behaviors are complex and nuanced
in voice-based communities [
50
]. Kou and Gui found that gamers
who ag and report other players doubt the transparency of ag-
ging mechanism, especially around whether and how it works,
so gamers generate distrust to moderation system [
58
]. But still,
researchers have generally reached a consensus that moderators
should keep dierent moderation procedures transparent such as
moderation notication and explanations [
45
,
51
,
73
], as well as
appeal process [26, 51].
Beyond moderation transparency from the angles of either mod-
erated users or human moderators, researchers have also paid atten-
tion to the fairness perception of moderation system. Since fairness
has scarcely been dened in a consensus, many researchers have
used multiple dimensions of fairness, such as outcome fairness,
retributive, procedural, and restorative justice, to understand users’
perceived fairness of moderation system. First, outcome fairness
means the extent to which users perceive the distribution of moder-
ation decisions (e.g., account suspension [
98
], visibility deduction
[
68
]) is fair. Several prior studies have found that users would per-
ceive content removal or account suspension as fair if they receive
a moderation explanation (e.g., [
44
,
98
]). Second, procedural jus-
tice refers to fair processes where users’ fairness perceptions are
inuenced by their experiences [
106
]. A recent study has found
that content creators experienced inconsistent punishments that
simultaneously violated the platform’s content rules, so creators
felt moderation as unfair [
68
]. Third, retributive justice describes
correct justice processes, where people violating rules require to
suer proportionally in return [
101
]. However, many researchers
have been concerned that retributive justice might not be the only
eective justice standard for adjudicating moderation cases (e.g.,
[
8
,
56
]). That is because users might not be able to eectively learn
from what they have done wrong by server punishments but con-
tinue being toxic [
43
]. Thus, one of the alternative justice models,
restorative justice, is appropriate to re-assess moderation cases.
This justice notion seeks to have both oenders and victims in the
justice process and allow their voice to be heard by decision-makers
[
101
]. That means, platforms need to communicate with punished
users [
67
], or as a recent study suggested, there at least should be
community eorts involving punished users, victims, and other
community members to justify moderation cases together [
104
].
Given these four dimensions of fairness notion from prior work,
we position our study in an integrative fashion of involving them
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
to interpret players’ perceived fairness of behavior moderation in
game. This can enable us to uncover more nuances that might be
missed if we use one single notion of fairness.
Along with studying user perceptions of moderation, researchers
have started to investigate how users cope with punishments (e.g.,
[
26
,
67
,
73
]). Coping is “the person’s constantly changing cognitive
and behavioral eorts to manage specic external and/or internal
demands that are appraised as taxing or exceeding the person’s re-
sources” [
27
]. Prior moderation literature has discussed how users
might or might not have enough resources to handle punishments
behaviorally or cognitively. For example, users are found to avoid
future punishments by tweaking their content [
13
,
26
,
35
] or cre-
ating closed, hidden groups to connect with and support other
punished users [
34
]. Users were also found to resist moderation
punishments by generating memes publicly to express complaints
[
92
] or initiating appeal procedures (e.g., [
68
,
98
]). Besides such
behavioral eorts in responding to punishments, users also choose
to make cognitive eorts, such as getting recovered and healing
through communication with other community members [
26
] or
collectively conducting sense-making on why punishments happen
[
67
]. When users do not have enough eort to behaviorally cope
with punishments, they choose to pay little attention to punish-
ments and accept what impacts the punishments bring to them
[11].
So, to better understand users’ coping eorts for behavior mod-
eration and punishments in the context of online gaming, we bor-
rowed the ve coping strategies delineated by Scherer et al., in-
cluding (1) problem-focused coping, (2) detachment, (3) wishful
thinking, (4) seeking social support, and (5) focusing on the posi-
tive [
83
]. Problem-focused coping means that people come up with
dierent solutions for the problem; detachment refers to the strate-
gized attitude of seeing the problem as nothing happening; wishful
thinking means that people wish the problem would go away or a
miracle will happen; seeking social support refers to talking to or
asking support from someone about the problem; focusing on the
positive denotes peoples’ actions of redirecting attention from the
problem to something positive or creative.
2.1 Research Gap: Moderation/Punishment
Experience in Game
A growing body of research has examined users’ moderation ex-
periences in game-related contexts (e.g., live streaming platforms
and audio-based community). Importantly, these game-related con-
texts are a unique type of online community in nature, and thus
fundamentally dierent from the game contexts in several ways.
These users, like many social media users, might complain moder-
ation decision-making as opaque or unfair (e.g., [
26
,
73
,
91
]), but
their behaviors are mostly presented as user-generated content
(e.g., videos [
69
] on YouTube, textual content on Reddit [
44
,
47
], or
audio on Discord [
50
]) in a relatively static way. That means, either
human moderators or moderation algorithms could nd ways (e.g.,
hash/keyword matching or classication [
38
]) to identify and legiti-
mately adjudicate whether they violate content policies [
12
,
45
,
79
],
and moderation decision-makers could further notify what they
have accused users for [
47
,
98
]. However, game players’ behaviors
are categorically more complex through a combination of in-game
communication, avatar actions, interactions with game design, etc.
Problematic user content such as hate speech that is commonly
found on social media is just one type of violation for moderation
in game; and players might commit toxic behaviors that exclusively
happen in online multiplayer games, such as sabotaging teamwork
or intentionally getting killed by enemies [
66
]. Or even the game
platform designs, such as matchmaking system or players’ per-
ceived loss due to merit-based game competition, could be part of
the reason that players become toxic and violate platform policies.
As such, player behavior is notoriously more dicult to adjudicate
than social media users’ behaviors which are mostly text, audio, or
video-based.
All of the aforementioned concerns introduce profound chal-
lenges to moderation fairness and, subsequently, players’ fairness
perception in multiplayer online games. Unsurprisingly, multiplayer
online games have long wrestled with what constitutes a fair moder-
ation decision. As early as the 1990s, MUD users would debate what
penalty was proper for a user who had committed a virtual “rape”
[
18
]. In contemporary multiplayer online games, game companies
such as Riot Games must deal with their player base’ reactions to
permanent bans of gaming celebrities [
39
]. Also, a great problem
fronted by both researchers and gaming companies is how to de-
sign better moderation systems to help reform player behaviors.
Researchers who have focused on social media or game-related mod-
eration have thought about rethinking moderation decision-making
procedures (e.g., involving users’ voice for procedural justice [
25
]
or expert review in moderation [
79
]) or designing moderation ex-
planations that can instruct users about content policies [47, 51].
However, given the complexity in player behaviors, it would be
hard to come up with design solutions to improve in-game modera-
tion system unless we advance the understanding of how players
experience punishments. Calling for more attention to players’
moderation experiences, we recognize the nuances of in-game pun-
ishment design, compared to moderation in other contexts such
as game-related or social media communities. For example, play-
ers might receive ranked rating deduction (i.e., game skill level
decrease) or barred entry from joining certain types of game (e.g.,
matchmaking or queue restrictions) from competitive games. While
game-related contexts such as communities on Twitch or Discord
conduct similar moderation mechanisms (e.g., chat restriction [
87
]),
they might not be enough or contextual for the in-game environ-
ment, which are usually competitive, merit-based for winning, and
toxic [
56
,
88
]. Thus, to reect on and implicate better punishment
design in game, we aim to ll the research gap of players’ mod-
eration experiences - their perceptions of and coping reactions to
punishment.
3 RESEARCH QUESTIONS & HYPOTHESIS
DEVELOPMENT
This section will discuss how we distill two specic research ques-
tions (RQ1 and RQ2) and corresponding hypotheses within two
hypothesized models from the prior work. Our rst hypothesized
model (H1-H3) for RQ1 describes the purposed relationships be-
tween punishment design (e.g., notication, explanation) and pun-
ished players’ perceptions of moderation (i.e., perceived trans-
parency and fairness) as well as their intended adoption of coping
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
Figure 1: Hypothesized model 1 of punishment experience for RQ1 (H1-H3). We removed “wishful thinking” after exploratory
factor analysis since its all survey items had signicant cross-loadings on other factors (see Section 5.1).
Figure 2: Hypothesized model 2 of punishment experience
for RQ2 (H4). We removed “wishful thinking” after ex-
ploratory factor analysis since its all survey items had sig-
nicant cross-loadings on other factors (see Section 5.1).
strategies for punishments, as summarized in Figure 1. Figure 2
summarizes the purposed relationships (H4) for RQ2 between the
perception of moderation, including perceived fairness and trans-
parency, and the intended adoption of coping strategies.
3.1 Punishment Design
Punishment design means the design construction where players
experience punishments. Prior work has broadly understood moder-
ation notication and explanation as important design components
or facets in moderation processes (e.g., [
47
,
56
,
67
,
98
]). For example,
in the issuance of moderation punishments such as account suspen-
sion, Suzor et al. found users felt confused about punishments since
they did not receive notications [
91
]. When users try to make
sense out of punishments, they request detailed explanations of
what policy they were deemed to violate by moderation system [
73
].
In the context of online gaming, such explanation can be detailed
as reasons for punishment (e.g., “reform card” in League of Legends
[
57
]), resource provision to cope with punishment (e.g., information
on how to appeal punishments [
98
]), and more to help punished
players understand moderation decision-making. When punished
users enter an appeal procedure of punishment, Vaccaro stressed
the importance of explanation provision to improve their perceived
fairness and trustworthiness of platform [
98
]. Also, throughout
moderation processes like receiving and appealing punishments,
game platforms might utilize dierent punishments to govern dif-
ferent identities of users, such as professional players, coaches, and
teams [
66
]. Thus, punishment notication, explanation, and named
punishments are three punishment design components important
to punished users and also shared by platforms. As these punish-
ment design components shape users’ moderation experiences, we
aim to understand how they aect players’ perceived transparency
and fairness, as well as the intended adoption of coping strategies
for punishments, as we discussed in Section 2. So, we ask:
•
RQ1 Does punishment design aect players’ perceptions of
behavior moderation and their post-moderation behaviors?
•
RQ1.1 Does punishment design aect players’ perceived
transparency of behavior moderation?
•
RQ1.2 Does punishment design aect players’ perceived
fairness of behavior moderation?
•
RQ1.3 Does punishment design aect players’ coping strate-
gies for punishments?
3.1.1 Moderation Transparency based on Punishment Design. Prior
work has broadly recognized the importance of punishment noti-
cations and explanations to improve moderation transparency (e.g.,
[
25
,
99
]). For example, researchers found that content moderation
in localized communities like subreddits would become opaque
when human moderators silently remove user content without no-
tication or specifying reasons [
51
]. Especially many qualitative
ndings show that users perceive moderation as opaque when they
do not receive notications (e.g., [
44
,
91
]) and explanations (e.g.,
[
67
,
73
]) of punishments. Such perceived opacity will be intensied
once punishment brings rippling eects to intervene in users’ com-
munication with online communities [
26
] and their online career
development (e.g., income) [
68
]. Users thus want to obtain enough
information about punishments. For example, on Facebook, they
request explanations of why moderation systems issue inconsistent
content removal decisions [
98
]. Especially a recent survey showed
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
that custom messages specifying punishment reasons would in-
crease users’ perceived transparency of moderation [
37
]. In this
sense, we assume that punishment notications and explanations
might allow players to perceive behavior moderation as more trans-
parent than no notication or explanation:
•
H1.1: Punishment notication provision positively aects
players’ perceived transparency of behavior moderation,
compared to no notication.
•
H1.2: Punishment explanation provision positively aects
players’ perceived transparency of behavior moderation,
compared to no explanation.
Besides, punishment types may inuence users’ perceived trans-
parency of moderation. Researchers have collectively understood
that account suspension (i.e., permanent ban in game or de-
platforming) is the most stringent punishment in moderation
[
46
,
56
,
67
,
73
,
91
], where users lose the ability to continue using the
original account to play games, post content, or communicate with
others. Users might instantly generate perceived uncertainty and
opacity toward moderation system due to such harsh punishment
[
73
]. However, encountering relatively lighter punishments like
content removal, users might take more time to make sense out
of the punishments. They might develop perceived opacity more
from the lack of notications of punishment or limited direct com-
munication with platforms than the punishment itself [
44
,
67
]. So,
based on this line of prior work, we propose that compared to the
relatively lighter punishments, such as content removal, other rela-
tively heavier punishments, like permanent ban might negatively
aect users’ perceived transparency of behavior moderation:
•
H1.3: Experiencing (a) restricted access to game features
(e.g., chat or matchmaking ban), (b) temporary ban, and (c)
IP or permanent ban negatively aect players’ perceived
transparency of behavior moderation, compared to content
or item removal.
3.1.2 Moderation Fairness based on Punishment Design. Platforms’
transparency eorts, such as oering punishment notications or
explanations, have been seen as an important path to users’ per-
ceived fairness of moderation. For example, Ma and Kou found
that content creators considered moderation as unfair when their
videos were disproportionately hidden by YouTube’s moderation
algorithms without notications [
68
]. Especially, creators felt the al-
gorithms did not involve their voice in moderation decision-making
procedures while moderation has already imposed negative eects
on their channel performance and livelihoods [
68
]. Similarly, in-
formation disclosure of moderation might decrease such perceived
unfairness. Jhaver et al. found that users receiving moderation ex-
planations considered content removal as fair than no explanation
on Reddit [
44
]. Vaccaro et al. uncovered that once explanations,
either written by algorithms or humans, are oered, users’ per-
ceived fairness of account suspension on Facebook would increase
[
98
]. This line of work shows how users will consider moderation
fair if notications and explanations are provided in moderation.
Given the dierent dimensions of fairness as we discussed in Sec-
tion 2.1, we propose that punishment notication and explanation
provision can positively aect users’ perceived fairness of behavior
moderation:
•
H2.1: Punishment notication provision positively aects
players’ perceptions of (a) outcome fairness, (b) retributive
justice, (c) procedural justice, and (d) restorative justice of
behavior moderation, compared to no notication.
•
H2.2: Punishment explanation provision positively aects
players’ perceptions of (a) outcome fairness, (b) retributive
justice, (c) procedural justice, and (d) restorative justice of
behavior moderation, compared to no explanation.
While little prior literature directly points out the relationship
between perceived fairness and behavior moderation, research in
other elds predicts that punishments might negatively aect per-
ceived fairness. For example, Xue et al. veried that in the enterprise
context, users’ perceived justice of punishments is negatively in-
uenced by actual punishments [
105
]. Also, prior work has shown
a positive correlation between transparency and fairness [
63
]. So,
as we propose the negative relationship between punishments and
perceived transparency, we predict the relationship between pun-
ishments and perceived fairness to be negative:
•
H2.3: Experiencing (1) restricted access to game features
(e.g., chat or matchmaking ban), (2) temporary ban, and (3)
IP or permanent ban negatively aect players’ perceptions
of (a) outcome fairness, (b) retributive justice, (c) procedural
justice, and (d) restorative justice in platform governance,
compared to content or item removal.
3.1.3 Coping Strategies based on Punishment Design. Prior mod-
eration research has showed that more degree of transparency in
moderation designs motivates more coping strategies adopted by
moderated users. For example, when users do not receive moder-
ation notications, they might make cognitive eorts to conduct
sense-making regarding why or how (e.g., algorithms or humans)
punishments happen [
67
]. When users receive explanations that
they perceive as unconvincing, they might request platforms to
re-examine previous punishment decisions through appeal proce-
dures [
4
,
26
,
98
]. Or they begin generating their own understanding
and rationales to justify why they experience punishments [
91
].
They might make more behavioral eorts to contact platform repre-
sentatives (e.g., human moderators) through third-party platforms
if they fail to directly contact them [
67
,
73
]. Even for detachment
factor, users who are informed of being blocked reacted to modera-
tion with indierence and think moderation does not matter [
48
].
In game, when players are notied of permanent ban, some does
not actively cope with it, while treating it as if nothing happened
and buying a new account to commit toxicity [
56
]. Prior work has
further showed that players can alter their behaviors (e.g., toxic lan-
guage) time after time, meaning that they might perform dierent
behaviors in dierent moment of game [
60
]. Thus, we predict that
the more actively platforms disclose information about moderation,
the more diverse coping strategies moderated players will adopt,
even though players might perform these strategies in dierent
temporal patterns. We propose:
•
H3.1: Punishment notication provision positively aects
players’ adoption of coping strategies for punishments, in-
cluding (a) problem coping, (b) seeking social support, (c)
detachment, (d) focusing on the positive, and (e) wishful
thinking, compared to no notication.
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
•
H3.2: Punishment explanation provision positively aects
players’ adoption of coping strategies for punishments, in-
cluding (a) problem coping, (b) seeking social support, (c)
detachment, (d) focusing on the positive, and (e) wishful
thinking, compared to no explanation.
However, we predict punishment types, especially harsher ones,
might not help users adopt diverse coping strategies. Much work
has shown that severe punishments could work against player’s
positive behaviors (e.g., collaboration, reforming past behaviors).
For example, a convicted person is not more likely to reform and
improve their behaviors when they are punished by stronger pun-
ishments than weaker ones [
10
]. A recent large-scale experimental
study with punished people also found that harsher punishments,
such as prison sentences, were not more eective in helping con-
victs reform or preventing them from re-oending [
41
]. Similar
situations happen in the context of content moderation. When
experiencing relatively heavier punishments such as account sus-
pension or community takedown, HCI researchers found users
might not actively cope with punishments but become more toxic
and hostile [
43
,
94
]. Game players who experience permanent ac-
count suspension also do not mean they become reformed players
[
56
]. While under lighter punishments like content removal, users
would generally decrease their frequency of posting spamming
and hate speech [
87
,
107
]. Thus, we propose that compared to con-
tent or item removal, other relatively heavier punishments might
negatively aect player’s adoption of coping strategies.
•
H3.3: Experiencing (1) restricted access to game features,
(2) temporary ban, and (3) IP or permanent ban negatively
aect players’ adoption of coping strategies for punishments,
including (a) problem coping, (b) seeking social support, (c)
detachment, (d) focusing on the positive, and (e) wishful
thinking, compared to content or item removal.
3.2 Perceived Transparency and Fairness as
Predictors of Coping Strategies
As discussed, perceived transparency and fairness of moderation are
two important user perceptions in the moderation literature. Prior
work has also uncovered users’ post-moderation eorts, such as
appealing moderation decisions (e.g., [
68
,
98
]) or making cognitive
eorts to make sense of why punishments happen [
67
]. However,
we have relatively little knowledge of how punished users’ percep-
tion of moderation is related to their behaviors afterward, especially
in game. So, we ask:
RQ2 Do players’ perceived transparency and fairness of behavior
moderation aect their coping strategies for punishments?
Obtaining an initial understanding of this question, we found
that prior work has recognized a positive relationship between
perceived transparency and users’ positive behaviors. For example,
employees’ perceived transparency of communication with em-
ployers positively aects employees’ altruism and collaborations
with others [
49
]. Users’ perceived transparency of privacy policy
is also a signicant positive predictor of their cognitive trust in
sharing health information with technologies [
21
]. In moderation
context, several researchers have indirectly uncovered that plat-
form’s transparency eorts (e.g., explanation provision) support
users’ positive behaviors. Jhaver et al. found that when content re-
moval explanations were provided, users improved their behaviors,
and thus fewer content removal cases happened to them [
47
]. Thus,
we predict player’s perceived transparency can support them in
coping with moderation punishments:
•
H4.1: Players’ perceived transparency of behavior moder-
ation positively aects their adoption of coping strategies
for punishments, including (a) problem coping, (b) seeking
social support, (c) detachment, (d) focusing on the positive,
and (e) wishful thinking.
Similarly, we predict that perceived fairness can motivate play-
ers’ adoption of coping strategies, which essentially are positive
behaviors or cognitive eorts. That is because many prior studies
have found that dierent dimensions of perceived fairness, includ-
ing procedural justice, distributive justice, and interactional justice,
positively aect people’s organizational citizenship behaviors (e.g.,
[
62
,
77
]). Organizational citizenship behaviors represent a person’s
positive and constructive actions that can contribute optimally to or-
ganizations. So, perceived fairness plays a generally positive role in
motivating people’s eorts and positive attitudes. Especially, prior
moderation literature has found a positive correlation between
perceived fairness of moderation and productive user behaviors
based on content removal explanation [
44
]. Increased perceived
procedural justice has also been shown to decrease users’ future
behaviors of violating social media platform’s content rules [
96
].
Reasonably, we propose a positive relationship between perceived
fairness and coping strategies.
•
H4.2: Players’ perceived outcome fairness of behavior mod-
eration positively aects their adoption of coping strategies
for punishments, including (a) problem coping, (b) seeking
social support, (c) detachment, (d) focusing on the positive,
and (e) wishful thinking.
•
H4.3: Players’ perceived retributive justice of behavior mod-
eration positively aects their adoption of coping strategies
for punishments, including (a) problem coping, (b) seeking
social support, (c) detachment, (d) focusing on the positive,
and (e) wishful thinking.
•
H4.4: Players’ perceived procedural justice of behavior mod-
eration positively aects their adoption of coping strategies
for punishments, including (a) problem coping, (b) seeking
social support, (c) detachment, (d) focusing on the positive,
and (e) wishful thinking.
•
H4.5: Players’ perceived restorative justice of behavior mod-
eration positively aects their adoption of coping strategies
for punishments, including (a) problem coping, (b) seeking
social support, (c) detachment, (d) focusing on the positive,
and (e) wishful thinking.
4 METHODS
The full survey is available as supplementary material, and we
summarize our survey design, procedure, and sample in this section.
Please note that we received 291 valid survey responses, but we
removed one out of 291 from inferential statistical analysis because
that participant’s response did not meet the minimum size for
inferential statistical analysis (see details in Section 5.2).
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
4.1 Survey Design
Our survey included three parts: (1) consent and screening, (2)
punishment experience, and (3) demographics. In the consent and
screening part, respondents read the consent sheet and indicated
their agreement to participate. They were also asked about their
age and experience with punishments (e.g., account, chat ban) from
online multiplayer games in the past. Participants who are under 18
years old or without experience with punishments were not given
the option to proceed with the survey.
In the punishment experience part, respondents were rst asked
to multi-select the punishment types they had experienced with
an option to manually type in more punishments. These punish-
ment types were shared by many prior studies around social media
moderation [
73
,
98
] and one recent work discussing punishments
in game [
66
]. Once the participants made their multi-selection, a
random punishment type from their multi-selection was presented.
Participants were then asked to answer a set of follow-up questions
on their perceived transparency and fairness on, as well as, coping
strategies for the randomly presented punishment type that they
had experienced. The reasons for the randomization were: (1) we
wanted to focus only on the punishment that the participants had
experienced so that their answers were not imaginary; (2) if the
participants had experienced multiple punishments, it would be
time-consuming to ask follow-up questions on each experienced
punishment (i.e., # of questions * # of punishments) and the random-
ization could make the survey more ecient; (3) the randomization
could control confounding factors that would bias the data. For
example, the randomization could avoid participants reporting the
punishments they remembered the most or the punishment they
believed was the most unfair. In the last part of the survey, we
asked about respondents’ demographics, such as age, race, gender,
and education levels. Also, we designed two attention check ques-
tions in dierent locations of our survey to ensure our data quality.
Participants who failed these two attention check questions were
excluded from our dataset.
4.1.1 Measurement Design. We measured respondents’ perceived
transparency, fairness, justice and adoption of coping strategies
regarding the random experienced punishment. We reminded the re-
spondents that the measurement questions were about the random
experienced punishment by stating “please rate your agreement
with the statements regarding the punishment decisions by [piped
game].” Perceived transparency was measured by ve items adapted
from Gray and Durcikova’s study [
19
] and Gonçalves et al.’s sur-
vey design [
37
]. The perceived fairness includes four dimensions,
outcome fairness, retributive justice, procedural justice, and restora-
tive justice, all of which were adapted from prior work. Outcome
Fairness was primarily measured by three items from Colquitt’s
study [
16
] and Gonçalves et al.’s survey [
37
] and one item we made
to summarize the outcome fairness notion. Retributive Justice was
measured by ve items adapted from Wenzel et al.’s study [
102
].
Procedural Justice was measured by ve items adapted from Nieho
and Moorman’s study [
75
], which measured perceptions of orga-
nization fairness. Restorative Justice was measured by ve items
adapted from Wenzel et al.’s study [102].
The factor group of coping strategies has ve factors, including
problem coping, detachment, social support, wishful thinking, and
positivity. We adapted all survey items of these ve factors from
Scherer et al.’s study [
83
]. It is worth noting that originally, we
adapted four items of wishful thinking from this work [
83
], but all
these four items had signicant cross loadings on the social support
factor in EFA results. So, we removed this wishful thinking factor
because all items did not converge into a single factor (see Section
4.4). Five-point Likert scales ranging from “strongly disagree” to
“strongly agree” were used for all the items. The content of each
item can be found in Table 2.
All the survey items were adapted to the context of gaming.
For instance, the source of the punishment was changed to the
particular game platform the respondent reported. For notions,
including perceived transparency, outcome fairness, and coping
strategies that are more related to personal and result-oriented
notions, our survey items were adapted to follow the research trend
that stresses subjective experiences of punishment (e.g., fairness
perceptions of account suspension punishment and reactions for
it [
44
,
47
,
91
,
98
]). For example, for perceived transparency, one of
our survey statements was “It is easy for me to see the status of
punishments.”
To reduce the social desirability bias, we used the method of
proxy subjects to frame the survey statements that read negatively
to respondents, namely the statements about justice notions, in-
cluding retributive, procedural, and restorative justice. Social de-
sirability is the tendency that study subjects, including survey re-
spondents, tend to deny socially undesirable things which place
the subjects in an unfavorable light [
74
,
80
]. One example is that a
person could admit fewer violations of the law than actually com-
mitted. One method to deal with the social desirability bias is to use
proxies, such as using someone who knows the respondents well
[
74
] or similar others [
33
], instead of the target person. The social
desirability bias might exist in the original statements of justice, i.e.,
“Overall, as a matter of justice, I should be punished”, which might
indicate negative and socially undesirable characteristics of the
respondents. To mitigate the bias, we used “the convicted players”
instead of “I” or “me” to avoid inquiring on whether punishment is
desired or not by respondent personally and present respondents
from mispresenting their punishment experience. This survey state-
ment adaptation consideration was also supported by prior work
that has assessed the perceived justice notions (e.g., [100, 103]).
4.2 Procedure
After this study was approved by our institution’s IRB oce, we
programmed our study design on Qualtrics. We rst ran a pilot
study with 15 respondents. These participants were compensated a
$ 2 gift card (i.e., $12 hourly payment rate) for their participation.
This pilot study helped us tweak some narratives of the questions
and make the survey more readable and digestible to participants.
Because of this change, after the pilot study, we did not include the
data of the pilot study for the actual analysis. We then launched
the survey on Prolic.co, an online participant recruitment ser-
vice, to recruit players of online multiplayer games. The reason
why we chose Prolic was that previous research has shown that
Prolic oers high data quality for social science experiments and
behavioral research [
22
,
78
]. To control the possible confounding
factors (i.e., culture and country), we only recruited participants
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
Table 1: Player proles (gender, education, age, race, and punishment types). All participants are from the US.
Gender Quantity (N=291) Percent
Female 103 35.40%
Male 173 59.45%
Non-binary / third gender 14 4.81%
Not Specied 1 0.34%
Education
A high school diploma or equivalent 54 18.56%
Bachelor degree 94 32.30%
Doctoral degree 4 1.37%
Less than a high school diploma 8 2.75%
Master’s degree 13 4.47%
Some college, no degree 73 25.09%
Two-year associate degree 45 15.46%
Race
Asian 30 10.31%
Black or African American 18 6.19%
Hispanic, Latino, or Spanish 17 5.84%
Mixed race 33 11.34%
White or Caucasian 193 66.32%
Age Mean Standard Deviation
31.26 0.55
Punishments Experienced Quantity Percent
Content or item removal 28 9.62%
IP or Permanent account ban 36 12.37%
Restricted access to game features 191 65.64%
Temporary account ban 154 52.92%
Warning 3 1.03%
who (1) understood English, (2) experienced punishments in online
multiplayer games, (3) and resided in the US. We compensated each
respondent who nished the survey and passed attention check
questions (N
=
291) with a payment rate of $12.54 per hour for com-
pleting the survey, which is higher than Prolic’s site-wide average
reward rate and the state minimum wage rate in the authors’ state.
The average time respondents used to complete the survey was
around 14.8 minutes. The survey data collection was completed in
May 2022.
4.3 Sample
We received a total of 432 responses, while 291 were complete and
also passed our attention check questions. Please note that we ran-
domized only one of the punishment types that respondents typed
in for customizing the survey questions for them (see survey design
in Section 4.1). We thus removed the response of one respondent
out of 291 from further inferential statistical analysis because that
respondent was the only one assigned to the punishment, “warn-
ing,” which was only manually typed in by three respondents (see
detail in Section 5.2). So here, we use a total of 291 valid responses
for descriptive statistical purposes, while later, we will use 290
responses for referential statistical analysis. Table 1 summarizes
the demographic information of the 291 respondents. Most play-
ers (65.64%) experienced restricted access to game features (e.g.,
chat ban, matchmaking restrictions) and temporary account ban.
52.92% of players experienced temporary ban, and 12.37% of players
experienced permanent or IP ban. Since many participants could
report more than one type of punishment, the sum percentage of
punishments experienced exceeds 100%, as shown in “Punishments
Experienced” in Table 1.
4.4 Data Analysis
We performed exploratory factor analysis (EFA), conrmatory fac-
tor analysis (CFA), and structural equation modeling (SEM) through
Mplus, a statistical modeling program for researchers to analyze
data. EFA was run rstly to check whether the factor structure
we drew from prior work t our survey data. In EFA, we used a
robust weighted least-square estimator (WLSMV) and an oblique
Geomin rotation method. The WLSMV estimator is better for or-
dered categorical indicators because it does not assume data in
the factors to be normally distributed. After we got a valid factor
structure from EFA, we further ran CFA to tone and build the nal
measurement model for the factors. We used the WLSMV estimator
again in CFA and tested the convergent and discriminant valid-
ity of factors. Convergent validity will be supported if indicators
(i.e., survey items) load signicantly on the corresponding factor
with standardized factor loading greater than 0.6, the Average Vari-
ance Extracted (AVE) higher than 0.5, and Cronbach’s Alpha > 0.6.
Discriminant validity will be supported if the correlation between
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
factors is smaller than 0.85 and smaller than the square root of the
AVE of each factor.
Based on the measurement model from CFA, we ran two SEMs
with a WLSMV estimator to answer RQ1 and RQ2, respectively.
SEM ts the measurement model and a set of linear regressions
between factors. In the rst SEM, we included all three indepen-
dent variables, including punishment notication, explanation, and
punishment types, as well as nine dependent variables, including
a group of perceived fairness factors and coping strategy factors
(we removed the “wishful thinking” factor after EFA, which was
detailed in Section 5.1) and perceived transparency, as shown in
Figure 1. The second SEM analysis involved ve independent vari-
ables, including a group of perceived fairness factors and perceived
transparency, as well as four coping strategy factors, as shown in
Figure 2.
5 FINDINGS
This section will discuss how our ndings answer RQ1 and RQ2. An-
swering RQ1, we found that punishment notication and explana-
tions both signicantly inuenced players’ perceived transparency
of behavior moderation, and explanation provision signicantly im-
proved all notions of perceived fairness. However, nearly all facets
of punishment design, including notication, explanation, and pun-
ishment types, did not aect how players cope with punishments
with one exception. Explanation provision signicantly aected
players to adopt problem coping strategy. Answering RQ2, we
found that both perceived transparency and fairness signicantly
aected players’ adoption of coping strategies for punishments,
while fairness notions like retributive, procedural, and restorative
justice played more critical roles in aecting players to adopt more
types of coping strategies than perceived transparency or outcome
fairness. Additionally, in a casual inference logic, we found that ex-
planation provision can only aect players to adopt problem coping
strategy when they perceive behavior moderation as transparent.
5.1 Measurement Models
We ran Exploratory Factor Analysis (EFA) on all 10 factors that we
adapted from prior work (see Section 4.3) to test whether our survey
data would support this 10-factor structure. EFA results showed
that a 9-factor structure had a better model t than the original 10-
factor solution. In the 10-factor solution, all four items of wishful
thinking had signicant cross-loadings on social support factor.
Also, the factor loadings of these four items on wishful thinking
were not at least two times higher than the loadings on the other
factors. In other words, the items of wishful thinking did not
converge into a single factor. Thus, based on EFA results, we
did not include wishful thinking for further analysis. We
correspondingly removed all hypotheses within H3 and H4 that
involve wishful thinking (e.g., H4.1(e), H4.2(e)). The results of EFA
further helped us remove a total of four items from three factors.
In detail, the third item of procedural justice had signicant cross-
loadings on perceived transparency. Also, the rst item of positivity,
as well as the rst and second items of detachment encountered, all
had signicant cross-loadings. Thus, we removed these four items
to run CFA, as we applied strikethrough to them in Table 2.
Our construct had acceptant t indices (RMSEA
=
0.054, which
is acceptable between 0.05 and 0.08 [
23
], 90% CI: [0.048, 0.058], CFI
=
0.975 > 0.95, TLI
=
0.972 > 0.95). Thus, the CFA results indicate
that our construct has acceptable goodness of t. The chi-square
statistics were signicant (
𝜒2=
1168.841, df
=
629, p<.001). Usu-
ally, a chi-square test with a p-value greater than 0.05 (i.e., non-
signicance) shows a good model t, and our results were contrary
to what we expected. However, researchers (e.g., [
9
]) have broadly
questioned the appropriateness of using chi-square test alone to
evaluate the overall model le because it is sensitive to study’s sam-
ple size and construct complexity. Thus, we alternatively used Root
Mean Square Error of Approximation (RMSEA), Comparative Fit
Index (CFI), and Tucker–Lewis index (TLI) together [
9
] to describe
the goodness-of-t of our construct. Our construct had acceptant
t indices (RMSEA
=
0.054, which is acceptable between 0.05 and
0.08 [
23
], 90% CI: [0.048, 0.058], CFI
=
0.975 > 0.95, TLI
=
0.972 >
0.95).
The convergent and discriminant validity of our measurement
model is supported. First, nearly all factor loadings are greater than
the requisite threshold of 0.6 [
97
] (see “factor loading” column in
Table 2). One exception is the third item of detachment, which is
smaller than 0.6. Therefore, we removed this item to run CFA again,
and we strikethrough it in Table 2. We also reported each latent vari-
able’s Average Variance Extracted (AVE) and Composite Reliability
(CR). Nearly all the AVEs are greater than 0.5, and CRs are greater
than 0.6, which indicates good convergent validity. One exception is
the AVE of detachment, which is 0.45 below the 0.5 threshold. How-
ever, the Composite Reliability (CR) of detachment is greater than
0.6, so the convergent validity of our construct/measurement model
is still adequate [
29
]. Besides, discriminant validity is supported
for our measurement model. The correlations between factors are
not only smaller than 0.85 but also smaller than the square root of
AVEs, as shown in Table 2.
5.2 Punishment Design
Given the variety of punishments each participant experienced,
we randomly assigned one of the multi-selected punishments to
further probe the context of the punishment. The most frequent
randomly assigned punishments were restricted access to game
features and temporary account ban, as shown in Table 4. Warning,
as a type of punishment, was assigned to only one participant who
chose to freely specify the punishment experienced, which was
“warning.” Thus, to make sure each punishment has enough data
points for the Structural Equation Modeling (SEM) analysis, we
removed this individual response. Our nal dataset then contained
a total of 290 valid responses for further analysis.
Every participant answered the game where they experienced
the randomly assigned punishment (see “Game Platform” column in
Table 4). Many participants mentioned the game platforms beyond
the options we provided. For example, they experienced restricted
access to game features in games of Ancient Anguish, Homeworld,
Lineage 2, and more. Also, others reported their temporary ac-
count ban happened in Alliance of Heroes, Gears of war, Magic:
The Gathering Online, Ragnarok Online, Sea of Thieves, and more.
After they answered the game platform questions, the rest of the
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
Table 2: Correlations between factors and the square root of AVEs. Note: each cell is the correlation coecient between two
factors with
∗
p < 0.05,
∗∗
p < 0.01,
∗∗∗
p < 0.001. All correlation coecients are smaller than the corresponding square root of
AVEs.
1 2 3 4 5 6 7 8 9
1Transparency
2Outcome Fairness 0.645***
3Retributive Justice 0.537*** 0.608***
4Procedural Justice 0.626*** 0.755*** 0.657***
5Restorative Justice 0.23*** 0.12 ns 0.3*** 0.165**
6Problem Coping 0.395*** 0.368*** 0.479*** 0.371*** 0.389***
7Detachment 0.198** 0.171** 0.32*** 0.325*** 0.253*** 0.239***
8Social Support -0.163 -0.209 -0.102 -0.239 0.217*** 0.078 -0.016
9Positivity 0.287*** 0.394*** 0.443*** 0.471*** 0.294*** 0.598*** 0.396*** 0.108
The square root of AVE 0.867 0.951 0.847 0.858 0.804 0.824 0.670 0.849 0.879
questions were automatically customized by the game name they
selected/typed.
As Table 5 shows, many players reported they received punish-
ment notications and explanations. 248 out of 291 participants
reported that they were notied of the punishment by the games.
These participants received notications by emails and pop-up
windows or in-game messages. Also, 217 out of 291 participants
reported that they received punishment explanations. However, 74
out of 291 participants reported they did not receive or were not
sure if receiving explanations. 26 out of 291 participants reported
that game platforms did not explain why punishment happened to
them even though receiving notications. Furthermore, seven out
of 291 participants reported that they did not receive punishment
notications but received explanations. Their responses to open-
ended questions indicated that many of them found explanations
when logging into game, while the game did not notify them of
punishment beforehand. For example, they said: “When I tried to
log in one day, it told me that I was temporarily suspended from
logging in to my account.” Another similar response was, “Could
not login or play upon trying.” These initial qualitative ndings
prompted us to dive deeper to understand how punishment design
like notication and explanation would aect the ways how players
perceive behavior moderation and moderation punishments.
5.3 RQ1: Punishment Design →
Transparency/Fairness Perceptions &
Coping Strategies
To answer RQ1 and its subsequent RQ1.1 to RQ1.3, we built an
SEM to model the hypothesized relationships between dierent
types of punishment designs (punishment types, notication, and
explanation) and (1) perceived transparency, (2) perceived fairness,
and (3) intent to adopt coping strategies. This SEM model has a
good model t:
𝜒
2
=
1331.714, df
=
789, p<.001; RMSEA
=
0.049 <
0.05 [
23
], 90% CI: [0.044, 0.053], CFI
=
0.969 > 0.95, TLI
=
0.964 >
0.95.
Table 6 summarizes the SEM results, as well as whether each
hypothesis is fully or partially supported (see “Results” column).
Overall, punishment types do not have a signicant association
with players’ perceived transparency and fairness or their coping
strategies. Providing punishment notication and explanation gen-
erally has positive eects on players’ perceived transparency and
fairness. We will elaborate on these results in the next subsections.
5.3.1 RQ1.1: Punishment Design
→
Perceived Transparency of Be-
havior Moderation. The eects of notication and explanation pro-
vision on participants’ perceived transparency of behavior mod-
eration are signicantly positive. This supports H1.1 and H1.2.
When participants are provided with punishment notications, they
consider behavior moderation as more transparent (
𝛽=
0.534***).
When participants are provided with punishment explanations, they
consider behavior moderation as more transparent (
𝛽=
1.537***).
Especially, the size of the coecient of punishment explanation pro-
vision is greater than the one of notication provision, indicating
punishment explanation plays a greater role in improving the per-
ceived transparency than notication. Besides, punishment types
do not signicantly aect perceived transparency. Thus, H1.3a-c
are not supported.
5.3.2 RQ1.2: Punishment Design
→
Perceived Fairness of Behavior
Moderation. The eects of explanation provision on outcome fair-
ness (H2.2a), retributive justice (H2.2b), procedural justice (H2.2c),
and restorative justice (H2.2d) are all signicantly positive. Besides,
neither notication provision nor punishment types have a signi-
cant eect on any dimension of perceived fairness. H2.3(1-3&a-d)
and H2.1a-d are thus not supported. In sum, punishment expla-
nations play an important role in aecting whether players perceive
behavior moderation as fair, while notication and punishment
types do not aect, which answers RQ1.2.
5.3.3 RQ1.3: Punishment Design
→
Coping Strategies for Punish-
ments. Nearly no punishment design has signicant eects on play-
ers’ coping strategies for punishments, with one exception. When
games provide explanations, players are more likely to initiate
“problem coping” to address the punishments (
𝛽=
0.55**), sup-
porting H3.2a. Other than this, transparency eorts from games,
such as providing punishment notications or explanations, gen-
erally did not motivate players to cope with the punishments by
adopting coping strategies such as seeking social support, focusing
on the positive, or detachment. Besides, punishment types are not
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
Table 3: Factor loadings of the factors of punishment experience (CFA results). Note: since the survey questions were based
on one random punishment participants experienced, [game] below represents where they experienced that punishment.
Strikethrough refers to a survey item that has signicant cross-loadings on other factors in EFA results.
Factors (AVE CR) Survey Items Factor
Loadings
Transparency [19, 37]
(AVE=0.751 CR=0.937)
Overall, [game] tries to be transparent on punishment decisions. 0.921
In general, I am notied about punishments from [game]. 0.837
It is easy for me to see the status of punishments. 0.726
I am being told the reason behind punishments. 0.934
Players like me are provided with information that is relevant to punishments.
0.899
Perceived
Fairness
Outcome Fairness [16, 37]
(AVE=0.905 CR=0.974)
The punishments I got so far in [game] are fair so far. 0.962
[game]’s punishment decisions are appropriate. 0.938
The punishments I experienced are proportional to what I have done. 0.954
[game] gave me the punishments I deserved. 0.951
Retributive Justice [102]
(AVE=0.717 CR=0.927)
Overall, as a matter of justice, the convicted players should be punished. 0.871
Justice is served at the moment that the convicted players are punished in
[game].
0.844
The only way to restore justice is to punish the convicted players. 0.86
The convicted players deserve to be penalized. 0.876
For the sake of justice, some degree of suering has to be inicted on the
convicted players.
0.78
Procedural Justice [75]
(AVE=0.736 CR=0.917)
The punishment decisions are made by [game] in an unbiased manner. 0.811
To make the punishment decisions, [game] collects accurate and complete
information.
0.903
[Game] claries punishment decisions and provides additional information
when requested by players
0.613
All punishment decisions are applied consistently across all aected players. 0.804
The decision-making process of punishments has followed ethical and moral
standards.
0.908
Restorative Justice [102]
(AVE=0.646 CR=0.916)
For justice to be reinstated, [game] needs to achieve agreement about the
values violated by the players.
0.87
To restore justice, the players and [game] need to rearm consensus on the
values and rules.
0.833
Without the players’ sincere acknowledgment of having acted inappropriately,
the injustice is not completely restored.
0.669
A sense of justice requires that the players and [game] develop a shared
understanding of the harm done by players’ behaviors.
0.785
Justice is restored as soon as the player has learned to endorse the values
violated by their behaviors.
0.785
For a sense of justice, players and [game] need to rearm the belief in shared
values.
0.864
Coping
strategies
Problem Coping [83]
(AVE=0.679 CR=0.913)
I know how to improve my behavior to avoid punishments in the future. 0.833
I try to analyze punishments in order to understand them better. 0.748
I could make a plan of action and follow it to improve my behaviors. 0.902
I could come up with a couple of dierent solutions to punishments. 0.754
I could analyze the punishments in order to understand them better. 0.871
Detachment [83]
(AVE=0.449 CR=0.619)
After punishments, I usually continue playing [game] as if nothing happened.
0.445
I was just unlucky to be punished by [game]. 0.536
I try to forget about the punishment I received. 0.158
I feel that time will make a dierence; the only thing to do is wait 0.652
I usually wait to see what will happen from punishments before I do anything
0.687
Social Support [83]
(AVE=0.721 CR =0.911)
I tend to talk to someone about the punishments I experience in [game]. 0.8
I tend to ask someone I trust for advice about the punishments. 0.973
I tend to talk to someone who could do something concrete for my
punishments.
0.843
I want to receive sympathy and understanding from someone. 0.765
Positivity [83] (AVE=0.772
CR=0.910)
Overall, I tend to focus on the positive after punishments. 0.703
The punishment can actually inspire me to do something positive. 0.816
The punishment encouraged me to discover what is important in playing
[game].
0.918
The punishment causes me to grow or change in a good way. 0.899
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
Table 4: Punishment design: The randomly assigned punishment and game where participants experienced it.
Randomly Assigned Punishments (Quantity, Percent) Game Platforms Quantity
Warning (1, 0.34%) Roblox 1
Content or item removal (11, 3.78%) World of Warcraft (WoW) 3
Apex Legends 1
Final Fantasy 1
Fortnite 1
Grand Theft Auto 1
League of Legends 1
Minecraft 1
Roblox 1
Runescape 1
Restricted access to game features (138, 47.42%) League of Legends 18
Other game 16
Fortnite 11
Apex Legends 10
World of Warcraft (WoW) 9
Minecraft 8
Overwatch 8
Dead By Daylight 7
Rocket League 7
Dota 2 6
Call of Duty 5
Runescape 5
Halo 4
Final Fantasy 3
Rainbow Six Siege 3
Valorant 3
Counter Strike Global Oensive (CS:GO) 2
Destiny 2 2
Smite 2
Splatoon 2 2
World of Tanks 2
Battleeld 1
Club Penguin 1
New World 1
Pokémon go 1
Team fortress 2 1
IP ban or Permanent account ban (20, 6.78%) Other game 4
Minecraft 2
PUBG 2
Apex Legends 1
Battleeld 1
Call of Duty 1
Club Penguin 1
Counter Strike Global Oensive (CS:GO) 1
Fortnite 1
Gaia Online 1
League of Legends 1
NBA 2k 1
Runescape 1
Team fortress 2 1
World of Warcraft (WoW) 1
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
Randomly Assigned Punishments (Quantity, Percent) Game Platforms Quantity
Temporary account ban (122, 41.92%) Other game 27
League of Legends 21
World of Warcraft (WoW) 21
Fortnite 11
Minecraft 8
Call of Duty 6
Apex Legends 4
Counter Strike Global Oensive (CS:GO) 3
Halo 3
Dota 2 2
Final Fantasy 2
Grand Theft Auto 2
Overwatch 2
Runescape 2
Club Penguin 1
Gaia Online 1
NBA 2K 1
Pokémon go 1
PUBG 1
Rainbow Six Siege 1
Roblox 1
Table 5: Punishment design: Self-reported punishment noti-
cation and explanation provision.
Explanation Provision
Yes No
Not sure Total
Yes
204
26 18 248
Notication
Provision
No 7 20 2 29
Not sure 6 2 6 14
Total
217
48 26 291
key factors for players to decide on coping strategies. H3.1a-d,
H3.3(1-3&a-d), and H3.2b-d are not supported.
Taken together, to answer RQ1, we found that when either pun-
ishment notications or explanations are oered, players would
consider behavior moderation as transparent. Especially, explana-
tion provision can signicantly improve players’ dierent dimen-
sions of perceived fairness such as restorative and procedural justice
as well as aect players to adopt problem coping actions for pun-
ishments. However, punishment types as one of the punishment
designs did not have signicant eects on either players’ perceived
transparency, fairness, or coping strategies.
5.4 RQ2: Perceived Fairness and Transparency
→Coping Strategies
Our second SEM (as shown in Table 7) has a good model t:
𝜒
2
=
1120.819, df
=
593, p<.001; RMSEA
=
0.055, which is acceptable
between 0.05 and 0.08 [23], 90% CI: [0.05, 0.06], CFI =0.975 > 0.95,
TLI
=
0.972 > 0.95. This model tests whether and how players’
perceived transparency and fairness of behavior moderation aect
players’ coping strategies for punishments.
We answer RQ2 by uncovering that perceived transparency posi-
tively aects problem coping (
𝛽=
0.168*, H4.1a) but does not aect
other coping strategies (H4.1b-d). So, H4.1 is partially supported.
While perceived fairness generally has positive inuences on cop-
ing strategies, perceived outcome fairness (H4.2a-d) does not have
signicant eects on players’ coping strategies. H4.2 thus is not
supported. Other than perceived outcome fairness, perceived re-
tributive justice positively aects problem coping (
𝛽=
0.303***,
H4.3a) and positivity (
𝛽=
0.176*, H4.3d) but does not signicantly
aect social support (H4.3b) and detachment (H4.3c). H4.3 are
thus partially supported. Furthermore, while perceived proce-
dural justice has a positive eect on detachment (
𝛽=
0.337, H4.4c)
and positivity (
𝛽=
0.32**, H4.4d), it has a negative eect on social
support (
𝛽=
-0.223*, H4.4b), which is contrary to what we hypoth-
esized, and does not aect problem coping (H4.4a). Thus, H4.4
is partially supported. Then, perceived restorative justice has a
positive eect on all types of coping strategies, including problem
coping (
𝛽=
0.261***, H4.5a), social support (
𝛽=
0.268***, H4.5b),
detachment (
𝛽=
0.169*, H4.5c), and positivity (
𝛽=
0.204***, H4.5d).
Thus, H4.4 is fully supported.
In sum, answering RQ2, we found that when players perceived
behavior moderation as transparent, they tended to proactively cope
with punishments (i.e., problem coping strategy). When players
think behavior moderation is conducted in an unbiased manner (i.e.,
procedural justice), they are more likely to adopt detachment and
positivity to cope with punishments but less likely to seek social
support. And when players think behavior moderation is conducted
in a correct, punitive manner (i.e., retributive justice), they tended
to adopt problem coping and positivity to cope with punishments.
Last, when players think the game platform arms consensus on
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
Table 6: The rst SEM (hypothesis testing) results for RQ1 (model 1). Note: The solid arrows (
→
) present signicant relationships,
and broken arrows (
→
) represent tested relationships that are non-signicant. (+) or (-) indicates a positive or negative eect
between factors. Coecient 𝛽with ∗p < 0.05, ∗∗ p < 0.01, ∗∗∗p < 0.001. n.s. means non-signicant.
Model 1
RQ Hypothesis# Coef (𝛽) Results
RQ1.1 H1.1 Notication provided →Perceived Transparency (+) 0.543*** Fully support
H1.2 Explanation provided →Perceived Transparency (+) 1.537*** Fully support
H1.3 baseline Content or item removal No support
(a) Restricted access to game features →Perceived Transparency
(+)
0.268 n.s.
(b) Temporary account ban →Perceived Transparency (+) 0.034 n.s.
(c) IP or permanent ban →Perceived Transparency (-) -0.094 n.s.
RQ1.2 H2.1 (a-d) Notication provided →Outcome Fairness (+), Retributive
Justice (+), Procedural Justice (+), Restorative Justice (-)
n.s. No support
H2.2 (a) Explanation provided →Outcome Fairness (+) 0.773*** Fully support
(b) Explanation provided →Retributive Justice (+) 0.475*
(c) Explanation provided →Procedural Justice (+) 0.645**
(d) Explanation provided →Restorative Justice (+) 0.358*
H2.3 (1&a-d) Restricted access to game features →Outcome Fairness (+),
Retributive Justice (+), Procedural Justice (+), Restorative
Justice (+)
n.s. No support
(2&a-d) Temporary account ban →Outcome Fairness (-), Retributive
Justice (-), Procedural Justice (+), Restorative Justice (+)
n.s.
(3&a-d) IP or permanent ban →Outcome Fairness (-), Retributive
Justice (-), Procedural Justice (+), Restorative Justice (+)
n.s.
RQ1.3 H3.1 (a-d) Notication was provided →Problem Coping (-), Social
Support (+), Detachment (-), Positivity (-)
n.s. No support
H3.2 (a) Explanation was provided →Problem Coping (+) 0.55** Partially support
(b-d) Explanation was provided →Social Support (-), Detachment
(+), Positivity (+)
n.s.
H3.3 (1&a-d) Restricted access to game features →Problem Coping (+),
Social Support (-), Detachment (+), Positivity (+)
n.s. No support
(2&a-d)
Temporary account ban
→
Problem Coping (+), Social Support
(-), Detachment (+), Positivity (+)
n.s.
(3&a-d)
IP or permanent ban
→
Problem Coping (+), Social Support (-),
Detachment (+), Positivity (+)
n.s.
its values and rules with them (i.e., restorative justice), they would
be more likely to adopt all coping strategies for punishments.
5.5 Perceived Fairness and Transparency Have
No Mediation Eects with One Exception
Since punishment design aects perceived transparency and fair-
ness and also perceived transparency and fairness aect coping
strategies, it is possible that perceived transparency and fairness
serve as mediators between punishment design and coping strate-
gies. To test this, we ran three more SEMs to test if introducing
mediators (i.e., perceived transparency and fairness) would make
the signicant eects of punishment design on coping strategies
insignicant [
40
]. Since only the eect of explanation on problem
coping was signicant (see RQ1.3 in Table 6), we thus tested the
mediation eect on this path only by introducing (1) perceived
transparency and fairness both as mediators, (2) perceived trans-
parency as the only mediator, and (3) perceived fairness as the
only mediator between punishment design and problem coping.
Since other eects of punishment design on coping strategies are
insignicant, there is no point in testing mediation for these paths.
Model (1) and (3) have poor model ts: RMSEA > 0.1, CFI < 0.9,
and TLI < 0.9, thus, do not support a mediation model. However,
Model (2) has a good model t: RMSEA > 0.054, which is acceptable
between 0.05 and 0.08 [
23
], 90% CI: [0.046, 0.062], CFI
=
0.972 > 0.95,
and TLI
=
0.964 > 0.95. When perceived transparency is introduced
as a mediator between punishment design and coping strategies, the
direct eects of punishment designs on coping strategies, including
the only signicant one (explanation
→
problem coping), are no
longer signicant, indicating that perceived transparency fully
mediates the relationship between explanation and problem
coping (shown in Figure 3). If players are given punishment ex-
planations, they will perceive more transparency of the behavior
moderation and inherently are more likely to improve the adoption
of problem coping strategy for punishments.
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
Table 7: The second SEM (hypothesis testing) results for RQ2 (model 2). Note: The solid arrows (
→
) present signicant
relationships, and broken arrows (
→
) represent tested relationships that are non-signicant. (+) or (-) indicates a positive or
negative eect between factors. Coecient 𝛽with ∗p < 0.05, ∗∗ p < 0.01, ∗∗∗p < 0.001. n.s. means non-signicant.
Model 2
RQ
Hypothesis#
Coef 𝛽Results
RQ2 H4.1 (a) Perceived Transparency →Problem coping (+) 0.168* Partially support
(b-d) Perceived Transparency →Social Support (-), Detachment (+), Positivity (-) n.s.
H4.2 (a-d) Outcome Fairness →Problem coping (+), Social Support (-), Detachment (-),
Positivity (+)
n.s. No support
H4.3 (a) Retributive Justice →Problem coping (+) 0.303*** Partially support
(d) Retributive Justice →Positivity (+) 0.176*
(b,c) Retributive Justice →Social Support (+), Detachment (+) n.s.
H4.4 (a) Procedural Justice →Problem Coping (-) -0.015 n.s. Partially support
(b) Procedural Justice →Social Support (-) -0.223*
(c) Procedural Justice →Detachment (+) 0.337**
(d) Procedural Justice →Positivity (+) 0.32**
H4.5 (a) Restorative Justice →Problem Coping (+) 0.261*** Fully support
(b) Restorative Justice →Social Support (+) 0.268***
(c) Restorative Justice →Detachment (+) 0.169*
(d) Restorative Justice →Positivity (+) 0.204***
Figure 3: Perceived transparency fully mediates the relation-
ship between explanation and problem coping with
∗
p < 0.05,
∗∗ p < 0.01, ∗∗∗p < 0.001.
6 DISCUSSION
We conducted a survey study to understand how players experi-
ence the punishment design of behavior moderation in multiplayer
online games, identifying interrelationships among three facets of
punishment design, players’ perceived fairness and transparency
of moderation decisions and players’ intended adoptions of coping
strategies for punishments. In this section, we will discuss how
our ndings help deepen understanding of behavior moderation in
the context of online games. Then, we will discuss how we should
consider moderation experience as part of player experience and de-
rive practical implications for moderation design and policymaking
from our ndings.
6.1 Extending Understanding of Moderation
Experience in the Context of Online Gaming
Prior work has focused broadly on understanding users’ expe-
riences with moderation systems on social media such as Red-
dit [
44
,
47
], YouTube [
67
,
68
], Facebook [
98
], and more. As our
study showed, game players are also concerned about the issues
such as transparency or fairness of moderation systems that many
HCI researchers have discussed in the social media context (e.g.,
[47, 51, 99]).
First, our ndings helped quantitatively conrm the importance
of punishment notications and explanations to improve the per-
ceived transparency and fairness of moderation in the context of
online gaming. Prior work has found that social media users per-
ceive the opacity of moderation decisions on social media because
they do not receive notications or explanations from platforms
(e.g., [
26
,
67
,
73
,
91
]). When users receive moderation explanations
such as appeal explanations [
98
] or reasons for account suspension
[
44
], their perceived transparency and fairness would be improved.
Resonating with this line of work, we found the positive eects
of punishment notication and explanation on players’ perceived
transparency. Importantly, extending the prior work, we found ex-
planation provision played a more critical role than notication in
aecting both perceived transparency with a larger eect size and
all notions of perceived fairness. That said, when online multiplayer
games construct more transparent, fairer moderation systems, spec-
ifying why players experience punishments would be key to helping
they understand (1) punishment as fair (i.e., outcome fairness), (2)
the punitive logic of moderation system as legitimate (i.e., retribu-
tive justice), (3) the procedures of moderation decision-making as
justied (i.e., procedural justice), and (4) games conrm the rules
and values within the same page with them (i.e., restorative justice).
Second, only punishment explanation from the three punishment
design components directly motivates players to actively cope with
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
punishments and indirectly drives players to do so through per-
ceived transparency. Prior work has uncovered that users might
conduct behavioral or cognitive eorts to avoid or resist punish-
ments on social media (e.g., [
4
,
26
,
91
,
98
]) but did not explicate
why users initiate such coping eorts for punishments. Our study
species one motivation: When players receive punishment expla-
nations, they would be more likely to analyze the punishments,
improve past behaviors, or make a plan to handle both (i.e., problem
coping strategy). So, by moving beyond prior work that stresses
the importance of moderation explanations [
44
,
67
,
98
], we empha-
size that explanation design is important not only because of its
impacts on improving perceived transparency of moderation but
also driving punished users to actively handle the negative eects
of punishments.
Third, punishment types as one of the punishment design compo-
nents do not play a role in aecting the perceived transparency and
fairness as well as intended adoption of coping strategies, which
are somewhat dierent from what we expected. We conjecture that
online gaming culture is dierent from social media platforms in its
closedness from scrutiny of the outside world and corporate owners
enjoying much power in making authoritative decisions that are
rarely challenged, and players are accustomed to this culture and
rarely challenge the severity of punishment [54].
While some HCI research showed that severe moderation deci-
sions such as account suspension impact social media users’ mod-
eration experiences (e.g., fairness perceptions [
44
,
98
]), our results
showed that players paid less attention to punishment severity com-
pared to punishment designs (e.g., notication, explanation). Such
new understanding of moderation experience exactly showed that
in the context of competitive online multiplayer games, where toxic
behaviors are prevalent and can be inuenced by game designs
(e.g., players’ powerless in matchmaking [
55
]), players might have
already normalized toxicity and become insensitive to punishment
severity [
5
,
56
]. This suggests the importance of punishment de-
sign – a design that could inform players of what procedure the
decision-maker, i.e., game platforms, conducts to make punishment
decisions and sequentially how to help players reform behaviors
[
57
] if they truly violate platform policies. So, not like social me-
dia users who request explanations and notications for certain
punishments (e.g., account suspension [
26
,
44
,
98
], revenue deduc-
tion [
67
]), game players purely request explanations to understand
how punishment decisions are made, which directly speaks to the
perceived justice of moderation they desired.
6.2 Foregrounding Justice Notions in
Investigating Moderation Experiences
Although this work is focused on the context of online gaming,
our ndings about perceptions and experiences of justice can form
meaningful conversations with moderation research in other con-
texts, and thus deepen our general understanding of moderation.
First, many HCI researchers have drawn from the notion of pro-
cedural justice to call for moderation system to increase people’s
perceived fairness of moderation decisions [
85
] and increase their
participation in the moderation decision-making process [
25
], as
well as to issue consistent moderation decisions across time, users,
and content policies [
68
]. Connecting with this body of work, which
is focused on social media contexts, we oer a new understand-
ing of users’ actions after they perceived the procedural justice of
moderation in game: Players were more likely to adopt detachment
and positivity but less likely to adopt social support. That means,
punished players treated punishment more as a typical procedure
they would go through rather than an emotionally challenging
event. In contrast, when players perceived little procedural justice
in moderation decision-making (i.e., game platforms oer limited
resources to help players understand the legitimacy of punishment
decision-making), they were more likely to seek help from peers,
i.e., social circles. This nding resonates with prior work that when
punished users on social media consider their voice is not involved
in moderation decision-making, they will ask for community sup-
port to make sense out of or learn about punishments [
26
,
67
]. And
importantly, our ndings convey an important message: designing
moderation procedures that involve punished players’ voice and
participation can not only enhance perceived procedural justice of
moderation but also decrease the chances of the perceived modera-
tion unfairness being generated and disseminated by players.
Moreover, as game and social media platforms typically adopt a
retributive/punitive justice logic on convicted users through punish-
ments [
36
,
56
,
73
], our ndings conrmed its eectiveness. When
players perceived the retributive justice of moderation, believing
that penalties were fairly issued, they would adopt problem coping
and positivity, two out of four coping strategies for punishments,
indicating the fairly good eects of perceived retributive justice on
helping players reect or reform behaviors. Also, players upholding
this notion would adopt more problem coping than focusing on
the positive, meaning that players accept penalty as a problem that
they should address instead of ignoring it. Thus, the punitive logic
still works in inuencing players so that they take punishments
seriously and seek to reform. And such eectiveness of retributive
justice further iterates the benet of taking procedural justice into
account in punishment design [
25
,
53
], as our ndings showed a
high positive correlation between procedural and retributive justice,
compared to other pairs in Table 2.
Beyond designing for both procedural and retributive justice,
our ndings conrmed the importance of restorative justice in
punishment design, which has been advocated by studies of non-
gaming contexts (e.g., [
85
,
98
,
104
]). We found that when players
perceived restorative justice, such as eorts and resources that help
them reform, they were likely to adopt all types of coping strategies.
And importantly, if games only ensure the perceived retributive
and procedural justice but not restorative justice, players will still
look for social support to cope with punishments. That means,
sometimes, social support is important for punished users to better
understand moderation decisions [
26
,
67
] but could occasionally
lead to collective circumventing moderation decisions or gaming
moderation systems [
13
,
34
]. Thus, online games need to ensure
perceived fairness, including perceived retributive, procedural, and
restorative justice, to eectively help players reect and cope with
punishments.
Punished players’ diverse needs for justice, when taken into
consideration together with prior ndings on punished users’ needs
for justice in other game-related and non-gaming contexts (e.g.,
[
44
,
69
,
99
]), raise a critical question regarding general moderation
research and practice – how moderation design could conceive
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
punished users as an important stakeholder group. From platform’s
perspective, punished players are deemed to be oenders who
violate platform policies (e.g., code of conduct). And researchers
would leverage this perspective to design moderation justice that
values oenders’ participation and voice (e.g., [
85
,
104
]). While
from punished players’ perspectives, the power imbalance between
game platforms and players in punishment decision-making is
apparent, where players experience punishment and bear with
its negative impacts on their player experience. Especially, as our
ndings showed that games did not explain well what and why
players were accused of, these punished players would be socially
stigmatized with a label or stereotype of toxic players or oenders
[
56
]. However, like many other social media users, players might
encounter hardships of contesting punishment decisions [
69
,
98
,
99
]
and justifying the punishments are legitimate on their own force
[
79
]. Thus, users are less motivated to put eort into clearing their
name, if they perceive the punishment decision-making as lacking
in justice.
Although sometimes, punished users can nd social support to
make sense of punishments, this is still extra labor and could be
attributed to inadequate and ineective punishment design. Like
users’ behaviors in audio-based communities [
50
], players’ behav-
iors might be complex and nuanced, that voluntary human moder-
ators nd tricky to adjudicate. Game publishers could do more to
educate and instruct punished players, as more researchers have
called for platform moderation to take more responsibilities such
as incorporating education in moderation (e.g., [
47
,
73
]). Our nd-
ings pointed out a pragmatic way – designing better punishment
explanations. That is because, as we found, without sucient or
informative explanations, players would not consider behavior mod-
eration as fair in terms of all justice notions, including procedural,
retributive, and restorative justice. But currently, relatively little
work has started to design moderation explanations except for sev-
eral situated in the social media context [
47
,
98
], so we call for
more HCI researchers’ attention on explanation design for more
transparent and fairer moderation in broader contexts.
6.3 Moderation Experience as Part of Player
Experience
Player experience (PX) research has growing attention to toxicity
in online games, as well as moderation techniques that could curb
toxicity [
2
,
5
,
55
,
89
]. Moderation experience is the other side of the
coin, concerning how those moderation techniques impact players
who are considered as toxic. In this regard, moderation experience
unambiguously belongs to player experience. When players engage
in online games, they interact with numerous, interlocking systems,
among which some govern the core gameplay, some manage in-
terpersonal communication, coordination, and teaming, and some
control behavior moderation systems. Although not part of the
core gameplay of a game, moderation experience cuts across many
facets of PX, such as social experiences (e.g., a player is temporarily
losing the ability to communicate with a chat restriction or seeks
social support from their fellow players), emotional experiences
(e.g., a player is frustrated due to not understanding an account
suspension), and player engagement (e.g., a player is no longer able
to play if their account is suspended).
Importantly, the purpose of our study is not to refute the neces-
sity and legitimacy of behavior moderation in multiplayer online
games. Rather, we are to identify punished players as a unique
player group that needs more scholarly and design attention. In-
deed, we see many connections between players’ interactions with
punishments and PX. When put in an adverse situation (i.e., being
punished), players have emergent needs. The self-determination
theory (SDT), widely used in HCI game research [
95
], establishes
three core needs as autonomy, competence, and relatedness. The
SDT holds relevance for us to understand moderation experience
in our study. First, the punished player has the ‘autonomy’ need
as they want to make decisions on their own, actively addressing
the problem of moderation penalty. Certainly, punishment design
aects this autonomy need. For example, the design of modera-
tion explanation could enhance players’ autonomy and encourage
them to take the problem-solving route; and better procedural
justice in punishment design could facilitate certain aspects of au-
tonomy while inhibit others. Second, the punished player has the
‘competence’ need, clearly shown in our ndings about players’
desire for more punishment information. Information helps grow
their competence in areas such as knowledge about moderation
decision-making processes, as well as normative standards for in-
game behavior. Third, the punished player has the ‘relatedness’
need, manifest in the social support coping strategy where people
are punished and subsequently turn to others for social support.
However, such relatedness need could be less if there is sucient
procedural justice so that players could count on the system to
make the right decisions.
6.4 Implications for Design: Rethinking
Moderation Design in Multiplayer Online
Games
Multiplayer online games usually follow a rudimentary, punitive
model in player behavior moderation, issuing a penalty and expect-
ing the punished player to either reform or leave the game. As a
pertinent example, the Fair Play Alliance [
76
], a global coalition
of game companies working together to promote healthy and safe
gaming environments, frames their primary solution to toxicity in
languages such as planning and building “a penalty & reporting
system” in their recent Disruption and Harms in Online Gaming
Framework [
24
]. The punitive model has severe limitations in such
dimensions as transparency and fairness [
47
,
98
], as demonstrated
by moderation researchers (most often in the context of social me-
dia moderation). Bridging the moderation literature and the HCI
game literature, our work points to the importance of moderation
design, especially in terms of providing explanations and notica-
tions. Without sucient information, it could be challenging for
players to understand why they are punished or to act accordingly.
As a result, simply sending a moderation penalty fails to realize the
full potential of creating a teachable moment [
72
] for players who
have committed toxicity, rendering a poor player experience.
More problematic is the situation when the moderation decision
is unjust, but the punished player has nowhere to resort to. Since
justice perceptions such as fairness and transparency aect players’
coping strategies, it is reasonable to assume that perceived injustices
in moderation decisions reversely aect players’ coping actions. In
CHI ’23, April 23–28, 2023, Hamburg, Germany Renkai Ma et al.
other words, moderation design’s insucient information provision
lowers players’ fairness and transparency perceptions, which in
turn could reduce their willingness and action to improve their
future in-game behaviors. Our ndings provided empirical support
for this observation: when perceived transparency and fairness are
low, players would count on their fellow players for help, but better
transparency and fairness could enable players to seriously consider
their penalties and take actions to reform their future behavior.
Moving beyond the simplistic moderation design, we could re-
think punishment design by drawing inspirations from game design.
Video games are known for presenting players with a challenge in
game and then supporting players to overcome it [
52
]: To defeat
the nal boss, the players are well prepared through the accumu-
lation of experience points, equipment, and the improvement in
game mechanics and knowledge (i.e., the needs of autonomy and
competence). In multiplayer online games, players team up with
others to accomplish a larger goal (i.e., the need for relatedness).
But if we consider punishment as a challenge, then players are left
on their own to cope with the challenge. Clearly, there is a large gap
where player needs could be met if moderation design utilizes what
we have already learned in PX about helping players to overcome
challenges. By drawing this analogy, we suggest that punishment
could be productively reframed as a challenge and call for better
design that could help players to overcome this challenge.
Specically, our ndings highlight several dimensions to rethink
moderation design in multiplayer online games: First, the informa-
tional dimension deals with what information should be provided
alongside a punishment and in what way. Our ndings showed
that if players do not understand why they are punished, they
could struggle to improve. In cases where they are wrongly con-
victed, access to the rationale behind their punishment is even more
important. Our ndings suggested that informational provision sig-
nicantly impacts players’ experiences with moderation decisions.
Explanation provisions could become a teachable moment and trig-
ger players’ subsequent actions of problem-solving. Thus, behavior
moderation systems could consider providing explanations when
issuing a penalty. Specically, explanation design should also con-
sider the level of granularity and detailedness. When an explanation
only refers to a vague community guideline, players would still
struggle to analyze their deeds [
57
]. Explanation design could in-
clude both high-level pointers to specic policies violated as well
as precise mappings between the policies and specic player behav-
iors in question. It could also be helpful if players are encouraged
to discuss their penalties with fellow community members through
collective sensemaking.
Second, the social dimension deals with players’ relatedness
needs. Our ndings found several occasions where players would
turn to social support as a coping strategy. When players are pun-
ished, they could be empowered to connect with fellow players to
better gure out how to overcome the negative event.
Third, the temporal dimension takes a developmental view of
players’ moderation experience. While most moderation design
stops at issuing a penalty, it is where punished players start to expe-
rience, feel, and react. These experiences are currently unaccounted
for in the moderation design and thus a missed opportunity. Thus,
restorative means could be designed around existing moderation
systems. For example, various forms of player support could be
designed for punished players. More mechanisms could be built
where punished players could be connected to helpful resources
that help them learn behavioral standards and other community
members who are willing to oer social support.
7 LIMITATIONS AND FUTURE WORK
Our study does not aim to dene what punishment design compo-
nents, including punishment explanation and notication as well as
punishment types, look like in real games, especially as our survey
respondents reported many online games. Even though we used
explicit language like “notication” and “explanation (i.e., reasons),”
we did acknowledge that players situated in dierent online games
might conceptualize punishment design dierently. Thus, future
work could explore and co-design with punished players to under-
stand what consists of a moderation explanation or notication
that can better center around players’ best interests. Also, even
though our sample size t the minimum standard (e.g., n>200) for
running factor analysis and SEM [
32
], we did recognize possibilities
for further work to study with more players.
We did not aim to assess whether and how players’ perceptions
and actions would be dierent among dierent game genres or
types, because existing literature has recognized that there lacks a
consensus on a taxonomy of game genres, and game categorization
methods might contain certain subjectivity or lack clarity [
15
,
42
].
For example, people might consider Overwatch either a shooter
or a ght game. Or it is also hard to categorize whether Call of
Duty as a shooter game or action-adventure game. If we categorize
games and conduct the comparison, it will remain questionable if
game category true-positively dierentiates players’ perceptions or
actions. But we do recognize a future work possibility by rst cate-
gorizing game genres systematically and then examining whether
game genres inuence players’ perceptions and actions.
8 CONCLUSION
Player behavior, usually a combination of in-game language and
avatar action, presents enormous challenges to behavior moder-
ation systems. Penalties issued from moderation systems impact
PX in profound ways but remain poorly understood. We thus con-
ducted a survey study with 291 players to understand how they
perceive and intend to behave around moderation systems in online
multi-player games to obtain a clearer understanding of how to
design more transparent and fairer punishment experiences. We
found that compared to moderation notication, explanation plays
a more critical role in improving players’ perceived transparency
and fairness of moderation. Also, compared to the perceived trans-
parency, the perceived fairness more signicantly aected players
to adopt dierent coping strategies for punishments. As we found
the importance of punishment explanation to perceived fairness,
we emphasize the indirect role of explanation provision to support
players in coping with punishments. These ndings not only extend
the understanding from prior moderation literature that frequently
focuses on the social media context but also help frame moderation
experience as part of player experience and rethink moderation
design in online multiplayer game.
Transparency, Fairness, and Coping: How Players Experience Moderation in Multiplayer Online Games CHI ’23, April 23–28, 2023, Hamburg, Germany
ACKNOWLEDGMENTS
This work is partially supported by NSF grant no. 2006854. We
appreciate all anonymous reviewers’ constructive feedback to make
this work rened and improved. We also appreciate 291 players’
participation in this survey study.
REFERENCES
[1]
Leigh Achterbosch, Charlynn Miller, and Peter Vamplew. 2017. A taxonomy of
griefer type by motivation in massively multiplayer online role-playing games.
Behaviour & Information Technology 36, 8 (August 2017), 846–860. DOI:https:
//doi.org/10.1080/0144929X.2017.1306109
[2]
Sonam Adinolf and Selen Turkay. 2018. Toxic Behaviors in Esports Games: Player
Perceptions and Coping Strategies. In Proceedings of the 2018 Annual Symposium
on Computer-Human Interaction in Play Companion Extended Abstracts - CHI
PLAY ’18 Extended Abstracts, ACM Press, New York, New York, USA, 365–372.
DOI:https://doi.org/10.1145/3270316.3271545
[3]
Aaron Alford. 2021. FIFA streamer loses his mind after being perma banned by
EA live on Twitch. invenglobal.
[4]
Anna Veronica Banchik. 2020. Disappearing acts: Content moderation and emer-
gent practices to preserve at-risk human rights–related content. New Media Soc
(March 2020), 146144482091272. DOI:https://doi.org/10.1177/1461444820912724
[5]
Nicole A. Beres, Julian Frommel, Elizabeth Reid, Regan L. Mandryk, and Madison
Klarkowski. 2021. Don’t you know that you’re toxic: Normalization of toxicity in
online gaming. Conference on Human Factors in Computing Systems - Proceedings
(May 2021). DOI:https://doi.org/10.1145/3411764.3445157
[6]
Lauren Bergin. 2021. Epic permanently bans renowned Fortnite leaker HYPEX
for incident 3 years ago. dexerto.com.
[7]
Apurba Biswas. 2022. Free Fire Max: Garena permanently bans over 1.7M ac-
counts for cheating in-game. insidesport.in.
[8]
Lindsay Blackwell, Jill Dimond, Sarita Schoenebeck, and Cli Lampe. 2017.
Classication and its consequences for online harassment: Design insights from
HeartMob. Proc ACM Hum Comput Interact 1, CSCW (November 2017), 1–19.
DOI:https://doi.org/10.1145/3134659
[9]
Kenneth A. Bollen and J. Scott Long. 1993. Testing Structural Equation Models.
Sage.
[10]
Jonathan E. Bone and Nichola J. Raihani. 2015. Human punishment is mo-
tivated by both a desire for revenge and a desire for equality. Evolution
and Human Behavior 36, 4 ( July 2015), 323–330. DOI:https://doi.org/10.1016/J.
EVOLHUMBEHAV.2015.02.002
[11]
Ragnhild Brøvig-Hanssen and Ellis Jones. 2021. Remix’s retreat? Content
moderation, copyright law and mashup music: New Media Soc (June 2021).
DOI:https://doi.org/10.1177/14614448211026059
[12]
Jie Cai, Donghee Yvette Wohn, and Mashael Almoqbel. 2021. Moderation Visi-
bility: Mapping the Strategies of Volunteer Moderators in Live Streaming Micro
Communities. In ACM International Conference on Interactive Media Experiences.
Association for Computing Machinery, New York, NY, USA, 61–72.
[13]
Stevie Chancellor, Jessica Pater, Trustin Clear, Eric Gilbert, and Munmun De
Choudhury. 2016. #thyghgapp: Instagram Content Moderation and Lexical Vari-
ation in Pro-Eating Disorder Communities. In Proceedings of the 19th ACM Con-
ference on Computer-Supported Cooperative Work & Social Computing - CSCW ’16,
ACM Press, New York, New York, USA. DOI:http://dx.doi.org/10.1145/2818048.
2819963
[14]
Lars Thøger Christensen and George Cheney. 2015. Peering into Transparency:
Challenging Ideals, Proxies, and Organizational Practices. Communication The-
ory 25, 1 (February 2015), 70–90. DOI:https://doi.org/10.1111/COMT.12052
[15]
Rachel Ivy Clarke, Jin Ha Lee, and Neils Clark. 2017. Why Video Game Genres
Fail: A Classicatory Analysis. Games Cult 12, 5 ( July 2017), 445–465. DOI:https:
//doi.org/10.1177/1555412015591900
[16]
Jason A. Colquitt. 2001. On the dimensionality of organizational justice: A
construct validation of a measure. Journal of Applied Psychology 86, 3 (June
2001), 386–400. Retrieved from https://psycnet.apa.org/buy/2001-06715-002
[17]
Christine L. Cook. 2019. Between a Troll and a Hard Place: The Demand Frame-
work’s Answer to One of Gaming’s Biggest Problems. Media Commun 7, 4
(December 2019), 176–185. DOI:https://doi.org/10.17645/MAC.V7I4.2347
[18]
Julian Dibbell. 1994. A Rape in Cyberspace or How an Evil Clown, a Haitian
Trickster Spirit, Two Wizards, and a Cast of Dozens Turned a Database into a
Society. Annu Surv Am Law (1994).
[19]
Alexandra Durcikova and Peter Gray. 2014. How Knowledge Validation Pro-
cesses Aect Knowledge Contribution. Journal of Management Information Sys-
tems 25, 4 (April 2014), 81–108. DOI:https://doi.org/10.2753/MIS0742-1222250403
[20]
Motahhare Eslami, Karrie Karahalios, Christian Sandvig, Kristen Vaccaro, Aimee
Rickman, Kevin Hamilton, and Alex Kirlik. 2016. First I “like” it, then I hide
it: Folk theories of social feeds. In Proceedings of the 2016 CHI Conference on
Human Factors in Computing Systems, Association for Computing Machinery,
New York, NY, USA, 2371–2382. DOI:https://doi.org/10.1145/2858036.2858494
[21]
Pouyan Esmaeilzadeh. 2019. The Impacts of the Perceived Transparency of
Privacy Policies and Trust in Providers for Building Trust in Health Information
Exchange: Empirical Study. JMIR Med Inform 7, 4 (November 2019). DOI:https:
//doi.org/10.2196/14050
[22]
Peer Eyal, Rothschild David, Gordon Andrew, Evernden Zak, and Damer
Ekaterina. 2021. Data quality of platforms and panels for online behavioral
research. Behav Res Methods 54, 4 (September 2021), 1643–1662. DOI:https:
//doi.org/10.3758/S13428-021- 01694-3/TABLES/13
[23]
Leandre R. Fabrigar, Robert C. MacCallum, Duane T. Wegener, and Erin J.
Strahan. 1999. Evaluating the use of exploratory factor analysis in psycho-
logical research. Psychol Methods 4, 3 (September 1999), 272–299. DOI:https:
//doi.org/10.1037/1082-989X.4.3.272
[24] Fair Play Alliance. 2020. Disruption and Harms in Online Gaming Framework.
[25]
Jenny Fan and Amy X. Zhang. 2020. Digital Juries: A Civics-Oriented Approach
to Platform Governance. In CHI Conference on Human Factors in Computing
Systems Proceedings (CHI 2020), Association for Computing Machinery, New
York, NY, USA, 1–14. DOI:https://doi.org/10.1145/3313831.3376293
[26]
Jessica L. Feuston, Alex S. Taylor, and Anne Marie Piper. 2020. Conformity of
Eating Disorders through Content Moderation. Proc ACM Hum Comput Interact
4, CSCW1 (May 2020). DOI:https://doi.org/10.1145/3392845
[27]