Designing Leaderboards for Gamification:
Perceived Differences Based on User Ranking,
Application Domain, and Personality Traits
Yuan Jia, Yikun Liu, Xing Yu
School of Informatics and Computing
Indiana University–Indianapolis (IUPUI)
Indianapolis, IN 46202 USA
{jiayuan, yikliu, yu64}@umail.iu.edu
Stephen Voida
Department of Information Science
University of Colorado Boulder
Boulder, CO 80309 USA
svoida@colorado.edu
ABSTRACT
Leaderboards, a common gamification technique, are used to
enhance engagement through social comparisons. Prior
research has demonstrated the overall utility of leaderboards
but has not examined their effectiveness when individuals
are ranked at particular levels or when the technique is
applied in different application domains, such as social
networking, fitness, or productivity. In this paper, we present
a survey study investigating how preferences for
leaderboards change based on individual differences
(personality traits), ranking, social scoping, and application
domains. Our results show that a respondent’s position on
the leaderboard had important effects on their perception of
the leaderboard and the surrounding app, and that
participants rated leaderboards most favorably in fitness apps
and least favorably in social networking contexts. More
extraverted people reported more positive experiences with
leaderboards despite their ranking or the application domain.
We present design implications for creating leaderboards
targeted at different domains and for different audiences.
Author Keywords
Gamification; leaderboards; motivational affordances;
personality; user interface design.
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI):
Miscellaneous.
INTRODUCTION
In the past few years, the trend of using gamification to
provide gameful, engaging and fun experiences has
proliferated into a variety of domains, such as education,
health, social networking, fitness, and workplace
productivity [22, 26]. Gamification is broadly defined as
“using game elements in non-game contexts” [8]. By
displaying ranks of comparisons of users’ performances,
leaderboards are one of the most widely used game elements
in gamification [12].
Previous research has shown that leaderboards are an
effective way to motivate users through competition
[5, 12, 23]. Additionally, leaderboards have been identified
as one of the ten key “ingredients” in game design [25], one
of the “seven primary game mechanics” [31], and one of the
“twelve things people like” from gamification [31]. However,
studies have revealed that leaderboards were only effective
in motivating some users; for some other users, they could
actually become a demotivating factor [5, 12, 13]. For
example, Codish and Ravid found that extraverted people
perceived leaderboards as being less playful than people who
were more introverted, based on their experiences in the
education domain [5]. In contrast, Jia et al.’s survey study
found that more extraverted people reported higher
preferences for leaderboards in personal informatics systems
[16]. Together, the results of these studies suggest that
personality influences people’s perceived preference for
leaderboards and also implied that people are motivated
differently by leaderboards when applied in different
domains.
Zichermann et al. summarized multiple ways of presenting
leaderboards in gamified applications, such as displaying the
user in the middle of what they term a “no-disincentive”
leaderboard, in or using a multilayered leaderboard when the
space of leaderboard participants is infinite [31]. In game
design, a study of leaderboards in the Olympic Games
showed that bronze medalists reported higher levels of
happiness with their performance than did silver
medalists [14]. In studies on digital games, researchers also
tested how players were motivated differently by appearing
at different leaderboard positions. For example, Butler’s
study showed that players were more likely to re-play a game
when they attained positions at the top or bottom of
leaderboards [4]. Another study from Sun and colleagues
identified an association between leaderboard positions and
players’ satisfaction ratings of a digital game. Players in this
study reported higher levels of satisfaction when they
appeared in the second, fourth or seventh position [27].
These studies demonstrated that people’s perceived
Permissio n to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
components of this work owned by others than the author(s) must be
honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. Request permissions from Permissions@acm.org.
CHI 2017, May 06–11, 2017, Denver, CO, USA
Copyright is held by the owner/author(s). Publication rights licensed to
ACM.
ACM 978-1-4503-4655-9/17/05…$15.00
DOI: http://dx.doi.org/10.1145/3025453.3025826
preference for leaderboards was also influenced by how their
performance was reflected by their positions on the
leaderboards in digital games. However, there have been no
studies of which we are aware on the topic of leaderboard
positions in gamification.
In this study, we explored how people perceive leaderboards
differently when they are ranked at different positions and
when this technique is applied in different domains. We
selected three positions on the leaderboard to study—top,
middle and bottom—and three domains in which
leaderboards have been widely applied but studied little——
social networking, fitness, and productivity. We also
examined the relations between personality traits and
people’s preferences for leaderboards. Our three main
research questions for this study are:
RQ1: Are users’ subjective perceptions of leaderboards
in gamified systems different when they are ranked
at different positions on the leaderboard?
RQ2: Do these perceptions differ when gamification has
been applied to different domains?
RQ3: What are the relations (if any) between users’
personality traits and their perceived preferences
for leaderboards, and are these relations affected
by position or application domain?
RELATED WORK
Leaderboards in Games and Gamification
Gamification has been defined in the research literature as
“the use of game design elements in non-game contexts” [8].
Deterding and colleagues defined “game elements” to
include those elements used in most games, that are readily
associated with games, and that play a significant role in
gameplay [8]. From the literature on games and gamification,
leaderboards were identified as one of the basic ingredients
for designing a great game [25]; they were also one of the
most-used game elements in gamification approaches [12].
Moreover, Reeves and Read listed leaderboards among “ten
ingredients of a great game” in the context of massive
multiplayer online (MMO) games. One of the “ingredients”
was “competition under rules that are explicit and enforced,”
which elicited an underlying motivation. Leaderboards also
brought a sense of fairness for players during the competition
[25].
In their book “Gamification in Design,” Zichermann and
Cunningham list seven primary game mechanics drawn from
the literature and existing gamified applications, including
points, levels, leaderboards, badges, challenges/quests,
onboarding, and engagement loops [31]. In addition to these
core game elements, the authors noted that “feedback”
critically influences players’ motivation and potentially ties
in with many other elements, such as points and leaderboards.
It implied that leaderboards can serve as a type of “feedback,”
rather than an outcome of their own accord [31]. Based on
the 42 different “fun” interactions listed by Radoff [24],
Zichermann and Cunningham categorized “12 things people
like from gamification” [31]. Three of these 12 were
associated with leaderboardsThe book also proposed three
underlying reasons why people were motivated by
leaderboards: leading others, getting attention, and gaining
status.
Mekler et al. conducted an empirical analysis to examine
whether leaderboards affect users’ behavior and intrinsic
motivation. Their findings indicated that leaderboard did not
affect users’ intrinsic motivation, but it was one of the
effective factors in increasing short-term performance in an
image annotation task [18].
Other research categorized leaderboards into two types: “no-
disincentive” and “infinite” leaderboards [27, 31].
Leaderboards, when used in social network websites like
Facebook, aim to create social incentives rather than
disincentives. One way to realize these kind of leaderboards
are to organize the names such that the user appears in the
middle, with better- and worse-performing individuals
bracketing his or her position. These instantiations of
leaderboards also often show the user how close he or she is
to attaining the next best score. Infinite leaderboards are
designed around the premise that a user’s score will be
beaten by another player sooner or later. Since it would be
impossible to allow every user to exist on the leaderboard
forever, these kinds of leaderboards are designed to present
rankings with multiple layers. For example, the mobile
gaming app Doodle Jump includes a leaderboard with a local
view, a friends view, and a global view [31].
Leaderboards in Different Domains
Leaderboards are widely used across multiple domains,
including social network websites, fitness tracking, and
productivity applications. To increase users’ engagement on
social network websites, leaderboards are usually designed
to present the rank of profile views or the number of online
activities undertaken. For example, Klout, a popular social
leaderboard, ranks its users according to their online social
influence via Klout score [2, 31]. Farzan et al. conducted a
study to understand the effects that a point-based incentive
system (i.e., points, “status” levels, and a leaderboard)
played in a social network site, and found that some users
were driven by leaderboards to keep up with others—an
effect that did not suffer significant decay even after the
leaderboard was removed. Their findings suggest that the
usage of leaderboards could play a role in transferring
extrinsic incentives to intrinsic motivations—at least for
some users [9].
Leaderboards are also popular in fitness applications (e.g.,
Fitbit’s companion app). In Wong and Kwok’s mobile health
app, a leaderboard displayed all users’ and groups’ step
records and rankings [30]. Anderson et al.’s study found that
leaderboards introduced a sense of playfulness and indirectly
induced participants to walk more [1].
Finally, some workplaces use gamification as a way of
improving productivity within the organization, namely
Enterprise Gamification [28]. Costa et al. found that
leaderboards were effective for improving some employees’
punctuality to regularly-scheduled work meetings [6].
However, several studies have also shown that leaderboards
could reduce work performance rather than enhance it
because they make the performance public for all to see in
the workplace [28]. For example, Mollick and Rothbard’s
study used leaderboards to motivate employees when
performing tedious and cumbersome tasks at work [21].
Their results showed that the usage of a leaderboard turned
work into a more pleasurable activity and enhanced
productivity when employees had provided consent to
interact with the leaderboard. But the effects from the
leaderboard were reversed in the no-consent condition [21].
Personality differences
Previous studies on gamification have found that
leaderboards might only be effective for “some” users [5, 9,
16]. Some researchers studied personality differences and
their influences on users’ motivation and behaviors. Kaptei
and Eckles studied personality and people’s online
purchasing behavior for e-commerce [15]. Arteaga et al.
applied personality differences in app interface redesign [3].
Nov and Arazy found significant relations between
conscientiousness and people’s participation in online
communities [23]. In a study of gamification applied to an
educational context, researchers found that personality
differences played a role in affecting people’s preferences for
leaderboards. For example, a learning-management system
that featured leaderboards motivated some students to take
extra courses and seek out more additional opportunities to
demonstrate achievement [20]. Codish and Ravid found that
extraverts reported a lack of playfulness in leaderboards
when applied to a course setting [5]. In a more recent study
1 https://www.surveymonkey.com/home/
by Jia and her colleagues, an online survey study of
personality and peoples’ preference on 10 types of
motivational affordances in gamification, results showed that
more extraverted people tended to prefer leaderboards in the
context of a habit-tracking application [16].
HCI studies on personality traits often use the “Big-Five
factors” as a primary scale [5, 16, 19]. The Big-Five is a
descriptive model of personality, which includes
conscientiousness (people actively organize and carry out
tasks), agreeableness (people who help others and expect
help in return), neuroticism/emotional stability (people who
have difficulty managing stress), extraversion (people who
seek out new opportunities and excitement), and
imagination/openness (people who devise novel ideas) [19].
STUDY DESIGN AND METHODS
This study investigates the relations among people’s self-
reported preferences both on leaderboards and the
corresponding applications when: 1) the user’s name is
shown at different positions on the leaderboard—namely at
the top, near the middle, or at the bottom; 2) these
leaderboards are applied to different domains, such as social
networking, fitness, and productivity systems within
organizations. We conducted a large-scale online survey
with 286 participants by using dynamic leaderboard
mockups, created with respondents’ self-reported names and
10 of their friends’ names. The survey was hosted via
SurveyMonkey1 and Amazon Mechanical Turk (AMT)2.
Survey Design
The survey contained four sections. The first section featured
a series of multiple-choice questions about the participant’s
demographic background, such as gender, age, educational
2 https://requester.mturk.com/
(a) (b)
Figure 1. The respondents’ experience of the mockups showing leaderboards applied to the fitness domain from our survey.
(a) Each respondent was asked to enter his/her name and 10 names of his/her friends. (b) A screenshot of the survey illustrating
the display configu ration of the mock-up for the situation of bottom position in the Fitness domain and our survey questions.
background, occupation, and ethnicity. Next, we asked
participants to complete an assessment of the Big-Five
factors of personality [7, 17]. We used the 50-item set of IPIP
Big-Five Factor Markers, which is a free and research
community-developed inventory.
The third part of the survey was designed to elicit feedback
regarding different leaderboards with the participant’s name
appearing at three different positions on leaderboards
situated within three domains. At the beginning, each
respondent was asked to enter his/her name and the names of
ten of his/her close friends (Figure 1a). To help respondents
understand the purpose of collecting names and how these
names were going to be used (and protected), the following
message was shown to all respondents:
In the following, you will be asked to give feedback on 9
different leaderboards. To generate leaderboards with
names that you are familiar with, you will be asked to
enter your name and any 10 of your friends’ names in the
next page. These names won’t be saved or shared with
researchers, and they are only used to generate the
interface mockups for the rest of this survey.
Based on these names, we automatically generated 9
interface mockups of various leaderboards for the
subsequent survey questions (Figure 1b). Specifically, each
respondent’s name was displayed in 3 positions on each
leaderboard (top, middle, bottom), with leaderboards applied
to one of three domains (social networking, fitness, and
productivity). These dynamic leaderboard interfaces were
generated by a SurveyMonkey feature called “Piping”. We
used the Latin Square method to counterbalance and avoid
any potential ordering effects in the study.
After viewing each leaderboard, each respondent was asked
to respond to questions that were designed to collect
information regarding the respondents’ opinions on (1) self-
assessed performance (based solely on the leaderboard
display), (2) the perceived enjoyment that the leaderboard
might impart, (3) the perceived feeling of motivation
provided by the leaderboard, (4) the participants’ willingness
to use an application like the ones illustrated by the mockups,
and 5) the participant’s perceived willingness for
recommending this application to their friends. Among these
5 questions, question 2 and 3 were designed to elicit feedback
about the leaderboard, and questions 4 and 5 were designed
to elicit feedback about the corresponding application
domains. These questions were adapted from survey
questions in previous research on people’s preferences for
game elements in gamification [16, 27].
At the end of each domain section (each containing 3
leaderboards), we asked 4 questions to elicit respondents’
opinions on: (1) for what reasons (if any) that the
leaderboards in that particular domain appeal to them, (2) for
what reasons (if any) that their positions on the leaderboard
appeal to them, (3) whether the inclusion of their friends’
names on the leaderboard matters, and (4) whether the
inclusion of their own names are on the leaderboard matters.
The fourth part of the survey consisted of only one open-
ended question: it was designed to gather respondents’
opinions on: 1) whether they felt that leaderboards appealed
to them differently in different domain, and, if so, why. The
survey took approximately 12 minutes to complete. The full
list of survey questions is included in the supplementary
materials.
Participant Recruitment
We recruited 286 respondents through Amazon Mechanical
Turk (MTurk). We chose to use MTurk for our study due to
the need for a large participant sample, the efficiency of
survey distribution, and its relatively low cost. Participants
were paid USD $1.00, the payment rate suggested by the
AMT platform for survey studies of this duration.
Figure 2. The interface of leaderboard mockups for social network and productivity domains in the survey.
RESULTS
Participant Demographics
To summarize the demographic information of the
respondents, we present their responses (expressed as
percentages of the overall sample population) to questions
regarding their age, gender, educational level, occupation,
and ethnicity (see Table 1). To support our subsequent
correlation analyses, respondents’ demographic responses
were coded into numerical variables. For age, 18–24 was
coded as 1, 25–34 as 2, and so on. For gender, male was
coded as 1 and female as 2; for educational level, the eight
response levels were coded from 1 to 8 from lowest
completed education level to the highest.
Before processing to our regression analysis, we used zero-
order correlations to test for correlations among independent
variables and respondents’ demographic variables (Table 2).
The independent variables of interest, i.e., the five IPIP
personality traits, were positively correlated with one
another. This result was consistent with prior literature [10].
The strongest correlation that we saw was between
conscientiousness and emotional stability (r = .481, p < .01).
This means that our participants who reported high levels of
emotional stability also tended to be more conscientious.
Participants with higher agreeableness levels also tended to
be more open to new experiences (r = .384, p < .01).
For gender, there was a positive correlation between the
coded gender variable and agreeableness (r = .271, p < .01)
and a negative correlation between the coded gender variable
and emotional stability (r = -.212, p < .01). This result shows
that for our sample (n = 286), males were more emotionally
stable but less agreeable than females. We found no
correlation between respondents’ personality characteristics
and their age, educational levels or ethnicity.
Positions on Leaderboards
A two-way ANOVA (repeated measure) with sphericity
corrections for each perception (enjoyment, motivation,
desire to use, and recommend to friends) was conducted. The
results show that position and domain, as two factors, did
play a role, individually, to affect people’s perceived
perceptions significantly on leaderboard and the
corresponding application (Table 4). The results also show
that the interaction between the two factors is significant for
each perception. Thus, to further determine the difference
between people’s perception at each level of each factor, we
conducted several t-tests. The detail of the ANOVA and
t-test results are presented in the supplementary materials.
Across 9 types of leaderboards, 3 positions ´ 3 domains,
respondents consistently reported significantly higher
preference for the leaderboards when their names appeared
in the “top” positions than when they appeared in the “middle”
positions, which were also consistently and significantly
higher than when they appeared in the “bottom” positions,
regardless of the application domain. This suggests that
respondents were able to understand each mockup presented
in the survey.
Total Participants (n = 286)
Age
18–24 (17.8%)
25–34 (50.3%)
35–44 (21.7%)
45–54 (8.0%)
55 and older (2.1%)
Gender
Female (47.2%)
Male (52.8%)
Educational
Level
Some high school (0.3%)
High school graduate/GED (10.1%)
Vocational/Associate degree (6.3%)
Some college (24.8%)
Bachelor degree (40.6%)
Some graduate school (2.8%)
Master degree (13.6%)
Ph.D., law, or medical degree (1.4%)
Occupation
Employed for wages (60.5%)
Self-employed (22.8%)
Student (7.3%)
Retired (0.7%)
Other (9.8%)
Ethnicity
White (65.4%)
Asian/Pacific Islander (19.2%)
Hispanic or Latino (5.6%)
Black or African American (7.3%)
Native American or American Indian (0.3%)
Other (2.1%)
Table 1. Participant Demographics
Mean
Std. Deviation
1
2
3
4
5
6
7
1. Extraversion
29.30
9.16
2. Agreeableness
38.19
7.05
.293**
3. Conscientiousness
36.26
6.85
.167**
.285**
4. Emotional Stability
32.86
8.62
.315**
.229**.
.481**
5. Imagination/Openness
38.40
5.86
.287**
.384**
.299**
.198**
6. Age
2.26
0.91
.033
.114
.080
.070
.012
7. Gender
1.47
.50
-.036
.271**
.044
-.212**
.120*
-.034
Table 2. Correlation matrix and descriptive statistics (n = 286). * indicates cells with p < .05 (2-tailed), ** indicates p < .01.
Leaderboards in Different Domains
We found some interesting results when comparing the
differences in reported preference based on position results
across domains. To be more specific, respondents rated
leaderboards highest in fitness apps and lowest in the social
networking context. From Table 3, on a scale from 1 to 5 (1
indicating strong disagreement and 5 indicating strong
agreement), we can see that when respondents’ names were
shown on the top or in the middle of the leaderboards,
participants provided significantly higher ratings for their
perceptions of Enjoyment, Motivation, Desire to Use the
application, and would Recommend to friends in the Fitness
and Productivity domains than they did for leaderboards in
the Social network domain. In addition, the only negative
perceptions (i.e., given a score below 3.0) that the
respondents reported when appearing in the middle position
were in the Social Network domain. This suggests that for
social network websites, people were only positively
affected by leaderboards when can readily interpret their
rank relative to other users.
People’s perceptions became much more negative when they
saw their names at the bottom of the leaderboards. However,
respondents still rated perceived Enjoyment, Motivation,
Desire to Use, and Recommend to friends positively for
leaderboards in the Fitness domain even when their
perceived performance was low. These results indicate that
people have positive experiences of leaderboards in the
fitness domain, regardless of their ranking.
Respondents were also asked about their opinions about
whether they would like to see their name on leaderboards
and whether they prefer competing only with their friends.
Figure 3 summarizes the results from these questions. This
figure illustrates that 1) showing users’ name on the
leaderboard was very important in both the fitness and
productivity domains; 2) people had even higher preferences
for seeing their names among the top three entries for
leaderboards in productivity domain; 3) respondents
generally rated leaderboards highly when competing among
their friends; and 4) compared to the other two domains,
respondents thought that the leaderboard feature in social
networking websites was least appealing, regardless of
whether their name or their friends’ names appeared in the
list.
Personality Type and Leaderboard Preferences
To explore the relationship between personality and users’
perception, we used structural equation modeling (SEM), a
mediational analysis, to test our proposed models. We
developed two measurement models showing the
relationship between exogenous variables and endogenous
variables as well as a structural model showing the
relationship between the latent personality traits and latent
users’ perception. For the measurement model of
personality, we used the test scores of the 50 questions from
the Big-Five personality inventory as the exogenous
variables. We assumed five latent variables (extraversion,
agreeableness, conscientiousness, emotional stability, and
imagination) for them. As to users’ perception, we assumed
a latent variable (perception) for the 4 measurements
(enjoyment, motivation, desire to use app, and recommend
to friend) that we used in our survey.
TopSoc
TopFit
TopPro
TopAvg
Performance
4.3 (0.9)
4.5 (0.9)
4.6 (0.7)
4.5 (0.9)
Enjoyment
3.3 (1.4)
3.9 (1.2)
3.8 (1.3)
3.7 (1.3)
Motivation
3.3 (1.4)
4.0 (1.2)
3.9 (1.2)
3.7 (1.3)
DesiretoUse
3.3 (1.4)
3.9 (1.2)
3.7 (1.3)
3.6 (1.3)
Recommend
3.2 (1.4)
3.8 (1.2)
3.6 (1.4)
3.5 (1.3)
MidSoc
MidFit
MidPro
MidAvg
Performance
3.2 (0.6)
3.3 (0.7)
3.2 (0.7)
3.2 (0.7)
Enjoyment
2.9 (1.2)
3.6 (1.1)
3.2 (1.2)
3.2 (1.2)
Motivation
3.0 (1.3)
3.6 (1.2)
3.5 (1.2)
3.4 (1.3)
DesiretoUse
2.9 (1.3)
3.6 (1.2)
3.2 (1.2)
3.2 (1.3)
Recommend
2.8 (1.3)
3.5 (1.2)
3.1 (1.3)
3.1 (1.3)
BotSoc
BotFit
BotPro
BotAvg
Performance
2.1 (1.2)
2.0 (1.2)
1.7 (1.1)
1.9 (1.2)
Enjoyment
2.5 (1.4)
3.1 (1.4)
2.5 (1.4)
2.7 (1.4)
Motivation
2.6 (1.4)
3.4 (1.4)
3.0 (1.5)
3.0 (1.4)
DesiretoUse
2.5 (1.4)
3.2 (1.4)
2.6 (1.4)
2.8 (1.4)
Recommend
2.5 (1.4)
3.2 (1.4)
2.6 (1.4)
2.8 (1.4)
Table 3. Descriptive results—reported as mean (SD)—for
respondents’ perceptions of leaderboards based on their
name appearing at three positions (top, middle, and bottom)
within three domains (social, fitness, and productivity).
Perception
Factor
F
value
p value
Enjoyment
Domain
0.97
3.29e-19 *
Position
0.68
6.18e-42 *
Domain: Position
0.88
2.39e-11 *
Motivation
Domain
0.94
1.33e-21*
Position
0.71
3.23e-26*
Domain: Position
0.92
8.93e-03*
Desire to use
the app
Domain
0.96
1.26e-19*
Position
0.72
2.54e-36*
Domain: Position
0.89
7.22e-07*
Recommend
to a friend
Domain
0.95
6.03e-20*
Position
0.67
4.15e-29*
Domain: Position
0.92
6.76e-08*
Table 4. Results from ANOVA. Significant codes (with
Greenhouse-Geisser correction): p< .05 ‘*’
For all the paths in the model, we estimated the path
parameters based on maximum likelihood, and the process
converged normally after 72 iterations. The overall badness-
of-fit of our model is significant (Chi-square < 0.001). Based
on examination of the path parameters, while the two
measurement models showed strong factor loadings, the path
parameters of the paths from the Big-Five personality traits
to users’ perception are fairly small, which suggests a weak
impact of personality on users’ perception of
leaderboards. Thus, from the SEM analysis, we find no
statistically significant casual relationships between
personality traits and perception on leaderboard. The detailed
results from the SEM analysis is included in the
supplementary materials.
We also conducted a multiple regression analysis. All
individual Beta (β) values from 36 regressions (4 perception
types ´ 3 positions ´ 3 domains) are summarized, and the
significant (p < .05/36 = .001) results presented in the
supplementary materials. Overall, more extraverted people
tended to have more positive perceptions of leaderboards in
the domains of social networking and productivity; people
with higher levels of agreeableness tended to express greater
enjoyment of leaderboards in the fitness domain. We found
no significant results for the personality traits of
conscientiousness, emotional stability, or imagination.
In the remainder of this section, we report significant
differences among perceptions (i.e., enjoyment, motivation,
desire to use the application, and would recommend to
friends) for each type of leaderboard. In addition, we also
report the qualitative results from our open-ended survey
questions.
For leaderboards in the social networking domain, when
respondents’ names were shown in the middle position, the
more extraverted people expressed more desire to use the
social networking websites (β = .216, p = .001) and were
more likely to recommend the websites to their friends
(β = .218, p = .001). When respondents’ names were at the
bottom of the leaderboard on social networking websites,
the more extraverted people reported stronger likelihood of
being motivated by the leaderboard (β = .218, p = .001); in
addition, for more extraverted people, they expressed more
desired to use the websites (β = .219, p = .001) and were
more likely to recommend it to their friends (β = .232,
p = .001).
Our qualitative results from the open-ended questions show
that leaderboards on social websites provide another mode of
connection, help monitor social influence status, and
increase communication among friends:
“Leaderboards on social networks help me assess the
reputation of people I may not know all that well.” (P146)
“I like the leaderboard just for the purpose of being
able to identify who I am staying in contact with, and
who wants to stay in contact with me.” (P224)
However, respondents reported that they use social media to
communicate with others rather than for competition, and the
social influence showed from the leaderboard does not
reflect reality since their social connections are not derived
solely from social websites:
“It doesn’t appeal to me because I don't see the point in
such a ranking, specially between friends. Feels like
added competition where there shouldn’t be any.” (P25)
“If I'm being honest, I don’t think I care for the ranking
system when it comes to a social network site. It doesn't
seem like it belongs on a social site.” (P105)
“The leaderboard feature in social networking
websites doesn’t appeal to me because it doesn’t reflect
Figure 3. Summarized results of respondents’ opinions on whether they would like to see their names on
leaderboards and whether they have preferences on competing only with their friends.
my real connections that I have people rather than on
some networking websites.” (P172)
With regards to leaderboards in the fitness domain, when
respondents’ names were shown at the bottom of the
leaderboards in fitness apps, the more agreeable respondents
rated Enjoyment of the leaderboard more highly (β = .227,
p = .001). From our qualitative results, respondents reported
that fitness itself can be competitive in nature. They also
reported feeling a sense of motivation from leaderboards in
this domain because leaderboards turn fitness activities into
a more fun competition. The leaderboard can also be seen as
a type of progress tracking, which is a good match to this
domain. Some sample comments from the open-ended
survey questions included:
“I always wanted to use a fitness app like this. It’s
addicting to keep watching your rank go up as you work
towards your fitness goals. It’s like when you work for
hours leveling on a video game only with real life
results.” (P118)
Consistent with the quantitative results, one reason that
respondents reported enjoying the leaderboards was that
people enjoy engaging in competition with their friends or
families on fitness activities:
“I like the competitive nature of it, plus, having friends
and family on the leaderboard is an extra incentive to
do well.” (P207)
“I have a Fitbit on my hand right now and I look at the
leaderboard from time to time to make sure my steps
don’t get too low. It really does motivate me because I
know my mom will get worried if she sees my numbers
go too low.” (P 32)
“It’s just interesting to know how well my performance
is compared to my friends. It makes doing activities
more exciting and motivating, to me. It motivates me to
compete.” (P256)
The results of leaderboards in the productivity domain
reveal that when respondents’ names were at the top of the
leaderboard, people who are more extraverted were more
likely to have positive perceptions of the leaderboards
(β = .222, p = .001) and the surrounding system (β = .233,
p = .000); for the personality traits of agreeableness,
emotional stability and imagination, people rated
leaderboards in this domain negatively, but this was not a
statistically significant difference. When respondents’ names
were shown in the middle of the leaderboard in productivity
applications, the more extraverted people still provided
positive ratings of the leaderboard and the application. From
our qualitative results, our respondents reported that they
liked the idea of incorporating leaderboards into team work
because it offers an incentive for doing a good job, it
provides a visual representation of work performance, and it
might be especially valuable when a deadline is approaching:
“This leaderboard lets me know how well I am doing
within my team and if I need to improve my
performance.” (P10)
“Gives real, easily quantifiable feedback on my
performance.” (P175)
“It is fun to see how well you are doing and makes work
feel a little more like a game which makes it a little
easier to enjoy what you are doing and feel motivated.”
(P148)
“I really do like to know how my output and quality of
work (of any kind) measures up to my peers. It’s good
to know whether I need to work harder or if I can relax
a bit and maintain.” (P192)
On the other hand, many negative comments from
respondents mentioned that the competition derived from a
leaderboard in a working environment reads more like a
“name-and-shame” feature instead of a “game-like” feature
since employees don’t have other options. They also felt that
employees should cooperate to reach a common goal instead
of competing with one another, that leaderboards might
foster animosity at work, and that some work cannot be
judged in a fair and objective manner upon which a
leaderboard visualization could be built:
“when I am down at the list I will have a motive to work
better, it’s a job, it's not optional...” (P12)
“This leaderboard does not appeal to me as I do not feel
my work can be judged adequately through it.” (P74)
“It does not appeal to me because I feel that
productivity in the workplace should be a matter
between each employee and their employer and not a
public matter between employees.” (P47)
DISCUSSION
In this section, we discuss the link between differences in a
person’s position on a leaderboard and their preferences for
leaderboards. We also delve into the relations between their
rank or position and their preferences for leaderboards across
different domains. Finally, we discuss how personality
differences could help to inform the design of leaderboards
in gamified applications.
Leaderboard Position and Domain Differences
In the gaming literature, leaderboard position was found to
be a factor that affects players’ game experiences. In the
example of leaderboards in Olympic games, researchers
explained the finding that bronze medalists reported higher
levels of happiness than silver medalists because of the
notion of “what could have been,” which implies that silver
medalists framed their thinking about the fact that they could
have won a gold medal, while and bronze medalists
understood their ranking as being better than not having
received any medal at all [14]. For leaderboards in digital
games, in the example of Gold Mine, Sun and colleagues
found that players reported higher satisfaction when they
appeared in positions 2, 4, and 7 [27].
Our results showed that respondents rated leaderboards
differently when they are ranked differently in different
domains. It indicates that unlike event-based competitions
like the Olympics or short-time-repetitive competitions in
digital games, leaderboards in gamified applications
typically present long-term competitions of various types of
domain-related activities. Thus, to design leaderboards in
gamified contexts, in addition to leaderboard position,
designers should also consider the impact of domain
differences.
For rankings on leaderboards, our results show that
respondents reported positive perceptions of leaderboards
only when they appeared in the top positons of the
leaderboard in the social-networking domain. However, the
results from the fitness and productivity domains revealed
that people liked leaderboards in fitness applications no
matter what their rank; and people had only negative
perceptions of leaderboards when ranked in the bottom of
leaderboards in the productivity domain. From our
qualitative results, one of the key differences among these
domains is the perceived fairness of the leaderboard in the
social and productivity domains. Unlike step count, the
metric typically used to determine ranking on fitness-
oriented leaderboards, respondents reported that their social
influence cannot be quantifiably reflected by the leaderboard
on social websites since not all of their contacts occurred in
a single social network application; and for productivity
domain, respondents reported that significant facets of their
work are simply not rank-able. In the research literature, a
design guide for leaderboards in game design mentioned
about that competition under rules should be fair and explicit
[25]. Thus, we suggest that the competitive activity used to
seed the leaderboard should be designed to bring a sense
of fairness for users.
In our results, respondents provided the lowest ratings for
leaderboards in the social networking domain. From our
qualitative results, respondents expressed a common concern
that they use social network sites primarily for
communication instead of a site for competition with others.
This finding is consistent with the findings from previous
studies of people’s experiences with social network games.
Wohn et al. mentioned that competition in social games
indirectly facilitate social interaction: people passively
obtained information about others’ performance from
leaderboards and treated this interaction as a “friendly
competition” [29]. Leaderboards introduce the concept of
competition to gamified systems, but social network domains
tend to emphasize an ultimate goal of facilitating interaction
among friends. It appears that among members of a close
social circle, it is not easy to encourage serious competition;
rather, competition manifests as friendly banter or a
lighthearted game. Thus, we suggest that leaderboards in
social networking contexts should be intentionally
designed to serve the purpose of facilitating
communication rather than just showing results of a
metricized competition. For example, leaderboards could
be designed to be less competition-oriented and instead focus
on expanding one’s social circle; showing long-time, no-
contact friends or shared-interest strangers on the board
might be a more effective use of these features than when
they simply display the performance of a close, stable group
of friends.
We also found that respondents rated leaderboards positively
in the fitness domain regardless of their position on those
leaderboards. Additionally, respondents reported
significantly higher preference for seeing their friends or
colleagues on these leaderboards and the lowest preference
for seeing strangers on them. Our qualitative results reinforce
these quantitative findings: people expressed more
enjoyment and motivation when competing against people
with whom they were familiar, such as family, friends, or
close colleagues. One reason is that the activity people are
competing with in this domain is almost always reflective of
their personal, daily routines. This may be why people are
more comfortable competing with their closer friends and
family members in this context. Competing with close
acquaintances leverages people’s universal desire to interact
with and be involved in the lives of their friends and family
members; additionally, it provides motivation for improving
one’s fitness levels because making unhealthy decisions can
in some ways be perceived as “letting down” those close
friends and family members. This duality is unique in the
fitness domain because fitness activities are both deeply
personal and influenced by the behavior of others. The
finding is consistent with the previous study from Hamari
and Koivisto, which found that users felt more attached to
gamified applications when they have more friends
participating in the gamified system [11]. Our finding also
supports Wong and Kwok’s hypothesis that people’s fitness-
or exercise-related motivation could be positively satisfied
through human-relatedness needs, such as social recognition
and affiliation [30]. People care more about who the
individuals are on these leaderboards than his/her own
ranking. Thus, we suggest that when designing a leaderboard
for a fitness app, designers should first understand who
should appear on the leaderboard rather than where to
position the user, focusing on supporting constructive
competitions among a small circle of close friends.
The results from the findings about productivity-oriented
leaderboards reveal that it is very important for respondents
to see their name on the leaderboard in this domain. And
people have even higher preferences for seeing their names
among the top three entries of these leaderboards. From our
qualitative results, respondents expressed the most negative
perceptions of these leaderboards when their names appeared
towards the bottom of the ranked list. Instead of introducing
a sense of “fun,” respondents thought that the competitive
tasks used to seed the rankings on productivity-oriented
leaderboards spur serious competition. They also expressed
concern that appearing at a low rank might have negative
consequences for how they were perceived among their
colleagues, or even to strain their relationship with their
employer. Our finding is consistent with the study by
Mollick and Rothbard, which found that employees
experienced less positive affect from leaderboards at work in
the “no-consent” condition [21]. Werbach and Hunter also
noted the negative effects of leaderboards in working
environments, pointing out that leaderboards can play a role
in “reducing the richness of a game to a zero-sum struggle
for supremacy [and] therefore inherently turns off some
people and makes them behave in less desirable ways” [28,
p76]. This might due to the sensitivity associated with
introducing (additional) competitiveness into workplaces.
On the other hand, in those successful documented examples
of using leaderboards in the productivity domain, the
competitive tasks around which rankings were based were
usually repetitive and boring [28]. For example, to reduce the
death rate from hospital-acquired infections, leaderboards
have been successfully applied in hospitals to motivate staff
to compete with one another in washing their hands often and
well, which turns hand washing into a competitive game.
Thus, we suggest that when designing leaderboards for the
productivity domain, the competitive tasks should be
selected from the set of simple and repetitive tasks
associated with the job. Additionally, designers might
strive to avoid showing the lowest-ranking employees on
workplace leaderboards. It is desirable that at workplaces
the design of the leaderboard should consider the dynamics
among co-workers and the impact that their introduction
might have on the overall office culture.
Personality-targeted Leaderboards
Our results show that more extraverted people tended to have
more positive perceptions of leaderboards. This finding is
consistent with previous studies by Jia et al. [16] and Nov &
Arazy [23]. Werbach and Hunter also mentioned that
leaderboards have the capability of showing progress that
other motivational affordances like points and levels cannot
[28]. One reason for extraverted people to prefer
leaderboards is because of their dynamic nature—they
reflect the ever-changing social landscape constituted by the
gamified system’s participants. Thus, we suggest, to appeal
to more extraverted users, designers should not only
design leaderboards as a way of showing rankings, but
also emphasizing changes.
In summary, based on the findings from our study, we can
provide several concrete suggestions for the design of
leaderboards in gamified applications. For these interfaces,
we propose that there are five questions that designers should
consider:
1. In what domain is the leaderboard going to be applied?
2. Does the competitive task on the leaderboard feature
rules that are fair and equally applicable to all
participants?
3. What are the relationships among the participant-
competitors?
4. Where should the active user be displayed on the
leaderboard — at the top, middle, or bottom of the list,
or does it not matter? In other words, how should the
user’s performance be communicated relative to the
other users of the system?
5. Will the task or activity that will be measured to seed
the leaderboard provide a dynamic enough competitive
landscape?
Limitations
Our study used regression results from an online survey. The
mockup leaderboards did not capture the wide range of
possible leaderboards application domains. Leaderboards
could be used in multiple domains and the social dynamics
between leaderboard players could vary among these
domains. Additionally, the results were gathered from a one-
time survey and thus our findings might not reflect actual
“after-use” user experiences. In order to constrain the
number of questions in our survey, we manipulated the user’s
ranking on the generated leaderboards to be at the top, in the
middle, and at the bottom, which does not reflect a person’s
real relationship to the domain or the task, given that he/she
did not put real effort into improving his/her ranking. Finally,
this study uses personality traits as indicator of preference on
leaderboard designs. In reality, other factors might play a
larger role in determining perceptions of gamification
designs.
CONCLUSION AND FUTURE WORK
Overall, this study contributes to the understanding of how
leaderboard positions affect people’s experiences of
leaderboards across different domains. We discovered that
for leaderboards in gamified applications, competition is a
media rather than purpose. We found that one primary
personality trait affect people’s perceived preferences on
leaderboards by a small amount—and did so in different
ways: extraversion. We developed several design guidelines
for leaderboards in specific domains and for specific
personality types.
In future work, we plan to explore the relations between the
dynamism of different leaderboard implementations and
people’s perceptions of those leaderboards. We anticipate
that this research will continue to guide the application of
motivational affordances to enhance users’ experiences with
a variety of gamified applications across many potential
domains.
ACKNOWLEDGEMENTS
We would like to thank our respondents for their time and
effort. We would also like to thank our anonymous reviewers
and PC members for their valuable suggestions in improving
our data analysis methods and effectively communicating the
results of this complex study.
REFERENCES
1. Ian Anderson, Julie Maitland, Scott Sherwood, Louise
Barkhuus, Matthew Chalmers, Malcolm Hall, Barry
Brown, and Henk Muller. 2007. Shakra: Tracking and
sharing daily activity levels with unaugmented mobile
phones. Mobile Networks and Applications 12, 2–3:
185–199.
2. Isabel Anger and Christian Kittl. 2011. Measuring
influence on Twitter. In Proceedings of the 11th
International Conference on Knowledge Management
and Knowledge Technologies, 31.
3. Sonia M. Arteaga, Mo Kudeki, and Adrienne
Woodworth. 2009. Combating obesity trends in
teenagers through persuasive mobile technology.
SIGACCESS Accessibility and Computing 94: 17–25.
http://doi.acm.org/10.1145/1595061.1595064
4. Charles Butler. 2013. The effect of leaderboard ranking
on players’ perception of gaming fun. In International
Conference on Online Communities and Social
Computing, 129–136.
5. David Codish and Gilad Ravid. 2014. Personality
based gamification-educational gamification for
extroverts and introverts. In Proceedings of the 9th
CHAIS Conference for the Study of Innovation and
Learning Technologies: Learning in the Technological
Era, 36–44.
6. João P. Costa, Rina R. Wehbe, James Robb, and
Lennart E. Nacke. 2013. Time’s up: Studying
leaderboards for engaging punctual behaviour.
In Proceedings of the First International Conference
on Gameful Design, Research, and Applications, 26–
33.
7. Paul T. Costa, Jr. and Robert R. McCrae. 1992. NEO
Personality Inventory–Revised (NEO-PI-R) and NEO
Five-Factor Inventory (NEO-FFI) Professional
Manual. Psychological Assessment Resources, Odessa,
FL.
8. Sebastian Deterding, Dan Dixon, Rilla Khaled, and
Lennart Nacke. 2011. From game design elements to
gamefulness: Defining “gamification”. In Proceedings
of the 15th International Academic MindTrek
Conference: Envisioning Future Media Environments,
915. http://doi.acm.org/10.1145/2181037.2181040
9. Rosta Farzan, Joan M. DiMicco, David R. Millen,
Casey Dugan, Werner Geyer, and Elizabeth A.
Brownholtz. 2008. Results from deploying a
participation incentive mechanism within the
enterprise. In Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems, 563–572.
10. Samuel D. Gosling, Peter J. Rentfrow, and William B.
Swann. 2003. A very brief measure of the Big-Five
personality domains. Journal of Research in
personality 37, 6: 504–528.
11. Juho Hamari and Jonna Koivisto. 2015 “Working out
for likes”: An empirical study on social influence in
exercise gamification. Computers in Human Behavior
50: 333–347.
12. Juho Hamari, Jonna Koivisto, and Harri Sarsa. 2014.
Does gamification work?: A literature review of
empirical studies on gamification. In Proceedings of
the 47th Hawaii International Conference on System
Sciences (HICSS ’14), 3025–3034.
13. Juho Hamari. 2013. Transforming homo economicus
into homo ludens: A field experiment on gamification
in a utilitarian peer-to-peer trading service. Electronic
Commerce Research and Applications 12, 4: 236–245.
14. Victoria Husted Medvec, Scott F. Madey and Thomas
Gilovich. 1995. When less is more: Counterfactual
thinking and satisfaction among Olympic medalists.
Journal of Personality and Social Psychology 69, 4,
603.
15. Maurits Kaptein and Dean Eckles. 2012. Heterogeneity
in the effects of online persuasion. Journal of
Interactive Marketing 26, 3: 176–188.
16. Yuan Jia, Bin Xu, Yamini Karanam, and Stephen
Voida. 2016. Personality-targeted gamification: A
survey study on personality traits and motivational
affordances. In Proceedings of the 2016 CHI
Conference on Human Factors in Computing Systems.
2001–2013.
17. Oliver P. John, Laura P. Naumann, and Christopher J.
Soto. 2008. Paradigm shift to the integrative Big Five
trait taxonomy: History, measurement, and conceptual
issues. In Handbook of Personality: Theory and
Research (3rd ed.), Oliver P. John, Richard W.
Robbins, and Lawrence A. Pervin (eds.). Guilford
Press, New York, NY, 114–158.
18. Elisa D. Mekler, Florian Brühlmann, Klaus Opwis, and
Alexandre N. Tuch. 2013. Do points, levels and
leaderboards harm intrinsic motivation?: An empirical
analysis of common gamification elements.
In Proceedings of the First International Conference
on Gameful Design, Research, and Applications, 66–
73.
19. Robert R. McCrae, and Paul T. Costa. 1989.
Reinterpreting the Myers-Briggs type indicator from
the perspective of the Five Factor model of personality.
Journal of Personality 57, 1: 17–40.
20. Rudy McDaniel, Robb Lindgren, and Jon Friskics.
2012. Using badges for shaping interactions in online
learning environments. In Proceeding of IEEE
International Professional Communication
Conference, 1–4.
21. Ethan R. Mollick and Nancy Rothbard. 2014.
Mandatory fun: Consent, gamification and the impact
of games at work. The Wharton School research paper
series.
22. Mark J. Nelson. 2012. Soviet and American precursors
to the gamification of work. In Proceedings of the 16th
International Academic MindTrek Conference, 23–26.
23. Oded Nov and Ofer Arazy. 2013. Personality-targeted
design: Theory, experimental procedure, and
preliminary results. In Proceedings of the 2013 ACM
Conference on Computer Supported Cooperative Work
(CSCW ’13), 977–984.
http://doi.acm.org/10.1145/2441776.2441887
24. Jon Radoff. 2011. Game on: Energize your business
with social media games. John Wiley & Sons.
25. Byron Reeves and J. Leighton Read. 2009. Total
engagement: Using games and virtual worlds to
change the way people work and businesses compete.
Harvard Business School Press, Boston, MA.
26. Katie Seaborn and Deborah I. Fels. 2015. Gamification
in theory and action: A survey. International Journal of
Human-Computer Studies 74, 14–31.
27. Emily Sun, Brooke Jones, Stefano Traca, and Maarten
W. Bos. 2015. Leaderboard position psychology:
counterfactual thinking. In Extended Abstracts of the
33rd Annual ACM Conference on Human Factors in
Computing Systems, 1217–1222.
28. Kevin Werbach and Dan Hunter. 2012. For the win:
How game thinking can revolutionize your business.
Wharton Digital Press.
29. Donghee Yvette Wohn, Cliff Lampe, Rick Wash,
Nicole Ellison, and Jessica Vitak. 2011. The “s” in
social network games: Initiating, maintaining, and
enhancing relationships. In Proceedings of the 44th
Hawaii International Conference on System Sciences
(HICSS 2011), 1–10.
30. Clara Choi-Ki Wong and Ron Chi-Wai Kwok. 2016.
The effect of gamified mHealth app on exercise
motivation and physical activity. In Proceedings of
Pacific Asia Conference on Information Systems
(PACIS), 389.
31. Gabe Zichermann and Christopher Cunningham. 2011.
Gamification by design: Implementing game mechanics
in web and mobile apps. O’Reilly Media.
