Content uploaded by Jorge Oceja
Author content
All content in this area was uploaded by Jorge Oceja on Nov 15, 2021
Content may be subject to copyright.
Proceedings of the
12th European Conference on Game-Based Learning
ECGBL 2018
Hosted By
SKEMA Business School
Sophia Antipolis, France
4 - 5 October 2018
Edited by
Dr Melanie Ciussi
Copyright The Authors, 2018. All Rights Reserved.
No reproduction, copy or transmission may be made without written permission from the individual
authors.
Review Process
Papers submitted to this conference have been double-blind peer reviewed before final acceptance
to the conference. Initially, abstracts were reviewed for relevance and accessibility and successful
authors were invited to submit full papers. Many thanks to the reviewers who helped ensure the
quality of all the submissions.
Ethics and Publication Malpractice Policy
ACPIL adheres to a strict ethics and publication malpractice policy for all publications – details of
which can be found here:
http://www.academic-conferences.org/policies/ethics-policy-for-publishing-in-the-conference-
proceedings-of-academic-conferences-and-publishing-international-limited/
Conference Proceedings
The Conference Proceedings is a book published with an ISBN and ISSN. The proceedings have been
submitted to a number of accreditation, citation and indexing bodies including Thomson ISI Web of
Science and Elsevier Scopus.
Author affiliation details in these proceedings have been reproduced as supplied by the authors
themselves.
The Electronic version of the Conference Proceedings is available to download from DROPBOX
http://tinyurl.com/ECGBL2018. Select Download and then Direct Download to access the Pdf file. Free
download is available for conference participants for a period of 2 weeks after the conference.
The Conference Proceedings for this year and previous years can be purchased from
http://academic-bookshop.com
E-Book ISBN: 978-1-912764-00-6
E-Book ISSN: 2049-100X
Book version ISBN: 978-1-911218-99-9
Book Version ISSN: 2049-0992
Published by Academic Conferences and Publishing International Limited
Reading
UK
Tel: +44-118-972-4148
www.academic-conferences.org
Gamifiying Wikipedia?
Jorge Oceja and Ángel Sierra-Obregón
University of Cantabria, Santander, Spain
jorge.oceja@unican.es
angel.obregon@alumnos.unican.es
Abstract: Gamification has been defined, with minor variations, as the use of game elements in non-game contexts. Fields
with gamified practices range from business (to increase sales, improve customer satisfaction or foster productivity) to
others, such as health, personal well-being and, regarding education, improvement of classroom management (in which
gamification aims to promote positive behaviours). The community around Wikipedia Project has decided to use some of
these practices to promote author participation and create new content. This work reviews some of these attempts, with a
focus on the experiences at Wikipedia Spain, analysing the impact of including different game elements and proposing
strategies to improve results in the future.
Keywords: Wikipedia, gamification, game elements, badges
1. Introduction
Although the term gamification may sound a little old-fashioned (Fuchs, Fizek & Ruffino, 2014), the use of game
elements in non-game contexts has become familiar to us in recent years. Situations in which we are asked to
set a happy (or sad) face as a rating for the cleaning service in a public toilet, or in which teachers award points
and badges to their students through apps such as ClassDojo, remind us that we live in what Raessens (2006)
defined as a ludification of culture. Even in this current edition of the ECGBL programme, 17 works include in
their title one of the terms gamification, gamified or gamifying.
Several authors have sought to identify the origin of the concept but without reaching a clear consensus. While
Werbach and Hunter (2012) mention that the “first use in its current sense occurred in 2003, when Nick Pelling,
a British game developer, established a short-lived consultancy company to create game-like interfaces for
electronic devices” (p. 24), Deterding, Dixon, Khaled and Nacke (2011) explain that “the first documented use of
the term gamification dates back from 2008" (p. 1). In any case, the concept became mainstream around 2010,
appearing in mass media and in several MOOCs.
Deterding, Sicart, Nacke, O’Hara and Dixon (2011) define gamification as game elements in non-game
contexts. However, academics fail to agree on what a game element is. The CCAE model (Conventions,
Components, Actions and Emotions) (Oceja & González-Fernández, 2016) tries to synthetize previous attempted
classifications of game elements, such as those of Hunicke, Leblanc and Zubek (2004), Werbach and Hunter
(2012) and Brathwaite and Schreiber (2008). This model distinguishes between iconographic conventions (i.e.
the most commonly used elements in simplistic gamification practices), components or objectual metaphors of
the real world, actions afforded to the players (including their consequences) and, finally, the emotions that
they can experience. Paradoxically, conventions have been the most common element, leading to experiences
based on rewards and rankings to foster different behaviours. However, different authors have highlighted the
risks involved in this behaviourist approach (Bogost, 2015; Fuchs et al., 2014; Versteeg, 2013).
Several authors (Werbach & Hunter, 2012; Zichermann & Linder, 2013) distinguish between two kinds of
gamification: the one that is used in business contexts (in order to increase productivity or sales or to improve
customers’ experience) and the one that aims to promote positive behaviours in what has been called
“gamification for good”. Recently, apps have appeared for promoting a healthy lifestyle (Fitocracy, Inc., 2016),
improving social and emotional well-being (McGonigal, 2015) or even for classroom management, such as
ClassDojo (Class Twist, Inc., 2016).
Wikipedia, the popular free-content encyclopaedia, and the Wikimedia Foundation (the project of which
Wikipedia is part) have not ignored this trend. For instance, Magnus Manske, an author, editor and software
author, developed a set of games to improve Wikidata1, the knowledge base that nourishes Wikipedia and other
projects from the Foundation (Manske, 2018). Also, the Wikipedia community and especially the different
1 https://tools.wmflabs.org/wikidata-game/
504
Jorge Oceja and Ángel Sierra-Obregón
chapters at the Wikimedia Foundation have created contests and game-like experiences to promote
participation all over the world.
Since the creation of Wikipedia in 2001, numerous scientific articles have been published on topics such as
vandalism (Kittur et al., 2007), neutrality (Göbel and Munzert, 2016), reliability (Rodrigues, 2012), “editing wars”
(Iñiguez et al., 2014), access inequalities (Lanamäki et al., 2015), the gender gap (Antin et al., 2011) and the racial
gap (Graham, 2011). However, there is a lack of studies focusing on the possibilities that gamification offers for
promoting participation.
This work reviews attempts by Wikimedia Spain to analyse the impact of including different game elements to
promote participation and proposing strategies to improve results in the future.
2. Methodology
This work is based on a quantitative approach. First, we identified every game-based event organized by
Wikimedia Spain, targeting both standard editors and photography uploaders. We identified the game elements
present in each activity and we used descriptive statistics and graphics to understand their evolution in terms of
numbers of participants.
In the case of standard contests, we also calculated the total amount of information generated (in bytes), as well
as the effectiveness of each event per user (by dividing bytes by the number of editors) and per week.
Then, inferential statistics (Levene’s test for equality of variances and independent samples t-test) allowed us to
compare the effectiveness of different rewards, namely non-tangible (such as badges) vs. economic.
3. Results
As already mentioned, Wikimedia Spain organizes two kinds of contests. One type of contest aims to generate
new textual content, and therefore it targets standard editors. The other type of contest is looking for new high-
quality photography uploads. Contests of the first type are based on a simple use of game elements, focusing
on conventions such as points (the number of bytes, the references created or the number of articles act as
points), rankings (sometimes shown at the end of the content and sometimes constantly updated by a bot2) and
rewards. Most often, these rewards translate into a sum of money that ranges from 75 to 200 euros, clothes or
books, but sometimes intangible rewards such as badges are given instead. This is the case in the Mes de la
Antártida and Desafío Wikificar competitions.
In the photography contests, a jury selects certain images for their quality, according to certain criteria, and
rewards the winners with money (100 to 800 euros), books or technology gadgets.
3.1 Descriptive statistics
Figures 1 and 2 show the evolution of these two types of contests in terms of numbers of participants.
Figure 1: Numbers of participants in the editing competitions organized by Wikimedia Spain
2 https://tools.wmflabs.org/superzerocool/desafio/wikificar2017
505
Jorge Oceja and Ángel Sierra-Obregón
The first contest took place in 2014, with 54 editors generating 12,328,000 bytes of new information. Since then,
participation has remained fairly stable, although the number of editors is still relatively small and the amount
of bytes generated has varied significantly. The most successful competition in terms of participation was Mes
de Asia, with a total of 122 editors generating 2,875,407 bytes, followed by the 2018 Wiki Loves the Olympics,
in which 66 editors generated 16,164,476 bytes of information. Some events in 2017 had the smallest
participation, namely Concurso día de la Poesía, with only three participants and 90,000 bytes, or Desafío
wikificar, with five participants and 14,616 bytes.
Although the number of participants may be a good measure of the success of a contest, we also looked at the
effectiveness per user, by dividing the total bytes of content generated by the number of editors. According to
this variable, Wiki Loves the Olympics 2018 was the most successful event (bytes per editor (b/e) = 244,916),
followed by Monuments Spain 2014 (b/e = 228,296) and Wiki Loves the Olympics 2016 (b/e = 207,578).
We also took into consideration the fact that the length of the contest may vary slightly. To calculate the
effectiveness per week, we took the previous quantity and divided it by the number of weeks. The results
indicated that Monuments of Spain Challenge 2014 (b/e/w= 57,074) was the most productive competition,
followed by Wiki Loves the Olympics 2016 (b/e/w = 34,596) and Wiki Loves the Olympics 2018 (b/e/w = 30,614).
It was striking that the two contests with badges instead of tangible awards (money, books, etc.) ranked among
the less successful, with Desafío wikificar being the least effective per user and per week (b/e = 2,923 and b/e/w
= 1,461), followed by Mes de la Antártida (b/e = 27,960 and b/e/w = 6,922). Therefore, we considered it relevant
to investigate whether there was any significant difference between the use of tangible (money or products)
and non-tangible rewards (badges).
In the case of the photography events, although they were referred to as competitions, game elements were
lacking. This competition offers only tangible awards (namely money) to the winners who have submitted the
best images. Conventions such as rankings or badges are absent, much less any kind of playful action or emotion.
Figure 2: Numbers of participants in photography competitions organized by Wikimedia Spain.
It seems that monetary rewards boosted participation by photographers during the first years (680 in 2011 and
739 in 2012, both within Wiki Loves Monuments), but that users tired of competing and participation decreased,
with only 68 participants in Wiki Loves Folk 2017 and a mere 58 in the Wiki Science competition 2017.
3.2 Inferential statistics
First, the means of bytes generated in contests with economic rewards (either money or physical products) and
those with game-like rewards such as badges were calculated.
Table 1: Means of bytes generated in contests with badges vs. contests with economic rewards
Rewards
N
Mean
Std. Deviation
Std. Error Mean
Bytes
Economic
460
99,132.89
344,043.56
16,041.12
Badges
46
24,998.02
85,749.57
12,643.08
As the difference between the means was quite noticeable, we performed both Levene’s test for equality of
variances and an independent t-test to assess if there was any significant difference between the means of bytes
generated in contests with economic rewards (either money or physical products) (M = 991,332.89) and those
506
Jorge Oceja and Ángel Sierra-Obregón
with game-like rewards such as badges (M = 24,998.02). Table 2 presents these differences, based on the
production of each user in these two different kinds of competitions.
Table 2: T-test results comparing the means of bytes generated per user in the two kinds of contests
Levene’s Test for
Equality of
Variances
t-test for Equality of Means
F
Sig.
t
df
Sig. (2-
tailed)
Mean
Difference
Std. Error
Dif.
95% Confidence Interval
of the Difference
Lower
Upper
Bytes
Equal
variances
assumed
5.87
.016
1.46
504
.146
74,133.87
50,926.1
-25,919.72
174,187.45
Different
variances
not
assumed
3.63
244.4
.000
74,133.87
20,424.61
33,903.14
114,364.59
Levene’s test was significant (p < .05), meaning that different variances cannot be assumed. Thus, we have to
pay attention to the test statistics in the second row of the table, which confirm that the difference between
the two groups is significant t(244.4) = 3.63, p < .05.
4. Discussion and future lines of research
Wikimedia Spain has tried to motivate its editors to create content in several ways, including the use of contests.
However, some of these events, although they may appear to be game-based, do not gamify the user experience
in any rich or complex way. Photography contests do not include game elements and standard editing
competitions focus only on conventions (the most basic game elements), failing to afford any meaningful playful
actions to the participants or, more importantly, the possibility of experiencing emotions. As several authors
have pointed out (Hunicke et al., 2004; Malone & Lepper, 1987), when participating in any experience, users
want to explore and investigate; they want to discover for themselves the territory that the representational
systems offer them. In addition, they want this process of discovery to challenge them and push them to find
solutions to problems. The simplistic use of conventions (rankings, points, rewards, etc.), reveals a lack of
understanding of the complexity latent in games. This imbalance between the richness of the medium and the
simplicity of most gamification practices appears frequently in the scientific literature (Bogost, 2015; Fuchs et
al., 2014). Because of the decisions adopted by the organizers, the contests have attracted modest numbers of
participation and achieved only moderate effectiveness, particularly the events that do not include game
elements, which show an alarming downward trend. The tendency is not as worrying in the gamified editors’
competitions, but even these competitions are at a standstill.
We also explored to what extent the presence of tangible rewards vs. badges impacts the user’s behaviours. The
two contests using badges ranked among the less effective ones, and our inferential statistics confirm that
participation and output are higher when money or goods are present. Still, previous studies showed that
experiences based exclusively on rewards (whether tangible or non-tangible) may not be effective in the mid-
and long term. Some of their risks are the hedonic adaptation of the users (Frederick & Loewenstein, 1999) or
the overjustification effect (Tang & Hall, 1995), which may explain why economic awards, although they may
yield better results in the short term, are not consolidating and increasing participation significantly. In recent
years, some authors have used words such as exploitation (Bogost, 2011) or dictatorship (Escribano, 2013) to
refer to practices where only behaviourist rewards are given. Furthermore, self-determination and cognitive
evaluation theory propose that intrinsic motivation for an activity can only be promoted by fulfilling the basic
psychological needs of autonomy, competence and relatedness with others (Deci & Ryan, 2008). In addition,
recent studies have shown that, in order to achieve this goal, gamified practices should provide users with
meaningful actions to be executed and emotions to be experienced (Oceja & González-Fernández, 2017).
We are aware of the limitations of this study, namely that some variables (themes, dates, etc.) have not been
deeply analysed. In the future, it would be interesting to carry out design-based research (or even some action
507
Jorge Oceja and Ángel Sierra-Obregón
research) in order to test the effectiveness of real game-based strategies that focus on playful actions, challenges
and discovery and that take the figure of editors as real players.
References
Antin, J., Yee, R., Cheshire, C., & Nov, O. (2011). Gender Differences in Wikipedia Editing. Proceedings of the WikiSym’11.
Mountain View, United States. Retrieved from
http://people.ischool.berkeley.edu/~coye/Pubs/Articles/GenderWikiSym2011.pdf
Bogost, I. (May 3, 2011). Persuasive Games: Exploitationware [Blog entry]. Retrieved from
http://www.gamasutra.com/view/feature/134735/persuasive_games_exploitationware.php
Bogost, I. (2015). Why gamification is bullshit. En S. Walz & S. Deterding, S. (Eds.), The Gameful World: Approaches, Issues,
Applications, (pp 65-80). Cambridge, MA: MIT Press.
Brathwaite, B., & Schreiber, I. (2008). Challenges for Game Designers. Cambridge, MA: Charles River Media.
Class Twist Inc. (2016). ClassDojo (Version 3.4) [Mobile app]. San Francisco, CA: ClassDojo.
Deci, E. L., & Ryan, R. M. (2008). Self-determination theory: A macrotheory of human motivation, development, and health.
Canadian Psychology/Psychologie Canadienne, 49(3), 182–185. https://doi.org/10.1037/a0012801
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining gamification.
En A. Lugmayr (Ed.), Proceedings of the 15th international academic MindTrek conference: Envisioning future media
environments (pp. 9–15). Tampere: Finland: ACM. Retrieved from http://dl.acm.org/citation.cfm?id=2181040
Deterding, S., Sicart, M., Nacke, L., O’Hara, K., & Dixon, D. (2011). Gamification. Using Game-design Elements in Non-
gaming Contexts. En D. Tan, B. Begoie y W. Kellog (Eds.), CHI ’11 Extended Abstracts on Human Factors in Computing
Systems (pp. 2425–2428). New York, NY: ACM. https://doi.org/10.1145/1979742.1979575
Escribano, F. (2013). Gamificación versus Ludictadura. Obra Digital: Revista de Comunicación, (5), 58–72.
http://dx.doi.org/10.25029/od.2013.22.5
Fitocracy Inc. (2016). Fitocracy (Version 3.1) [Mobile app]. New York: NY: Fitocracy Inc.
Frederick, S., & Loewenstein, G. (1999). Hedonic adaptation. In D. Kahneman, E. Diener, & N. Schwarz (Eds.), Well-being:
The foundations of hedonic psychology (pp. 302-329). New York: Russell Sage Foundation.
Fuchs, M., Fizek, S., & Ruffino, P. (2014). Rethinking Gamification. Lüneburg, Germany: Meson Press.
Göbel, S., & Munzert, S. (April 4, 2016). Political advertising on the Wikipedia market place of information. University of
Konstanz. Social Science Computer Review. Retrieved from
https://papers.ssrn.com/sol3/papers2.cfm?abstract_id=2720141
Graham, M. (2011). Wiki Space. Palimpsests and the politics of exclusion. In Lovink, G., & Tkacz, N. Critical Point of View: A
Wikipedia Reader. Amsterdam: Institute of Network Cultures.
Hunicke, R., Leblanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. In D. Fu y J Orkin
(Eds.), Proceedings of the Challenges in Games AI Workshop, 19th National Conference of Artificial Intelligence (pp. 1–
5). Menlo Park, CA: AAAI Press.
Iñiguez, G., Török, J., Yasseri, T., Kaski, K., & Kertész, J. (2014). Modeling social dynamics in a collaborative environment. EPJ
Data Science, 3(1), 1-20. doi: 10.1140/epjds/s13688-014-0007-z
Kittur, A., Suh, B., Pendleton, B. A., & Chi, E. H. (2007). He says, she says: Conflict and coordination in Wikipedia. 25th
Annual ACM Conference on Human Factors in Computing Systems (CHI 2007), 453-462. San José, United States.
Lanamäki, A., Iivari, N., Rajanen, M., & Hedberg, H. (2015). Battle over media choice: Multiplex tensions in the online
community of Wikipedia. Twenty-Third European Conference on Information Systems (ECIS), Münster, Germany.
Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude,
Learning, and Instruction, 3(1987), 223–253.
Manske, M. (2018). Magnus Manske, the page you all have waited for! Retrieved from http://magnusmanske.de/
Oceja, J., & González-Fernández, N. (2016). Actors, Elements, and Innovative Interfaces in Game Experiences: CCAE as a
Model For Analysing Game Elements. In T. Connolly y L. Boyle (Eds.). European Conference on Games Based Learning
(p. 509-514). Peasley, Scothand: ACI. Retrieved from
https://www.researchgate.net/publication/308678482_Actors_Elements_and_Innovative_Interfaces_in_Game_Expe
riences_CCAE_as_a_Model_For_Analysing_Game_Elements
Oceja, J. (2017). Diseño de experiencias de juego para la promoción de la competencia cívica (Doctoral dissertation).
Retrieved from https://repositorio.unican.es/xmlui/handle/10902/12426
Raessens, J. (2006). Playful identities, or the ludification of culture. Games and Culture, 1(1), 52–57.
Rodrigues, F. (2012). Mass collaboration or mass amateurism? A comparative study on the quality of scientific information
produced using Wiki tools and concepts (Doctoral thesis). University of Évora.
Tang, S.-H., & Hall, V. C. (1995). The overjustification effect: A meta-analysis. Applied Cognitive Psychology, 9(5), 365–404.
https://doi.org/10.1002/acp.2350090502
Versteeg, M. J. J. M. (September 9, 2013). Ethics & Gamification design: a moral framework for taking responsibility (M.A
Thesis). Retrieved from http://dspace.library.uu.nl/handle/1874/281831
Werbach, K., & Hunter, D. (2012). For the Win: How Game Thinking Can Revolutionize Your Business. Philadelphia, PA:
Wharton Digital Press.
Zichermann, G., & Linder, J. (2013). The Gamification Revolution: How Leaders Leverage Game Mechanics to Crush the
Competition. New York, NY: McGraw-Hill Education.
508
