Content uploaded by Sami Hyrynsalmi
Author content
All content in this area was uploaded by Sami Hyrynsalmi on May 09, 2017
Content may be subject to copyright.
The Dark Side of Gamification:
How We Should Stop Worrying and Study also the Negative Impacts
of Bringing Game Design Elements to Everywhere
Sami Hyrynsalmi
Tampere University of Technology
Finland
sami.hyrynsalmi@tut.fi
Jouni Smed, Kai K. Kimppa
University of Turku
Finland
jouni.smed@utu.fi, kai.kimppa@utu.fi
Abstract: There are always two sides to every story. This statement is true also for the most recent hype term
gamification – i.e., bringing game design elements into non-game contexts – that has been used to improve users’
motivation and performance in various domains. Previous studies on gamification have mainly taken a positive
approach towards the phenomenon and its implications. To depart from the existing research, this tertiary literature
review assesses the negative effects of gamification (such as game addiction and ethical issues). The systematic
literature review method is followed in collection of 22 literature studies published on gamification. The analysis
of these secondary studies show that while several researchers acknowledge possible problems and consequences,
there is a clear research gap in understanding the negative impacts of gamification. We categorize the presented
negative implications to limiting and harmful issues. Finally, this study calls for further work assessing and defin-
ing the limitations and borders for the ethical use of game design elements in everyday life as well as for growing
our understanding for harmful issues.
Keywords: Gamification, Literature study, Tertiary review, Addiction
1. Introduction
An old idiom of English language states that “there are two sides to every coin”, which emphasises
that there can be two different but closely related features for the same idea. For example, punishments
and rewards can both be used to guide people towards set goals. Recently, this idiom has been in a cen-
tral role in the design of new information systems as well as new services, where the aim has not been
in forcing either the employee or the customer to use the invented solution but, rather, in making them
enjoy the use of the artefact by adding elements that were previously met in games and traditionally not
in workplace or other environments. The term ‘gamification’ was coined by Nick Pelling
1
in 2002 to
describe such use of game design elements in non-game contexts (Huotari & Hamari, 2012).
Gamification has become popular among information and software system researchers as well as prac-
titioners during the last few years. For example, Google trends
2
indicates that in the beginning of
2010s, there was practically no (search) interest towards the concept. The search activity started to
grow in 2011 and peaked in 2014. Since then, curiosity towards the concept has remained stable. Fur-
thermore, one can easily find numerous examples of gamified solutions in a business domain such as
Visual Studio Achievements, JIRA Hero, in a well-being area such as Nike+, Zombies, Run!, Super-
better as well as in an educational sector (see e.g. Hidalgo-Céspedes; Marín-Raventós; & Lara-
Villagrán, 2016).
Gamification has been showed to improve motivation of, e.g., improving motivation and performance,
while there are some caveats (Hamari; Koivisto; & Sarsa, 2014). Adding game design elements into
educational systems seems to be useful in motivating students, improving their skills and learning (de
Sousa Borges; Durelli; Reis; & Isotani, 2014). The use of gamification has been even discussed in li-
1
Conundra Ltd. http://www.nanodome.com/conundra.co.uk/ Accessed January 30th, 2017.
2
Google Trends for the search term ’gamification’ - https://www.google.com/trends/explore?q=gamification Accessed January
30th, 2017.
braries (Brown, 2014) as well as in software development environments (Pedreira; García; Brisaboa; &
Piattini, 2015).
While gamification has been mainly used on improving users’ interests on virtuous issues and tasks,
there are also clearly dark issues where it has been applied. As an example, the rumored Russian game
‘Blue Whale’
3
is claimed to give increasingly more dangerous tasks to players. To proceed in the game,
50 daily tasks have to be completed and documented, and to “win” the player has to complete the final
task, committing a suicide. Although there is no conclusive proof that this game actually exists, it gives
an extreme example of how gamified elements could be used for harmful purposes. Similarly, there are
reports that gamification techniques have been used to motivate anarchists to steal or damage CCTV
cameras in Berlin, Germany (Versteeg, 2013). However, these are examples of extreme malpractices of
any kinds of technologies, methods and tools.
In addition, there are naturally limits to where gamification works. That is, there are domains and tasks
where applying gamification is not clearly virtuous or immoral. As gamification is often used to moti-
vate the user, it does not add anything extra if there is enough motivation already (e.g. meeting
friends). Certain areas of application require utmost speed, usability and urgency (e.g. paramedics or
firefighters), and adding gamified elements there could have serious negative results. There is also the
consideration, when gamification clashes with ethics (e.g. casinos, gambling, game addiction). This is
the grey area where we focus in this paper.
The old English idiom, that every coin has two sides, translates badly to Finnish language. The idiom
can be translated word-by-word to Finnish but its meaning changes: the verbatim translation states that
every story has two sides. The same applies for ‘gamification’: no change in a complex system can be
done without consequences (c.f. ‘inseparability postulate’ by Nurminen & Forsman, 1994). The objec-
tive of this study is to uncover what are researchers’ perception of the negative side effects caused by
adding game elements to everyday life. The study is loosely motivated by the observation that addic-
tion to games is a growing, although it still is a niche problem in modern societies (see e.g. Søraker,
2016). Thus, an interesting question arises on how the gamification research community has strived to
solve the problem of bringing potentially addictive gaming elements into the design of everyday things.
To generalize this question, this study seeks to answer to the research question:
RQ: How researchers have perceived the negative side effects of applying gamification?
To seek an answer to the question, we study the extant literature of gamification. We use systematic
literature review (SLR) approach to collect existing meta-studies of gamification. As there is already a
plethora of existing literature surveys, we decided to focus on these to map the current knowledge from
all fields. The meta-studies are used to analyse what is the current knowledge on gamification’s un-
wanted impacts.
This study is structured as follows. In the following section, we will present the search and analysis
protocol used in this study. The third section presents results found from the analysis of selected sec-
ondary studies. The last two sections, discuss the implications of this study, propose new avenues for
future inquiries and, finally, conclude this study with key arguments as well as limitations.
2. Research approach
The aim of this study is to map existing knowledge on negative impacts of gamification. We use Sys-
tematic literature review (SLR) method to gather all relevant tertiary studies for this study. In SLR, we
follow the guidelines given by Kitchen and Charters (2007). For the selected tertiary studies, we per-
form content analyses and attempt to find all relevant evidence and primary studies related to the RQ.
3
The Sun - ‘Blue Whale’ suicide game linked to 130 teen deaths is just tip of the iceberg in the world’s suicide capital Russia
https://www.thesun.co.uk/news/3002981/blue-whale-game-suicide-russia-rules-challenge-social-media/ Accessed April 19th,
2017.
This study uses a simple three steps research process. In the first phase, we construct a search term,
inclusion and exclusion criteria and decide the used publication forums and databases. In the second
phase, we apply the search term and select the studies to be included. In the final phase, the chosen
studies are analysed and issues raised in them are collected and categorized. In the following we will
go through these steps with more detail.
There are two main approaches for selecting primary studies in a systematic literature review method:
either a manual search to selected publication forums or an electronic search to the selected databases
(Kitchenham & Charters, 2007). We decided to use the latter in order to maximize the coverage of the
search. If we would have decided to concentrate on pre-selected publication forums and stick to those
only, we would have unnecessary restricted the potentially small set of tertiary studies. The selected
electronic publication databases are given in Table 1 with the number of hits returned and selected
studies by each database.
In the first phase, we constructed a search term that we applied to all selected electronic publication
databases. The search term used contains two parts: First, we request that the term ‘gamification’ ap-
pear in the publication. Second, the study has to be some sort of a literature study. Therefore, at least
one of the most common keywords related to literature reviews have to be present in the publication.
The general search term is:
gamification AND ("literature review" OR "literature study" OR "sys-
tematic review" OR "systematic mapping")
The search was targeted, when possible, to the title, abstract, and keywords. Naturally, the search term
was adapted to each of the electronic publication databases used according to the specific features of
the database. The searches for all databases were done at January 27th, 2017.
In the second phase, all the found tertiary studies were evaluated based on the title and abstract. We
used simple inclusion and exclusion criteria: Studies were included if they were written in English,
published in a peer-reviewed forum, focused on some aspect of gamification and the research approach
was a literature survey. Both systematic and non-systematic literature studies were included. We ex-
cluded posters, commentaries, extended abstracts and prefaces; studies written with other language
than English as well as studies that did not either focus on gamification or were not literature surveys.
In the final phase, the selected studies were analysed. We used content analysis. In this, all negative
aspects of gamification found in the tertiary studies were marked and referred primary studied were
sought. The researchers then synthesized the results found by grouping similar issues raised into cate-
gories. Finally, in the discussion among the authors, gaps in the existing literature were spotted and
proposed topics for future inquiries were formed.
3. Results
In the following, we will first present a categorization of found negative aspects that fall into two main
groups: limiting issues and harmful issues. In the latter parts of this section, we will focus more on the
issues belonging to the harmful category.
In total, we selected 22 literature studies assessing gamification from various viewpoints. The selected
secondary studies are shown in Table 2. Out of the selected studies, six are journal articles and the rest
Table 1: The publication databases used and the number of matches as well as the number of in-
cluded publications (contains duplicates).
Database
Hits
Included
ACM Digital Library
6
5
AISel
5
4
IEEE Xplore Digital Library
7
6
ScienceDirect
5
5
Scopus
6
6
Wiley Online Library
1
0
All together
30
26
are published as part of conference proceedings. The studies cover a broad area of different research
fields from education and library science to information systems science and software engineering.
Therefore, the decision to use electronic search in a wide set of different publication databases seems to
be justified.
Despite a large number of systematic and disorganized literature reviews published on gamification, we
did not find a prior meta-study focusing solely on the negative aspects of gamification. On the contrary,
most of the secondary studies have mainly a positive view towards gamification. That is, the studies
were focusing on the found positive implications and effects of the gamified solutions. We share this
same observation with Bui et al. (2015) who state that most of the primary studies that they went
through have rarely addresses potential downsides of gamification. In addition, Kim and Werbach
(2016) argue in their study that the ethical side of gamification has been left without much attention.
Most of the analysed secondary studies had only little if any discussion on the negative effects of gami-
fication. The negative observations summarized in the secondary studies can be categorized roughly
into two main themes: The first group is formed from the worries about gamification limiting the full
capabilities of an artefact. The second group contains discussions on the harmful implications of gami-
fication. To put it simply, the observations in the former group are related to gamification not produc-
ing the best results. The discussion in the latter group is related on clearly negative impacts of gamifi-
cation to for example, the behaviour of the users.
First, several authors have expressed their worries about limitations of gamification. For example, users
might be optimizing the end-result of the ‘game’ (e.g., positions in leader boards), and not the task at
hand (Knaving & Björk, 2013; Silpasuwanchai; Ma; Shigemasu; & Ren, 2016). Gaming elements, that
are lucrative for a single person, can present competing interests against teamwork, thus hindering the
team to achieve the best performance (Marlow; Salas; Landon; & Presnell, 2016). Both of these two
previous limitations fall within “you get what you measure” problem. In addition, gamified solutions
can also be found demotivating due to, e.g., frustrating simplicity or childishness of a request task (Au-
gustin; Thiebes; Lins; Linden; & Basten, 2016).
Second, only a few authors have discussed the harmful consequences of gamification. Harmful conse-
quences, as the name suggest, are questionable – and potentially unethical – side effects of the gamified
features. For example, as pointed out by Bui et al. (2015), gamified solutions could encourage users to
perform behaviours only when rewarded. Furthermore, losses in productivity can be faced when gami-
fied elements distracts users from the main purpose of the system (Thiebes; Lins; & Basten, 2014).
To generalize, the observations in the limiting category are more or less related to unsuccessful imple-
mentation of gamified features or failed deployment of the new system. That is, if positions in the lead-
er board are not reflecting all wanted tasks, it is likely a signal of failed requirements elicitation work.
Similarly, if gamified elements do not support learning for all kinds of students, alternative solutions
should be studied and supported. Even though these are extremely important aspects of successful de-
ployment of a new system or a process, these are mainly limiting either the system or user to achieve
the full potential value that the gamified solution could offer. The main dichotomy between the two
groups is that limiting elements are usually issues that can be fixed more easily than the issues belong-
ing to the harmful category.
The issues belonging to the latter of the two groups, the harmful issues, are more complex and the
number of primary as well as secondary studies assessing these is scarce. The secondary studies ana-
lysed refer to some primary studies touching the topic. For example, Nicholson (2012) argues for
bringing user-centred design principles into gamification discussion. However, he also discussed the
problem that gamification might replace internal motivation with pursuit of extrinsic rewards. This
would, ultimately, diminish the initial goal of gamification practices: to increase the motivation of the
users.
By far, the most thought-provoking article has been written by Kim and Werbach (2016). In their
study, they present a two-by-two framework for mapping gamification ethics. The main question of the
taxonomy’s four categories are whether or not (1) gamification practices take unfair advantage of
workers; (2) they infringe users autonomy; (3) do they harm users or others; and (4) whether gamified
practices have a negative effect on the moral character of the involved users?
Table 2: The selected secondary studies.
ID
Selected study and the authors
1
Are We Playing Yet? A Review of Gamified Enterprise Systems
(Augustin;Thiebes;Lins;Linden;& Basten, 2016)
2
A Literature Review of How Videogames Are Assessed in Library and Information Science and
Beyond (Brown, 2014)
4
Gamification – A Novel Phenomenon or a New Wrapping for Existing Concepts?
(Bui;Veit;& Webster, 2015)
5
A systematic mapping on gamification applied to education
(de Sousa Borges;Durelli;Reis;& Isotani, 2014)
6
Gamification in education: A systematic mapping study
(Dicheva;Dichev;Agre;& Angelova, 2015)
7
Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification
(Hamari;Koivisto;& Sarsa, 2014)
8
Learning principles in program visualizations: A systematic literature review
(Hidalgo-Céspedes;Marín-Raventós;& Lara-Villagrán, 2016)
9
Gamification for health and wellbeing: A systematic review of the literature
(Johnson, et al.., 2016)
10
Designing for Fun and Play: Exploring Possibilities in Design for Gamification
(Knaving & Björk, 2013)
11
Eliciting teamwork with game attributes: A systematic review and research agenda
(Marlow;Salas;Landon;& Presnell, 2016)
12
A Literature Review of Gamification Design Frameworks
(Mora;Riera;González;& Arnedo-Moreno, 2015)
13
Fun beliefs in digital games from the perspective of human nature: A systematic review
(Normal;MdNor;& Ishak, 2014)
14
Gamification in software engineering – A systematic mapping
(Pedreira;García;Brisaboa;& Piattini, 2015)
15
Competing or Aiming to Be Average?: Normification As a Means of Engaging Digital Volunteers
(Preist;Massung;& Coyle, 2014)
16
Serious games and active healthy ageing: A pilot usability testing of existing games
(Pyae;Raitoharju;Luimula;Pitkäkangas;& Smed, 2016)
17
Developing a Conceptual Model for Facilitating the Issuing of Digital Badges in a Resource
Constrained Environment
(Salerno;Ouma;& Botha, 2015)
18
A Descriptive Literature Review and Classification Framework for Gamification in Information
Systems
(Schlagenhaufer & Amberg, 2015)
19
Developing a Comprehensive Engagement Framework of Gamification for Reflective Learning
(Silpasuwanchai;Ma;Shigemasu;& Ren, 2016)
20
Gamifying Information Systems - A Synthesis of Gamification Mechanisms and Dynamics
(Thiebes;Lins;& Basten, 2014)
21
A Systematic Mapping Study on Gamified Software Quality
(Vargas-Enríquez;García-Mundo;Genero;& Piattini, 2015)
22
A survey of gamification for healthcare system
(Wen;Hsien;& Huang, 2015)
4. Research gaps
This study seeks an answer to the research question how researchers have perceived gamifications’
negative implications and to uncover what do we know about the dark side. Based on the literature
study of 22 secondary studies and selected primary studies, the answer is simple: Not much. The most
complete work has been presented by Kim and Werbach (2016), while they also acknowledge that their
framework is not complete and there could be more moral issues that should be noted.
Nevertheless, at least some researchers are aware of the negative impacts and possible side effects.
Several of the studied literature reviews mention that there are clear limitations and potentially unwant-
ed side effects that should be noted. However, only a few studies have taken a step forward concretiz-
ing the ethics of gamification, and there are clear ethical and practical questions still left unanswered.
In the following, we will present few of the open research avenues regarding the negative implications
of gamification.
First, we can take an assumption that where gamification is applied, there are also similar problems that
have been faced with games. Thus, the questions related to cheating – that have seen in all single-
player, multiplayer and online multiplayer games – are present also in a gamified solutions. Second, if
games can exploit their users, also gamified solutions can follow that path. However, if gamification
solutions follow the rules of games, then researchers could follow, for example, the guidelines set by
Søraker (2016) regarding exploitation in games. He has suggested that the ‘gamers’ should be educated
on the incentives of the systems so that they would be aware of possible problems by themselves. This
could be a reasonable approach to study as nowadays the number of gamified solutions is growing rap-
idly.
Third, one of the most interesting questions faced during this study is: can gamified solutions be used
with employees who have a history or tendency towards game or gambling addiction? Also, how ethi-
cally justified is it to use gamified solutions in systems and services marketed for children (see also e.g.
Sprenkels and van der Ploeg, 2011)? These questions have been amazingly little discussed in the litera-
ture. We were also surprised that health and well-being related secondary studies selected for this as-
sessment do not cover this issue.
Fourth, what can and cannot be gamified? We assume that applying gamification in the work of para-
medics or firefighters might not be a good solution as unnecessary steps in their work might rather
cause harmful impacts. However, where is the line between useful gamification and adding game de-
sign elements into tasks that can cause dangers, remarkable losses or that are ethically questionable?
Furthermore, can gamification cause threatening situations and what would be an ethically justified
way of handling these? As an example of blind trust to technology, we could consider cases where a
car driver has uncritically trusted in a GPS navigator and driven his or her car to a lake
4
.
5. Discussion and Conclusions
This study addressed the extant knowledge on the negative effects of gamification by using a literature
study. We used systematic literature review to collect the articles and we focused on existing literature
studies on gamification. In total, 22 secondary studies were analysed for this tertiary study. As a result,
we found little prior discussion on the negative impacts of gamification. Few of the existing studies
have addressed the topic; however, this study seems to be the first secondary study to be devoted to this
theme.
We classified the present worries into two categories: limiting and harmful issues. After the categoriza-
tion of the issues and analysis of the presented taxonomy, we noted that there are inherited similarities
with Brooks’ (1987) classic categorization of software engineering fields’ complexity. While the simi-
larity is unintentional, it serves well in concretizing the difference between the two categories. In
Brooks’ observation, accidental complexities are software engineering issues that engineers create and,
thus they can be fixed. For example, inefficient tools, processes and methods belong to this complexity
group. Essential complexity, on the contrary, rises from the basic nature of the software. For example,
4
Lauren Hansen. 8 drives who blindly followed their GPS into disaster. The WEEK. http://theweek.com/articles/464674/8-
drivers-who-blindly-followed-gps-into-disaster Accessed January 31th, 2017.
the changeability of software causes that everything can be changed almost any time in a software en-
gineering project – even after the deployment of the product.
First in our categorization, there are issues limiting to achieving the full capability of either gamified
solutions or their users. While these issues are important for the successful deployment and use of any
gamified solution, they are, by following Brooks’ (1987) taxonomy, accidents – issues that can be fixed
with relatively reasonable amount of work. Second, there are harmful issues that clearly represent the
negative side effect of gamification. These are, in Brooks’ (1987) taxonomy, the essential aspects of
gamification; prevalent issues that are visible in the horizon of the decade regardless of new tech-
niques, taxonomies and methods developed.
The key argument of this study is that the gamification research field should do the same that software
engineers did after Brooks’ seminal paper: We should move our attention from tackling limiting prob-
lems to study and understand harmful issues. Even though removing the limiting issues is important for
the acceptance and successfulness of any gamified solution, the open questions on applicability and on
borders of gamification lies in the latter, more important category.
Naturally, there are certain limitations for this study that are noteworthy to discuss. First, instead of
focusing on primary studies, we used secondary studies as the main study objects in this tertiary re-
view. The decision was justified with the existence of several literature studies and with the observa-
tion that gamification research is widely applied in several disciplines from tourism studies to library
sciences. Nevertheless, this study is limited by the quality and discussion presented in the selected sec-
ondary studies. Further work is needed to map primary studies in order to draw a richer picture of the
field.
Second, while we strictly followed the guidelines for conducting SLRs, the search term, selected data-
bases as well as inclusion and exclusion criteria are crucial for the generalization of a literature re-
view’s results. Whereas these were carefully chosen by following the existing literature surveys (e.g.
Hamari et al., 2014), the decision made has likely still ruled out relevant venues and studies.
This study opens interesting avenues for future inquiries. We showed that there is a clear research gap
on understanding what the limits of gamification are and proposed four topics for future work. The list
is not complete and not all of the presented questions might be, in the end, relevant in the gamification
context. However, this study aims to wake up the discussion about the limitations and harmful aspects
of gamification for the gamification research community and beyond.
Finally, while the subtitle of this study humorously refers to Stanley Kubrick’s classical movie, there is
also a darker tone present in the topic. The history of video games has been full of strange prejudices
and ‘studies’ (see e.g. Anderson, 2004, amongst others by him and his ilk). The aim of this study is not
to encourage users to avoid gamified solutions; instead, we call the researchers to also address the
darker side of gamification and to honestly investigate the negative impacts of gamifying everyday
solutions.
References
Anderson, C. A. (2004). An update on the effects of violent video games. Journal of Adolescence 27,
113-122.
Augustin, K., Thiebes, S., Lins, S., Linden, R., & Basten, D. (2016). Are We Playing Yet? A Review
of Gamified Enterprise Systems. PACIS 2016 (p. Paper 2.). AIS.
Brooks, F. P. (1987). No Silver Bullet: Essence and Accidents of Software Engineering. Computer,
20(4), 10-19.
Brown, R. T. (2014). A Literature Review of How Videogames are Addressed in Library and Infor-
mation Science and Beyond. The Journal of Academic Librarianship, 40, 447-451.
Bui, A., Veit, D., & Webster, J. (2015). Gamification - A Novel Phenomenon or a New Wrapping for
Existing Concepts? Thirty Sixth International Conference on Information Systems (pp. 1-21). AIS.
de Sousa Borges, S., Durelli, V. H., Reis, H. M., & Isotani, S. (2014). A Systematic Mapping on Gami-
fication Applied to Education. SAC'14 (pp. 217-222). ACM.
Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamificatio in Education: A Systematic
Mapping Study. Educational Technology & Society, 18(3), 75-88.
Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does Gamification Work? -- A Literature Review of Em-
pirical Studies on Gamification. Hawaii International Conference on System Science (pp. 3025-3034).
IEEE.
Hidalgo-Céspedes, Marín-Raventós, G., & Lara-Villagrán, V. (2016). Learning principles in program
visualizations: a systematic literature review. 2016 IEEE Frontiers in Education Conference (FIE) (pp.
1-9). IEEE.
Huotari, K., & Hamari, J. (2012). Defining Gamification - A Service Marketing Perspective. 16th In-
ternational Academic Mindtrek Conference (pp. 17-22). Tampere, Finland: ACM.
Johnson, D., Deterding, S., Kuhn, K.-A., Staneva, A., Stoyanov, S., & Hides, L. (2016). Gamification
for health and well-being: A systematic review of literature. Internert Interventions, 6, 89-106.
Kim, T. W., & Werbach, K. (2016). More than just a game: ethical issues in gamification. Ethics in
Information Technology, 18, 157-73.
Kitchenham, B., & Charters, S. (2007). Guidelines for performing Systematic Literature Reviews in
Software Engineering (version 2.3). Keele, Staffs, UK: Keele University.
Knaving, K., & Björk, S. (2013). Designing for Fun and Play: Exploring possibilites in design for gam-
ification. Gamification'13 (pp. 131-134). ACM.
Marlow, S. L., Salas, E., Landon, L. B., & Presnell, B. (2016). Eliciting teamwork with game attrib-
utes: A systematic review and research agenda. Computers in Human Behavior, 55, 413-423.
Mora, A., Riera, D., González, & Arnedo-Moreno, J. (2015). A literature review of gamification design
frameworks. 2015 7th International Conference on Games and Virtual Worlds for Serious Applications
(VS-Games) (pp. 1-8). IEEE.
Nicholson, S. (2012). A User-Centered Theoretical Framework for Meaningful Gamification.
Games+Learning+Society 8.0, (pp. 1-7). Madison, WI.
Normal, M. J., MdNor, K., & Ishak, B. I. (2014). Fun Beliefs in Digital Games from the Perspective of
Human Nature: A Systematic Review. ISTMET 2014 (pp. 359-364). IEEE.
Nurminen, M.I., & Forsman, U. (1994). Reversed Quality Life Cycle Model. Human Factors in Or-
ganizational Design and Management – IV (pp. 393-398). North-Holland, Amsterdam.
Pedreira, O., García, F., Brisaboa, N., & Piattini, M. (2015). Gamification in software engineering -- A
systematic mapping. Information and Software Technology, 57, 157-168.
Preist, C., Massung, E., & Coyle, D. (2014). Competing or Aiming to be Average? Normification as a
Means of Engaging Digital Volunteers. CSCW'14 (pp. 1222-1233). ACM.
Pyae, A., Raitoharju, R., Luimula, M., Pitkäkangas, P., & Smed, J. (2016). Serious games and active
healthy ageing: a pilot usability testing of existing games. International Journal of Networking and
Virtual Organizations, 16(1), 103-120.
Salerno, C., Ouma, S., & Botha, A. (2015). Developing of a Conceptual Model for Facilitating the Is-
suing of Digital Badges in a Resource Constrained Environment. SAICSIT'15 (pp. 1-8). ACM.
Schlagenhaufer, C., & Amberg, M. (2015). A Descriptive Literature Review and Classification
Framework for Gamification in Information Systems. ECIS 2015 Complete Research Papers (p. Paper
161). AIS.
Silpasuwanchai, C., Ma, X., Shigemasu, H., & Ren, X. (2016). Developing a Comprehensive Engage-
ment Framework of Gamification for Reflective Learning. DIS 2016 (pp. 459-472). ACM.
Sprenkels, I. & van der Ploeg, I. (2011) Eeny, Meeny, Miny, Masquerade! Advergames and Dutch
Children; A Controversial Marketing Practice. Ethicomp 2011
Søraker, J. H. (2016). Gaming the gamer? -- The ethics of exploiting psychological research in video
games. Journal of Information, Communnication and Ethics in Society, 14(2), 106-123.
Thiebes, S., Lins, S., & Basten, D. (2014). Gamifying information systems - a synthesis of gamification
mechanism and dynamics. Proceedings of the European Conference on Information Systems (ECIS)
(pp. 1-17). AIS.
Vargas-Enríquez, J., García-Mundo, L., Genero, M., & Piattini, M. (2015). A systematic mapping
study on gamified software quality. 2015 7th International Conference on Games and Virtual Worlds
for Serious Applications (VS-Games) (pp. 1-8). IEEE.
Versteeg, C. (2013) Ethics & Gamification design: a moral framework for taking responsibility. Mas-
ter’s thesis. The Netherlands, Utrecht University.
Wen, C.-H., Hsien, H.-Y., & Huang, H.-C. (2015). A Survey of Gamification for Healthcare System.
ICOT'2015 (pp. 117-121). IEEE.
