Content uploaded by Seiji Isotani
Author content
All content in this area was uploaded by Seiji Isotani on Feb 01, 2015
Content may be subject to copyright.
Does Gamification Work for Boys and Girls? An
Exploratory Study with a Virtual Learning
Environment
Lais Z. Pedro
University of São Paulo (USP)
São Carlos – SP - Brazil
laiszp@icmc.usp.br
Aparecida M. Z. Lopes
University of São Paulo (USP)
Fatec Jahu – Jaú – SP – Brazil
cidazem@icmc.usp.br
Bruno G. Prates
Federal Institute of São Paulo
São Carlos – SP - Brazil
brunogenovaprates@gmail.com
Julita Vassileva
University of Saskatchewan,
Saskatoon - SK- Canada
jiv@cs.usask.ca
Seiji Isotani
University of São Paulo (USP)
São Carlos – SP - Brazil
sisotani@icmc.usp.br
ABSTRACT
The development and use of Virtual Learning Environments
(VLE) has increased considerably over the past decades.
Following that trend, many research findings have shown the
benefits of using VLE during the learning process. Nevertheless,
there are important problems that hinder their use requiring
further investigation. Among them, one of the main problems is
the inappropriate use of these systems by students. The boredom,
lack of interest, monotony, lack of motivation, among other
factors, ultimately causes students to behave inappropriately and
lead them to a lower performance. In this context, the proposed
study investigates whether it is possible to reduce undesirable
behaviors and increase performance of students through the use of
game mechanics (i.e. gamification). We develop a VLE, E-Game,
that can turn on/off several game mechanics, such as points,
badges, levels and so on. A case study was conducted with two
groups of students to investigate their behavior during their
interaction with E-Game with and without gamification. The
results indicate that the gamification implemented by E-Game
contributed to improve student performance in the case of boys.
Yet, improvement was not observed in the case of girls.
Furthermore, it was not possible to conclude whether the use of
gamification helps to prevent inappropriate student behavior, and
therefore, further studies and experiments are needed.
Categories and Subject Descriptors
• Interactive learning environments; Interactive games
General Terms
Experimentation, Human Factors
Keywords
e-learning, gamification, gender difference, gaming the system
1. INTRODUCTION
The design, implementation and use of game mechanics in non-
game contexts is known as “Gamification” [5]. It involves the use
of aesthetics, mechanics and dynamics in contexts unrelated to
games, to increase motivation and support behavior change.
Currently, there is an increasing interest in using gamification in
educational contexts due to the amount of applications and
research in this area [12,16]. This growing interest can be
explained mainly by the potential of gamification to influence,
engage and motivate people [14].
Studies about the use of games and game mechanics in education
have been conducted for decades [17]. Nevertheless, the interest
in the topic has increased considerably in the past few years due to
the potential of using game mechanics in virtual learning
environments (VLE) to reduce students’ dropout rates and
increase their motivation and participation in online learning
activities [15].
Several researchers suggest that students who use VLE,
particularly intelligent tutoring systems, often learn more and
improve their performance and knowledge retention compared to
students in conventional classrooms [1,11,14,20]. However, in
some cases, factors such as lack of student motivation, boredom
or dissatisfaction with a discipline directly affect the proper use of
these systems [2]. A behavior called by Baker et al. [2] as
“Gaming the System” manifests itself when students ignore the
essence of the learning activities, and find ways to complete them
mechanically without learning the content. Numerous efforts have
been described to detect this type of behavior [3, 4, 6], to perform
some sort of content adaptation that makes it difficult and even
eliminate the continued use of inappropriate behavior [7].
However, no one so far has attempted to prevent this type of
behavior by increasing students’ motivation and desire to learn as
well as their commitment to their learning process.
According to Cytowic [8] cognition, memory and decision
making capacity of the individual are intrinsically linked to
emotions. Studies show that students who feel anxious, upset or
depressed do not assimilate information properly and because of
that, eventually develop inappropriate behaviors that hinder
learning [3, 13]. In contrast, students who feel motivated,
challenged and intrigued tend to get better results. Yet, keeping
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
SAC’15, April 13-17, 2015, Salamanca, Spain.
Copyright 2015 ACM 978-1-4503-3196-8/15/04…$15.00.
http://dx.doi.org/ 10.1145/2695664.2695752
students motivated throughout the whole learning process is
considered one of the major challenges in all forms of learning.
According to Vassileva [21], it is possible to incorporate
mechanisms and tools in the design of applications that can
motivate users and change their behavior in a desirable way.
In this context, we intend to investigate whether the use of
gamification can help to reduce the inappropriate behavior of
students and also improve their performance in VLE. For that, we
developed the E-Game, a gamified educational virtual
environment that rewards students’ successful performance with
points, badges, and levels. A controlled study was conducted with
two groups of students in order to investigate their behavior when
interacting with E-Game with and without game mechanics. The
study aims to analyze the use of game mechanics in educational
systems to improve students’ performance during the learning
process and to reduce the occurrence of undesirable behaviors,
such as Gaming the System.
2. RELATED WORK
In this section, we present an overview of related work about
undesirable behaviors occurring during the use of virtual learning
environments and about increasing students’ motivation through
gamification and game mechanics. We conclude by presenting a
brief overview of the game mechanics chosen to use in this work.
Undesirable behavior
There are many behaviors in which learners may engage during
the use of an educational system. To identify those behaviors,
several studies were considered [2,3,4,6]. According to the
literature, several behaviors, such as gaming the system and lack
of interest, are considered undesirable because they affect
negatively the learning process during the use of an educational
system. Baker et al. [2] describe a family of behaviors that he
called “Gaming the System”, which makes the student ignore the
content to be learned to find short cuts to mechanically perform
learning activities proposed by the system. In other words, the
student has found different ways to cheat the system to get the
right answer, or to get a better performance without learning the
content. In this aspect, “Gaming the System” is considered the
most problematic undesirable behavior, since it affects directly the
process of learning and by cheating the virtual learning
environment, it is difficult to verify whether the learner is
studying properly, but not learning, or behaving like someone
who is studying properly, but in fact he/she is not.
Motivation through Gamification
Gamification is the integration of game mechanics in non-game
environments to increase audience engagement, loyalty and fun
[10]. Although the term of “Gamification” is new, it is directly
related with the concept of games and game mechanics, which has
accumulated a number of patterns, rules and feedbacks that create
user engagement, are motivational and can be applied to develop
game-like mechanics in any application, including educational
environment [21]. According to Vassileva [21], the most
commonly used game mechanics are summarized in Table 1.
PATTERNDESCRIPTION
OwnershipAllowing the user to own things, such as points,
token, badges. It creates loyalty to the system.
A
chievementsProviding a virtual or physical representation of
having accomplished something that can be easy,
difficult, surprising, funny, and accomplished alone
or as a group.
Status Computing and displaying rank or level or a user.
Community
collaborationand
quests
Posing challenges to the users related to time-limit
or competition, that can be resolved by working
together.
Kapp [15] defines different patterns that he calls “Game
Elements”, which the games are based on, and the combination of
which largely determine the success or failure of the game.
According to Kapp [15], some of the common Game Elements are
players, abstraction, rules, feedback, quantifying results,
emotional results, storytelling, among others.1
Those elements used independently don’t make a game
interesting. However, combining game elements can make a
difference to increase the motivation and the interest in the
system. Given the research about Gamification, some of the game
mechanics were chosen to be used in the development of a virtual
learning environment, which is part of this study. The game
mechanics, combined with patterns of game mechanics, used in
this work are shown in Table 2.
PATTERN GAMEMECHANICS
Ownerships Points
Badges
Achievements Feedback
Emotional Results
Challenge
Rules
Status Ranking
This combination has the goal to avoid or reduce the
externalization of undesirable behaviors such as Gaming the
System by motivating the student and keep the loyalty between
the player and the game.
3. EVALUATING THE GAMIFICATION
This section presents the development of a gamified educational
virtual environment, E-Game, which implemented all the
gamification mechanics chosen in Table 2. Also, it describes the
experiment to test the impact of gamification on undesirable
learner behaviors and learner achievement that we carried out
using E-Game as a tool. Finally, it discusses the experimental
results.
3.1 Development of E-Game
E-Game was designed to be an environment for support math
teaching and learning process, to virtually support a classroom
and distance learning environment. Moreover, the environment
includes the concepts of gamification in an attempt to prevent or
reduce the student’s behaviors of Gaming the System. It is
possible to create courses and add tasks for each course, where the
students login and complete the tasks assigned to them in a fun
environment. Also, E-Game supports video uploads so the teacher
can add to help the students during their tasks.
The following game mechanics were implemented:
1 Different authors call game mechanics as “game elements”. For
this work, the terminology and differences are not relevant.
Table 2 – Patterns and Game Mechanics chosen
Table 1 – Patterns of Game Mechanics
•POINTS: Each user collects points for each question
answered correctly the user will receive (10 points).
•BADGES: The badges are indirectly related to the points.
The student receives different badges according to the number of
questions he/she answers correctly.
•FEEDBACK: In each question, the system shows 3 buttons
that the user can click: “Help”, “Check” and “Continue”. Before
continuing to the next question, the user has to check if the answer
is correct pressing the “Check” button, and the system gives the
immediate feedback to the user, showing the correct, in case the
answer is wrong. Also, the user can click on the “Help” button,
and it will show a popup with some tips to help answer the
question. However, the tips have a cost for the user of 5 points
from their points score. In this way students are discouraged from
requesting Help without even trying to answer the question (one
of the gaming behaviors identified by Baker et al. [2]).
•RANKING: The ranking of students is based the points won
by them so far. The sidebar menu hosts the ranking, so it is visible
all the time. The ranking also shows the avatar chosen by each
student, and how many points they have.
•EMOTIONAL RESULTS, CHALLENGE, RULES: those
game mechanics are intrinsically implemented and it appears
during the use of the system itself. The emotional results are
consequence of the feedback and the questions to be answered;
the challenge and rules are explained in the beginning of the
system before they started, and could be checked at anytime
during its use.
Figure 1 show the instructions page after the login, where it
explains how the learning environment works. Currently, it
implements with only 3 game mechanics, yet E-Game allows
incorporating different ones, if needed. In the left side (Figure 1),
the menu bar shows the points, progress bar, badges and ranking
of each student logged in. The top contains the number of correct
and incorrect questions during the use, and also a link to the
profile page, which can be accessed anytime.
3.2 Experiment
In order to evaluate the effects of game mechanics and game
mechanics to discourage the student to game the system, we have
developed an experiment using the E-Game in a school from the
Catholic School System, in Saskatoon, Canada. More specifically,
we designed the experiment to examine whether the gamified E-
Game (i.e. which has all the game mechanics implemented)
reduces the occurrence of students’ behaviors of gaming the
system and improves the students’ performance in comparison to
the non-gamified version (i.e. a version of E-Game without the
game mechanics). Thus, our research questions (RQ) can be
formulated as the following:
RQ
1
: Does gamification with points, badges, feedback, and
ranking increase the motivation and help to discourage the student
gaming behaviors during the use of the educational system?
RQ
2
: Do the game mechanics increase the learning performance
of the students during the use of the educational system?
3.2.1 Goal Definition
The case study was conducted in a class of 16 seventh graders (7
girls and 9 boys, ages 12-13) who were using the educational
system E-Game. The class was divided in two groups of 8
students randomly, one using the gamified educational system and
the other group the system without game mechanics. Since we are
aware that with 16 students the results can hardly get statistical
significance and cannot be generalized, this case study has mainly
an exploratory purpose, seeking to preliminary test several
hypotheses and discover the influences that gamification may
have on student performance and motivation.
The purpose of this case study is to evaluate the game mechanics
implemented in E-game in terms of improvement of motivation
(thereby, reducing undesirable behaviors). Specifically, we
investigate whether a gamified learning environment will increase
the motivation in the students and reduce the externalization of
Gaming the System and improve the performance during the
learning process. The experiment provides insight into how many
game mechanics enhanced the use of educational systems by
increasing the motivation and reducing the undesirable behaviors
externalized during its use.
The metrics used to compare the two groups under investigation is
the score (points) obtained by the subjects during the use of E-
Game and analysis of a questionnaire to measure motivation
applied at the end of the task.
3.2.2 Hypothesis Formulation
We formalized the research question (RQ
1
) into hypothesis so that
tests can be carried out:
Null hypothesis, 1H
0
: there is no difference in terms of
motivation and reducing undesirable behaviors in a gamified
educational system between boys and girls (measured in terms of
the score achieved by the questionnaire and the number of ‘help’)
which can be formalized as:
1H
0
: µ
gamified system =
µ
non-gamifi ed system
Alternative hypothesis, 1H
1
: there is a significant difference in
terms of motivation and reducing undesirable behaviors in a
gamified educational system between boys and girls (measured in
terms of the score achieved by the questionnaire and the number
of ‘help’):
1H
1
: µ
gamified system ≠
µ
non-gamified system
The research question RQ
2
is formalized by the following
hypothesis:
Figure 1 – E-Game Educational System: screenshot of
the Instructions page
Null hypothesis, 2H
0
: there is no difference in terms of
performance in a gamified educational system between boys and
girls (measured in terms of the score achieved by the number of
right questions answered) which can be formalized as:
2H
0
: µ
gamified system =
µ
non-gamifi ed system
Alternative hypothesis, 2H
1
: there is a significant difference in
terms of performance in a gamified educational system between
boys and girls (measured in terms of the score achieved by the
number of right questions answered):
2H
1
: µ
gamified system ≠
µ
non-gamified system
3.2.3 Case Study Design
Aimed at verifying our conjecture, we applied a standard design
with one factor and two treatments [22]. The main factor
(independent variable) of the underlying case study is the game
mechanics. The treatments of levels of this factor are two versions
of the system E-Game, a gamified and a non-gamified version. In
this experiment setup, the main dependent variable (or outcome
variable) is the points of the subjects, which is defined by the
number of questions they correctly answered using E-Game.
Furthermore, the scores of the motivational questionnaire are used
as dependent variables as well to analyze some factors as the
subjects interest/enjoyment, perceived competence, perceived
choice and pressure/tension during the use of the educational
system.
3.2.4 Procedure
The case study design is composed by the following steps: (i)
personal questionnaire, (ii) intervention and (iii) motivation
questionnaire. In the first step, the subjects were required to fill
out a questionnaire with their own information, and it was used as
a registration form into the system. The system store and
anonymize the identification information (using an arbitrary
number for participation identification).
In the second step, the students used the learning environment
during one hour for the first time. Although one hour is not
sufficient in some environments, in E-Game was enough to
measure the goal definition of this work. Initially, the system
showed a page explaining how the activity works and the system
rules. Next, the students started solving multiple-choice math
questions based on material they have studied in school, such as
evaluation of algebraic expressions and equations. During their
work on the questions, the students using the gamified system
could see their ranking within the group and their progress bar in
the left sidebar of the interface. . After completing all the
questions, the gamified environment shows the final profile page
with their scores, badges, quantity of right and wrong questions
and the ranking. The non-gamified environment only shows a
final page with the quantity of right answers. Both systems ask the
student to go to the next step and answer a brief questionnaire
about themselves as game players and about their motivation in
performing the activity (solving the math problems).
i. Personal Questionnaire
The personal questionnaire contained the results of a general
questionnaire. The student population has ages between 12 and 13
years and have economic and educational equality.
ii. Intervention
During the intervention, both groups completed 30 Math
questions. For each question answered correctly, the subject won
10 points and to get a tip for a question, the subject had to pay 5
points. For each sequence of 5 questions answered correctly, the
subject won a badge. The ranking was updated after each question
answered, showing the top 5 students with higher points.
iii. Motivational Questionnaire
The last step of the procedure was the motivational questionnaire,
which contained 22 short questions proposed by Deci et al. [9].
The Intrinsic Motivation Inventory (IMI) is a multidimensional
measurement device intended to assess participants’ subjective
experience related to a target activity in laboratory experiments. It
has been used in several experiments related to intrinsic
motivation and self-regulation [18,19]. The task evaluation
questionnaire measures the motivation into 4 categories:
interest/enjoyment, perceived competence, perceived choice and
pressure/tension.
3.2.5 Analysis of Results
This section presents our experimental findings based on the
results described in the previous sections. The analysis is divided
into two subsections: (1) results and (2) hypothesis testing.
3.2.5.1 Results:
From Figure 2, it can be seen that:
1) Along both metrics (points collected and number of right
answers), there was insignificant difference between the
Gamified and the Non-Gamified groups. The Non-Gamified
group collected 426.25 points altogether and answered
correctly 42.7 questions. The Gamified group collected 423
points and answered correctly 42.6 answers. It is clear that
there were very few, if any people asking for help, so no
gaming of the system was observed.
2) The female subjects in the non-gamified condition
outperformed all other sub-groups obtaining higher scores in
both metrics analyzed (number of right answers: 23 and
number of points: 228.75). However, the male subjects in
the non-gamified condition had the lowest scores among all
fours sub-groups (19.7 and 197.5, respectively). The male
and the female subjects in the gamified condition had nearly
equal scores(210 and 21 vs 213 and 21.6, respectively).
3) The standard deviations for both metrics of the
underperforming groups in each condition (boys in the non-
gamified condition and girls in the gamified condition) were
much higher than the standard deviations of the respective
metrics in of the better performing groups. The non-
gamified condition, the standard deviation of boys is 63.78
and girls is 26.54. However, the gamified group the
numbers is 14.35 for boys and 37.41 for girls.
Figure 2 - Data from the intervention.
This suggests that the game mechanics implemented in E-Game
had different effects for students with different genders.
Considering only the female subjects, the use of game mechanics
reduced the learning performance compared with the traditional
environment (i.e. with no game mechanics). With male subjects it
had the opposite effect; the game mechanics improved their
performances.
The gender differences in the impact of the gamification
mechanics are more pronounced in of the results of the motivation
questionnaire. Figure 3 and Figure 4 show the results of the task
evaluation questionnaire, in the aspects of interest/enjoyment,
perceived competence, perceived choice and pressure/tension.
Figure 3 - Results of task evaluation questionnaire for girls
Figure 3 - Results of task evaluation questionnaire for boys
In Figure 3, we observed that girls had a lower perceived
competence in a gamified environment compared to a non-
gamified environment. Nevertheless, the use of game mechanics
offered positive aspects such as interest/enjoyment and perceived
choice, since the girls gave higher scores for these factors in the
gamified environment. However, they also felt more pressure and
tension during the use of the gamified system.
On the other hand, Figure 4 shows that boys had higher scores in
all aspects in the gamified environment, including more
pressure/tension during the use of the educational system. They
had significantly higher perceived competence in the gamified
environment than in the non-gamified (the values are nearly
reversed to those of the girls). The boys felt they had more choice
in the gamified environment than girls, but also more pressure/
tension (15 for boys and 13 for girls).
The big differences between the two environments between the
genders were in the areas of perceived competence and
interest/enjoyment. Yet overall the non-gamified environment
received better scores from the girls, where they felt more
interest/enjoyment, more perceived competence and choice, and
less pressure/tension than the boys.
We didn’t find evidence for Gaming the System during the study.
This behavior was identified by Wood et al. [23] as a typical game
the system behavior where students rapidly and repeatedly asks
for tips (help) to show the correct answer. Figure 5 shows the
averages of total counts from the tips (help) that each student
asked for. Since there was no reward and no pressure on the
subjects to complete the study, there was no reason for them to
engage in cheating behaviors. Thus our results cannot answer the
question if the gamification mechanics reduced faming behaviors.
3.2.5.2 Hypothesis testing:
Aimed at testing the hypotheses in subsection 3.2.2, we performed
a T-test, which is a parametric test used to compare two
independent samples [21] and the number of subjects is less than
30. However, the results showed some extra information that
wasn’t previewed in the hypothesis, but it cannot be ignored, that
is the difference between genders. For that reason, each
hypothesis will be tested considering this factor.
Analyzing the 1H
0
in the context of male subjects, we obtained
the following results: t = 5.9744, 3 degrees of freedom at 5%
significance level, p-value = 0.0094. Since p-value is smaller than
0.05 we can refute 1H
0
for the boys. Therefore, there are
evidences to say that for the male subjects the use of game
mechanics provide positive results towards motivational
standards. Applying the t-test in the context of female gender, we
have the following results: t = 0.0168, at a 5% significant level, p-
value= 0.9877. It turns out we cannot reject 1H
0
for this group.
That is, it is not clear statistically whether the gamified
environment is more helpful towards motivation than non-
gamified environment.
The 2H
0
obtained the following results in context of female group:
t = 1.2388, 1 degree of freedom at 5% significance level, p-value
= 0.4323, which is higher than 0.5, meaning no statistically
relevance to refute the null hypothesis. The male group also
doesn’t have statistically significance, with the following values: t
= 0.0664, at 5% significance level, p-value = 0.9531, which is
higher than 0.5. The results mean that there is no difference in
terms of performance in a gamified educational system (measured
in terms of the score achieved by the number of right questions
answered).
Although most of the results are not statistically significant due to
the small number of subjects, it is important to run statistical
analysis such those presented in this section to identify trends in
the data and to provide insights that can help to improve the
correct usage of game mechanics in learning environments. We
were able to find statistically significant evidence, even with the
small number of subjects, that boys are more strongly motivated
by gamified environment.
4. CONCLUDING REMARKS
Educational virtual environment is the key for a successful future
in the learning aspects with the new generation of students.
Figure 4 - Results of the tips
However, keeping them motivated throughout the whole learning
process is considered one of the major challenges in all forms of
learning. One solution for this problem is introducing game
mechanics into those environments, making them more
interesting, fun and enjoyable. To check the impact of these
mechanics in a learning environment, we developed E-Game, a
platform that can be used to support learning with and without the
use of game mechanics.
The controlled study using a gamified and a non-gamified version
of E-Game in a small class showed with statistical significance
that game mechanics had a positive motivational effect with the
male students. Although the learning performance among the two
groups and among genders didn’t show statistically significant
difference, the numbers of points earned in the gamified system
were higher compared to those in the non-gamified system. We
did not observe behaviors of Gaming the System in either of the
systems. We also found that the game mechanics implemented in
E-Game did not have any effect (on motivation and performance)
in the female students, which suggests that the studies of
gamification in the context of learning should consider gender
differences to draw better conclusions about their impact on
motivation and learning performance. In future research, we will
extend this case study to an experiment and further analyze the
gender issue with respect to previous experience with games, in
the context of gamified learning environments.
5. ACKNOWLEDGMENTS
The authors would like to thank the financial support provided by
FAPESP (grant number 2014/03115-6 and 2011/10926-2) and
CNPq. We are also thankful for the teacher Tracy Bazylak for
making the experiment possible.
6. REFERENCES
[1] Anderson, J. R., Corbett, A. T., Koedinger, K. R., Pelletier,
R. (1995). Cognitive tutors: Lessons learned. In The Journal
of the Learning Sciences, 4, 167-207.
[2] Baker, R. S. J. D., Corbett, A. T., Roll, I., & Koedinger, K.
R. Developing a generalizable detector of when students
game the system: User Modeling and User-Adapted
Interaction, 2008, 18(3), 287-314.
[3] Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A.,
Koedinger K. Why students engage in “Gaming the System”
behavior in interactive learning environments. Journal of
Interactive Learning Research, 2008, 19(2), 185–224.
[4] Beal, C. R., Qu, L., Lee, H. Mathematics motivation and
achievement as predictors of high school students' guessing
and help-seeking with instructional software. Journal of
Computer Assisted Learning, 2008, 24(6), 507-514.
[5] Borges, S. S., Reis, H. M., Durelli, H. S. V., Isotani, S. A
Systematic Mapping on Gamification Applied to Education.
In Proceedings of the ACM Symposium on Applied
Computing, 2014, 216-222.
[6] Cetintas, S., Si, L., Xin, Y. P., Hord, C. Automatic Detection
of Off-Task Behaviors in Intelligent Tutoring Systems with
Machine Learning Techniques. IEEE Transactions on
Learning Technologies, 2010, 3(3), 228-236.
[7] Cheng, R., Vassileva, J. Adaptive Reward Mechanism for
Sustainable Online Learning Community. In Proceedings of
the 12th International Conference on Artificial Intelligence
in Education, 2005, 152-159.
[8] Cytowic, R.E. Synesthesia: A Union of the Senses. Springer-
Verlag, New York, 1989.
[9] Deci, E. L., Eghrari, H., Patrick, B. C., & Leone, D. (1994).
Facilitating internalization: The self-determination theory
perspective. Journal of Personality, 1994, 62, 119-142.
[10] Deterding, S., Dixon, D., Khaled, R., Nacke, L.E. From
Game Design Elements to Gamefulness: Defining
“Gamification”. In Mindtrek 2011 Proceedings, ACM Press,
Tampere, 2011.
[11] Devedzic, V. Semantic Web and Education. Springer, 2006.
[12] Domínguez, A., Saenz-De-Navarrete, J., De-Marcos, L.,
Fernández-Sanz, L., Pagés, C., Martínez-Herráiz, J.
Gamifying learning experiences: Practical implications and
outcomes. Computers and Education, v.63, p.380–392, 2013
[13] Goleman, D. Emotional Intelligence: why it can matter more
than IQ. London, Bloomsbury, 1996.
[14] Isotani, S., Inaba, A., Ikeda, M., Mizoguchi, R. An ontology
engineering approach to the realization of theory-driven
group formation. International Journal of Computer-
Supported Collaborative Learning, 2009, v. 4, p. 445-478.
[15] Kapp, K. The Gamification of Learning and Instruction:
Game-Based Methods and Strategies for Training and
Education. San Francisco. Pfeiffer. ISBN: 978-1-118-09634-
5, 2012, p.336.
[16] Lee, H., Doh, Y. Y. A study on the relationship between
educational achievement and emotional engagement in a
gameful interface for video lecture systems. In Proceedings -
2012 International Symposium on Ubiquitous Virtual
Reality, ISUVR 2012, 2012, p. 34–37.
[17] Malone, T. W., Lepper, M. R. Making learning fun: A
taxonomy of intrinsic motivations for learning, v. 3 Hillsdale,
N.J.: Erlbaum, 1987, p. 223–253.
[18] Ryan, R. M. Control and information in the intrapersonal
sphere: An extension of cognitive evaluation theory. Journal
of Personality and Social Psychology, 1982, 43, 450-461.
[19] Tsigilis, N., & Theodosiou, A. Temporal stability of the
Intrinsic Motivation Inventory. Perceptual and Motor Skills,
2003, 97, 271-280.
[20] VanLehn, K., Burleson, W., Chavez-Echeagaray, M. E.,
Christopherson R., Gonzalez-Sanchez, J., Hidalgo-Ponet, Y.,
Zhang L. The Affective Meta-Tutoring Project: How to
motivate students to use effective meta-cognitive strategies.
In Proceedings of the 19th International Conference on
Computers in Education. Chiang Mai, Thailand: Asia-Pacific
Society for Computers in Education. November, 2011.
[21] Vassileva, J. Motivating participation in social computing
applications: a user modeling perspective. User Model User-
Adap Inter. 22:177-201. Springer Science, 2012.
[22] Wohlin, C. P. Runeson, M. Höst, M. C. Ohlsoon, B. Regnell,
and A. Wesslen. Experimentation in Software Engineering:
An Introduction. Kluwer Academic Publishers, 1999.
[23] Wood, H. Help seeking, learning and contingent tutoring.
Computers and Education, v. 33, n. 2-3, p. 153–169, 1999.
