Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
211
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
International Journal of Learning, Teaching and Educational Research
Vol. 20, No. 5, pp. 211-231, May 2021
https://doi.org/10.26803/ijlter.20.5.12
Gamification in Higher Education
(Case Study on a Management Subject)
Andrea Bencsik
J. Selye University, Komarno, Slovakia
https://orcid.org/0000-0001-8204-3706
Adriana Mezeiova
J. Selye University, Komarno, Slovakia
https://orcid.org/0000-0001-5878-7334
Bernadett Oszene Samu
Jedlik Anyos Mechanical and Informatics School, Hungary
https://orcid.org/0000-0002-6176-8510
Abstract. In today’s education systems, new solutions are required for
educators to raise and maintain the interest of young people (from
primary school to higher education). The aim of the study is to present a
self-developed gamification solution and its application in higher
education in economics. The method, the process, and experiences
presented in the study were tested within the framework of a
management subject. The gamification model, based on an extensive
literature review, was elaborated with the help of a self-developed
method. Prior to the development of the process, students’ opinions on
their experiences and expectations for current educational methods were
surveyed. After the end of the semester, our students were asked on their
feedback, and a national survey was conducted in higher education
institutions about the experiences with gamification solutions. The
positive consequences of the application of our own model, can be traced
in the students’ continuous and year-end performance (a higher level of
task solutions and better grades) and also in their feedback.
Although the subject of the test semester was a management-type subject,
the logic of the model can be applied within the framework of any other
subject and in any higher education institution as well.
Keywords: flow; gamification; gamification model; higher education;
management; motivation
212
http://ijlter.org/index.php/ijlter
1. Introduction
Raising and maintaining the attention of young people has been a problem for
years on all levels and in all fields of education. It is a global problem and can be
found all around the world. Beyond the digital world, young people are not
interested in anything, they cannot concentrate on one thing for more than a few
minutes, and they constantly hang out on screens – opinions generally say. A
group of researchers (Wang et al, 2014; Barak, 2020) believe that this
inattentiveness (disorganisation) is inherent in our digital world, as the human
brain is unable to develop as fast as the technology it creates. It is also an everyday
experience that they are able to listen to music, search in browsers, chat with
friends while watching TV. It indicates that they can still pay attention to what
interests them. Constant opportunities for expression and interactive activities are
also expected in different education systems. Teachers and lecturers face this
challenge; in education, interactive techniques should be used that meet the needs
of young people (Steigerwald, 2016). Young people in educational institutions
should be prepared for the future, where they will have to deal with complex,
multidisciplinary problems and approach global challenges from a new
perspective (Schwab, 2016).
21st century higher education is not only about acquiring knowledge in a single
field of science. Higher-level skills such as critical thinking, creative problem
solving, teamwork and communication, and also soft skills are becoming
increasingly valuable. Due to the multiplication of data and information, the
selection and critical evaluation of, and the appropriate decision-making based on
relevant information is essential for the success of the society of the future. Today,
lecturers in higher education not only have to pass on information, knowledge
content, and students – unlike former students – not only take and internalise that.
Thanks to modern technological tools, students are capable of quickly finding
anything on their smart devices but this knowledge is superficial. The task of the
teacher is to make underlying content and connections understood. Young people
should be taught to process information so as to understand its meaning and
interplay, and their conceptual and practical skills. The role of teachers is
becoming increasingly important in this process (Zivkovil, 2016; Kereluik, Mishra,
Fahnoe & Terry, 2014; Kivunja, 2014).
Higher education institutions are aware of the need to increase investments in
education, although they are not able to do so at the same level. This is also the
reason for the increase in the number of researchers seeking effective ways of
education, teaching-learning in recent years. The results of research by
Deslauriers and colleagues (Deslauriers, Schelew & Wieman, 2011) showed about
10 years ago that traditional presentation-oriented education does not provide 21st
century key skills. In interactive education, students become equal participants of
research-based learning, exploratory innovation, and the process of learning from
mistakes. A most important benefit of this new solution is that they learn to think,
decide and act in context, thus developing their professional identity. As a result,
they become more ambitious not only in educational institutions but also in real
life (Holmes et al, 2015).
213
http://ijlter.org/index.php/ijlter
Freeman and colleagues (2014) have shown that interactive methods increase the
efficiency of education by giving students a deeper understanding of concepts,
and as a result of activity, they spend a significant part of their time in the
classroom. During the course of education, they do activities that require
extensive information gathering using electronic devices or answering questions,
filling out worksheets on web interfaces, analysing problems and constantly
communicating. Such interactive techniques make education more attractive,
authentic and satisfying – full of challenges that young people have to address –,
thus increasing the efficiency of learning (Talbot, Doughty, Nasim, Hartley & Le,
2016).
One method that meets the above requirements is gamification. Several studies
support (Han-Huei Tsaya, Kofinasb & Luo, 2018; Goksuna & Gursoy, 2019) that,
with the help of games, both children and adults absorb knowledge much deeper
and more thoroughly. Huotari & Hamari (2017) approach gamification as a
service developing process, where the generation of a game-like experience
supports user value creation. The application of game elements in higher
education is often biased, many consider it frivolous, although research has
shown its positive effects (Nah, Zeng, Telaprolu, Ayyappa & Eschenbrenner,
2014). The experimental teaching method illustrated below attempts to test this
contradiction.
In this research, which is a kind of case study, answers to the following research
question were sought: What impacts does gamified education have on the
students’ in-class and out-of-class activity? How do they relate to the new
method? Is the positive impact of gamified education on learning results
traceable? What do higher education lecturers think of the possible applications
of gamification? Answering the questions seemed possible with the help of our
own ‘experimental’ education. The study summarises the experiences that present
the gamified solution of a management subject taught in economic higher
education. The method preferred creativity more than a solution involving
financial investments. Education required Internet access, laptops, tablets or any
kind of smart devices (available to all students without exception). No further
specific software is needed. The experiences of the experimental education show
that the majority of the participating young people enjoyed and found the
gamified solution useful, which was also confirmed by the end-of-semester
grades. Colleagues from other educational institutions surveyed had mixed
feelings and varied opinions about gamification as a method, but few have their
own experience.
2. Literature review
2.1. The impact of media on the human brain
The young generation studying at universities is also called Generation Z. The
first ones (born between 1995 and 2000) who were there at the beginning of the
digital era. Children born in the second wave (between 2005-2010) were already
born into the digital world. In this world, you already have to be present on social
media sites, on-demand entertainment (whenever you want it) is self-evident. For
them, info-communication technology (ICT) is a natural part of life, which fills all
214
http://ijlter.org/index.php/ijlter
segments of society. They enjoy it and want to make the most of the opportunities,
they live in the moment, talk less and keep things brief (Tari, 2015; Seemiller &
Grace, 2016).
Several studies have discussed the impacts of media on the human brain. They
showed that media changes our habits and routines (Valkenburg, Joche, &
Walther, 2016; Uncapher & Wagner, 2018; Crone & Konijn, 2018). Carr (2010)
described his feelings that his brain was constantly “hungry”, and demanded the
Internet to feed it in its own way, but the more it received, the “hungrier” it
became. Today, we know that our brain is constantly changing, and adapts to the
slightest changes in circumstances and behaviour. This wonderful property is
called plasticity. Neuroplasticity is the most important result of evolution, which
allows individuals to adapt to changed circumstances and reorganise themselves
throughout their lives or even in the course of a few days (Hanson, 2017; Price &
Duman, 2019). During adolescence, significant changes take place in the brain.
Nerve cells that are not used regularly die, so in some cases high-performing
students in secondary school fail in higher education because weak functions of
their brains are overburdened by increased strain (Carr, 2010). More studies
indicate the fact that when having to switch between two tasks confuses our brain
and increases cognitive load, and also the possibility of not registering important
information (Kirschner & Karpinski, 2010; Junco & Cotten, 2012).
As a result of the use of multimedia devices for several hours a day, young
people’s brains have adapted and their brain maps have rearranged. You could
say: their brains ‘work’ differently from their parents. Today’s young generation
is heavily burdened by academic traditions: the curriculum (content that we
teach) and the teaching methods (how we teach). A kind of reaction to this is the
use of interactive teaching methods in higher educational institutions.
2.2. Interactive teaching methods
Active/interactive reading improves the learning process as it provides students
with more starting points. These are important because students come to school
with different knowledge or culture. Innovative education offers more
opportunities for correcting misconceptions, providing timely feedback, or
integrating different viewpoints through debate and discovery (Holmes et al.,
2015) as the responsibility for the education, which combines different
educational methods to meet various professional directions and student
aspirations (Deslauriers et al., 2011; Hoellwarth & Moelter, 2011; Freemann et al.,
2014; Von Korff et al., 2016). According to Tamim and colleagues (Tamim,
Bernard, Borokhovski, Abrami & Schmid, 2011), shifts towards students. These
methods are able to adapt to the abilities of individuals, trainings can become
more effective and provide participants with a higher level of satisfaction. One of
the solutions to active/interactive learning is gamification.
2.3. Gamification
The term gamification was born from the word ‘game’ and the suffix ‘-fication’
transform into something, in the digital media industry. It was first used by
Pelling (2011) in 2002, but gained public awareness only in 2010. The most
significant difference between game and gamification is that while games are
215
http://ijlter.org/index.php/ijlter
always self-serving, and the focus is on fun and gaming experience, gamification
always has some out-of-the-game, useful purpose.
According to the most widely accepted definition, gamification is the application
of game elements and mechanisms in an out-of-the-game environment (Deterding
et al., 2011). According to Kapp (2012), gamification is a strategy that aims to
change user behaviours in a positive direction by applying elements of game
design and game aesthetics. The basic idea is that human activity is more efficient
if the task to be performed is enjoyable and there is joy both in the work process
and the result. The two definitions are consistent.
To understand the definition, we need to distinguish two types of games, for
which there are two separate words. ‘Play’ means the free, spontaneous, self-
directed game of childhood (Santayana, 1955). By contrast, “game” means a game
subject to rules, with a purpose and quantifiable outcome (Salen & Zimmerman,
2004). The French thinker, Caillois (2001), conveys the same difference by the
terms ludus and paidia. Paidia (from the Greek pais “kid” root) is an exuberant,
spontaneous manifestation of the playful instinct, while ludus (play) borrowed
from Latin is characterised by brainwork, instructions and limitations.
Gamification uses game design elements that ban be categorised as ludus. The
creators of the definition illustrate this concept on a twice two-element matrix (see
Figure 1). On one axis you find playfulness as opposed to gamefulness, while on
the other axis holism as opposed to construction from elements. According to the
definition, gamification does not use complete games (in other words, full-fledged
games), only game elements, and not playful design, but gameful design.
Source: Deterding et al., 2011
Figure 1. Gamification and related concepts
Gamification occurs in an out-of-the-game context, so it does not create or
complete games, but supports other systems with game elements, for instance, in
216
http://ijlter.org/index.php/ijlter
the educational or business spheres. The aim of gamification is therefore not to
entertain but to increase engagement, motivation and user experience.
The creators of the definition distinguish three types of game elements: game
technology, playful design and game-based procedures, of which gamification
uses only game design elements. Subsequently, gamification is not linked to the
use of digital technologies (Deterding et al., 2011; Dulova Spisakova, 2017).
The logic of gamification is a blend of game design tools and psychology. It can
be divided into three levels, which include regulatory, behavioural and emotional
components that are also called MDA (Mechanics, Dynamics Aesthetics) model.
Breaking down to factors, you can formulate the three elements of the
gamification system (Kim, 2015).
For game mechanics, various rewarding tools are listed, such as collectable points,
badges, achievable levels, challenges and missions, virtual assets and gifts. Game
dynamics denote human desires such as reward, status, performance, self-
expression, competition and altruism (Dicheva, Dichev, Agre & Angelova, 2015;
Aliyu, 2020). Aesthetics is the third and final elements of the MDA model, which
describes what reactions the game process evokes in the player. These emotions
can come from: trying out something new, completing a challenge, exploration
and adventure, belonging to a community, the opportunity of self-expression,
immersion in a fantasy world (Kusuma. Wigati & Utomo, 2018). Players react
emotionally to each part of the game, so they will have game experience and
develop their cognitive and social competencies (Deterding et al., 2011; Kapp,
2012).
A few years ago, game researchers only studied the negative effects of games.
They paid great attention to addictive attributes (Gentile et al., 2011), and game-
induced aggressive behaviour (Bushman, Rothstein & Anderson, 2010; Rozsa,
2019). Today, they endeavour to exploit the positives.
Based on a decision by the U.S. Food and Drug Administration (FDA) released in
June 2020, experts used Akili’s game called EndeavorRx to treat children with
Attention Deficit Hyperactivity Disorder (ADHD). The FDA’s decision is a
milestone in the growing field of digital therapies, as this is the first time an
authority authorised a video game therapy for any health condition. The studies
have tested the new method with more than 600 children over 7 years. Clinical
trials results showed that one-third of the participating children had a measurable
improvement in their attention deficit after playing 25 minutes a day, five times a
week for four weeks. The manufacturer claims that the game is able to activate
and strengthen certain neural networks through targeted algorithms. The
following period may bring new challenges for the company, as they must
convince doctors that the game is worth prescribing to children, and health
insurers to cover treatment costs (FDA, n.d; Collins et al., 2020; Mura,
Gontkovicova, Dulova Spisakova & Hajduova, 2019).
217
http://ijlter.org/index.php/ijlter
Sheldon’s (2012) experiment is often mentioned as a successful gamification
example in higher education, who gamified the university course on the example
of MMORPG (Massively Multiplayer Online Role-Playing Game). In his solution,
students moved levels based on points, based on which they received their final
grades. He recognised that the simplest point and level systems, and the
immediate feedback motivate students. Ryan and his colleagues claim that
planning a good game is difficult, and planning a good educational game is even
more difficult (Ryan, Costello & Stapleton, 2012).
Various simulation programs belonging to the ‘serious game’ category mentioned
above are used at universities, especially in teaching economics subjects. These
games are usually used in teaching corporate decision-making subjects and
developing leadership competences. The next chapter presents our own gamified
teaching method.
3. Methodology
The research was carried out in three phases.
1. Survey of student opinions motivation and attitude (questionnaire survey)
2. Development and implementation of a gamified course (the combined use
of gamification tools and the evaluation of results in the teaching of a
management subject)
3. A national survey on the use of gamification in higher education
(questionnaire survey)
The study describes the second phase in detail, the first and final phases are
presented outlined only for clarity.
3.1. Participants
As a first step, in the 2019/2020 school year, a quantitative research was
conducted among students (260 participants) studying at the Faculty of
Economics of a higher education institution, aimed to map students’ learning
motivation factors and preferred classroom tasks. Hypothesis testing revealed
significant differences between the liking of gamified solutions and solving real
problems (Pearson 0,341, 2 sign. 0,000<0,01), between internal motivation and
gamified solutions (Pearson 0,129, 2 sign. 0,000<0,01), and between internal
motivation and the liking of real problems (Pearson 0,466 2 sign. 0,000<0,01).
Summarising the results, students driven by internal motivation liked real
problems and were glad to solve them in a gamified form [50].
The second step
Based on the results of this phase, we prepared the gamified curriculum for 4th
year students studying for a master’s degree, for a seminar course related to a
management subject, which was taught in the spring semester of 2020. The two
main topics of the course were knowledge management and change management.
The students had sufficient preliminary studies, and acquired the special
knowledge of the subject necessary for the seminar during theoretical lessons. The
course was taken by 68 students, divided into two seminar groups. Seminar
classes were conducted on a weekly basis in two lessons.
218
http://ijlter.org/index.php/ijlter
3.2. Research design and procedures
The logical process of the course is shown in Figure 2 below.
Source: Authors’ own construction
Figure 2. The logical process of the gamified course
After planning the process, the ‘persona generation’ phase helps with thinking.
Its aim is to get to know participant motivation, which provides guidance for
planning ‘challenges’. In our research, persona generation was aided by the
evaluation of student questionnaires (Phase 1 of the research). Accordingly, the
characteristics of our ‘persona’ are: members of Generation Z, their learning is
mainly driven by internal motivation, do not mind having to work harder if the
task is interesting, like to solve real problems with the help of games, like to
perform and present less, play some inline game every day and like strategy and
logic games best.
The course began with an introductory session, where students got familiar with
the new method, tasks, logic of progress, achievable levels, expected outcomes,
potential pitfalls and opportunities for success.
3.3. Instruments
At the beginning of the semester, a virtual classroom was created on the Google
Classroom site for students to join. With the help of Google Docs Editor, an Excel
table called ‘Progress Indicator’ was created, in which everyone could keep track
of their own and others’ progress, completed tasks, the current status of points,
levels and badges.
In the preparation for the course, students formed small groups of 7-8 people, and
worked in a permanent composition throughout the semester. The background of
gamified tasks was a self-invented company, which was freely chosen and built
on students’ creativity. They formulated the vision, mission, scope of activities,
organisational framework and operation of the company, distributed the most
important positions and named the main problem, for the solution of which the
tasks received during the semester provided support. Nine companies were
established, the problems of which could be solved in a ‘customised way’ through
solving compulsory tasks. The activities and experienced problems of the
established companies are collected in Table 1.
219
http://ijlter.org/index.php/ijlter
Table 1. Students’ companies and main problems
Company activity
Problem
Marketing service
Few orders
Sale and cultivation of fruit and vegetable goods
Overchemicalised products
Manufacturing of bakery products, delivery to
chain stores
Outdated technology, high costs,
labour shortage
Catering unit – sale of craft beers
Unreliable suppliers, halting the
supply of raw materials
Plumbing, electrical installation
Unskilled workforce
Diamond trading
Legal regulation as an obstacle
Events management
Few orders
Fitness
Few visitors, high costs
The tasks solved during the semester, built on each other in their logic, served to
solve a company problem defined by the students, built on the theoretical
curriculum (change and knowledge management). The first part of the tasks was
aimed at identifying problems, delimiting them in the organisations of companies,
then trying to solve them.
In addition to problem solving, based on the logic of Probst’s model (Probst, 1998)
(keeping in mind the rules of the relationships between the system approach and
the processes), students had to work out conditions and steps for the
implementation of a knowledge management system that ensures successful
organisational operation. As each group elaborated the tasks for their own
company, creative and unique solutions were created.
All the companies operated in the same ‘virtual market’, so it was an additional
task to find a company among the others, with which they could enter into a
mutually beneficial cooperation agreement to support the solution of the main
problem. (Technique used was free to choose, which aroused great enthusiasm,
and serious ‘business negotiations’ were held in the lesson. There was a group
that reached a mutual agreement with all the other companies.)
In addition, problem solving and task completion was supported by film
screenings, analyses, situational games and personality tests. They could test their
own progress during the semester in the Kahoot program, using individual and
group quizzes we had created.
3.4. Analysis technique
Some tasks were solved during lessons, while others had to be uploaded to the
Google Classroom site. During the semester, they could collect 400 points (the so-
called XP points known from video games), which belonged to eight levels
(rookie, interested, inquisitive, knowledge-thirsty, eager-to-learn, hardworking
apprentice, master, grandmaster). A virtual badge was also associated with each
level, and progress could be tracked by following XP points on the ‘Progress
Indicator’ interface. After completing a task, immediate feedback was provided
with help of the evaluation of results and the collected points. At most fewer
points, but no negative evaluation was not given, which provided them with
motivation with an opportunity to reach a higher level. Students had the
220
http://ijlter.org/index.php/ijlter
opportunity to earn extra points by proposing a solution to a problem of a real
company of their choice. Twice during the semester and in the final session, they
demonstrated their progress in problem solving to the other companies in a
presentation. By the end of the semester, all small businesses had met their main
goal (problem solving), albeit at different levels.
The success of the method was measured in two ways. At the end of the semester,
the participating students were asked to complete a brief questionnaire. Using
closed-ended questions, students assessed the semester on a 4-point Likert scale,
answering how much they agree with the provided statements (1-Not at all, 4-
Totally agree). In the last, open-ended question, they were asked to give their
honest opinions, suggestions, comments about the lessons. Another element of
back-testing was the comparison of their learning results with those of the
students in the previous year’s non-gamified course. The detailed evaluation
results are presented in the next chapters.
3.5. Evaluation of the semester
During course evaluation, 49 students expressed their opinions by answering
questions summarised in 5 groups. In the first group of questions, respondents
were asked to provide a general evaluation of the gamified seminar. The
summarised results are shown in Table 2.
Table 2. General evaluation of the seminar (n=49)
The second group of questions investigated how the students experienced solving
tasks through a self-created company, assuming a real market situation, which led
to a solution to the main problem. The statistical evaluation of the results is shown
in Table 3 below.
Table 3. Students’ reception of a self-created company (n=49)
Statements
Mean
Median
Modus
Standard
deviation
% of
students
I was not bored in class
3.15
3.00
3.00
.994
71.5
The classes were interesting
3.04
3.00
3.00
.856
73.4
I liked attending the classes
3.02
3.00
3.00
.892
73.5
I would have attended the
classes even if it had not been
compulsory
2.71
3.00
2.00
.995
57.1
Statements
Mean
Median
Modus
Std. dev.
% of
students
I would prefer if there was
a similar opportunity for
subjects as well
3.31
3.00
3.00
.748
89.8
I think gamification is a
good idea
3.30
3.00
4.00
.832
89.8
I liked that we could be
creative
3.23
3.00
4.00
.857
85.7
221
http://ijlter.org/index.php/ijlter
Student feedback supported the results of the exploratory research. The results
show that students evaluated the tasks developed for their own companies very
positively. They could be creative as there was no predefined ‘correct’ solution.
They found it interesting and exciting to solve a real problem and use their own
ideas.
In the third group of ideas, students evaluated the system of points and badges.
The statistical results of answers are summarised in Table 4.
Table 4. Evaluation of the system of points and badges (n=49)
Statements
Mean
Median
Modus
Standard
deviation
% of
students
I think it is a good idea
2.90
3.00
4.00
1.021
71.4
I would prefer a similar
solution built in other
subjects as well
2.90
3.00
4.00
1.115
67.4
Made the subject more
interesting
2.80
3.00
3.00
.935
67.4
Increased the spirit of
competition
2.55
3.00
3.00
.98
55.2
The results show that the system of points and badges had the least effect on
motivation and competitiveness, but it still increased these in more than half of
the students. Nearly three quarters found it a good idea and the majority believed
they could better track their or progress and would have preferred to meet such
an opportunity in other subjects as well.
The fourth group of questions explored student experiences with the Google
Classroom site. The statistical results of the answers are summarised in Table 5.
Table 5. Evaluation of the use of Google Classroom (n=49)
Statement
Mean
Median
Modus
Standard
deviation
% of
students
Handling it did not cause
any problems
3.56
4.00
4.00
.733
93.9
It was good to have all
the information in one
place
3.56
4.00
4.00
.611
95.9
I think it is a good idea
3.56
4.00
4.00
.705
87.8
I would prefer if there
was a similar solution in
other subjects as well
3.28
4.00
4.00
.935
77.5
I liked that I could think
about the task
3.15
3.00
3.00
.772
83.7
They were more
interesting to me
3.14
3.00
3.00
.714
87.5
I thus better understood
the curriculum
3.02
3.02
3.00
.978
75.5
222
http://ijlter.org/index.php/ijlter
The students were very positive about the online solution, and they quickly
became familiarised with its handling. They took advantage of the opportunities
provided by the site, and constantly followed their own and the others’ progress.
In the last, fifth group of questions, the students were asked to compare the
gamified solutions of the course with the seminar lessons of other subjects. The
statistical results of the answers are shown in Table 6.
Table 6. Comparison of the gamified seminar lesson with other seminar lessons (n=49)
Statements
Mean
Median
Modus
Standard
deviation
% of
students
The tasks were more
creative
3.19
3.00
3.00
.77
83.7
The sessions were more
interesting
3.08
3.00
3.00
.731
81.2
The sessions were not
more boring
1.66
2.00
1.00
.788
89.7
The lessons were the
same as before
1.92
2.00
2.00
.731
81.5
As the next step of evaluation, the study results of the students attending the
gamified lessons were compared with those who attended the traditional course.
Table 7 shows that the results of those studying with the new solution have
significantly improved compared to the students attending the traditional course.
Table 7. Study results of gamified and traditional courses
Traditional course
Gamified course
Evaluations
Number of
students
%
Number of
students
%
A
5
8.2
13
19.1
B
10
16.4
13
19.1
C
10
16.4
21
30.9
D
18
29.5
9
13.2
E
15
24.6
7
10.3
FX
2
3.3
1
1.5
Did not attend
1
1.6
4
5.9
Total
61
100. 0
68
100.0
As the third step of the research, we asked 24 colleagues in higher education
institutions to share their experience with and evaluations of gamification. 273
responses were evaluated during the questionnaire survey. Experiences with the
application are summarised in Table 8.
Table 8. Lecturer opinions about gamified education
Application of the gamified method
N
%
Have not applied yet
142
52.0
Have applied, but mainly prefer traditional education
77
28.0
I apply traditional and gamified methods alternately
48
18.0
The application of the gamified method predominates
6
2.0
Total
273
100.0
223
http://ijlter.org/index.php/ijlter
The third step
It is interesting how two further responses illustrate the domestic picture of the
method.
As an answer to the question ‘What subject/subjects have you gamified or would
like to gamify?’ in most cases, foreign language education was marked, followed
by the field of economics (management, marketing, corporate economics, micro
and macroeconomics), and then mathematics. The diversity of the mentioned
subjects confirms the wide applicability of the method: oral surgery, history,
pedagogy, conflict management, communication, research methodology,
bioinformatics, chemistry, constitutional law, mechanics, optics, heritage
protection and food safety, etc.
Based on the respondents’ opinion (273 participants), the advantages and
disadvantages are summarised in Table 9 below.
Table 9. Advantages and disadvantages of gamification
Lecturer experience shows that the application of gamification in higher
education makes knowledge transfer easier, attracts student attention, positively
affects competitiveness, motivation and in-class activity. Preparation time was
mentioned as the biggest disadvantage, but less think that the more preparation
time is disproportionate to the usefulness of the method. Of course, not everyone
agreed on the application of the method, as other answers show (not exhaustive
list): gamification hides the importance of lexical knowledge, older students may
find it too childish, teacher training is missing, it is not the task of the educator to
motivate, students get too comfortable. However, few colleagues have direct
personal experience. In the next chapter, we summarise our experiences in the
light of former research results.
4. Discussion
Recent research works have proven the positive effect of gamification on user
behaviour and motivation, but also that it does not have similar impact on
everyone (Barak, 2020; Bencsik, Mezeiova, Seres Huszarik, & Tobias Kosar, 2019;
Gokuüna & Gursoy, 2019). Our work has also confirmed them. The majority of
Advantages
Mean
Disadvantages
Mean
Makes the learning process
more enjoyable
4.32
It requires more preparation time on
the part of the educator
4.23
Attracts student attention
4.27
Few publications discuss practical
implementation
3.59
Positively influences student
motivation
4.07
There is not enough theoretical
information about the topic
3.55
Increases student activity
4.07
Its administration is more
complicated
3.53
Increases competitiveness
3.66
Too student-centred
3.33
Knowledge is easier to transfer
3.60
The educator’s preparation time is
disproportionate to the usefulness of
the method
3.23
224
http://ijlter.org/index.php/ijlter
studies report a positive effect, but emphasise that gamification largely depends
on users and context (Hamari, 2013).
The elements of game dynamics and game technics are closely related. The
elements of mechanics brought dynamics with them, namely, points served as
tools of rewarding, levels indicated current status, challenges satisfied their desire
for performance, virtual assets helped in self-expression, ranking lists increased
competitive instinct, gifts supported their being altruistic. These elements appear
as fundamental expectations in the summary of research by Fromann, (2017) and
Bunchball (2011). The solution we have developed meets these needs and practical
experience has confirmed their raison d’être.
Aesthetics, as the third element of the model, describes the players’ (students)
emotional reactions during the game. These feelings can be achieved in more
ways, depending on what they can be derived from. The method we have
developed could provide the following from the ‘roots’ defined by Kusuma et al.,
2018): trying something new, completing the challenge, belonging to the
community, an opportunity for self-expression, immersion in the world of
fantasy.
Fromann (2017) stated that there is no miracle recipe for a successful game or for
participants to enjoy the game. He says gamification’s immersive effects can be
achieved through enforcing three conditions (or participant expectations). These
(optimal workload, ideal levelling ideal reward system) were kept in mind when
planning the semester.
Our results are supported by several studies that have shown that the application
of the problem-based learning (PBL) model increases activity, and improves
students’ problem solving skills, (Simamora, Simamora & Sinaga, 2017) critical
thinking skills (Najah, Rohmah & Susilo, 2019) and verbal communication skills
(Kumar & Bervell, 2019). Several studies have shown that the ease of use and
usefulness of Google Classroom has a positive effect on its spread in education
(Zichermann and Cunningham, 2011; Wang et al, 2014; Wijava, 2016). This way
of transferring information and knowledge (using Google Classroom) can be
effectively used in educational activity inside and outside the classroom
(Supriyanto, Setiawan & Budiarti, 2018).
In their study, Laskowski and Badurowicz (2014) report the results of a
gamification course with students of an IT course where no significant
relationship was found between class attendance and end-semester results.
Students in the gamified course achieved worse results than the participants of
the traditional course. A higher course attendance and an increase in the
willingness to solve homework were recorded as positive results. Campillo-Ferrer
and his colleagues (2020) used the well-known Kahoot program as a gamification
tool in a Spanish university course. Their experiences were particularly positive
in terms of students’ active participation, their social relationships, the
development of their interactivity, and their motivation to learn and solve tasks.
Also building on the Kahoot program, experimental education was conducted by
225
http://ijlter.org/index.php/ijlter
Varannai and his colleagues (2017) who experienced a positive attitude, an
increased student performance, and the acceptance of the use of the program. The
positive effects of the Kahoot program were also reported by Prieto, Palma, Tobias
& Leon (2019) in relation to teaching an operations management subject. These
results support our research results, despite only one method was used from the
tools that were part of the gamification training. Although experiences are rather
positive, the studies note that the relationship between the invested amount of
time, energy and costs and the results achieved needs to be weighed.
According to several researchers, education lacks a consistent understanding of
processes used for gamifying learning activities (Borges, Durelli, Reis & Isotani,
2014; Ibanez, Di-Serio & Delgado-Kloos, 2014). There is no trained teaching team,
which is true in our case as well. We and our colleagues, who apply them, prepare
in the form of self-education. In many previous cases, inconsistency led to the
failure of game experiments in education, which resulted in undesirable and
unexpected effects on the learning processes and study results (Hakulinen &
Auvinen, 2014; Hanus & Fox, 2015); Dominguez et al., 2012). Therefore, special
attention should be paid to teacher preparedness. In addition, users may not be
able to take advantage of opportunities and focus too much on end results (e.g.
achieved position or ranking), and less on the tasks (Knaving & Bjork, 2013;
Silpasuwanchai, Shigemasu & Ren, 2016). Gamified processes can also inspire
users to behave appropriately only when it is rewarded (Bui, 2015). For many
users, due to its simplicity or childishness, it may be demotivating (Augustin,
Thiebes, Lins, Linden & Basten, 2016).
5. Conclusion
We considered the logic of the MDA (Mechanics, Dynamics, Aesthetics) model,
well-known from the gamification theory, as the basis for the course design,
keeping in mind the mechanics, dynamics and aesthetics of the gamified system,
which elements are recommended by several earlier research works (Hamari,
Koivisto & Sarsa, 2014; Zichermann & Cunningham, 2011). Based on these
theoretical models, we built the gamified semester from elements that meet the
expectations formulated by Fromann (2017) and Bunchmall (2011). They included
challenges for participating groups, all ‘companies’ started with an equal chance,
there was an opportunity to gain a reputation for their performance, the tasks had
to be completed on time, to which obtainable points were pre-assigned. The
opportunity to advance between levels was ensured, which could be tracked on a
list prepared for this purpose. The system rewarded extra performance. During
gamification, the combination of these elements – provided that it is foreseeable
and known to the participants – ensure the successful achievements of goals
(Bunchmall, 2011). The students were familiar with all the tasks, expectations, and
they chose the main scope of company activities and the problem to be solved.
This ensured that they were able to realise “the main goal” to be achieved (solving
their own company’s problem) by the end of the semester.
The gamified teaching solution contributed to raising participants’ result to a
higher level, but, of course, the student receptivity was not the same and the
achieved results did not represent the same shift for everyone.
226
http://ijlter.org/index.php/ijlter
The mere implementation of the mechanisms of game techniques does not
automatically lead to a significant increase in activity, but the users who actively
kept track of their badges and those of their peers showed increased user activity.
The methods used during the semester – solving the problem of a fictitious
company invented by the students by associating special tasks – can be classified
as problem-based learning (PBL). The students participating in the experiment
liked that they could be creative (85,7%), and believed they could better
understand the curriculum (75,5%). The lessons were interesting and they were
glad to attend, they would be happy if further subjects were taught in a similar
way. The built-in player feedback (badges) made the progress more interesting
and traceable.
67.3% of the students felt that the Progress Indicator helped to assess their level of
progress along with that of their peers. This tracking had a positive impact on
their motivation and the building of healthy competitiveness). Managing the
Google Classroom interface used in the course, on which they could track their
own progress, points and badges, cause no difficulty, students easily learnt to use
it (93,7 The course was considered more interesting and creative compared to
traditional education. Learning outcomes at the end of the course showed a higher
level than those in traditional education.
Overall, it was seen during the course that young people are most motivated by
internal motivation, and they like to solve real problems in a gamified way
(Bencsik et al., 2019), which facts were confirmed by our former research.
According to the responses of 273 employees of 24 educational institutions, the
advantage of the application of gamification is that it makes knowledge transfer
easier, attracts the students’ attention, has a positive impact on competitiveness
and motivation, but, at the same time, requires much more effort from educators.
After concluding our research (experimental teaching), we do not claim that the
developed solution is perfect, but, compared to previous semesters, the students
were able to achieve better results. Conclusions have been drawn from the
experiences, which will be incorporated into the gamified solution of the next
course.
6. Limitations of the research
The most significant limitation was the opportunity to apply the method. We
managed to try the new method on a relatively small course, so our results are
true only for the observed course and the students participating in the experiment.
Thus, the results cannot be generalized. Another limitation to mention was the
lecturers’ inexperience, which may distort student opinions. It is also possible that
what they felt as a problem of the method was actually a consequence of the
lecturers’ inexperience. It is also a problem and influences the success of the
course that during the course the students cannot yet get involved in the use of
similar methods in the case of other subjects.
7. References
Aliyu, M. O., Akinwale, A. A., & Shadare, A. O. (2020). Psychological contracts and
discretionary behaviour in Nigerian academics, Acta Oeconomica Universitatis Selye
9(1), 24-44. https://doi.org/10.36007/Acta.2020.9.1.2
227
http://ijlter.org/index.php/ijlter
Augustin, K., Thiebes, S., Lins, S., Linden, R., & Basten, D. (2016). Are we Playing Yet? A
Review of Gamified Enterprise Systems. PACIS 2016 Chiayi, Taiwan, Proceedings.
2. https://aisel.aisnet.org/pacis2016/2
Barak, M. (2020). Teaching Problem-Solving in the Digital Era. In: Williams P., Barlex D.
(eds) Pedagogy for Technology Education in Secondary Schools. Contemporary Issues in
Technology Education. Springer, Cham. https://doi.org/10.1007/978-3-030-41548-
8_13
Bencsik, A., Mezeiova A., Seres Huszarik, E., & Tobias Kosar, S., (2019). Higher Education
Knowledge Transfer Based on Gamification. In: ICERI 2019: Conference
Proceedings, 12th International Conference of Education, Research and Innovation
3311-3320.
Borges, S., Durelli, V.H.S., Reis, H.M. & Isotani, S. (2014). A Systematic Mapping on
Gamification Applied to Education. In: SAC '14: Proceedings of the 29th Annual
ACM Symposium on Applied Computing, pp. 216–222.
https://doi.org/10.1145/2554850.2554956
Bui, A., Veit, D., & Webster, J. (2015). Gamification – A novel phenomenon or a new
wrapping for existing concepts? Thirty Sixth International Conference on
Information Systems, Fort Worth 2015, Proceedings pp. 1-22.
Bunchball, (2011). Winning with Gamifcation: Tips from the Expert’s Playbook [White
Paper].
https://www.biworldwide.com/gamification/?utm_source=bunchball.com&ut
m_medium=referral&utm_campaign=domain_redirect
Bushman, B. J., Rothstein, H. R., & Anderson, C. A. (2010). Much ado about something:
Violent video game effects and a school of red herring: Reply to Ferguson and
Kilburn. Psychological Bulletin 136(2), 182–187. https://doi.org/10.1037/a0018718
Caillois, R. (2001). Man, Play, and Games. Illinois, USA: University of Illinois Press.
Carr, N. G. (2010). The shallows: What the Internet is doing to our brains. New York.
Collins, S. H., DeLoss, D. J., Canadas, E., Lutz, J., Findling, R. L., Keefe, R. S., & Faraone,
S. V. (2020). A novel digital intervention for actively reducing severity of
paediatric ADHD (STARS-ADHD): a randomised controlled trial. The Lancet
Digital Health, 2(4), 168-178. https://doi.org/10.1016/S2589-7500(20)30017-0
Crone, E. A., & Konijn, E. A. (2018). Media use and brain development during
adolescence. Nature communications, 9(1), 1-10. https://doi.org/10.1038/s41467-
018-03126-x
Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved Learning in a Large-
Enrollment Physics Class. Science, 332(6031), 862–864.
https://doi.org/10.1126/science.1201783
Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamification in education: A
systematic mapping study. Journal of Educational Technology & Society, 18(3) 75–88.
Dominguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernandez-Sanz, L., Pages, C., &
Martinez-Herraiz, J.J. (2012). Gamifying learning experiences: Practical
implications and outcomes. Computers & Education, 63. 380-392.
https://doi.org/10.1016/j.compedu.2012.12.020
Duova Spisakova, E., Mura, L., Gontkovicova, B. & Hajduova, Z. (2017). R&D in the
context of Europe 2020 in selected countries. Economic Computation and Economic
Cybernetics Studies and Research, 51(4), 243 – 261.
FDA (n.d.) Permits Marketing of First Game-Based Digital Therapeutic to Improve Attention
Function in Children with ADHD. https://www.fda.gov/news-events/press-
announcements/fda-permits-marketing-first-game-based-digital-therapeutic-
improve-attention-function-children-adhd
228
http://ijlter.org/index.php/ijlter
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., &
Wenderoth, M. P. (2014). Active learning increases student performance in
science, engineering, and mathematics. In: Proceedings of the National Academy of
Sciences of the United States of America, 111(23), 8410–5.
https://doi.org/10.1073/pnas.1319030111
Fromann, R. 2017. Jatekoslet – a gamifikacio vilaga, (Life of Players – the Word of Gamification).
Budapest, Hungary: Typotex Elektronikus Kiadó Kft.
Gentile, D. A., Choo, H., Liau, A., Sim, T., Li, D., Fung, D., & Khoo, A. (2011). Pathological
video game use among youths: a two-year longitudinal study. Pediatrics, 127(2),
319-329. https://doi.org/10.1542/peds.2010-1353
Goksuna, D.O., & Gursoy, G. (2019). Comparing success and engagement in gamified
learning experiences via Kahoot and Quizizz, Computers & Education, 135. 15-29.
https://doi.org/10.1016/j.compedu.2019.02.015
Hakulinen, L., & Auvinen, T. (2014). The Effect of Gamification on Students with Different
Achievement Goal Orientations. In: International Conference on Teaching and
Learning in Computing and Engineering, IEEE, (pp. 9–16)
https://doi.org/10.1109/LaTiCE.2014.10
Hamari, J. (2013). Transforming homo economicus into homo ludens: A field experiment
on gamification in a utilitarian peer-to-peer trading service. Electronic commerce
research and applications, 12(4), 236-245.
https://doi.org/10.1016/j.elerap.2013.01.004
Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? a literature review of
empirical studies on gamification. In: Proceedings of the 47th Hawaii International
Conference on System Sciences, Hawaii, USA.
https://doi.org/10.1109/HICSS.2014.377
Han-Huei Tsaya, C., Kofinasb, A., & Luoa, J. (2018). Enhancing student learning
experience with technology-mediated gamification: An empirical study.
Computers & Education, 121. 1-17. https://doi.org/10.1016/j.compedu.2018.01.009
Hanson, R. (2017). Positive neuroplasticity: The neuroscience of mindfulness, Advances in
Contemplative Psychotherapy, Oxfordshire, England: Routledge.
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A
longitudinal study on intrinsic motivation, social comparison, satisfaction, effort,
and academic performance. Computers and Education, 80, 152–161.
https://doi.org/10.1016/j.compedu.2014.08.019
Hoellwarth, C. & Moelter, M. J. (2011). The implications of a robust curriculum in
introductory mechanics, American Journal of Physics, 79(5), 540–545.
https://doi.org/10.1119/1.3557069
Holmes, N. G., Wieman, C. E. & Bonn, D. A. (2015). Teaching critical thinking. In:
Proceedings of the National Academy of Sciences of the United States of America, 112(36),
11199–204. https://doi.org/10.1073/pnas.1505329112
Huotari, Kai., & Hamari, J. (2017). A definition for gamification: anchoring gamification in
the service marketing literature. Electronic Markets, 27(1), 21–31.
https://doi.org/10.1007/s12525-015-0212-z
Ibanez, M.B., Di-Serio, A. & Delgado-Kloos, C. (2014). Gamification for Engaging
Computer Science Students, In: Learning Activities: A Case Study, IEEE Transactions
on Learning Technologies, 7(3), 291–301.
https://doi.org/10.1109/TLT.2014.2329293
Junco, R., & Cotten, S. R. (2012). No A 4 U: The relationship between multitasking and
academic performance, Computers & Education, 59(2), 505-514.
https://www.learntechlib.org/p/66713/
Kahoot program n.d. https://kahoot.com/ignite/
229
http://ijlter.org/index.php/ijlter
Kapp, K. M. (2012). The gamification of learning and instruction: game-based methods and
strategies for training and education. New Jersey, USA: John Wiley & Sons.
https://doi.org/10.1145/2207270.2211316
Kereluik, K., Mishra, P., Fahnoe, C., & Terry, L. (2013). What knowledge is of most worth:
Teacher knowledge for 21st century learning. Journal of digital learning in teacher
education, 29(4), 127-140. https://doi.org/10.1080/21532974.2013.10784716
Kim, B. (2015). Game Mechanics, Dynamics, and Aesthetics. Library Technology Reports
51(2), 17-19.
Kirschner, P. A., & Karpinski, C. (2010). Facebook® and academic performance, Computers
in human behavior, 26(6), 1237–1245. https://doi.org/10.1016/j.chb.2010.03.024
Kivunja, C. (2014). Innovative pedagogies in higher education to become effective teachers
of 21st century skills: Unpacking the learning and innovations skills domain of
the new learning paradigm, International Journal of Higher Education, 3(4), 37-48.
Knaving, K., & Bjork, S. (2013). Designing for fun and play: exploring possibilities in
design for gamification. In: Nacke, L.E., Harrigan, K., Randall, N. (eds).
Proceedings of the first International conference on gameful design, research, and
applications (pp. 131-134). https://doi.org/10.1145/2583008.2583032
Kollins, S. H., DeLoss, D. J., Canadas, E., Lutz, J., Findling, R. L., Keefe, R. S., & Faraone,
S. V. (2020). A novel digital intervention for actively reducing severity of
paediatric ADHD (STARS-ADHD): a randomised controlled trial, The Lancet
Digital Health. https://doi.org/10.1016/S2589-7500(20)30017-0
Kumar, J. A. & Bervell, B. (2019). Google Classroom for mobile learning in higher
education: Modelling the initial perceptions of students, Education and Information
Technologies, 24(2), 1793-1817. https://doi.org/10.1007/s10639-018-09858-z
Kusuma, G., Wigati, E., & Utomo, Y. (2018). Analysis of Gamification Models in Education
Using MDA Framework, Procedia Computer Science 135, 385-392.
https://doi.org10.1016/j.procs.2018.08.187
Mura, L., Gontkovicova, B., Dulova Spisakova, E. & Hajduova, Z. (2019). Position of
Employee Benefits in Remuneration Structure. Transformations in Business &
Economics, 18(2(47)), 156-173.
Nah, F.F.H., Zeng, Q., Telaprolu, V.R., Ayyappa, A.P., & Eschenbrenner, B. (2014).
Gamification of Education: A Review of Literature, In: Nah F.FH. (ed) HCI in
Business. HCIB 2014. Lecture Notes in Computer Science, 8527. New York City, USA:
Springer Cham. https://doi.org/10.1007/978-3-319-07293-7_39
Najah, A. A., Rohmah, A. F., & Susilo, H. (2019). The Implementation of Problem Based
Learning (PBL) Model Improving Students’ Oral Communication Skill through
Lesson Study, Journal of Physics: Conference Series 1227(1), 012004, IOP Publishing.
https://doi.org/10.1088/1742-6596/1227/1/012004
Pelling, N. (2011). The (short) prehistory of gamification, Funding Startups (& other
impossibilities). http://nanodome.wordpress.com/2011/08/09/the-
shortprehistory-of-gamification/
Price, R. B., & Duman, R. (2019). Neuroplasticity in cognitive and psychological
mechanisms of depression: An integrative model, Molecular psychiatry, 1-14.
https://doi.org/10.1038/s41380-019-0615-x
Prieto, M.C., Palma, L.O., Tobias, P.J.B. & Leon, F.J.M. (2019). Student Assessment of the
Use of Kahoot in the Learning Process of Science and Mathematics. Education
Science, 9(55), 1-13. https://doi.org/10.3390/educsci9010055
Probst, G. J. (1998). Practical knowledge management: A model that works, Cambridge
Massachusetts, USA: Prism-Cambridge.
Rozsa, Z. (2019). Should universities in the field of management studies influence the
ethnocentrism of students? Acta Oeconomica Universitatis Selye 8(2), 41-51.
230
http://ijlter.org/index.php/ijlter
Ryan, M., Costello, B., & Stapleton, A. (2012). Deep learning games through the lens of the
toy, In: Meaningful Play 2012 Conference Proceedings, East Lansing, MI: Michigan
State University, (pp. 1-29).
Santayana, G. (1955). The Sense of Beauty: Being the Outline of Aesthetic Theory, New York,
USA: Courier Corporation Dover Publications.
Salen, K. & Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals. Cambridge
Massachusetts, USA: The MIT Press.
Schwab, K. (2016). The Fourth Industrial Revolution: What it means and How to Respond.
Retrieved from https://www.weforum.org/agenda/2016/01/the-fourth-
industrial-revolution-what-it-means-and-how-to-respond/
Seemiller, C. & Grace, M. (2016). Generation Z goes to college, John Wiley & Sons.
Sheldon, L. (2012). The Multiplayer Classroom: Designing Coursework as a Game, Boston, USA:
Course Technology PTR.
Silpasuwanchai, C., Ma, X., Shigemasu, H., & Ren, X. (2016). Developing a comprehensive
engagement framework of gamification for reflective learning. In: Proceedings of
the 2016 ACM Conference on Designing Interactive Systems (pp. 459-472).
https://doi.org/10.1145/2901790.2901836
Simamora, S. J., Simamora, R. E., & Sinaga, B. (2017). Application of problem based
learning to increase students’ problem solving ability on geometry in class X SMA
Negeri 1 Pagaran, International Journal of Sciences: Basic and Applied Research
(IJSBAR), 36(2), 234-251.
Steigerwald, J. (2016). Entanglements of instruments and media in investigating organic
life, Studies in History and Philosophy of Science Part C: Studies in History and
Philosophy of Biological and Biomedical Sciences, 57, 107-111
https://doi.org/10.1016/j.shpsc.2016.03.001
Supriyanto, A., Setiawan, A. C. & Budiarti, E. M. (2018). Social Media Diffusion of Google
Classroom in Teaching and Learning. In: International Conference on Education and
Technology (ICET 2018). Atlantis Press. (pp.1-4). https://doi.org/10.2991/icet-
18.2018.1
Talbot, R., Doughty, L., Nasim, A., Hartley, L. & Le, P. (2016). Theoretically Framing a
Complex Phenomenon: Student Success in Large Enrollment Active Learning
Courses. In: Jones, D. L., Ding, L., and Traxler, A.L. (eds). Proceedings Physics
Education Research Conference 2016. Part of the PER Conference series, Sacramento,
CA: July 20-21, 2016. (pp. 344–347). https://doi.org/10.1119/perc.2016.pr.081
Tamin, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C. & Schmid, R. F. (2011). What
Forty Years of Research Says About the Impact of Technology on Learning: A
Second-Order Meta-Analysis and Validation Study, Review of Educational Research,
81(1), 4–28. https://doi.org/10.3102%2F0034654310393361
Tari, A. (2015). Z generacio: Klinikai pszichologiai jelensegek es tarsadalom-lelektani szempontok
az informacios korban. (Generation Z; Clinical Psychological Phenomenons and
Viewpoints of Socio-mental in the Age of Informatics) Budapest, Hungary: Tercium.
Uncapher, M. R. & Wagner, A. D. (2018). Minds and brains of media multitaskers: Current
findings and future directions, In: Meyer, D.E. (ed). Proceedings of the National
Academy of Sciences, 115(40), 9889-9896. https://doi.org/10.1073/pnas.1611612115
Valkenburg, P. M., Jochen, P. & Walther, J. B. (2016). Media effects: Theory and
research. Annual review of psychology, 67, 315–338
https://doi.org/10.1146/annurev-psych-122414-033608
Von Korff, J., Archibeque, B., Gomez, K. A, Heckendorf, T., McKagan , S. B., Sayre, E. C.
Schenk, E. W., Shepherd, C. & Sorell, L. (2016). Secondary analysis of teaching
methods in introductory physics: A 50k-student study, American Journal of Physics.
84(12), 969–974. https://doi.org/10.1119/1.4964354
231
http://ijlter.org/index.php/ijlter
Wang, S., Hsu, H., & Campbell, T. (2014). An investigation of middle school science
teachers and students use of technology inside and outside of classrooms:
considering whether digital natives are more technology savvy than their
teachers, Education Tech Research 62, 637–662. https://doi.org/10.1007/s11423-
014-9355-4
Wijaya, A. (2016). Analysis of factors affecting the use of Google Classroom to support
lectures. In: Abdillah, L. (ed.) Proceedings of The 5th International Conference on
Information Technology and Engineering Application (ICIBA2016), (pp. 61-68). Bina
Darma University.
Zichermann, G., & Cunningham, C. (2011). Gamification by Design: Implementing Game
Mechanics in Web and Mobile Apps, Sebastopol, CA: O’Reilly Media.
ZivkoviĿ, S. (2016). A model of critical thinking as an important attribute for success in
the 21st century. Procedia-Social and Behavioral Sciences, 232, 102-108.
https://doi.org/10.1016/j.sbspro.2016.10.034
