Content uploaded by Omer Kirmaci
Author content
All content in this area was uploaded by Omer Kirmaci on Apr 24, 2024
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=ujrt20
Journal of Research on Technology in Education
ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/ujrt20
The impact of scenario-based online gamified
learning environment tailored to player types
on student motivation, engagement, and
environment ınteraction
Ömer Kırmacı & Ebru Kılıç Çakmak
To cite this article: Ömer Kırmacı & Ebru Kılıç Çakmak (28 Mar 2024): The impact of scenario-
based online gamified learning environment tailored to player types on student motivation,
engagement, and environment ınteraction, Journal of Research on Technology in Education,
DOI: 10.1080/15391523.2024.2323447
To link to this article: https://doi.org/10.1080/15391523.2024.2323447
Published online: 28 Mar 2024.
Submit your article to this journal
View related articles
View Crossmark data
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION
The impact of scenario-based online gamified learning
environment tailored to player types on student motivation,
engagement, and environment ınteraction
Ömer Kırmacıa and Ebru Kılıç Çakmakb
aDistance Learning Research & Implementation Centre, Kırklareli University, Kırklareli, Turkey; bComputer &
Instructional Technologies Education, Gazi University, Ankara, Turkey
ABSTRACT
This study evaluates the impact of player-type-adapted scenario-based gam-
ified online learning environments on students’ motivation, engagement,
and interactions. In this context, a quasi-experimental design with a
pretest-post-test paired control group was adopted from quantitative
research methods. Within the scope of the research, elective scenarios
(learning environment) adapted to Bartle’s player typologies were devel-
oped and applied for six weeks. Findings reveal no significant variance in
the experimental group’s motivation and engagement, suggesting the
design’s efficacy. The stable motivation levels, despite the six-week duration,
and minimal dropout rates underscore this implication. Particularly, social-
izer type students exhibited increased interactions, driven by cooperation
and peer pressure, indicating the nuanced effectiveness of player-type tai-
lored learning environments in fostering engagement and participation.
Introduction
Games have long played a pivotal role in enriching educational experiences. However, the concept
of gamification extends beyond merely integrating games into teaching. It entails the strategic
use of game elements to enhance student engagement and active participation in academic
content, thereby fostering a dynamic and interactive learning experience. This involves students
interacting with the learning environment and maintaining this interaction over time (Kapp
etal., 2014). Trend analysis studies indicate that many works have found gamification to support
students’ active participation, motivation, and academic achievement (Kasurinen & Knutas, 2018;
Meşe & Dursun, 2018; Şahin & Samur, 2017). Meta-analysis studies by Yıldırım and Şen (2019),
Özcan (2019), R. Huang et al. (2020) also affirm the positive impact of Gamified Learning
Environments (GLE) on academic success and learning outcomes. However, some studies (e.g.
Hanus & Fox, 2015; Mekler, 2015; Mekler et al., 2013) indicate a decline in academic perfor-
mance linked to motivation when gamification elements are not used cautiously.
The effectiveness of specific gamification elements (components and mechanics) varies among
individuals (Kocadere & Caglar, 2018). Some elements may even adversely affect motivation and
engagement (Ferro, 2018; Klock et al., 2015; Kocadere & Caglar, 2018). Therefore, when designing
Gamified Online Learning Environments (GOLE), it is crucial to consider the interaction between
students (players) and gamification elements, selecting those that positively influence individuals’
motivation and engagement.
© 2024 ISTE
CONTACT Ömer Kırmacı kirmaciomer@gmail.com Distance Learning Research & Implementation Centre, Kırklareli
University, Kırklareli, Turkey.
https://doi.org/10.1080/15391523.2024.2323447
ARTICLE HISTORY
Received 13 October
2023
Revised 1 February 2024
Accepted 4 February
2024
KEYWORDS
Tailored Gamification;
player types; scenario
based learning
2 KIRMACI AND KILIÇ ÇAKMAK
In recent years, “Tailored Gamification” which involves designing GLEs based on learner
characteristics, has gained popularity (Klock et al., 2020; Oliveira, et al., 2022). Research
by Santos et al. (2021) and Şahin and Samur (2017) highlights the abundance of gamifi-
cation elements and underscores the importance of selecting and utilizing these elements
based on teaching and learning theories, as well as individual. Consequently, researchers
have categorized gamification elements within the context of students’ individual charac-
teristics (e.g. Hunicke etal., 2004; Kapp, 2017; Marczewski, 2015). Player typologies are at
the forefront of these categories.
Various researchers have introduced player typologies according to different perspectives
and characteristics such as personality/character, game-playing attitude, and behavior (e.g.
Bartle, 1996; Ferro etal., 2013; Marczewski, 2015; Nacke et al., 2011). In this context, some
studies have identified appropriate components that motivate and support player types in
GLE (e.g. Ferro, 2018; Kocadere & Caglar, 2018; Marczewski, 2017). However, limited studies
are determining the impact of GLEs tailored to player types and individual differences.
While numerous studies explore the effect of GOLEs on motivation and engagement, few
delve into how individual preferences, aligned with player types, influence these factors.
Thus, there is a need for studies investigating the design of GOLE adapted to player types
in higher education online courses, particularly focusing on how these designs can increase
participation, motivation, and engagement.
This study investigated the impact of scenario-based learning environments, specifically tai-
lored to accommodate various player types, on learners’ motivation, engagement, and interaction
in the course. The study aims to address the following questions:
1. How do student interactions in a player-type-adapted scenario-based GOLE differ in terms
of variability from those in other scenarios and a control group?
2. What is the variation in motivation scores among students participating in the player-type
adapted scenario-based GOLE compared to those in the different scenarios and the control
group?
3. What is the variation in engagement scores among students participating in the player-type
adapted scenario-based GOLE compared to those in the different scenarios and the control
group?
Literature review
Individual differences and player types
Player types are prominent among individual characteristics (Klock et al., 2020; Oliveira et al.,
2022). Numerous gamification design models and frameworks are based on the player types
identified by Bartle and Marczewski, with Bartle’s classification of player types forming the
foundation for other classifications (Mora etal., 2017). Bartle analyzed player personalities within
a coordinate plane based on two dimensions. He differentiated between unilateral action (acting)
and interaction (interacting) variables based on the player’s behavioral attitude, and between
environment (world) or other players (people) variables in the context of the quality of the
player’s behavior (Figure 1). Consequently, four distinct player types were identified:
• interact with the environment (Explorers),
• act within the environment (Achievers),
• interact with people (Socializers), and
• act against people (Killers) (Bartle, 1996).
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 3
There are two fundamental characteristics of player typologies. Firstly, there are no strict
boundaries between player types, and in some cases, alongside a player’s dominant characteristics,
recessive traits can emerge, and variations can be observed over time (Ferro etal., 2013; Kocadere
& Caglar, 2018). Secondly, the mechanics that positively influence players can vary depending
on the gamification environment and player type, and the mechanics triggered by a gamification
component can change according to the player type (Bartle, 1996; Klock et al., 2015; Werbach
& Hunter, 2012). For example, according to a study by Kocadere and Caglar (2018), while a
leaderboard triggers the challenge mechanic in achiever players, it triggers the status mechanic
in killer players.
Scenarios and storytelling in GOLE
Scenarios and storytelling play a pivotal role in GOLE, addressing student queries such as “How
is this useful to me?” and “Why am I learning this?”. These elements are influential in both
educational and gaming contexts (Kapp, 2017). In gamification, stories boost the user’s intrinsic
motivation and facilitate identification with the narrative while completing tasks (Rasmusson,
2017; Yılmaz, 2017). Stories and narratives are crucial for creating a sense of flow in gamified
environments. However, uninteresting story themes and gamification plots fail to engage players.
In educational settings, scenarios can significantly increase student engagement with the
content (Arabacıoğlu, 2012; Schank etal., 1994). In multiplayer online environments, scenarios
aim to captivate players and sustain their engagement (Rasmusson, 2017; Toledo Palomino et al.,
2019). For this to occur, it’s essential for scenarios to reflect real-life experiences, and tasks
should be based on real-life problems. In this context, goal-based scenarios can be easily inte-
grated into teaching environments and stories based on real-life problems have proven effective
(Arabacıoğlu, 2012).
Schank et al. (1994) explain the principles that should be considered in creating goal-based
scenarios. These principles include thematic consistency, realism, fostering intrinsic motivation,
appropriate difficulty levels, and providing timely and adequate feedback (Clark & Mayer, 2012;
Schank et al., 1994).
In GLE, scenario narration can be executed in two ways: embedded and emergent (Toledo
Palomino etal., 2019). Embedded narratives typically exhibit fixed and linear progression, while
Figure 1. Bartle player type coordinate.
4 KIRMACI AND KILIÇ ÇAKMAK
emergent narratives advance by branching according to the player’s choices. However, due to
ease of implementation, embedded scenarios are often preferred.
Method
In the study, the impact of Gamified Online Learning Environments (GOLEs) on students’
motivation, engagement, and environment interaction in higher education distance courses was
examined utilizing a pretest-post-test matched control group quasi-experimental design. This
quantitative method is often employed when the random assignment of participant groups is
infeasible (Cresswell, 2014; Özmen, 2014). In educational research contexts like this, existing
groups, such as schools and classes, are commonly studied. Although convenient sampling is
typically used in quasi-experimental designs, researchers mitigate potential biases by matching
experimental and control groups on specific variables to ensure comparability, albeit not a sub-
stitute for random assignment (Büyüköztürk et al., 2015). In this specific study, groups were
paired based on the dependent variables: motivation and engagement scores.
Participants
The participants of the study were 158 engineering faculty students who took the “Office
Applications in Engineering” course, which was taught entirely online, in a higher education
institution during the fall semester of 2021–2022. These students are predominantly in their
first year and, therefore, have not previously taken a course via online distance education.
The experimental and control groups were delineated based on their respective academic
programs. Specifically, the experimental group was composed of students from the software and
electrical-electronics engineering programs, whereas the control group encompassed individuals
from the mechanical, civil, and mechatronics engineering programs. In this way, it was aimed
to prevent the exchange of information between the experimental and control groups that
threatened internal validity (Cresswell, 2014; Fraenkel et al., 2013; Özmen, 2014). Table 1 shows
the distribution of the experimental and control groups in terms of programs and some
information.
In the application process, the experimental group was asked to select a scenario adapted
according to player types. Within this scope, the experimental group was divided into four
sub-groups. The control group, however, continued the learning process as a single group.
Figure2 shows the number of participants distributed across groups and the distribution methods
of all groups.
A pretest analysis of the 153 students was conducted to measure whether the experimental
and control groups were equivalent in terms of their engagement and motivation scores
(5 students left the course at the beginning of the implementation for various reasons). These
analyses examined whether the groups had a normal distribution and homogeneity. Table 2
Table 1. Distribution of Experimental and Control Groups.
Gender n x (age) n x (age)
Experimental group Software engineering Male 52 19.18 121 19.18
Female 21 19.5
Electric-electronical
engineering
Male 34 19.05
Female 14 19
Control group Mechanical engineering Male 822.12 32 20.9
Female 318.33
Civil engineering Male 520.8
Female 327.6
Mechatronics engineering Male 10 19.27
Female 319.6
Total 153 19.53
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 5
shows the normality and homogeneity tests for the experimental and control groups based on
the variables of engagement and motivation.
The Shapiro-Wilk test was used to check the normality of the distribution of the groups
(p > .05). The Levene test was employed to test the homogeneity of variances. Although the
variances were found to be homogeneous for the engagement variable (p > .05), the motivation
variable was not homogeneous (p < .05); therefore, the Welch test was used for motivation (Alpar,
2020) (p > .05). Consequently, the variances were considered homogeneous for both variables in
the experimental and control groups, and the analysis proceeded. The pretest results of the
experimental and control groups are shown in Table 3.
Accordingly, it was observed that there was no significant difference between the experimental
and control groups in terms of motivation and engagement scores (p > .05). Thus, it was under-
stood that the experimental and control groups were equivalent in terms of achievement, moti-
vation, and engagement, and the implementation process was initiated.
Instruments
During the study conducted in the Online Learning Environment (OLE), students’ motivation
levels were assessed using the “Motivated Strategies for Learning Questionnaire (MSLQ),” adapted
into Turkish by Büyüköztürk et al. (2004), and initially created by Pintrich et al. (1993). The
motivation levels were measured before and after the application to discern variances. The
questionnaire, consisting of two distinct dimensions, evaluates academic motivation and learning
strategies specifically for university students. For this research, only the 31-item “Motivation
Figure 2. Participant allocation and demographic overview.
Table 2. Normality and Homogeneity Tests of Pretest Dependent Variables.
Dependent
variables Group nSkewness Kurtosis Shapiro–Wilk Levene Welch
Engagement Experimental 121 −0.018 0.856 0.172 0.453 0.243
Control 32 0.322 −0.260 0.513
Motivation Experimental 121 −0.139 −0.323 0.880 0.008 0.179
Control 32 −0.695 0.669 0.167
Table 3. t Test Results for Pretest Data.
NMean Std. deviation df tp
Engagement Experimental 121 3.51 10.87 151 −1.078 .283
Control 32 3.63 9.40
Motivation Experimental 121 5.16 19.93 151 −1.207 .229
Control 32 5.31 13.41
6 KIRMACI AND KILIÇ ÇAKMAK
Scale (MS)” was utilized. The scale’s Turkish adaptation has a total score correlation coefficient
of 0.85 with the original English version, confirming its consistency.
Additionally, students’ engagement levels were gauged using the “Student Engagement Scale
in Online Learning Environments,” adapted into Turkish by Ergün and Koçak Usluel (2015),
and initially developed by Sun and Rueda (2012). This scale, validated and reliable, comprises
three factors - Behavioral, Affective, and Cognitive - totaling 19 items. It was administered
before and after the application to identify changes in engagement levels.
GOLE
In this study, the topic of MS Word in the “Office Applications in Engineering” course taught
in the Engineering Faculty was designed. Accordingly, the designed course consists of four
components: Elective Scenarios, Elective Tasks, Interactive Videos, and Quizzes (Figure 3).
Elective Scenarios. This study developed narrative plots tailored to each player type, as clas-
sified by Bartle’s Action-Interaction and People-World dimensions. The Explorers were catered
to with discovery-centered plots, while Socializers received stories emphasizing collaboration
and social engagement. Killers were immersed in competitive, confrontation-themed narratives,
and Achievers were presented with task-centric stories highlighting individual achievements.
The narratives aimed to induce a flow state and sustained engagement for each player type,
enriched by integrating an emergent narrative method (Toledo Palomino etal., 2019), enabling
Figure 3. GOLE based on selective scenarios adapted to player types.
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 7
customized, branched story choices aligning with individual preferences and player
characteristics.
Elective Tasks. A task pool adapted to the scenarios according to weekly objectives and levels
was established. Students can select any task that aligns with their level within their scenario.
As seen in Figure 3, each task in each level (chapter) is equal in terms of learning outcomes.
They differ only in the context of emergent narratives, stories that are shaped according to the
learner’s choice. Accordingly, the Word course is planned over four levels, each covering weekly
topics. In the control group, tasks are given directly, not within the scope of any scenario.
Supportive Interactive Videos. Supportive videos, segmented according to the knowledge and
skills needed for weekly tasks/assignments, were added for both experimental and control groups.
The videos are categorized by topic. Students can watch any video they want according to the
given assignment. In the experimental group, course videos are provided in connection with the
scenario tasks. Afterwards, the student can watch any video according to the task of the week.
Some instant questions for reinforcement are included in the videos. The presentation and
selection screen of the videos can be seen in Figure 4.
Figure 4. Supportive interactive videos.
8 KIRMACI AND KILIÇ ÇAKMAK
Quizzes. Weekly quizzes were added to reinforce the learners’ knowledge. The quiz assessments
vary according to the player type scenarios. In the Killers group, a score table is created with
a leaderboard for every quiz. For the Explorers, students who attain a certain score are awarded
valuable items within the scope of the player’s journey.
Gamification components
Within the scope of the research, a GOLE design suitable for each player type has been made.
In this context, gamification components have been determined to be compatible with scenarios
that align with the character and behaviors of the player types. Table 4 contains the gamification
components used in the GOLE and their descriptions.
Procedure
The application covers 6 wk during the Fall term of 2021–2022 for the “Office Applications in
Engineering” course. All students taking the course use the created GOLE for the first time. To
familiarize themselves with the system and be prepared for the application process, all students
Table 4. Utilization of Components in Accordance with Player Typology.
Player type Component Description Mechanics
Killer Points The gamification score provided progress and the
player’s journey. Activities such as tasks and
supportive videos related to the outcome affected
the achievement score.
Progression
Leaderboard Shows the top 10 people on the leaderboard. The
ranking is based on the gamification score.
Competition
Scoreboard Students are ranked according to their score after a
certain activity.
Challenge
Locked level Levels limit the player’s journey. Enough points must
be accumulated to advance to the next level.
Challenge
Explorer Virtual goods The player’s progress will be based on whether they
have hidden or designated objects.
Progression
Hidden objects Some objects are hidden in certain places in the
learning environment. Students can find these
objects and collect them in their crates.
Mystery
Curiosity
Locked level Chapters are unlocked when students find or possess
some specific objects.
Curiosity
Trade Collected objects can be exchanged for items. Transaction
Socializer Team The students in the scenario are divided into groups
and the students who are divided into groups earn
points for their team.
Cooperation
Chat rooms Team members can use the chat room to strategize
and collaborate to earn points.
Cooperation
Points Event and achievement points are used to progress
through the story. In some events, additional
activities are planned to earn bonus points and
event points.
Progression
Leaderboard (with
team points)
The leaderboard is calculated as team points to build
team spirit
Status
Badge Badges prepared according to the type of activities
(forum, task submission, task evaluation) will be
obtained. These badges will be given as part of the
story.
Reward
Achiever Progress bar Level-based progress bar is used. It informs about the
completion of the levels on a point basis.
Progression
Locked level Transfers are subject to certain conditions. Challenge
Points Progress and the player’s journey were carried out with
the gamification score. Activities related to the
learning outcome such as tasks and lesson videos
affected the achievement score.
Resource Acquisition
Display board It is a component where the learner can see the tasks
they have performed and their level of achievement
in order to see their own situation.
Status
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 9
started under the control group conditions instead of the application subject for the first 3 wk.
The application process began in the fourth week. The calendar for the application process is
found in Figure 5.
Weeks 1–3: All students were introduced to a learning environment that did not include
gamification elements to help them get accustomed to the learning ecosystem. This involved
access to course videos, tasks, and assessments.
Week 4: An online meeting was organized to introduce the students to the application process.
Detailed information about the scenarios was provided during this meeting. Students were asked
which scenario they wanted to continue with and were required to choose one. The necessary
steps for registering for the selected scenario were explained. Students were given one week to
register for their preferred scenarios. As a result of the selections made, 19 students chose the
“Business Competition” scenario for the Killer player type, 56 chose the “An Engineer’s Success”
for the Achievers, 34 opted for the “A Traveler’s Adventure” for Explorers, and 12 selected the
“Helpful Student” scenario for Socializers. Meanwhile, 32 students entered the application process
in the control group.
Weeks 5–9: The application process began with the experimental group students registering
for their scenarios. During this phase, the researcher provided support for issues experienced
by the learners regarding scenario selection and progression through the GOLE. Similarly, in
the control group, an instant messaging system was established to solve the problems experienced
by students in the learning environment.
Week 10: A post-test application was conducted after the application process. Focus group
interviews were then conducted to gather students’ opinions on the scenario-based GOLE, adapted
according to player types.
Findings
How do student interactions in a player-type-adapted scenario-based GOLE differ in terms
of variability from those in other scenarios and a control group?
In this section, the interactions of the experimental group students with the GOLE are examined.
Firstly, the average access numbers for all groups were reviewed to observe their variations
throughout the application process. Table 5 displays the average number of accesses to the GOLE
portal by students in all groups.
Figure 5. Application process and procedures.
10 KIRMACI AND KILIÇ ÇAKMAK
Accordingly, it is observed that the number of accesses by all students increased during the
application process. The highest increase is noted in the socializers’ group, while the lowest is
in the control group. When the overall averages are examined, socializers are the player type
that accesses GOLE the most. (xSocializer = 5.32) The control group has the lowest average access
number (xControl =1.53).
Figure 6 presents a graph showing the average number of entries into the learning
environment by students participating in player-type scenarios and those in the control
group.
Upon examining the access values of the groups, it’s evident that the Socializers have the
highest number of accesses, setting them apart from the others. The other player types are
observed to have similar access values.
Achievers, despite having fewer accesses to the system than the control group in the 3rd
week, appeared to align with the Killers and Explorers in the subsequent weeks. The Killers saw
an increase in access numbers after the application process, particularly in the 11th and 12th
weeks. However, there was a sharp decline in the following week, dropping below the Socializers
once again.
In addition to the students’ access to the system, their interactions with the activities have
been examined. The average number of times students viewed these activities, which are present
in every scenario, is provided in Table 6.
The data reveals that scenario activity had the lowest average views at 4.16 across all groups,
while task activity attracted the highest average views at 19.60. In terms of scenarios, students
in the Explorers group averaged the highest views at 5.33, while the Achievers group averaged
the lowest at 3.21. For course videos, the Achievers group averaged the highest views at 21.99,
while the Socializers averaged the lowest at 7.64. However, in task activity, the Socializers led
with an average of 23.47 views, and the Achievers had the fewest at 16.40.
Table 5. Number of Student Access to the GOLE during the Course Period.
Pre-application Application Post-application Average
Experimental group 2.03 5.59 2.55 3.96
Killer 1.72 5.05 3.01 3.75
Achiever 1.54 5.17 1.75 3.41
Explorer 1.98 4.66 2.11 3.36
Socializer 2.89 7.5 3.31 5.32
Control group 1.17 2.21 0.6 1.53
Figure 6. Average number of entries of scenarios into the learning environment.
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 11
The Killers scenario recorded the highest average views for quiz questions at 21.54. In con-
trast, the Explorers scenario marked the lowest at 14.29. Weekly access data exhibited a decline
in scenario activity views after the first two weeks for all player types, indicating reduced viewing
frequency. There was no data for the control group in the scenario activity, but their viewing
data for video, evaluation, and task activities were noted.
In the course video activity, the Achievers scenario saw a peak in views at 64.44 in the first
week, normalizing in the following weeks. The Task activity, enabling students to view, upload, and
grade tasks, indicated a high initial view count for the Socializers and Killers scenarios, which
decreased in subsequent weeks. The control group consistently registered lower task view counts.
The Quiz activity’s average access numbers reveal distinct patterns among different student
scenarios. The Killers, assigned the “Business Competition” scenario, register the highest view
counts, experiencing a peak in the initial three weeks before declining in the fourth week, yet
maintaining a lead over other groups. The Achievers, under the “An Engineer’s Success” scenario,
follow a similar trend. In the Explorers’ scenario, “The Traveler’s Adventure,” students’ interest
peaked in the second week, after which engagement decreased. On the other hand, in the
“Helpful Student” scenario of the Socializers, access to the quiz activity was lower than in the
other scenarios. There was no significant change in the average access of Socializers. The control
group’s interaction diminishes over the first three weeks and rebounds in the final week. A
general decline in view counts is noted across all activities toward the application process’s
conclusion, with the Quiz activity being the least affected (Figure 7).
Table 6. Activity Viewing Averages by Scenarios.
Scenario Video Task Quiz
Experimental group Explorer 5.33 9.81 17.39 14.29
Achiever 3.21 21.99 16.40 15.43
Killer 3.84 17.19 21.15 21.54
Socializer 4.28 7.64 23.47 14.37
Average 4.16 14.15 19.60 16.41
Control group –4.47 10.05 11.14
Figure 7. Participation graphs of experimental and control groups according to the activity.
12 KIRMACI AND KILIÇ ÇAKMAK
To assess the level of student interest and dropout during the application process, the ratio
of students who viewed the activities to the total number of students in each group was exam-
ined. The ratio of students who viewed the activities can be seen in Table 7.
The Killers’ scenario recorded the highest scenario viewing rate at 97.25%, while interest in
the Explorers’ scenario declined from universal engagement to a 2% viewing rate by the fourth
week. In the video activity, 48.75% of the control group viewed, compared to a larger proportion
of the experimental group. Socializers exhibited the highest viewing rate at 96%, while Achievers
showed the lowest at 87%.
For task activities, 91% of the experimental group viewed, contrasting with 65% of the control
group. Socializers had the highest viewing rate at 98%, while Explorers had the lowest at 79%.
In the quiz activity, the experimental group again outperformed the control group with 88%
participation versus 68%. Participation was highest among Killers at 95% and lowest for
Explorers at 79%.
Overall, the experimental groups fully viewed initial activities, unlike the control group.
Participation peaked with Killers and Socializers at 95%, with Explorers recording the lowest at 77%.
What is the variation in motivation scores among students participating in the player-type
adapted scenario-based GOLE compared to those in the different scenarios and the control
group?
ANCOVA (Analysis of Covariance) analyses were employed to determine the impact of the
chosen scenarios on students’ motivation. Some statistics relating to each scenario and control
groups are provided in the table. According to the data, a decline in motivation scores is observed
Table 8. Statistics of Scenario Groups Motivation Pretest Post-test Scores.
Process Group nMean Std Skewness Kurtosis Shapiro–Wilk Levene Welch
Pretest Socializer 12 5.12 0.61 0.581 0.250 0.917 0.046*0.237
Killer 19 5.25 0.61 −0.799 −0.880 0.006*
Achiever 56 5.26 0.63 0.049 −0.375 0.738
Explorer 34 4.98 0.71 −0.059 −0.379 0.972
Control 32 5.32 0.43 −0.794 0.964 0.102
Post-test Socializer 12 4.93 0.37 0.812 1.222 0.749 0.112 –
Killer 19 4.92 0.63 −0.356 −0.847 0.374
Achiever 56 4.98 0.83 −0.309 −0.267 0.705
Explorer 34 4.81 0.79 0.318 −0.430 0.707
Control 32 5.10 0.53 0.284 −0.681 0.567
*p <. 05.
Table 7. Proportion of Students Viewing Activities.
Activity type Activity Explorer (%) Achiever (%) Killer (%) Socializer (%)
Control group
(%)
Scenario Scenario 1 100 100 100 100 None
Scenario 2 88 97 100 100
Scenario 3 62 84 100 92
Scenario 4 271 89 92
Video Video 1 100 100 100 100 59
Video 2 88 97 100 100 59
Video 3 83 81 89 92 41
Video 4 83 70 84 92 36
Task Task 1 100 100 100 100 70
Task 2 81 95 100 100 68
Task 3 71 92 95 100 64
Task 4 62 81 84 92 57
Quiz Quiz 1 100 100 100 100 82
Quiz 2 88 97 100 100 75
Quiz 3 79 81 95 92 59
Quiz 4 50 65 84 75 57
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 13
in all scenarios. The ANCOVA test was utilized to ascertain whether the difference in motivation
among the scenarios is statistically significant.
In Table 8, for the conduction of the ANCOVA test, the distributions of pretest and post-test
scores of the scenario groups are observed to be normal, except for the Killers group, where
the Shapiro-Wilk test is significant. However, when skewness and kurtosis values are examined,
they are seen to be within the ±1.5 limit values. The variances between the scenarios are not
homogeneously distributed (p < .05), but no significant difference is observed in the Welch’s
alternative test (p > .05). Therefore, we can state that the pretest and post-test scores between
the scenarios are normally distributed, and the variances are homogeneous. In another analysis,
a moderate positive linear relationship between the pretest and post-test scores is observed. The
homogeneity of the regression slopes has been tested. For this, the interactions with the pretest
scores of the scenario variable have been examined. Because this result is not statistically sig-
nificant (F(4-143) = 1.906, p> .05), it is decided that all assumptions have been met. The adjusted
post-test values for the scenarios can be seen in Table 9.
Accordingly, when pretest scores are controlled, it is seen that the control group has the highest
motivation score (xcontrol=5.05). Explorers are observed to have the highest motivation among the
Table 9. Scenario Groups Adjusted Motivation Post-test Scores.
Group nPost-test Adjusted post-test
Socializer 12 4.93 4.96
Killer 19 4.92 4.90
Achiever 56 4.98 4.93
Explorer 34 4.81 4.97
Control 32 5.10 5.05
Table 10. ANCOVA Analysis Results of Scenario Groups According to Motivation Dependent Variable.
Source
Type III sum
of squares df
Mean
square Fpη2
Pretest 11.459 111.459 34.268 .000 0.193
Scenario 2.517 40.629 1.882 .117 0.050
Scenario*pretest 2.550 40.637 1.906 .113 0.051
Error 47.816 143 0.334
Total 3835.862 153
Table 11. Statistics Related to Scenario Groups’ Engagement Pretest Post-test Scores.
Process Group nMean Std Skewness Kurtosis Shapiro–Wilk Levene
Pretest Socializer 12 3.63 0.55 0.506 0.498 0.621 0.927
Killer 19 3.23 0.61 −0.495 1.048 0.249
Achiever 56 3.61 0.58 −0.080 1.170 0.106
Explorer 34 3.49 0.52 0.378 0.601 0.674
Control 32 3.64 0.50 0.263 −0.256 0.576
Post-test Socializer 12 3.62 0.41 −0.530 −0.036 0.790 0.205
Killer 19 3.15 0.66 0.048 −0.859 0.604
Achiever 56 3.48 0.73 −0.651 0.445 0.110
Explorer 34 3.36 0.56 0.706 0.773 0.310
Control 32 3.56 0.54 0.535 −0.582 0.138
Table 12. Scenario Groups Adjusted Engagement Post-test Scores.
Group nPost-test Adjusted post-test
Socializer 12 3.62 3.58
Killer 19 3.15 3.30
Achiever 56 3.48 3.42
Explorer 34 3.36 3.40
Control 32 3.56 3.49
14 KIRMACI AND KILIÇ ÇAKMAK
scenario groups (xexplorer=4.97). A decline in the motivation level of all scenario groups is noted.
The ANCOVA analysis results, provided in Table 10, are used to determine whether the score
differences that occurred in the scenario groups are statistically significant according to the
scenarios.
According to the ANCOVA results of the scenario groups, it is observed that there is
no significant difference in the adjusted motivation scores of the students in the scenarios
based on their pretest motivation scores (F(4-143)=1.88, p > .05). In other words, this
result can be interpreted to mean that the scenarios in the GOLE do not have a significant
effect on motivation.
What is the variation in engagement scores among students participating in the player-
type adapted scenario-based GOLE compared to those in the different scenarios and the
control group?
ANCOVA analyses were employed with the aim of examining the significance of changes in the
engagement scores of scenario groups. Accordingly, some statistics for each scenario group are
provided in Table 11, it can be observed that all scenarios exhibit a decrease in engagement
scores. ANCOVA tests were conducted to determine whether there was a significant difference
in motivation among the scenarios.
It is observed that the assumptions of normality and homogeneity of variances are met for
the ANCOVA test. On the other hand, due to the existence of a high positive linear relationship
(r=.70 p<. 01) and the lack of significant combined effects of the scenario variable’s control
changes (F(4-143) = 1.301, p>. 05), all assumptions are accepted to have been met.
Examining the descriptive statistics results, we can state that a decline is observed in engage-
ment scores across all scenarios, similar to what is noted in motivation scores. When pretest
results are accepted as a standard control variable, the adjusted post-test scores associated with
the scenario groups can be seen in Table 12.
Accordingly, it is observed that the Socializers have the highest level of engagement
(xSocializer= 3.58). This value is noted to be higher than the controlled pretest score. In other
words, when the pretest scores of the scenario groups are controlled, it is observed that
the engagement scores of the Socializers are higher than those of other groups. ANCOVA
analyses have been conducted to test the significance of this difference. In Table 13, the
results of the ANCOVA analysis are displayed, determining the significance of the effect
of the scores between scenarios on post-test engagement scores when the pretest values of
the engagement variable are controlled.
According to the ANCOVA results, it is observed that there is no significant difference in
the adjusted post-test engagement scores of students in the scenario groups based on their pretest
engagement scores (F(4-143)=1.293; p > .05). In other words, this result can be interpreted as
the GOLE not having a significant effect on engagement in the context of both the scenario
groups and the control group.
Table 13. ANCOVA Analysis Results of Scenario Groups According to Engagement.
Source
Type III sum of
squares df Mean square Fpη2
Pretest 16.766 116.766 79.389 .000 0.357
Scenario 1.092 40.273 1.293 .276 0.035
Scenario*pretest 1.099 40.275 1.301 .273 0.035
Error 30.199 143 0.211
Total 1872.02 153
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 15
Discussion and conclusion
Online distance education has been gaining popularity and spreading widely in recent years.
However, issues like dropout and non-completion devalue online distance education and create
a perception of failure. Researchers attribute these problems to the lack of a learning environ-
ment that considers motivation and engagement elements, conforms to individual characteristics,
and promotes active student participation. On the other hand, there’s an increasing recognition
of the need for its flexible structure. Therefore, the necessity to design OLEs that address these
issues appropriately is evident. In this regard, GOLEs offer solutions to these problems, aiming
to enhance the quality and effectiveness of online distance learning by prioritizing interaction,
motivation, and engagement.
However, it’s emphasized that motivation and engagement should be considered, and individual
differences should be noted in the design of all OLEs, including GOLEs. The absence of a
significant difference in the motivation and engagement scores of the learners in the experimental
group in the current research is seen as a positive outcome. Furthermore, the higher task com-
pletion and participation rates of the students in the experimental group compared to the control
group reinforce this positive situation. As a result of the active participation observed in the
experimental group compared to the control group, the achievement of learning outcomes and
objectives is expected as a natural outcome of this research.
In the distribution of learners, the achievers’ group is the most preferred (n = 56), followed
by explorers (n = 34), kil lers (n = 19), and socializers (n = 12). This contrasts with Zichermann
and Cunningham’s (2011) assertion that most individuals are socializers. The prevalence of the
achievers’ group suggests that the emphasis on “achievement” and the portrayal of the hero as
an “engineer” in their scenario influenced learners’ choices. This indicates that factors such as
scenario topic, title, and character traits significantly sway participant preferences, aligning with
Schank et al. (1994)’s principles of scenario-based learning environments.
When examining the interaction of students with the GOLE, it is observed that their partic-
ipation progressively increased during the pre-application process. The experimental group’s
interaction with the learning environment has significantly increased compared to the control
group with the commencement of the application process. The spike in interaction in the first
week of the application process can be attributed to scenario selection and registration activities.
However, a meaningful increase in interaction compared to the control group is evident after
the sixth week in the experimental group. It can be inferred that the GOLE has enhanced
interaction, corroborated by a study by B. Huang etal. (2019).
In the context of player typologies, socializers exhibit the highest level of interaction with GOLE.
This observation aligns with the inherent characteristics of socializers in gamified settings, as
outlined by Bartle (1996) and Kocadere and Caglar (2018), who emphasize their predilection for
interaction-focused activities. Predominantly, socializers are involved in team activities centered
around tasks, which underscores the success of the collaborative environment specifically designed
for them. This finding is further corroborated through discussions with socializer group, confirming
the beneficial impact of a collaborative setting in enhancing the interactive experiences of socializers.
Conversely, in the killer player type, the dominant mechanics are competition and challenge.
These mechanics are triggered by specific components introduced in the environment. Iacono
et al. (2020) incorporated only leaderboards in the learning environment designed for killers.
In the current study, competition and challenge are supported by multiple components, including
a score table that lists Quiz scores, distinct from other scenarios. The higher interaction in Quiz
illustrates the effectiveness of instant challenges. The higher participation of killers in task activity
also indicates a result linked to the competition mechanic. However, some adverse situations
have emerged from the peer review activity that triggers the competition mechanic in the killer
group. Incidents like inquisition into evaluators’ identities, investigations in other social media
and communication groups, and disputes arising from scoring have been observed. These situ-
ations suggest a negative impact on student motivation.
16 KIRMACI AND KILIÇ ÇAKMAK
Peer assessment was intended to enhance active participation among learners. However, in
the killer group, it led to unintended negative consequences associated with competition. Tenório
et al. (2016) note that peer assessment in competitive environments can lead to a decrease in
motivation. This suggests a need for control over the competitive element within the killer group.
Especially in situations where learners can intervene in the process, minimizing or controlling
such interventions can help manage the scale of competition. For instance, structuring the
assessment criteria and process to be transparent, explainable, clear, and as immune to student
interpretation or intervention as possible, can prevent competition from reaching negative extents.
Measures like these are supported by studies by Xiong and Suen (2018) and Kurnaz (2021).
Also, the controlled inclusion of competitive elements in groups that highly value competition
can centralize focus and enhance motivation. For example, adding only a leaderboard or score
table can concentrate competition around point accumulation.
The achievers’ scenario saw the highest interaction in the video activity. Given that the video
activity is based on individual work, it can be said to support achievers’ individual struggles.
Achievers are also known to value task-based progress, explaining their high averages in task
and video interactions. Additionally, one of the reasons achievers chose their scenario was its
individual nature, further supporting this observation. Thus, it can be concluded that the indi-
vidual progress tracking offered in the designed scenario is apt for the achiever group.
In analyzing the weekly variations in system interactions, we notice a general decrease in
user engagement over the initial weeks. However, an exception is observed in the form of an
increased frequency of quiz interactions within the ‘killers’ scenario. This uptick suggests that
the competitive element in this scenario resonates effectively with the ‘killers’. The most pro-
nounced drop in engagement occurs in video-based activities. It has been observed that students
often resort to external resources, outside the GOLE, to complete their tasks. This tendency
underscores a critical issue in online distance learning: the insufficiency of effective educational
materials, as noted by Kil and Uşun (2021), Sánchez and Martí (2017), and A. B. Yılmaz and
Karataş (2022). Therefore, it is advisable to incorporate external resources into OLEs, as recom-
mended by Lee et al. (2019), to guide learners toward a broader array of information sources.
Within the context of the GOLE developed for this study, the task-based structure necessitates
content that aligns with specific tasks. Hence, interactive lesson videos are tailored to align with
the characteristics demanded by these tasks. This strategy ensures that learning materials are
not only abundant but are also contextually relevant, enhancing the overall engagement and
learning outcomes for students navigating the tasks within the gamified environment. It under-
scores the need for adaptive content strategies that are responsive to the dynamic learning paths
unfolding within gamified learning contexts.
At the conclusion of the application, it is observed that there is no significant change in the
students’ motivation and engagement. In the literature, it is noted that the novelty effect can
lead to a reduction in motivation (Hamari et al., 2014; Tondello et al., 2017). According to a
study by Rodrigues et al. (2022), motivation in GOLE applications decreases in the first four
weeks but begins to rise after the sixth week. Within the scope of this study, the fact that
learners’ motivation did not decrease strengthens the view that the design was effective. In this
regard, the six-week duration of the application process might be a reason for the lack of sig-
nificant effect on motivation.
Additionally, the experimental groups in the online courses show a significant increase in
activity participation and a decrease in dropout rates, which is quite remarkable. Nevertheless,
positive trends in student task performance and increased engagement in various activities
indicate a high level of active participation in the gamified course designed in the study. This
finding is encouraging and satisfactory for both instructors and researchers. Especially for stu-
dents involved in the socializer player type, it is observed that all tasks were completed, and
interactions increased due to collaboration and peer pressure. This observation underlines the
impact of social dynamics on student engagement and completion rates, emphasizing the role
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 17
of peer interactions in enhancing both motivation and engagement to the tasks within a gamified
learning context. This could potentially offer insights into optimizing future designs of GOLEs
to bolster student engagement and learning outcomes over extended periods.
Limitations and future directions
This study has a few primary limitations. Firstly, the study’s design allowed each student to
progress in a learning environment tailored to their individual player type. To facilitate this,
equivalent learning environments were created, each corresponding to different player types
but aligned in terms of learning outcomes and gains. Although this approach was intended
to enhance engagement and motivation by providing a personalized learning experience, it
also introduced the challenge of developing multiple learning environments for a single edu-
cational objective. This could be perceived as impractical and is acknowledged as a limitation
of the research. However, the observed increase in active participation and positive outcomes
in terms of motivation and engagement suggest the effectiveness of this tailored approach.
Additionally, the potential for reusability and adaptability of e-learning environments should
be noted. With minor updates to scenarios and tasks, these environments can remain effective
over time.
The second limitation concerns the sample size. Despite having a considerable number of
experimental groups, the overall number of participants was limited. To address this, researchers
selected a course taught across an entire faculty to maximize participant numbers. However, for
future research, especially in the context of massive online courses, a larger participant group
would be beneficial to enhance the external validity of the findings.
Finally, this research focused on addressing specific challenges encountered in online courses,
such as deficiencies in interaction, motivation, and engagement, through a gamification design.
As such, variables like course achievement were not included in this study. The impact of the
player-type-adapted, scenario-based GOLE on student achievement is an area ripe for future
investigation. By exploring this aspect, researchers can gain a more comprehensive understanding
of the effectiveness of tailored gamification strategies in online learning environments and their
influence on various learning outcomes.
Acknowledgments
This study was produced from the corresponding author’s dissertation, no. 749641.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Notes on contributors
Dr. Ömer Kırmacı completed his undergraduate education at Marmara University Computer Education and
Instructional Technology Department. He completed his master’s degree in Computer Education and Instructional
Technology at Çanakkale OnsekizMart University. He completed his doctorate degree in Computer Education and
Instructional Technology at Gazi University. His research interests are Human Performance Development, Human
Computer Interaction and Interaction Design in Online Learning. Since 2012, he has been working at Kırklareli
University Distance Education Application and Research Center.
Ebru Kılıç Çakmak is a Professor at Gazi University’s Department of Computer Education and Instructional
Technology. She holds an MSc. and Ph.D. in Educational Technology from Ankara University. Since September
2018, she has been the Head of the Department of Computer Education and Instructional Technology at Gazi
Education Faculty. Her research focuses on e-learning, online learning environment design, instructional design,
18 KIRMACI AND KILIÇ ÇAKMAK
technology integration in education, project management, and scale development. Prof. Kılıç Çakmak has taught
both undergraduate and graduate courses in various formats, improved distance education content, and actively
participated in seminars and scientific committees at national and international events. She has an extensive pub-
lication record in indexed journals, with 68 citations in Web of Science, and over 2000 citations to her work in
national refereed journals. Additionally, she has contributed to numerous national and international projects as a
researcher, consultant, and instructor, including EU and ABD national projects.
ORCID
Ömer Kırmacı http://orcid.org/0000-0002-0954-1263
References
Alpar, R. (2020). Uygulamalı İstatistik ve Geçerlik-Güvenirlik [Applied statistics & validity-reliability] (6. Baskı). Detay
Yayıncılık.
Arabacıoğlu, T. (2012). Farklı İletişim Ortamlarıyla Yürütülen Senaryo Temelli Öğretim Programının Temel Bilgi
Teknolojileri Dersi Erişilerine Etkisi [The effect of scenario-based curriculum with different communication media
on basic information technologies course achievement]. Adnan Menderes Universitesi.
Bartle, R. (1996, August). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD Research, 1(1),
19. http://www.arise.mae.usp.br/wp-content/uploads/2018/03/Bartle-player-types.pdf
Büyüköztürk, Ş., Akgün, Ö. E., Kahveci, Ö., & Demirel, F. (2004). Güdülenme ve Öğrenme Stratejileri Ölçeğinin
Türkçe Formunun Geçerlik ve Güvenirlik Çalışması [The validity and reliability study of the Turkish version of
the motivated strategies for learning questionnaire]. Kuram Ve Uygulamada Eğitim Bilimleri, 4(2), 207–239.
Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2015). Quantitative research. In
Scientific research methods (19th ed., pp. 173–238). Pegem Akademi.
Clark, R. C., & Mayer, R. E. (2012). Design of scenario-based e-learning. In Scenario-based e-learning: Evidence-based
guidelines for online workforce learning (pp. 35–45). Center for Creative Leadership.
Cresswell, J. W. (2014). Designing research. In J. W. Cresswell (Ed.), Research design qualitative quantitative and
mixed methods approaches (4th ed., p. 398). SAGE Publications. https://doi.org/10.1007/s13398-014-0173-7.2
Ergün, E., & Koçak Usluel, Y. (2015). Çevrimiçi Öğrenme Ortamlarında Öğrenci Bağlılık Ölçeği’nin Türkçe Uyarlaması:
Geçerlik ve Güvenirlik Çalışması. Eğitim Teknolojisi Kuram ve Uygulama, 5(1). https://doi.org/10.17943/etku.64661
Ferro, L. S. (2018). An analysis of players’ personality type and preferences for game elements and mechanics.
Entertainment Computing, 27(March), 73–81. https://doi.org/10.1016/j.entcom.2018.03.003
Ferro, L. S., Walz, S. P., & Greuter, S. (2013). Towards personalised, gamified systems: An investigation into game
design, personality and player typologies. In 9th Australasian Conference on Interactive Entertainment Matters of
Life and Death – IE’13 (pp. 1–6). ACM Publication. https://doi.org/10.1145/2513002.2513024
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2013). How to design research in education and evaluate (8th bs., C.
53). McGraw-Hill. https://doi.org/10.1017/CBO9781107415324.004
Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? - A literature review of empirical studies on
gamification. In Proceedings of the Annual Hawaii International Conference on System Sciences (pp. 3025–3034).
IEEE Publications. https://doi.org/10.1109/HICSS.2014.377
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on
intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education,
80, 152–161. https://doi.org/10.1016/j.compedu.2014.08.019
Huang, B., Hwang, G.-J J., Hew, K. F., & Warning, P. (2019). Effects of gamification on students’ online inter-
active patterns and peer-feedback. Distance Education, 40(3), 350–379. https://doi.org/10.1080/01587919.2019.
1632168
Huang, R., Ritzhaupt, A. D., Sommer, M., Zhu, J., Stephen, A., Valle, N., Hampton, J., & Li, J. (2020). The impact
of gamification in educational settings on student learning outcomes : A meta - analysis. Educational Technology
Research and Development, 68(4), 1875–1901. https://doi.org/10.1007/s11423-020-09807-z
Hunicke, R., Leblanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. In
AAAI Workshop Papers (pp. 1–5). Association for the Advancement of Artificial Intelligence (AAAI) Publications.
Iacono, S., Vallarino, M., & Vercelli, G. (2020). Gamification in corporate training to enhance engagement: An
approach. International Journal of Emerging Technologies in Learning (İJET), 15(17), 69–84. https://doi.org/10.3991/
ijet.v15i17.14207
Kapp, K. M. (2017). Gamification designs for ınstruction. In R. Reigeluth, C. M. Beatty, & B. J. Myers (Ed.),
Instructional design theories and models, vol IV: The learner-centered paradigm of education (pp. 351–383).
Routledge Publications.
Kapp, K. M., Blair, L., & Mesch, R. (2014). The gamification of learning and ınstruction fieldbook: Ideas into practice.
Wiley.
JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION 19
Kasurinen, J., & Knutas, A. (2018). Publication trends in gamification: A systematic mapping study. Computer
Science Review, 27, 33–44. https://doi.org/10.1016/j.cosrev.2017.10.003
Kil, G., & Uşun, S. (2021). An analysis of distance education problems experienced in higher education: Â
meta-synthesis study. Yuksekogretim Dergisi, 11(3), 638–648. https://doi.org/10.2399/yod.20.644139
Klock, A. C. T., Gasparini, I., Pimenta, M., & de Oliveira, J. P. M. (2015). “Everybody is playing the game, but
nobody’s rules are the same”: Towards adaptation of gamification based on users’ characteristics. Bulletin of The
Technical Committee on Learning Technology, 17(4), 22–25.
Klock, A. C. T., Gasparini, I., Pimenta, M. S., & Hamari, J. (2020). Tailored gamification: A review of literature.
International Journal of Human-Computer Studies, 144(September 2019), 102495. https://doi.org/10.1016/j.
ijhcs.2020.102495
Kocadere, S. A., & Caglar, S. (2018). Gamification from player type perspective: A case study. Educational Technology
& Society, 21(3, SI), 12–22.
Kurnaz, F. B. (2021). Online peer assessment in teacher education. Journal of Educational Technology and Online
Learning, 4(4), 835–853. https://doi.org/10.31681/jetol.987902
Lee, D., Watson, S. L., & Watson, W. R. (2019). Systematic literature review on self-regulated learning in massive
open online courses. Australasian Journal of Educational Technology, 35(1), 28–41. https://doi.org/10.14742/
ajet.3749
Marczewski, A. (2015). Even Ninja monkeys like to play: Gamification, game thinking and motivational design.
CreateSpace Independent Publishing Platform. https://www.gamified.uk/wp-content/uploads/2016/01/Loyalty.pdf
Marczewski, A. (2017). Periodic table of gamification elements. Gamified UK. https://www.gamified.uk/2017/04/03/
periodic-table-gamification-elements/periodic-table-of-gamification-elements/
Mekler, E. D. (2015). The motivational potential of digital games and gamification – The relation between game ele-
ments, experience and behavior change. University of Basel. http://edoc.unibas.ch/44253/1/diss_emekler_ub_noCV.
pdf#page=62
Mekler, E. D., Brühlmann, F., Opwis, K., & Tuch, A. N. (2013). Do points, levels and leaderboards harm intrinsic
motivation? In Proceedings of the First International Conference on Gameful Design, Research, and Applications -
Gamification ‘13 (pp. 66–73). ACM Publications. https://doi.org/10.1145/2583008.2583017
Meşe, C., & Dursun, Ö. Ö. (2018). Influence of gamification elements on emotion, ınterest and online participation.
Ted Eğitim Ve Bilim, 43(196), 67–95. https://doi.org/10.15390/EB.2018.7726
Mora, A., Riera, D., Gonzalez, C., Arnedo-Moreno, J., González, C., & Arnedo-Moreno, J. (2017). Gamification: A
systematic review of design frameworks. Journal of Computing in Higher Education, 29(3), 516–548. https://doi.
org/10.1007/s12528-017-9150-4
Nacke, L. E., Bateman, C., & Mandryk, R. L. (2011). BrainHex: Preliminary results from a neurobiological gamer
typology survey. In J. C. Anacleto, S. Fels, N. Graham, B. Kapralos, M. Saif El-Nasr, & K. Stanley (Eds.),
Entertainment computing – ICEC 2011 (pp. 288–293). Springer.
Oliveira, W., Hamari, J., Shi, L., Toda, A. M., Rodrigues, L., Palomino, P. T., & Isotani, S. (2022). Tailored gamifi-
cation in education: A literature review and future agenda. Education and Information Technologies, 28(1), 373–
406. https://doi.org/10.1007/s10639-022-11122-4
Özcan, Ş. (2019). Eğitimde Oyunlaştırma Üzerine Yapılan Araştırmalara İlişkin Bir Meta Analiz Çalışması[A
meta-analysis of research on gamification in education]. Fırat Üniversitesi.
Özmen, H. (2014). Deneysel Araştırma Yöntemi [Experimental research method]. In M. Metin (Ed.), Eğitimde
Bilimsel Araştırma Yöntemleri [Scientific research method in education]
(1st ed., pp. 47–76). Pegem Akademi.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & Mckeachie, W. J. (1993). Reliability and predictive validity of the
motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–
813. https://doi.org/10.1177/0013164493053003024
Rasmusson, T. (2017). Interactive storytelling, gamification and online education: Storytelling made easy. International
Journal on Innovations in Online Education, 1(3). https://doi.org/10.1615/IntJInnovOnlineEdu.2017018913
Rodrigues, L., Pereira, F. D., Toda, A. M., Palomino, P. T., Pessoa, M., Carvalho, L. S. G., Fernandes, D., Oliveira,
E. H. T., Cristea, A. I., & Isotani, S. (2022). Gamification suffers from the novelty effect but benefits from the
familiarization effect: Findings from a longitudinal study. International Journal of Educational Technology in
Higher Education, 19(13), 1–25. https://doi.org/10.1186/s41239-021-00314-6
Şahin, M., & Samur, Y. (2017). Dijital Çağda Bir Öğretim Yöntemi: Oyunlaştırma [A teaching method in the dig-
ital age: Gamification]. Journal of Ege Education Technologies, 1(1), 1–27.
Sánchez, A., & Martí, J. (2017). Drivers and barriers to adopting gamification: Teachers’ perspectives. Electronic
Journal of E-Learning, 15(5), 434–443.
Santos, A. C. G., Oliveira, W., Hamari, J., Rodrigues, L., Toda, A. M., Palomino, P. T., & Isotani, S. (2021). The
relationship between user types and gamification designs. User Modeling and User-Adapted Interaction, 31(5),
907–940. https://doi.org/10.1007/s11257-021-09300-z
Schank, R. C., Fano, A., Bell, B., Jona, M., Schank, R. C., Fano, A., Bell, B., Jona, M., Schank, R. C., Fano, A., &
Bell, B. (1994). The design of goal-based scenarios. Journal of the Learning Sciences, 3(4), 305–345. https://doi.
org/10.1207/s15327809jls0304
20 KIRMACI AND KILIÇ ÇAKMAK
Sun, J. C. Y., & Rueda, R. (2012). Situational interest, computer self-efficacy and self-regulation: Their impact on
student engagement in distance education. British Journal of Educational Technology, 43(2), 191–204. https://doi.
org/10.1111/j.1467-8535.2010.01157.x
Tenório, T., Bittencourt, I. I., Isotani, S., Pedro, A., & Ospina, P. (2016). A gamified peer assessment model for
on-line learning environments in a competitive context. Computers in Human Behavior, 64, 247–263. https://doi.
org/10.1016/j.chb.2016.06.049
Toledo Palomino, P., Toda, A. M., Oliveira, W., Cristea, A. I., & Isotani, S. (2019). Narrative for gamification in
education: Why should you care? In Proceedings of the IEEE 19th International Conference on Advanced Learning
Technologies, ICALT 2019 (pp. 97–99). IEEE Publications. https://doi.org/10.1109/ICALT.2019.00035
Tondello, G. F., Orji, R., & Nacke, L. E. (2017). Recommender systems for personalized gamification. In M. Tkalcic
& D. Thakker (Eds.), UMAP 2017 - Adjunct Publication of the 25th Conference on User Modeling, Adaptation
and Personalization (pp. 425–430). ACM Publications. https://doi.org/10.1145/3099023.3099114
Werbach, K., & Hunter, D. (2012). Game thinking. In How game thinking can revolutionize your business. Wharton
Digital press.
Xiong, Y., & Suen, H. K. (2018). Assessment approaches in massive open online courses: Possibilities, challenges
and future directions. International Review of Education, 64(2), 241–263. https://doi.org/10.1007/s11159-018-9710-5
Yıldırım, İ., & Şen, S. (2019). The effects of gamification on students’ academic achievement: A meta-analysis study.
Interactive Learning Environments, 29(8), 1301–1318. https://doi.org/10.1080/10494820.2019.1636089
Yılmaz, A. B., & Karataş, S. (2022). Why do open and distance education students drop out ? Views from various
stakeholders. International Journal of Educational Technology in Higher Education, 19(1), 28. https://doi.
org/10.1186/s41239-022-00333-x
Yılmaz, E. A. (2017). Oyunlaştırma [Gamification] (4th ed.). Abaküs.
Zichermann, G., & Cunningham, C. (2011). Player motivation. In Gamification by design: Implementing game me-
chanics in web and mobile apps. O’Reilly Media Publication. https://doi.org/10.1017/CBO9781107415324.004
