Content uploaded by Ovidiu Noran
Author content
All content in this area was uploaded by Ovidiu Noran on Feb 09, 2018
Content may be subject to copyright.
Content uploaded by Ovidiu Noran
Author content
All content in this area was uploaded by Ovidiu Noran on Feb 09, 2018
Content may be subject to copyright.
On Gamification in Action Learning
Ovidiu Noran
Griffith University
Nathan Campus
Nathan QLD4111 AUSTRALIA
+61-7-37355382
ABSTRACT
The advent of globalisation brought about by advances in
information and communication technology has triggered major
changes in the way people work, live and study. The modern
teaching endeavour must meet the needs of a fundamentally
changing learning environment and student cohorts, while
preserving delivered knowledge quality in order to meet the
required learning objectives. Flexible teaching (distance/online,
intensive mode, afternoon classes etc.), online resources and tools,
teachers with industry experience and especially new teaching
models matching the new cohorts profile requirements can
provide a solid platform for a new paradigm in the higher
education domain.
This research has investigated how a customised application of
‘gamification’ (taking game-based elements and applying them in
non-gaming contexts) to higher education can improve the
learning and teaching experience and student engagement and
thus help towards reducing student attrition. Gamification of the
educational endeavour has been studied in the past; however, in
this study it is applied within a novel iterative and combined
Action Research, Experiential Action Learning, Plan, Implement,
Review, Improve and Plan, Do, Study, Act approach that attempts
to make contributions to both learning and teaching theory and
practice.
Categories and Subject Descriptors
• Computing Education
Keywords
Action Learning; Gamification; Action Research
1. INTRODUCTION
The current global dynamic and competitive economic
environment enabled by fast evolving technologies requires
companies to become agile so as to readily and continuously
adapt in order to stay abreast of competition. As a result, the
current and future workforce needs to acquire an increasing
amount and variety of skills, in a shorter time span. Another
emerging global issue is population ageing, which is putting
significant pressure on healthcare and social services worldwide.
Governments typically react to this challenge by extending the
retirement ages; as a result, people need to work for longer and
often must study additional degrees at a mature age in order to
keep up with employer demands, re-enter the workforce or switch
employment areas.
These requirements are reflected in the evolution of student cohort
profiles and their learning needs and habits, as people from
increasingly varied backgrounds and age brackets are entering
higher education. These students often have to sustain themselves
and sometimes also their family, which reduces the time available
for tuition, especially travel required for face-to-face interaction.
As information is fast becoming ubiquitous, the capability to
recall on short notice large pieces of memorised knowledge is
increasingly replaced by the ability to find and filter accurate and
relevant data for the task at hand.
Another aspect is the reluctance (sometimes accompanied by a
certain incapacity) to engage in a mainly one-sided and theoretical
learning exercise by the new cohorts; nowadays, the new
generation of students expect that learning should be engaging
and practical with real life-type tasks and rather take the form of
discussions and negotiations with the teacher, in a collaborative
exercise [1]. They expect learning material to be free, available
anyplace, anytime and to reflect the state-of-the-art in the domain
so they are ‘job-ready’ upon graduating. Unfortunately however,
new technology is also highly distracting, presents the danger of
over-reliance on infrastructure (e.g. mobile networks, ‘the Cloud’)
and may create a perception that hard work and commitment are
no longer required to gather the required knowledge; this mindset
often results in superficial learning, disappointment and high
attrition rates [2, 3]. While this is particularly true of the first year
of tertiary education, when students experience a significant
change in learning and teaching style from high school to
university, student retention is affected throughout all programs.
In order to suitably prepare the future generations for the
challenges that lay ahead, the teaching endeavour must match the
fundamental changes experienced by the learning environment
and the student cohorts, without compromising the quality and
quantity of knowledge provided to the students [4]. Flexible
teaching (distance/online, intensive mode, afternoon classes etc.),
online resources and tools, teachers with industry experience and
especially new teaching models matching the new cohorts profile
requirements can provide a solid platform for a new paradigm in
the higher education domain.
The research described in this paper has investigated the
practicality of a teaching model involving the ‘gamification’
(taking game-based elements and applying them in non-gaming
contexts [5]) of computing courses in tertiary education so as to
improve learning and teaching experience and student
engagement and thus assist in reducing attrition. Gamification of
the educational endeavour is not a new proposal; however, this
study applies the concept in a novel iterative and combined Action
Research (AR), Experiential Action Learning (AL) [6], Plan,
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
ACE-ACSW’15, February 1–2, 2015, Canberra, ACT, Australia.
Copyright 2010 ACM 1-58113-000-0/00/0010 …$15.00.
DOI: 10.475/1234
Copyright. Citation Information: Noran, O. (2016) On Gamification in Action Learning, Proceedings of the
Australasian Computer Science Week Multi-conference (ACSW 2016), Canberra, Australia, Article 15, ACM Digital
Library, pp. 1-9
Implement, Review, Improve (PIRI) and Plan, Do, Study, Act
(PDSA [7]) approach (see Figure 1).
Critical
Reflection
(Study)
Identify
Improvements
(Act)
Plan (Plan):
design /
change
approach
Implement
(Do),
Participate
Action
Iterations
Body of
Knowledge
Enter cycles
Exit the cycles
Figure 1. Combined AR / AL / PIRI / PDSA approach
In line with the ‘action’ character present in some paradigms
belonging to the set of research methods adopted, the author has
been directly involved in the application of the practical research
deliverables; therefore, he has sought to clearly identify his
research stance and biases, including those specific to learning
and teaching [8].
The research has involved two cycles, each involving a critical
literature review, identification of the problems and changes
necessary, making the required changes, teaching using the
modified material and approach, observing the effects and
gathering data and finally reflecting on the results obtained and
proposing further changes. The main focus of each research cycle
spanned over one semester of teaching, although elements of
gamification have been present in previous offerings of the
courses involved, albeit in a less systematic manner. Due to space
limitations, the scope of the current paper is limited to the
description of one cycle.
2. GAMES AND GAMIFICATION
Games, challenges and competitions are a part of the human
character and thus constitute a significant aspect of human
society. Companies have long tried to leverage this fact and use
games towards engaging clients or encouraging employees to
work together more effectively [9]. Game concepts and mechanics
(e.g. [10-12]) have been and still are being used in a variety of
areas, including military, marketing, health, business and
education, with varying degrees of success. The advances of
Information and Communication Technologies have provided a
solid platform for the development of an evolved games industry
and for the encouragement of games research.
As with any emerging area of research and practice, there is still
debate and divergence in regards to the domain ontology- for
example, the true meaning of the term ‘gamification’[13-15].
Generally, the term is perceived as describing the application of
game concepts in various other contexts in order to encourage and
motivate problem solving and better performance using typical
human tendencies towards competition, achievement, status,
altruism, self-expression, etc. [16]. Jenkins et al [17] propose a
distinction between the terms ‘game’ and ‘gaming’: the first
facilitates learning by playing games, while the second involves
the use of game concepts and social, technological, theatrical,
virtual reality etc. knowledge.
The more recent technology advances have enabled the creation
of digital distribution, rights management, multiplayer, and
communications platforms featuring community features (friend
lists and groups, ‘Cloud’ game settings saving and sync with local
devices, in-game voice and chat functionality and so on).
Importantly, some such platforms (see e.g. Steam for Schools
[18]) may feature ‘altered’ functionality that can be used in other
domains (e.g. educational); likewise, ‘modified’ games can be
used for teaching science and critical thinking, role playing etc.
(see e.g. Portal 2 [19]).
Gamification takes specific forms depending on the application
area. For example, in the business or military areas gamification is
represented by so-called ‘serious games’, which are generally
used for training and as such are very specific and work-related
[20, 21]. In marketing, game concepts can take the shape of
rewards, loyalty points and virtual currency [22] that can be used
towards future purchases - thus reinforcing the gaming behaviour.
In education, ‘gamified courses’ attempt to engage students [23]
e.g. by developing games in the process of learning. The concept
of ‘learning games’ is used in academia to “foster habits of mind
and understanding” leading to the acquisition of knowledge [20]
and can be used in formal, informal or self-teaching learning
environments.
Gamification as a new area displays typical ‘teething’ issues; thus,
there is criticism in relation to the misuse of the gamification
concept resulting in a false sense of achievement [5], omitting the
storytelling element, lack of game developer involvement,
erroneous or narrow implementation (‘points-ification’ [24]) and
flawed business research results confusing those unfamiliar with
the gamification concept [25]. Such criticism also emphasizes the
potential danger of gamification detracting from, or hindering the
objectives rather than assisting in their achievement. Therefore,
the rationale for the context and the use of gamification must be
carefully considered [15] and reflected in the components of the
game design ‘schema’, such as e.g. espoused by Salen and
Zimmerman [26] – namely the rules, play and especially culture.
Gamification is easier to implement where the users know the
system or work in the environment in question (see Microsoft’s
gamified environment for testing system usability, or SAP’s
gamified data entry processes and online community network [27,
28]).
The above-mentioned research shows that gamification is clearly
not a trivial or generic ‘one size fits all’ exercise. It has to be
accomplished in a tailored manner, appropriate to the specific
environment (industry, education, military, etc.) and audience
type (e.g. considering age, background, culture and so on).
Gamification has the potential to revolutionise human
competencies and human-system interfaces; however, “the hard
work – strategizing – hasn’t been done yet” [29].
3. GAMIFYING LEARNING AND
TEACHING AS AN INNOVATION
EXERCISE
As gamification of the learning and teaching environment is a
relatively new and original concept, the innovation acceptance
attributes identified by Rogers [30] can be used in order to
determine avenues to increase the approval of this endeavour by
students and higher management, resulting in broader
championing and wider application of the concept.
Thus, Rogers’ first attribute is Relative Advantage, i.e. how a
gamified approach is ‘better’ than previous approaches. In
education, gamification may translate in an improved student
learning experience and motivation reflected in better retention
and student experience reports. However, there are many factors
influencing retention rates, not all of which are related to the use
of a specific learning and teaching approach. In addition, there is
significant debate around the accuracy and indeed, the
appropriateness of the assessment tools used to gather such
information [31-33]. While a trend of increased engagement of
students resulting in the improvement of student experience and
retention may develop, generally one should not expect immediate
results but rather progressive improvement.
The second attribute is Compatibility, which here appears to apply
to what needs to be taught, values and prior experience: i.e.
gamification concepts have to be assimilated into the teaching
material and students’ comprehension. In the author’s view, this
aspect requires deep knowledge of the course material and
gamification on the part of the teacher so that appropriate
gamification concepts are matched to suitable course sections, at
the appropriate time. Compatibility may also refer to the shared
technology used to provide the gaming experience - in this case,
competing and often incompatible technologies.
The third attribute is Complexity. ‘Gamifying’ a course should not
increase its complexity, but rather lower it by providing a learning
curve and smaller, easier to achieve sub-goals that lead to feelings
of achievement and investment resulting in students staying
enrolled in the course and successfully completing it.
Trialability or Replayability (fourth attribute) can be used to
reduce the uncertainty surrounding the acceptance of gamification
concepts and also to provide self-training opportunities. Thus,
positive experiences in trials using gamified learning objects are
highly likely to increase the acceptance of this strategy.
Observability, the last attribute listed by Rogers, appears to relate
in this context to the results of prior gamification attempts, where
previous positive experiences would encourage stakeholders to
embrace the concepts proposed. Feedback gathered from past
‘gamified’ offerings can be used towards this aspect.
The acceptance of gamification may also be assisted by the
previous exposure of the target audience to gamification concepts.
For example, widely used gamification-based marketing strategies
may result in students’ previous exposure to specific artefacts
such as reward points, achievement badges, virtual currency etc.
Technology can facilitate the introduction of gaming concepts in
education; however, it demands appropriate teacher training in
order to maximize the usefulness of the devices [34, 35] and
poses an additional acceptance challenge for educators and
educational institutions.
4. IDENTIFYING POSSIBLE
IMPROVEMENT ACTIONS BASED ON
GAMIFICATION CONCEPTS
This section of the paper attempts to build a pool of gamification-
based potential improvements that can be applied to higher
education learning and teaching. The action part of this research
has applied a selection of improvements from the above-
mentioned collection, matching the specifics of the target courses.
As earlier stated in the paper, literature shows that an important
cause of poor student retention rates is the lack of engagement.
This can be caused by several factors, some of which can be
addressed by gamification. For example, students that like their
program, but whose motivation is low or it has withered over time
could be a target for gamification-based student engagement
interventions. On the other hand, students who have joined a
program they were not attracted to because of poor advice (e.g. by
parents or advisors), low job prospects, lack of career alternatives
etc. are not likely to respond to such changes in course strategy.
The following subsections identify avenues of innovative
intervention leading to potential improvements in the learning and
teaching effort, based on the analysis of gaming concepts
identified during the critical literature review. Figure 2
summarizes the gaming concepts identified while at the same time
highlighting potential dependencies, which may be useful in their
deployment within learning and teaching practice.
Replay /
Infinite play
Fulfilment
Sub-goals
Engagement
Investment
Progression
/ Skill
Storytelling
Actions,
Events
Game State
Rewards
Figure 2. Relations between various gaming concepts
4.1 Good Storytelling: Rules, Actions, Events
Storytelling featuring well-thought sequences of events requiring
various actions [10] is an essential feature of every successful
game. Similarly, an engaging course should be able to clearly
articulate through its published profile (and by its teaching team)
what the course is about, the knowledge it provides, its place in a
program or degree and the actions and events and ‘achievement’
(knowledge) level required in order to successfully complete it.
The profile is the public ‘image’ of the course presented to the
potential students and the basis for their decision to select it for a
program of study. Importantly, it is also the ‘contract’ between the
student and the institution stipulating the obligations (‘game
rules’) that the two parties must abide by. Notably, good
storytelling should extend to course content and its assessment
items; students should clearly understand the way a particular
lecture, exercise or assessment item contributes to the overall
level of and type of knowledge required in their future career.
Students also must develop an understanding of what constitutes
‘quality’, and how to evaluate it [36]. Unfortunately, based on the
review of a significant number of course profiles, often this is not
the case. In the case studies embedded in the research cycles
storytelling has been used in order to more accurately reflect
course contents, learning objectives and achievement level
requirements.
4.2 Student Engagement: Competition or
Cooperation?
Engagement through competition and cooperation (with variations
such as time, space and team constraints) is a concept well
established in games [5, 15, 23, 37, 38]. However, competition
(involving individuals or groups of students) must be used with
caution by teachers as it may bring unwanted shortcomings that
outweigh the benefits; thus, it can increase attention on ‘what it
takes to win’ at the expense of learning as a goal in its own right.
In addition, competition in team situations may determine group
members to see each other less as peers in a learning community
and more as the means to win [39]. Dialogue and reflection may
be reduced or disappear altogether. There is also debate over
‘healthy vs. unhealthy’ competition [40], whereby in a healthy
competition the valuable goals are the learning rather than the
competitive ones and the competition is of short duration with no
long term effects, with all participants understanding these
features and having a fair chance of winning. The results of
questionnaires administered in the research cycles have revealed a
positive attitude of students towards competition.
Cooperation may be a better alternative, encouraging dialogue and
reflection and divergent ideas leading to novel ways to
accomplish tasks. However, this approach also presents pitfalls;
for example, some team members may rely on the others to do
most of the work, leading to conflict, need for mediation and even
expulsion of members, with disrupting effects on the team.
In conclusion, both approaches are useful in the context of careful
management. The onus is on the teacher to identify the best option
(or a mix thereof) for each specific practical scenario.
4.3 Sub-Goals: Investment and Engagement
through Fulfilment and Progression
Goals and sub-goals are paramount concepts in a game. They
have the role of keeping a player ‘invested’ and thus engaged in
playing a game by providing a sense of gradual progression and
fulfilment. In learning and teaching, such perceptions are essential
to student involvement, engagement and ultimately, retention. A
learning curve is also present, typically steeper in technical areas.
Therefore, similar to the game concepts that include low and high
stake goals, students should be provided with formative and
summative assessment items featuring progressively increasing
degree of difficulty. Progressive achievements will provide “the
connection of a certain act with a certain situation and resultant
pleasure” as espoused by Thorndike [41] based on his view of
learning as the process of forming associations and bonds.
Smaller, on-going assessment items are also important in order to
provide a degree of self-assessment – fact acknowledged by
students in the questionnaires used within the case studies with
remarks like “the quizzes [on-going assessment] in this course
provides a barometer of my knowledge level”. While self-
assessment promotes beneficial student learning reflection [42,
43], it must be noted that summative assessment is also required
because, similar to game situations, many students do not pay
suitable attention to items that do not carry rewards (marks). This
is particularly true of first-year and low performing students who
are still transitioning from the high-school study model and / or
may not have yet formed appropriate study habits. In-class small
assessment items spread throughout the teaching period encourage
students to focus, participate, spend time on task [44] and thus
promote a consistent learning approach. Such items also provide
appropriate and timely feedback on students’ knowledge level
[45, 46].
Progression reflecting the skill of players is often accomplished in
gaming environments using achievement badges, virtual goods
that players can trade (or use toward purchasing other games),
detailed statistics etc., with players typically being able to control
the accessibility of such information (player profile) themselves.
In the learning and teaching environment, such information is
provided (e.g. through ‘running totals’, percentages etc.), albeit
subject to significantly tighter privacy rules. Thus, marks and
statistics for a student are typically visible just to the teacher,
admin personnel and student in question. Only overall statistics
can be made public.
4.4 Replayability, Risk, Rewards, Game State
Commonly, non-achievement of a sub-goal in a modern game
does not translate in automatic failure; rather, it is limited to some
temporary hindrance, activates an alternative scenario or allows
another attempt; any of these options may also incur a penalty. By
analogy, should a student should be allowed to repeatedly attempt
assessment items with low or no penalty, especially when the
assessment closely reflects the skills that students should have
upon graduating? Many games typically provide (or game
modifications are available for) penalty-free ‘training’ modes so
that the player can hone their skills and achieve the desired
proficiency level; however, these training modes do not apply to
the entire game. In learning and teaching, this approach can be
emulated by ‘risk-free’ formative partial assessment items,
followed by summative over-arching assessments.
In this context, an increasing trend in modern games is the
replayability concept, whereby the player encounters different
environments and actors in each game instance [47]; this concept
can also be (and increasingly is) utilised in learning and teaching
by the use of large question and scenario banks, randomly
selected for each assessment item sitting. This approach can also
be applied for summative assessments in order to prevent
academic integrity concerns.
Modern games allow saving the state of a game at critical
milestones (‘checkpoints’) so that the player does not have to start
from the beginning every time. This works towards maintaining
the investment level high; in learning and teaching this can be
achieved by recording, saving and restoring student progress e.g.
in online tests used for formative assessment.
4.5 Reflection v. Observation: Depth v Agility
In many multiplayer online games, latency is a crucial factor;
thus, awareness of the situation enabling prompt reaction (agility)
may often be the difference between a win or lose situation.
However, even in such situations, the lack of extensive knowledge
of the game universe of discourse (maps, strategy but also
unintended game behaviour such as glitches) gained through
many hours of play can make a significant difference. Thus, while
low latency gives agility, thorough game knowledge gives depth
of understanding; fortunately, the two aspects are not mutually
exclusive but rather complementary.
These concepts are highly relevant to learning and teaching. For
example, using AR/AL cycles is a thorough approach involving a
significant amount of time (typically several semesters or years).
As a result, curriculum and teaching improvements eventuating
from reflections on the results gathered during a specific iteration
will only benefit subsequent offerings. On the other hand,
observing class mood and participation during live lectures and
workshops and while administering various assessments is a
valuable rapid feedback tool at teacher’s disposal. For example,
‘secondary emotions’ analysed according to Russel’s [48]
‘circumplex model of affect’ (see Figure 3) can provide feedback
that the teacher may promptly use to tune the teaching approach in
real time e.g. by applying more or less emphasis on various
gamification aspects. Such observations can successfully
complement the AL approach: AL provides depth and method
while observation provides agility.
Valence
Arousal
Figure 3. Secondary Emotions
(based on [48]; arrows show desired trend)
4.6 Evaluating the Results in Gaming vs.
Teaching & Learning
This section aims to identify problems and potential solutions in
achieving meaningful interpretations of data gathered in games,
extrapolate the solutions and apply them to learning and teaching.
Achievements and status are important reasons for playing games,
be it single or multiplayer, off- or online. In recognition of this
fact, games have always made some attempt to record player
performance and typically use it to entice players to increase their
investment and engagement. Thus, the competitive aspect of
human nature is leveraged by providing rewards such as badges
and / or statistics, skill levels etc. The sophistication of
performance evaluation has steadily increased so as to match the
growing games' complexity and IT infrastructure capability.
Multiplayer technology and online gaming enabled by Cloud-
based deployment and management platforms [49] are now
affordable and accessible to any household with an average speed
connection. Game state (configuration, levels played) and player
information shown in statistics such as game playing
achievements, level of assistance to the team, level of efficiency,
preferred games and number of hours played in each, advanced
skills (such as being a developer of game maps and artwork) etc.
is now accessible irrespective of location and specific hardware.
Players can retrieve their game state and player status and
continue playing a game from where they left off.
Evaluation is very important in games, even for those who do not
care much for achievements levels. This is because many games
and/or platforms feature match-making, where only players with
comparable experience are allowed to play together. In addition,
the status of a player determines the availability of bonuses
(virtual cash, weapons, skills, etc.) and often, the rank and level of
team acceptance within multiplayer games.
One must note however that the evaluation mechanism used in
games is prone to drawbacks, especially when attempting to
interpret the results. For example, a player may misuse the
evaluation mechanism by leaving a game running for long periods
just to achieve ‘hours of play’ and a higher level; colluding with
others to get specific achievements (e.g. win games, beat specific
records etc.); leaving a game before losing so the loss is not
recorded; ‘stacking’, i.e. building an overly skilled team
(generally considered unethical) e.g. by using messaging and
other ‘back door’ means. There are also objective factors that can
distort statistics. Thus, several team members may leave in-game
and be replaced with weaker or stronger players; this will reflect
on the achievement of the entire team. Finally, the specific area
(e.g. map) of a game and players’ knowledge of that area may also
play a significant role in skewing the scores.
To summarise, it is quite hard to draw a conclusion in regards to
the skills of a player just by looking at the statistics, as many
subjective and objective factors may distort the information. What
can be done to improve the situation? In order to see if a player
really achieves good results one could a) get involved in games
with the player (primary information) or b) seek the opinion of
other players (secondary information), thus using an incremental
evidence gathering method.
Within the educational realm, evaluation is an important part of
the PIRI (Plan, Implement, Review, Improve) cycle, the basis for
many Frameworks for Quality Assurance and principles for
excellence in learning and teaching (e.g. [50]). Unfortunately, the
main vehicle currently used in many institutions to gather the
necessary information, namely Student Experience of Course
(SEC) and Teaching (SET), are a rather limited and often
inaccurate quantitative reflection of the reality.
Thus, SEC and SET are typically difficult to interpret [51] and
present only one side of the story. Nulty [31] rightfully argues that
typically, online surveys (widely used in tertiary education
providers) achieve low response rates (even with various
incentives and instructor encouragement); therefore, “relying
heavily on student evaluations of courses and teaching is
inadequate at best and misleading at worst” (ibid.). In addition,
Yorke [33] points out that “[…] when students come together
with differing prior experiences […], a variation is introduced into
the responses that can be extremely difficult to identify”. These
are important factors to consider especially for courses that must
be taken (‘core courses’) by undergraduate and postgraduate
students (many of which are international and from a non-English
speaking background). Similar to the distortions identified in the
games realm, variations due to the actual teacher (e.g. whether
sessional of staff, mature or early career) and the maturity of the
course offering material itself further complicate proper SEC and
SET results interpretation. The open-ended sections of these
reports may offer some insight as to the cause of the results
distribution; however they are typically only filled in by a
minority of respondents. Smith [52] proposes an SEC / SET
evaluation model whose second stage supplies “an interpretive
guidance system that helps lecturers (but also the executive)
understand and interpret” the results; however, currently this
concept does not appear to be properly implemented.
One must also recognise that, similar to gaming evaluation
mechanisms, SEC and SET are also prone to wilful manipulation,
as scores typically important consequences (e.g. career
progression) for the teachers involved. Therefore, supplementary
or ‘easier’ assessment items, more lenient marking, etc. may
potentially be used by teachers specifically to obtain better scores.
Potential solutions resemble those investigated in the gaming
realm. Thus, frequent informal feedback (verbal and written, in
confidence) sought from the students before, during and after
classes can help build ‘incremental’ evaluation (such as Nulty’s
[32] ‘A-Z’ model), and increase course ‘agility’ by making on the
fly adjustments to the teaching style. However, such feedback
must be taken with a ‘grain of salt’ in the absence of anonymity
and the possibility of students ‘observing back’ and potentially
attempting to please the teacher. The context of such feedback
must also be carefully prepared so as to make it a non-threatening,
engaging, and fun experience integrated in the learning and
teaching conversation [53].
5. CASE STUDY
Systems Analysis and Design (SAD) is a large course offered at
three campuses of a major tertiary education provider. SAD is a
second year core course that aims to impart knowledge of the
various activities undertaken within each of the system
development life cycles. For this reason, the course is a core (i.e.
compulsory) in several IT programs and is also part of a set of
‘primers’ offered to non-IT background students undertaking IT
and IT-related postgraduate programs. The course enrolment is
typically large, with a mostly undergraduate cohort. The course
has been offered for many years; the material is mature, the
teacher highly knowledgeable and having industry experience.
5.1 Identified Areas of Improvement
Reflection on the teaching practice and evaluation reports for this
course has yielded several areas of improvement (the relevant
gaming concepts are presented in italics):
lack of early assessment and feedback for students. Students
were not required to produce any assessed item until mid-
semester and thus did not get feedback and possible early
warning in relation to their performance (sub-goals, risk);
lack of continuous assessment so as to provide an on-going
indication of their performance and a feeling of achievement
and involvement (sub-goals, rewards, investment, fulfilment);
lack of self-study, non–assessed, formative resources paced in
parallel with the material (replayability, rewards, risk);
insufficient practice in hands-on work and incentives to do so
(sub-goals, rewards);
insufficient explanation of the learning objectives and their
relevance to the students’ future career (storytelling, game
goals, culture);
insufficient explanation of the assessment items and their
relation to the learning objectives and future career (game
rules, goals);
insufficient explanation of the learning strategies and their
relation to the learning objectives and future career (game
rules, goals);
insufficient interactivity (player, multiplayer, culture).
5.2 Proposed Changes
In line with the gamification concepts identified as suitable as a
result of the critical literature review, a range of changes were
proposed as detailed below.
Storytelling, Game rules, Goals: At the outset of the course,
explain why the course is required – for students’ careers and also
how it helps them to understand the more advanced courses. Go
through the course outline together with the students and explain
key areas such as Course Description, Aims, Learning Objectives,
Assessment Plan and timing and how each assessment item will
contribute to the Learning Objectives of the course. This helps
student understanding and motivation (‘why do I have to do this
course?’) and creates a perception of required engagement. The
Learning Strategies are also to be explained and justified to the
students in relation to the required graduate outcomes.
Sub-goals, Risk: A low-points assessment item is introduced in
the third week of study serving as a ‘wake-up call’ and also to
reinforce the requirement of continuous study. This is followed by
a series of assessed, low-points quizzes, administered every 2-3
weeks to reinforce the need to keep studying and also to provide a
feeling of achievement and encourage engagement. The risk
associated with failing any individual quiz is low; however,
repeated failure may lead to significant loss of points.
Replayability, Rewards, Risk: Redesign the former sample
assessment items so as to become self-assessment items. Thus, the
solutions for these items are no longer to be released from the
start; students need to solve them first. The solutions are also to be
redesigned so as to contain explanations of ’why is it so and not
otherwise’ (and thus promote the previously mentioned ‘dubito
ergo cogito…’ approach), rather than a set of laconic statements
describing the correct answers.
Sub-goals, Rewards: Adequate hands-on practice (training) to be
introduced via additional computer labs. They are to be spread
over the period of study and cover a larger amount of topics, to
the benefit of students’ practical experience. Students must attend
these labs in order to progress through the course and send their
work to the teacher for perusal.
The above proposed changes are to be complemented by the
introduction of ‘refreshers’ in the form of brief questions at the
beginning of each class, relating to the material previously taught.
‘Jogging’ the students’ memory is likely to be more productive as
they have to think of the answer, provide and defend it rather than
passively looking on as they are shown the solutions.
Game Frequent Asked Questions (FAQ): The answers to the most
asked questions are also to be made available online, alleviating
the requirement for the teacher to answer similar questions over
and over and instead focus on material delivery quality and
student engagement.
5.3 Teaching the Modified Structure
The learning activities and assessment sections were revised in
line with the proposed changes, using a conceptual development
view of teaching [54] and keeping the emphasis on student
development and making people understand [55].
As planned, storytelling was involved in order to explain students
the purpose (goals) and expectations of the course (game rules).
The learning and teaching strategy involved a balanced mix of the
game collaboration and competition approaches in class
discussions and debates, as well as in the negotiation of
assessment marking criteria in order to alleviate large class
alienation and loss of motivation syndromes [56]. In line with the
game state perspective, in each lecture and workshop students
were shown where they were in respect to the entire course. A
diagram showing the areas of the main system development life
cycles covered by the course was used for this purpose. This was
preferred to text as due to a variety of factors such as language,
culture etc., student perception and interpretation may have not
reflected the intended meaning of teacher’s utterances. The
lectures followed the progression of the SAD development life
cycle phases, presenting and comparing the two main approaches
(‘traditional’ and object-oriented) used in the industry. In line with
the AL approach adopted, the students were encouraged to reflect
and criticise the two approaches and ask themselves e.g. ‘what is
different about them?’, ‘are they really that different?’ and ‘is any
of them best, or just more suitable depending on the particular
system being analysed and designed, and in view of the
competencies of the people involved in the development project?’
The teacher’s experience, personal characteristics and integrity
(player profile) as defined by Palmer [57] have been used to
inspire students to learn as their beliefs essentially influence their
development [58]. Storytelling was also used in course delivery,
based on teacher’s vicarious experience of relevance [59], aiming
to prepare students for real situations that sometimes involve
‘wicked problems’ requiring likewise competencies [60].
The workshops also used storytelling by following a fictitious
company through its quest to (re)-develop some of its essential
systems. All the concepts learned and discussed in the lectures
were gradually applied to this scenario using a constructivist
approach.
In line with game sub-goals, fulfilment and progression aspects,
students were constantly given analogies they could relate to and
were encouraged to apply their previous experience(s) and newly
gained knowledge to try to solve increasingly difficult problems
and critically assess the solutions produced [61].
Computer labs provided functional [62] hands-on Computer
Aided Software Engineering tool experience for students and
(according with game sub-goals, rewards and replayability
concepts) featured assessment items allowing multiple solutions
which promoted creativity in model development.
Large classes typically feature students with a variety of learning
styles [63], moving up towards the ‘qualitative understanding’
[64] featuring Bloom’s content analysis and evaluation [65] at
different paces. This situation was tackled by cycling through
several teaching approaches featuring high and low complexity,
practical examples and theory, moving ahead but periodically
returning to test and deepen students’ understanding of the
material: in essence, using appropriate storytelling to cater for a
large and varied audience.
Perry [66] found that students’ learning and knowledge
perspective evolves from an absolutistic (objective / black and
white) form to a ‘relativistic’ one where personal values and
interpretations of the evidence prevail. In line with game
progression and replayability notions, this perspective was taken
into account by steadily increasing the complexity of the exercises
(‘low-risk’ sub-goals) featuring multiple solutions and introducing
debates as to why solutions are not unique and why there is no
such thing as a ‘best’ solution for all scenarios.
The above-mentioned low-risk assessment items were spread
along the teaching period in line with the gaming sub-goals, risk
and rewards and progression concepts. Sample assessment items
were also created; their solutions were only disclosed after
students attempted to solve them (according to the game training
concept).
5.3.1 Gathering Feedback
Keeping in mind the caveats of evaluation examined in the
gaming analogies descried in Section 4.6, feedback has been
gathered at the end of the semester but also in the middle of the
teaching period using a variety of methods (survey, focus group,
informal) so as to enable triangulation and at the same time to
catch any early issues (see section 4.5) and correct them. SEC and
SET reports have also been used, with specific focus on students’
answers to open-ended questions.
5.4 Reflection on the Feedback
The feedback has reinforced issues previously identified in the
literature review such as perceived advantage brought by
gamification and need for compatibility with learning objectives.
Many students, although previously exposed to games and gaming
concepts, were still between the persuasion and decision stage of
acceptance of new paradigms and technologies [30]. The
inclusion of free text fields in the questionnaire used has proven
beneficial; many students wrote beyond the allocated space in an
attempt to strongly make their point, thus confirming their
engagement with the gamification concept. The majority of
respondents believed that gamification can provide a more
competitive edge to their future career and strongly endorsed the
need for some competition elements in learning and teaching, as
well as the need for taking controlled risks. The respondents also
appeared to support the integration of gamification elements
throughout the course; in addition, most declared themselves
happy to participate in pilot studies. There was some
inconsistency and polarisation among the opinions regarding
learning objects and concepts that should not be part of a ‘well-
gamified’ course; the likely underlying reason for this could be
the novelty of the gamification style employed and the lack of
proper reflection on the concepts proposed.
Practical examples, reflecting the author’s industry experience and
the state-of-the-art in SAD, appear to have kept students alert and
interested; thus, student feedback has confirmed the importance of
teacher’s practical experience [59] in inciting students interest and
promoting deep learning [67]. Another aspect positively perceived
by the students was the teacher’s willingness to stop at any time
and explain more if students appeared puzzled. This approach
requires the teacher to be the equivalent of ‘seasoned gamer’, so
as to master the course material (game rules, maps etc.) well, be
able to stop and resume at will and provide novel perspectives on
the material. Should this not be the case (e.g. in the case of core
courses taught by inexperienced sessional staff), the likely result
would be a worsened learning and teaching experience. This issue
can be addressed by proper course ‘match-making’: only sessional
staff (players) with certain level of knowledge (gaming
experience) may be allowed to teach courses (play games) of
certain importance, size and complexity.
6. CONCLUSIONS AND FURTHER WORK
This paper has presented the phases of a research cycle spanning a
teaching semester, comprising the identification of potential
improvements based on the learning and teaching and
gamification bodies of knowledge, planning the required changes
to a target course, implementation of the changes, teaching the
gamified course, feedback gathering and reflection on the results.
It appears that gamification does benefit students by increasing
student engagement and motivation and helping them achieve the
desired learning objectives. However, proper gamification
requires researching, interpreting, customising and using relevant
gamification principles in learning and teaching practice.
Accordingly, in this research gamification has been employed in
the context of a multi-pronged and iterative Action Research,
Experiential Action Learning, Plan, Implement, Review, Improve
and Plan, Do, Study, Act framework.
An important aspect also true for gamification is that the quality
of teaching can be enhanced by aligning objectives, teaching
styles and milestone deliverables [68]; however, the research,
implementation, action (teaching) and reflection leading to this
alignment must be ongoing in order to ensure agility in the face of
constant change in the industrial and academic environments.
The research approach must be refined and tested in subsequent
cycles in order to better assist the learning and teaching effort in
its endeavour to prepare students for the global dynamic and
competitive environment faced by companies and the workforce.
7. REFERENCES
[1] Bovill, C., K. Morss, and C. Bulley, Should students
participate in curriculum design? Discussion arising from a
first year curriculum design project and a literature review.
Pedagogical Research in Maximising Education, 2009. 3(2).
[2] Angelino, L.M., F.K. Williams, and D. Natvig, Strategies to
Engage Online Students and Reduce Attrition Rates. The
Journal of Educators Online, 2007. 4(2): p. 1-14.
[3] Torenbeek, M., E. Jansen, and A. Hofman, The effect of the
fit between secondary and university education on first-year
student achievement. Studies in Higher Education, 2010.
35(6): p. 659-675.
[4] AQF Council. Australian Qualifications Framework. 2013;
Available from:
http://www.aqf.edu.au/Portals/0/Documents/2013%20docs/A
QF%202nd%20Edition%20January%202013.pdf.
[5] Deterding, S., et al., From game design elements to
gamefulness: defining “gamification”, in MindTrek. 2011:
Tampere, Finland. p. 9-15.
[6] Kolb, D., Experiential Learning. 1984, Englewood Cliffs,
N.J.: Prentice-Hall.
[7] Deming, W.E., Out of the Crisis. 1986: MIT Center for
Advanced Engineering Study.
[8] Noran, O., 'Gamified' Action Learning Environments: Case
Studies and Reflections. Master Thesis 2013.
[9] DeLoitte. Gamification goes to work: Moving beyond points,
badges, and leaderboards 2013 [cited 2013 Jun]; Available
from: https://documents.deloitte.com/techtrends2013.
[10] Björk, S. and J. Holopainen, Patterns in Game Design. 2004:
Charles River Media.
[11] Adams, E., Fundamentals of Game Design 2nd ed. 2009,
Thousand Oaks, CA: New Riders Publishing.
[12] Schell, J., The Art of Game Design: A book of lenses. 2008:
CRC Press.
[13] Marczewski, A., Foreword, in Gamification: A Simple
Introduction & a Bit More. 2012, e-Book. p. 46.
[14] Erenli, K. The impact of gamification: a recommendation of
scenarios for education. in Interactive Collaborative
Learning (ICL). 2012. Villach, Austria.
[15] Kapp, K., The gamification of learning and instruction:
game-based methods and strategies for training and
education. 2012, San Francisco: John Wiley & Sons.
[16] Van Grove, J. Gamification: How Competition Is
Reinventing Business, Marketing & Everyday Life. 2011.
[17] Jenkins, H., B. Camper, and A. Chisholm, From Serious
Games to Serious Gaming, in Serious Games: Mechanisms
and Effects, U. Ritterfeld, M. Cody, and P. Vorderer, Editors.
2009, Routledge: London. p. 448-468.
[18] Wilde, T. Valve rolls out "Steam for Schools" to teach math
and physics lessons with Portal 2 PC Gamer, 2012.
[19] Valve Software. Teach with Portals. 2011 [cited 2015
August]; Available from: http://www.teachwithportals.com/.
[20] Klopfer, E., S. Osterweil, and Salen, K. Moving learning
games forward: The education arcade. 2009; Available from:
http://education.mit.edu/papers/MovingLearningGamesForw
ard_EdArcade.pdf.
[21] Ritterfeld, U., M. Cody, and P. Vorderer, Serious Games:
Mechanisms and Effects. 2009, London: Routledge.
[22] Hamari, J. and V. Eranti, Framework for Designing and
Evaluating Game Achievements, in Think Design Play: 5th
International Conference of the Digital Research Association
(DIGRA)2011: Utrecht, Netherlands.
[23] Muntean, C. Raising engagement in e-learning through
gamification. in The 6th International Conference on Virtual
Learning (ICVL 2011). 2011. Cluj-Napoca, Romania.
[24] Robertson, M. Can’t play, won’t play. 2010; Available from:
http://www.hideandseek.net/2010/10/06/cant-play-wont-
play//.
[25] Slavin, K. In a World Filled With Sloppy Thinking. 2011
[cited 2013 June]; Available from:
http://slavin.tumblr.com/post/6353625142/in-a-world-filled-
with-sloppy-thinking-this.
[26] Salen, K. and E. Zimmerman, Rules of play - game design
fundamentals, ed. T.M. PRess. 2004, Cambridge.
[27] Yolton, M. Gamification: much more than just fun and
games. 2013.
[28] Schacht, M. and S. Schacht, Start the game: Increasing user
experience of enterprise systems following a gamification
mechanism, in Software for People: Fundamentals, Trends
and Best Practices, Maedche et. al, Editor. 2012, Springer
Verlag: Berlin.
[29] Zichermann, G., My Take on Gamification, in Gamification
goes to work: Moving beyond points, badges, and
leaderboards 2013, DeLoitte.
[30] Rogers, E.M., Diffusion of innovations. 2003, New York:
Free Press.
[31] Nulty, D., The adequacy of response rates to online and
paper surveys: what can be done? Assessment & Evaluation
in Higher Education, 2008: p. 1-13.
[32] Nulty, D., Workshop 6 - Innovation and Evaluation, in
7014GIH: Curriculum Design, Innovation, Assessment and
Evaluation – Innovation and Evaluation. 2011, Griffith
University.
[33] Yorke, M., ‘Student experience’ surveys: some
methodological considerations and an empirical
investigation. Assessment & Evaluation in Higher Education,
2008. 34(6): p. 721-730.
[34] Glover, D. and D. Miller, Running with technology: the
pedagogic impact of the large-scale introduction of
interactive whiteboards in one secondary school. Journal of
Information Technol for Teach Educ, 2001. 10: p. 257-278.
[35] Smith, H., et al., Interactive whiteboards: boon or
bandwagon? A critical review of the literature. J Comput
Assist Learn, 2005. 21: p. 91-101.
[36] Sadler, D.R., Indeterminacy in the use of preset criteria for
assessment and grading. Assessment & Evaluation in Higher
Education, 2009. 34(2): p. 159-179.
[37] Malone, T. and M. Lepper, Making learning fun: A taxonomy
of intrinsic motivation for learning, in Aptitude learning, and
instruction, R.E. Snow and M.J. Farr, Editors. 1987,
Lawrence Erl-baum Associates: London.
[38] Salen, K. and E. Zimmerman, Rules of play - game design
fundamentals. 2004, Cambridge: The MIT Press.
[39] Johnson, D.W. and R. Johnson, Conflict resolution, peer
mediation and peacemaking, in Handbook of classroom
management., C.M. Evertson and C.S. Weinstein, Editors.
2006, Lawrence Erlbaum Associates: Mahwah, NJ. p. 803-
832.
[40] Shindler, J., Transformative Classroom Management:
Positive Strategies to Engage All Students and Promote a
Psychology of Success. 2009: Jossey-Bass.
[41] Thorndike, E., Animal Intelligence: An Experimental Study
of the Associative Processes in Animals, in Psychological
Review, Monograph Supplements. 1898, MacMillan: New
York.
[42] Boud, D., HERDSA Green Guide No 5. Implementing student
self-assessment. 2nd ed. Vol. The Higher Education Research
and Development Society of Australasia (HERDSA). 1991,
Campbelltown.
[43] Black, P. and D. William, Assessment and classroom
learning. Assessment in Education, 1998. 5(1): p. 7-74.
[44] Chickering, A.W. and Z.F. Gamson, Seven principles for
good practice in undergraduate education. American
Association for Higher Education Bulletin, 1987: p. 3-7.
[45] Newell, A. and H.A. Simon, Human Problem Solving, ed.
Engelwood-Cliffs. 1972, Engelwood Cliffs, N.J.: Prentice
Hall.
[46] Newstead, S.E., The Use of Examinations in the Assessment
of Psychology Students. Phsychology Teaching Review,
1992. 1(1): p. 22-23.
[47] Booth, M. Replayable Cooperative Game Design: Left 4
Dead. 2009.
[48] Russell, J.A., A circumplex model of affect. Journal of
Personality and Social Psychology, 1980. 39: p. 1161-1178.
[49] Breckon, N. Valve Announces Steam Cloud; Online Network
to Store Saved Games Indefinitely. Shacknews, 2008.
[50] Griffith University. Principles to Promote Excellence in
Learning and Teaching Practices at Griffith University. 2010
[cited 2012 August]; Available from:
http://www.griffith.edu.au/__data/assets/pdf_file/0006/12020
1/PrinciplesLandT.pdf.
[51] Abrami, P.C., S. d'Apollonia, and P.A. Cohen, Validity of
Student Ratings of Instruction: What We Know and What We
Do Not. Journal of Educational Phsychology, 1990. 82(2): p.
219-231.
[52] Smith, C., Building effectiveness in teaching through
targeted evaluation and response: connecting evaluation to
teaching improvement in higher education. Assessment &
Evaluation in Higher Education, 2008. 33(5): p. 517-533.
[53] Pask, G., Conversation, cognition and learning. 1975, New
York: Elsevier.
[54] Ramsden, P., Ways of understanding teaching (Chapter 2), in
Learning and Teaching in Higher Education, P. Ramsden,
Editor. 2003, Routledge Falmer: London. p. 19-38.
[55] Edgerton, R., P. Hutchings, and K. Quinlan, The Teaching
Portfolio: capturing the scholarship in teaching. American
Association for Higher Edutation. 1991: Stylus Publishing.
[56] Hejmadi, M.V., Improving the Effectiveness and Efficiency
of Teaching Large Classes: Development and Evaluation of a
Novel e-Resource in Cancer Biology. Bioscience Education
Journal, 2007. 9.
[57] Palmer, P.J., The heart of a teacher, in The courage to teach:
Exploring the inner landscape of a teacher's life. 1998,
Jossey Bass: San Francisco. p. 9-33.
[58] Kember, A Reconceptualisation of the Research into
University Academics' Conceptions of Teaching. Learning
and Instruction, 1997. 7(3): p. 255-275.
[59] Hodgson, V., Lectures and the experience of relevance, in
The experience of learning: implications for teaching and
studying in higher education, F. Marton, D. Hounsell, and
N.J. Entwistle, Editors. 1997, Scottish Academic Press:
Edinburgh. p. 159-171.
[60] Knight, P. The assessment of 'wicked' competences. 2006;
Available from: http://www.open.ac.uk/cetl-
workspace/cetlcontent/documents/460d21bd645f8.pdf.
[61] Paul, R.W. and L. Elder, Critical Thinking: Basic Theory and
Instructional Structures Handbook. 2nd ed. 2000: Foundation
for Critical Thinking.
[62] Leinhardt, G., K. McCarthy Young, and J. Merriman,
Integrating professional knowledge: the theory of practice
and the practice of theory. Learning and Instruction: , 1995.
5.
[63] Larkin-Hein, T. and D. Budny, Research on Learning Style:
Applications in the Physics and Engineering Classrooms.
IEEE Transactions on Education, 2001. 44(3): p. 276-281.
[64] Biggs, J. and K. Collis, Evaluating the Quality of Learning:
the SOLO taxonomy. 1992, New York: Academic Press.
[65] Bloom, B.S., Taxonomy of Educational Objectives,
Handbook I: The Cognitive Domain. 1956, New York: David
McKay Co Inc.
[66] Perry, R.P., Perceived Control in College Students:
Implications for Instruction in Higher Education, in Effective
Teaching in Higher Education: Research and Practice, R.P.
Perry and S.J. C., Editors. 1997, Agathon: New York.
[67] Ramsden, P., Approaches to Learning (Chapter 4), in
Learning and Teaching in Higher Education, P. Ramsden,
Editor. 2003, Routledge Falmer: London. p. 39-61.
[68] Biggs, J., Teaching For Quality Learning at University:
What the Student Does (Society for Research into Higher
Education). 2003: Open University Press.