Gamification in mobile and workplace integrated microlearning

Abstract

Business analysts and human resource executives consider MicroLearning and Gamification as two of the currently hottest topics in corporate learning and workplace learning. MicroLearning is seen as a solution for the limitations in dedicated learning time of employees, already stressed in their daily work routines. Gamification fosters users engagement and creates higher intrinsic motivation to learn. In this case study, we report and analyze the learning behavior of 175 employees using gamified MicroLearning over the course of seven months. The study setup allows observing behavior with and without the extrinsic motivation of an employee competition. While we observe a large increase in activity compared to groups studied in previous work, we did not see an increase in the amount of average daily learning sessions. However, we could identify a shift in learning times from working hours towards high attention periods in evenings and weekends. Consequently, we regard gamification as a means to raise user engagement, which yet bears challenges for the deployment in corporate environments to shape learning behavior as intended.

Full text

Gamification in Mobile and Workplace Integrated
MicroLearning
Bernhard Göschlberger
Research Studios Austria FG
Linz, Austria
goeschlberger@researchstudio.at
Peter A. Bruck
KnowledgeFox GmbH
Research Studios Austria FG
Salzburg, Austria
bruck@researchstudio.at
ABSTRACT
Business analysts and human resource executives consider Mi-
croLearning and Gamication as two of the currently hottest topics
in corporate learning and workplace learning. MicroLearning is
seen as a solution for the limitations in dedicated learning time of
employees, already stressed in their daily work routines. Gamica-
tion fosters users engagement and creates higher intrinsic motiva-
tion to learn. In this case study, we report and analyze the learning
behavior of 175 employees using gamied MicroLearning over the
course of seven months. The study setup allows observing behavior
with and without the extrinsic motivation of an employee com-
petition. While we observe a large increase in activity compared
to groups studied in previous work, we did not see an increase in
the amount of average daily learning sessions. However, we could
identify a shift in learning times from working hours towards high
attention periods in evenings and weekends. Consequently, we
regard gamication as a means to raise user engagement, which
yet bears challenges for the deployment in corporate environments
to shape learning behavior as intended.
CCS CONCEPTS
•Applied computing →E-learning
;Collaborative learning;
•In-
formation systems →
Collaborative and social computing systems
and tools;
KEYWORDS
MicroLearning, Gamication, Workplace Learning
ACM Reference Format:
Bernhard Göschlberger and Peter A. Bruck. 2017. Gamication in Mobile and
Workplace Integrated MicroLearning. In iiWAS ’17: The 19th International
Conference on Information Integration and Web-based Applications & Services,
December 4–6, 2017, Salzburg, Austria. ACM, New York, NY, USA, 8 pages.
https://doi.org/10.1145/3151759.3151795
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic permission
and/or a fee. Request permissions from permissions@acm.org.
iiWAS ’17, December 4–6, 2017, Salzburg, Austria
©
2017 Copyright held by the owner/author(s). Publication rights licensed to Associa-
tion for Computing Machinery.
ACM ISBN 978-1-4503-5299-4/17/12.. .$15.00
https://doi.org/10.1145/3151759.3151795
1 INTRODUCTION
As with other forms of learning, successful MicroLearning is very
dependent on intrinsic motivation as a driver for learners to en-
gage in learning activities. Reports on the eects of competitive
game design elements and the commercial success of player-versus-
player quiz games
1
inspired this research project on competitive
gamication for MicroLearning. For this, we used and adopted the
commercially available MicroLearning solution from Knowledge-
Fox GmbH and its gamication extension called KnowledgeMatch
2
.
In this paper, we present a case study with 175 users in a corporate
education scenario in a multi-regional Austrian bakery and bakery
products company and report the evaluation of the deployment
of KnowledgeMatch. The objective in the product design of Knowl-
edgeMatch has been to improve user engagement and motivation
by adding competitive game-play to the existing KnowledgeCard
system used by the KnowledgeFox MicroLearning solution. While
the standard system already incorporates many game design ele-
ments, it does not include one to one or one to many competition
or elements of social gamication. Recent research [
11
] suggests
that social gamication amplies positive eects of game design
elements when considering retention rates, accuracies in answers
and test scores. In our study, we were looking for and expected
even stronger eects on user engagement.
Before we describe the case study in detail in Section 5 we will
provide background on MicroLearning and gamication in Sec-
tion 2, on the used MicroLearning system in Section 3, and on
KnowledgeMatch in Section 4. Finally, we present the results of the
case study in Section 6 and before presenting our conclusions in
Section 7.
2 BACKGROUND
The presented work is the latest in a list of research eorts that look
at improvements in corporate learning. In the following, we intro-
duce and point out how our research builds upon the established
concepts of MicroLearning and gamication.
2.1 MicroLearning
MicroLearning refers to a didactic concept and approach that uses
digital media to deliver small, coherent and self-contained content
for short learning activities.
While such a denition is widely undisputed and can be found
in most publications, dierent interpretations and foci developed
as the popularity of MicroLearning grew over the past decade and a
1Quizduell — Das Buch. riva Verlag, 2014, ISBN 978-3-86883-436-9.
2http://www.knowledgefox.net and http://www.knowledgefox.net/knowledgematch

iiWAS ’17, December 4–6, 2017, Salzburg, Austria Bernhard Göschlberger and Peter A. Bruck
half. We follow the denition, that MicroLearning refers to charac-
teristics of Micro-Content (nuggets) and MicroLearning activities
(steps). Micro-Content is small (e.g. ts on small screens) and topi-
cally focused (single fact or concept), it has a simple structure and
is easy to grasp (reduced cognitive load). MicroLearning activities
are short (seconds rather than minutes), interactive (require user
input), provide feedback (as a direct reaction to user input), and
trigger reection on part of the learners. Thus, learners choose their
own pace and integrate learning activities into their daily routines.
We, therefore use the concept of Integrated MicroLearning[
7
] or
Workplace Integrated MicroLearning in the context of corporate
learning. As mobile devices accompany learners throughout the
day Integrated MicroLearning is often done using mobile devices
and thus closely related to mobile learning.
Although often used for learning following formal curricula
and highly specied learning objectives, MicroLearning happens
informally, meaning between other activities, on the move, during
waiting moments — often driven by knowledge needs or inner
impulse [
10
]. Such impulses may be supported or even triggered by
a MicroLearning system using learning pushes as a teaser to call up
attention and/or raise interest [
2
]. As an interactive, informal form
of learning focusing on small feedback loops and instant reection
MicroLearning can be viewed as a specialized form of self-regulated
learning [3].
2.2 Gamication
Deterding et.al. dene gamication as "[.. . ] the use of game design
elements in non-game contexts." They identify ve dierent levels
of game design elements:
(1) game interface design patterns
(2) game design patterns and mechanics
(3) game design principles and heuristics
(4) game models
(5) game design methods
Over the past years, gamication has received a lot of attention,
inuencing many dierent application domains. Many attempts
have been undertaken to systematize the plethora of elements.
The Octalysis Framework[
4
] provides such an approach, clustering
game design elements into eight categories. The model positions
those eight categories along two continuous scales:
(1)
left-brain (logic, calculation) vs. right-brain (creativity, so-
cial)3
(2)
white hat (positive reinforcement) vs. black hat (negative
punishment)
Figure 1 illustrates the Octalysis Framework with left-brain asso-
ciated categories on the left, right-brain associated categories on
the right, white hat associated categories on the top and black hat
associated categories on the bottom.
Many elements in MicroLearning didactics are also game design
elements. This is not surprising as one of the rst publications on
MicroLearning as we understand it today[
7
] was largely inspired
by Prensky’s work on Digital Game-Based Learning[
15
]. In this re-
gard, a notable example for the overlapping area of MicroLearning
3the left/right brain model is a symbolic model and not neurologically sound
Figure 1: The Octalysis Framework [4]
and gamication is Duolingo
4
, which besides of being a popular
language learning application also has the ambition to use lan-
guage learners to crowdsource translations[
17
]. A general view of
the overlapping concepts in the context of corporate learning is
provided by Decker et.al.[
5
]. They analyze the applicability and
suitability for MicroLearning of various game design elements for
didactic requirements or goals on a theoretical level. Below, we
use the categories of the Octalysis Framework to systematically
introduce the game design elements that are relevant in the context
of this work and relate them to Decker’s results.
2.2.1 Epic Meaning and Calling. Games oftentimes use a narra-
tive to relate the user’s activity to a higher goal or greater meaning.
In learning environments learning goals typically provide such a
meaning. To engage the learners by relating them to the greater
meaning learning goals are often presented at the beginning — just
like in video games where players are engaged by the ambition to
ultimately save the princess.
2.2.2 Development and Accomplishment. Games typically re-
quire users to complete challenges,missions or quests and reward
badges or other rewards upon success. Points that players can use
to compare themselves with others are another type of reward.
Oftentimes leaderboards and rankings are directly incorporated
to trigger such a comparison directly. For goals with quantiable
progress, indicators such as progress bars inform the user about his
goal achievement. Decker concludes that these game elements are
well suited for MicroLearning.
2.2.3 Empowerment of Creativity and Feedback. Games often
grant users room to unleash their creativity and require them to
solve problems through exploration. To successfully direct the user’s
creativity towards solutions, games provide instant feedback on
4https://duolingo.com

Gamification in Mobile and Workplace Integrated MicroLearning iiWAS ’17, December 4–6, 2017, Salzburg, Austria
the user’s actions — a mechanic that is closely related to the self-
regulated learning loop. A similar game mechanic deals with longer
loops. It is described as Milestone Unlocks that users want to ac-
complish before stopping. They are dened by players themselves
beforehand, achieved after a sequence of actions and reected upon
afterward. A user might, for example, try to reach a certain level
before leaving a game, plays until the level is reached and reects
by checking unlocked rewards or skills. Again this mechanic is
directly comparable to the cycle of self-regulated learning.
2.2.4 Ownership and Possession. When players acquire some-
thing of perceived value in a game they develop the motivation to
keep it. This holds true for typical elements like in-game currencies
or resources, for in-game reputation, and also for an avatar or a
prole picture.
2.2.5 Social Influence and Relatedness. Social components are
main motivation drivers in games. Collaboration and Competition
can be found in many dierent forms in games. Decker postulates a
limited suitability of collaborative mechanics for MicroLearning and
does not cover competitive mechanics besides the aforementioned
leaderboards.
2.2.6 Scarcity and Impatience. A typical game design includes
some form of scarce resource. Scarcity makes games hard and there-
fore challenging and interesting. It forces players to focus and by
doing so involves them also emotionally. The scarce resource might
be game specic or as simple and universal as time. A Countdown
Timer is a typical example that creates a scarcity of time to accom-
plish something. Decker states that a Countdowns align well with
the didactic goals for MicroLearning activities.
2.2.7 Unpredictability and curiosity. Unforeseeable events in
games increase thrill and challenge. In unpredictable environments,
players are driven by their curiosity. What’s in that box, behind
that door, in that cave? While not covered by Decker, it might be
dicult to excessively apply the element of surprise in the context
of learning. That said, the diverse set of activities, rewards and
lesson goals in Duolingo are partially suited to excite somebody’s
curiosity.
2.2.8 Loss and Avoidance. Game design elements in this cate-
gory make use of punishment of negative or unwanted behavior.
Players lose lives, achievements, progress, points, items or skills in
certain conditions. Decker denies the suitability of such elements
for MicroLearning in the context of corporate learning. Conversely,
Duolingo uses a variety of implementations of loss and avoidance
such as skill level depletion and streaks. In section 4 we will present
an application of loss and avoidance for depletion of points of
achievements in KnowledgeMatch.
3 THE MICROLEARNING SYSTEM FORMERLY
KNOWN AS KNOWLEDGEPULSE
As mentioned KnowledgeMatch is an extension of an established
MicroLearning system, originally developed under the brand name
KnowledgePulse. It was re-branded to KnowledgeFox
5
two years
ago in the course of increased market success. In this paper, we use
5https://knowledgefox.net
the original name to emphasize the coherence and continuation
of the research eorts. Over the past few years, the system has
served as a platform for long-term research eorts and has been
described and evaluated at various stages in [
2
,
16
]. These results
also serve as a baseline for this case study. For consistency of ar-
gument, we present here briey the core concepts and features of
KnowledgePulse without going into technical details.
3.1 System Overview
KnowledgePulse is a multi-tenancy client-server application with a
strong focus on mobile usage. A tenant is called an organization and
consists of users with dierent roles such as organization adminis-
trator, user administrator, content author, and learner. It provides
currently native clients for Android, iOS, Windows Mobile
6
and
Windows Desktop as well as a responsive web application. Content
and learning progress is synchronized across all clients if Internet
access is available, but all native clients support learning oine.
3.2 Knowledge Cards
KnowledgePulse builds upon the concept of a Knowledge Card as
a base unit for learning. As a rst step Knowledge Cards present
a question or MicroLearning activity that requires user input as
a response. Learners can optionally request a hint as a support in
answering the question or solving the activity. In direct response
to the users answer or input the Knowledge Card provides direct
and immediate feedback on the user’s performance. Additionally,
learners are provided with an in-depth explanation of the answer
and content referred to in the Knowledge Card. Content authors
can group Knowledge Cards into Lessons and Courses. Lessons
have a predened order to ensure that prerequisites are understood
before more advanced topics or concepts are tackled. Each course
and each lesson have an introductory text. This text should be used
to describe the content and learning goals. From a gamication
standpoint, this can be viewed as Epic Meaning and Calling. Also, the
checkpoint character of lessons that need to be completed to unlock
the next lesson can be viewed under the aspect of Development and
Accomplishment and in relation to learning goals under the aspect
of Empowerment of Creativity and Feedback.
3.3 Sequencing and Spaced Repetition
Resembling a mental bookmark, KnowledgePulse uses the notion
of an Active Lesson. The sequencing of Knowledge Cards within
the Active Lesson is determined by a probabilistic spaced repetition
algorithm loosely based on the work of Leitner[
12
]. Consequently,
the likelihood that the system presents a certain Knowledge Card is
determined by a function of previous performances of the user and
decreases with increasing prociency. Such sequencing strategies
draw o classical psychological research[
6
] and are supported by
more recent neurological ndings[
9
]. We wish to point out that the
concept of the scaolded feedback and reection loop of a Knowl-
edge Card is distinctively dierent from Ebbinghaus’ experiments
6Windows Phone 8, Windows 10 Mobile

iiWAS ’17, December 4–6, 2017, Salzburg, Austria Bernhard Göschlberger and Peter A. Bruck
on memory capacity
7
and related ashcard research solely building
on a stimulus-response model8.
3.4 Learner Involvement and Contribution
MicroLearning in our denition follows the paradigm of small
structured, living content rather than static, predened, monolithic
compendia. To involve learners and encourage the reection pro-
cess, KnowledgePulse fosters user contributions. It supports learner
created Knowledge Cards and direct user feedback on Knowledge
Cards to benet from the learners’ reection processes and em-
brace participatory learning design. The mobile clients make use of
platform features such as smartphone cameras in the card creation
process to enable users to easily create rich learning content.
3.5 Learning Teaser
A central concept in the development of KnowledgePulse is the
integration of learning into daily routines - be it in the workplace
or elsewhere. The very rst prototype was a screen-saver based
application called Lernschoner. Instead of colorful animations, it
presented users with a learning question as part of an early version
of the Knowledge Card. The presented question acts as a teaser to
users and triggers the intrinsic motivation to start a learning activity.
This design has been rened and is an integral component in the
current solution of KnowledgePulse, providing platform-specic
notication mechanisms.
3.6 Gamication
As mentioned in Section 2.2 MicroLearning has roots in game-based
learning and KnowledgePulse incorporates various gamication
traits. Badges hallmark completed courses and a short thumbs-up
animation rewards correct answers as good performances on single
activities. The most essential and at the same time very subtle
gamication element is the visualization of the learning progress.
An unobtrusive progress bar at the top of the screen constantly
indicates the progress within the Active Lesson. A click on the bar
or a respective menu item reveals more detailed information using
Leitner’s learning box as an illustrating model.
A more recent feature is called Intervention Screen. As Knowl-
edge Cards are constructed around a teasing question, a sequence
of Knowledge Cards leads to a pull eect captivating learners. This
counteracts the idea of short, integrated, and spaced learning ac-
tivities. The Intervention Screen intervenes after a sequence of 10
learning steps and triggers reection on the learning activity. It
presents the performance of the learner on the past sequence using
achievement visualization as often used in games and a random
motivational quote related to learning (see Figure 2).
4 KNOWLEDGEMATCH
KnowledgeMatch was designed as an additional feature to the exist-
ing system. The following section provides a concise summary of
the functionality of KnowledgeMatch and the main design decisions.
7
Ebbinghaus conducted lab experiments on retention of nonsense syllables as a func-
tion of time [
6
]. His results are known as the forgetting curve and have been evaluated
and even replicated several times [14]
8ash card instruction, ash card drill and incremental rehearsal [1, 8, 13]
Figure 2: Intervention Screen
4.1 Topic Selection
While interactive quiz games select arbitrary topics and subjects to
increase thrill, tension, and challenge, curricular learning activities
are planned and thoughtfully arranged for learners. The focus of
player versus player quiz games is in the choice of opponents.
Conversely, learning is focused on learning goals and therefore on
the choice of content.
KnowledgeMatch keeps a match focused on one particular topic,
namely a course. Learners set learning goals by subscribing to
courses. As pointed out earlier users dene an active course in the
standard learning mode to mark their current focus. For Knowledge-
Match however limiting users to matches on their current active
course seemed too restrictive to yield sucient opponents. Conse-
quently, eligibility is dened through course subscription. A user
can either be subscribed to a course or not — and thus eligible for a
KnowledgeMatch on a particular range of topics.
The journey of a match initiating user starts with the selection
of a course. The content of the course denes the topic of a match.
Only then the user is asked to choose an opponent eligible for the
chosen topic. What to learn has priority over who to learn it with.
4.2 Match Design and Turn-taking
KnowledgeMatch was designed as an asynchronous competition
with alternating player turns. A central goal was to engage and
motivate learners to frequently use the system for short learning
activities in spare minutes that occur during the day — for instance
in a coee break while waiting for a meeting or while commut-
ing. The turn-taking strategy fosters these small but more frequent
interactions. A single player should interact with ve to ten Knowl-
edge Cards before the game switches to the opponents turn, to
ensure that a typical session does not exceed two to ve minutes.
Such short learning sequences align with the concept of Integrated
MicroLearning.
During a match, we refer to the interaction with a single Knowl-
edge Card as a move. A move has a time limit indicated by a count-
down clock on the top of the screen. The organization administrator
can set the time limit tenant wide. After the user has submitted
his answer or the time has expired the solution and the in-depth
explanation are displayed (see Figure 3). A user can briey pause,
read and reect as this screen has no time constraint. The instead
the timer displayed at the top of the screen is frozen at the time the
move was submitted. In the left upper corner, the current round is

Gamification in Mobile and Workplace Integrated MicroLearning iiWAS ’17, December 4–6, 2017, Salzburg, Austria
Figure 3: Knowledge Card in match mode after the answer
was submitted
indicated while the results from previous moves are displayed in
the upper right corner.
As dierent mental eects and types of motivation can be ob-
served depending on whether a player opens a new round or tries
to equalize or surpass the opponent’s performance, round openings
alternate between players. KnowledgeMatch uses alternating round
openings and a round size of ve Knowledge Cards. Matches consist
of three rounds to reduce the likelihood of being decided before the
nal round to prevent players from dropping out.
The rst round is opened by the initiating player and closed by
the accepting player, who in turn starts the second round thereafter.
The second player receives a match invitation and subsequently
plays the rst and second round before his turn ends. The initiating
player is then notied and plays the second and third round, com-
pleting his part of the match. The second player is then notied
and completes his third round yielding a match result. The initiat-
ing player gets a notication about the match completion and the
result.
4.3 Multiple Matches, Knowledge Card
Selection and Sequencing
As a result of the asynchrony of KnowledgeMatch a single match
can include plenty of idle time for a player. To avoid blocking
motivated users, there is no restriction to start new additional
matches. Multiple matches can be played at the same time — even
with the same opponent.
The Knowledge Card selection for a match needs to be dened
beforehand as both players receive the same cards. The implemen-
tation of KnowledgeMatch denes the Knowledge Card selection
when a match is initiated. To provide a selection that grants fair
chances to both players from a game point of view, both players
should have the same experience regarding the selected cards. The
card selection is based on both players learning records associated
with a single Knowledge Cards and the number of its selections
for other matches (played or yet to play). While the system uses
elaborated and well established didactic concepts for sequencing in
standard learning mode, algorithms can only control the frequency
but not the sequence of Knowledge Cards in KnowledgeMatch.
In general, quiz game players expect to receive new content
rather than repetition. Players perceive items that are reoccurring
frequently as monotonous or even assume that software bugs cause
the observed selection behavior. Quiz game item selection and
spaced repetition, therefore, work quite dierently.
As the selection of Knowledge Cards for a match cannot deter-
mine their sequence, KnowledgeMatch does not implement spaced
repetition. Rather, the card selection strategy focuses on improving
the game experience by preferring unknown and less known cards.
This leads to an even distribution with constant spacing between
repetitions instead of an increasing one.
4.4 Prole Page and Avatar
KnowledgeMatch allows users to personalize their game presence
by selecting avatars or setting prole images to increase the experi-
ence of ownership and relatedness. On the prole page, learners
can congure their name, upload or take a picture or choose an
avatar. Avatar sets can be dened for each individual organization
(tenant), in order to allow companies to use their corporate iden-
tity or branding as part of the avatar collection. Before a match
is started learners see the avatars or prole images of both play-
ers, their own and their opponent’s. The visual representation of
the players are also shown on the result, intermediate result and
leaderboard screens.
4.5 Points and Leaderboard
KnowledgeMatch also introduced a leaderboard into the existing
system as an additional means of gamication. The analysis of
dierent existing ranking systems (chess, football, tennis. .. ) showed
two central, reoccurring features:
•
the amount of awarded points depends on competitiveness
(e.g. opponent strength, tournament prestige) and
•awarded points expire or deplete.
In terms of the Octalysis Framework, these features fall into the
categories Development and Accomplishment and Loss and Aversion.
In the case of KnowledgeMatch, up to 10 points are awarded for
victories and up to 5 points for ties. The amount of awarded points is
based on opponent strength (similar to ELO-system). Additionally,
one point is awarded for playing a complete match as an incentive
to participate. The depletion or expire strategy decreases points
of achievement by 1/365 which leads to a stabilization for players
with a constant performance after one year.
5 CASE STUDY: LEARNING — FRESHLY
BAKED
In this paper, we describe a case study conducted at a super-regional
Austrian bakery and bakery products company. The company has
established human resource development programs for onboard-
ing and continuous training. Most of the training was conducted
face-to-face by internal trainers or external experts. Several new
approaches such as a new talents program, conventional e-learning,
and MicroLearning were evaluated. The learning system was set up

iiWAS ’17, December 4–6, 2017, Salzburg, Austria Bernhard Göschlberger and Peter A. Bruck
and a sales division was selected to do a pilot project
9
. Eleven per-
sons were trained for content authoring on-site and a MicroLearn-
ing course on content authoring was made available to them. After
the rst phase of two months for content authoring, the rst in-
crease in system usage could be detected after three months when
about 75 new users were registered.
Accompanying a broader rollout of the system an employee-
challenge was organized in project month 5
10
. Our case study fo-
cuses on the impact of this extrinsic motivation on learning activi-
ties. We investigate usage patterns in standard learning mode and
match mode over the course of seven months and compare pre-
challenge activities and activities during the organized challenge.
For the employee-challenge, 175 users of 15 dierent regional
sales and key accounting teams were invited to participate. Team
sizes were ranging from 6 to 18 members. Participants were located
at operating sites in dierent parts of Austria (96 employees) and
Germany (79 employees). Within the selected sales division 78 em-
ployees were eld service salespersons, 47 were back-oce employ-
ees and 20 persons were sales managers or directors. The remaining
persons held other sales related positions such as sales assistant.
The participant group was balanced with respect to gender, as 89
were female and 86 were male employees. The age ranged from 19
to 64 years with an average of 42.9 (Q0.25 =35.28,Q0.75 =50.29).
For the employee-challenge, 386 Knowledge Cards of four ex-
isting courses were combined into a new course. This course was
congured to be available only for matches and not for standard
learning mode. The original four courses remained available for
standard learning mode. The content covered knowledge on sales,
service, products, production process, branding, the company itself,
economic indicators, internal key performance indicators, salary
payment and remuneration schemes.
As an incentive to participate the HR department announced
prizes to the top three contestants. Scores were based on the sum
of individual and average team points. The scoring scheme and
prizes were communicated upfront. The prizes were an IPad 5,
a
e
400 travel voucher and a
e
150 consumption voucher. The
vouchers were redeemable at restaurant, hotel and catering partners
of the company. Additionally, for all active users, two tickets to a
symposium including accommodation were raed o.
Employees were encouraged to use KnowledgeMatch during the
working hours, but not while being with customers.
6 RESULTS
After project month seven and the completion of the employee-
challenge, we conducted the analysis of the recorded data.
6.1 Client Usage
Participants could choose freely which device and clients to use.
The KnowledgePulse server records client information for all web
service calls allowing a detailed investigation of client usage. The
client usage distribution extracted from all recorded actions in
Table 1 shows the signicance of mobile devices for MicroLearning.
Almost 9 out of 10 recorded user actions were executed on mobile
devices. As the web application client is responsive, it cannot be
9The project and the evaluation study were started in December 2016
10The challenge was held from May 1st to July 31st 2017
Table 1: Device distribution
Client Learning Steps (relative)
Android 75,85%
iPhone/IPad 12,63%
Web 9,94%
Windows 1,58%
clearly determined to what extent it might have been used on mobile
devices.
6.2 Overall Activity
A simple indicator of user engagement used in literature is the
overall user activity. Previous research reports 10-20 learning steps
on average per day and six daily learning sessions for the most
active users [2].
6.2.1 Pre-Challenge Activity. In the present case study, 102 users
had already completed at least a single learning activity and 57
users completed at least one course before the challenge kick-o.
Overall a total of 164 course completions, where 37 learners had
completed two or more courses and 12 learners even completed ve
or more. On average 494 learning activities per active user were
recorded in that time frame. These activities comprise standard
learning mode as well as match mode. Although not promoted
yet, KnowledgeMatch was already actively used by 51 users and 28
matches were started on average per day in the month before the
start of the company-wide challenge. This amounts to exactly 50%
of the 102 users that had made at least a single learning step till
then. About 37% of all pre-challenge learning activities were made
in KnowledgeMatch mode.
Another important indicator regarding the motivational assess-
ment of MicroLearning is the number of learning daily learning
sessions. To analyze learning sessions we extracted contiguous
sessions from a stream of activity data. Two adjacent activities
belong to the same session if the timespan between is below a
dened threshold of one minute. This threshold has been dened
iteratively by statistical comparison of intra-sequence intervals
and inter-sequence intervals, nally yielding an average interval of
more than 5 hours between two sessions. On average users started
2.35 learning sessions per active day. The median of average daily
sessions is 1.86 and 70% of all users are below the average daily
sessions. The most active 5% of all users (in terms of daily sessions)
average 5.5 or more sessions per active day.
6.2.2 Activities during the Challenge. When the game challenge
with KnowledgeMatch started 175 employees were invited to par-
ticipate. The number of monthly active learners increased subse-
quently from 97 before to 130 at the end of the challenge period.
Overall 136 users (77% of all invited users) were active within in this
period, i.e. completed at least one learning activity. Of those active
users, 95 (70%) participated in at least one match. As an expected
result of the promotional activity, the growth in match participants
of 44 (+86%) surpassed the growth in active users of 34 (+33%). 92
users played at least one match completely (all 3 rounds, i.e. 15
cards). During the challenge period, an average of 585 matches per
day was created and match activity peaked at almost 6000 matches

Gamification in Mobile and Workplace Integrated MicroLearning iiWAS ’17, December 4–6, 2017, Salzburg, Austria
Figure 4: Usage Times of KnowledgeMatch (Jan-Jul)
in one of the last weeks. During the challenge, 95% of all recorded
learning activities were made in KnowledgeMatch.
The average learning sessions per active day dropped to 2.22
with a median of 1.8, where the 0.95-quantile decreased to 4.5.
At the end of the challenge, 90 users had completed at least one
course, 67 users had completed at least two courses and 39 users
had completed ve courses or more.
6.3 Usage Patterns
The aggregated overall activity described in the previous section
gives an indication of user engagement. Below we try to iden-
tify dierences in usage patterns between standard learning mode,
learning with KnowledgeMatch, and extrinsically motivated Knowl-
edgeMatch learning during the challenge.
6.3.1 Time of Usage. Bearing in mind that learners were encour-
aged to use the MicroLearning system during their working hours,
the distribution of activities over time of day is surprising. While
matches were played evenly distributed during the working hours
(7 am to 4 pm), a signicant increase of match activity occurred
after 5 pm peaking in the evening between 8 pm and 9 pm. Figure 4
shows this distribution of match turns over the course of a day. A
comparison between typical working hours and after work hours
shows that 37% of all match activity was recorded between 7 am
and 4 pm while 53% of all match moves were made between 5 pm
and midnight. The challenge and incentives provided by the com-
pany did not inuence this distribution as a comparison with data
recorded before the challenge shows: 37% of all match moves were
made between 7 am and 4 pm while 57% were made after 5 pm.
The distribution of activities in the standard learning mode in the
same period, however, was distinctly dierent. As shown in Figure 5
about 70% of the learning activities in the standard learning mode
was recorded between 7 am and 4 pm while only 22% occurred after
5 pm in the pre-challenge time frame.
6.3.2 Weekday Activity. For the whole observation period, the
weekday did not inuence activity. Similarly to the time distribu-
tion, the data shows activities on days o such as Sundays. In fact,
Figure 5: Usage Times of Standard Learning Mode (Jan-Apr)
Monday Tuesday Wednesday Thursday Friday Saturday Sunday
Day of Week
KnowledgeMatch Activity (% of weekly moves)
0 5 10 15
pre−challenge moves
challenge moves
Figure 6: Distribution of KnowledgeMatch activity before the
challenge(Jan-Apr) and during the challenge(Mai-July)
the average activity on Sunday was even slightly above the average
daily activity. In the pre-challenge period, the weekday distribu-
tion of activities between standard learning mode and Knowledge-
Match shows no signicant dierence regarding workdays and
weekend. The overall weekend activity amounts to 27% and 24% of
the KnowledgeMatch activity was recorded on weekends. During
the challenge, a shift of activity towards the weekend occurred. Fig-
ure 6 shows a comparison of activities in KnowledgeMatch before
the challenge and during the challenge. Note that the illustration
shows the relative activity as the absolute amount of activity is
magnitudes higher during the challenge for all weekdays. During
the challenge, 37% of all KnowledgeMatch activities and 36.4% of all
activities were recorded on weekends.
7 CONCLUSION
From our results, we conclude that KnowledgeMatch strongly con-
tributes to user engagement. We found increased activity in the pre-
challenge time frame compared to previous results that can largely

iiWAS ’17, December 4–6, 2017, Salzburg, Austria Bernhard Göschlberger and Peter A. Bruck
be explained by a 37% share of KnowledgeMatch activities. Although
the overall activity increased, the number of daily learning sessions
did not. Also, not all users respond to gamication as 50% of active
users were reluctant to play before the challenge. Those who did
play also used KnowledgeMatch outside their working hours. The
analyzed data shows that learners played KnowledgeMatch espe-
cially in the evening. It is interesting to note that learners did that
voluntarily and may have treated it as a leisure time activity.
The employee-challenge advertised KnowledgeMatch and oered
extrinsic motivation through incentives. This lead to a large in-
crease in activity. However, the number of learning sessions per
active day decreased slightly in comparison to pre-challenge activ-
ities. The extrinsic motivation led to a stronger concentration of
activities in fewer sessions compared to the pre-challenge phase.
This concentration of activities in fewer but longer sessions is coun-
teracting spaced repetition in MicroLearning didactics. Another
noteworthy eect of the challenge was the shift of activities towards
the weekend.
This raises issues whether corporate learning activities are to
be considered as part of work and whether regulations regarding
working hours should apply. While this is the case for mandatory
courses and training, it is not for training oered learners for use
on a voluntary basis. However, sometimes there might be a thin line
between mandatory and voluntary. Employees might feel obligated
to learn material that is not mandatory but might be tracked by
corporate Learning Analytics. Conversely, it can be argued that
users might not want to be restricted in learning to 9-to-5 workdays
in a knowledge economy. After all, knowledge is a personal asset
beneting learners, especially in the long run as much if not more
than companies.
In our case study, gamication led to more after work activity
in the pre-challenge condition. We believe that this due to many
participants perceiving KnowledgeMatch as fun rather than work.
To conclusively validate this assumption future work on gamied
MicroLearning should investigate and record employees’ views and
perceptions.
For Mobile and Workplace Integrated MicroLearning the in-
crease of average daily learning sessions remains a goal for future
improvements. Gamication has proven to be a successful approach
to shape user behavior and will be a central pillar in our eorts to
do so according to didactic paradigms.
REFERENCES
[1]
Diane M. Browder and Maura L. Roberts. 1993. Guidelines for ash card in-
struction. Journal of Behavioral Education 3, 3 (01 Sep 1993), 235–245. https:
//doi.org/10.1007/BF00961553
[2]
Peter A Bruck, Luvai Motiwalla, and Florian Foerster. 2012. Mobile learning with
micro-content: a framework and evaluation. Bled eConference, 25th Bled eConfer-
ence eDependability: Reliable and Trustworthy eStructures, eProcesses, eOperations
and eServices for the Future (2012), 17–20.
[3]
Deborah L Butler and Philip H Winne. 1995. Feedback and self-regulated learning:
A theoretical synthesis. Review of educational research 65, 3 (1995), 245–281.
[4]
Y.K. Chou. 2015. Actionable Gamication: Beyond Points, Badges, and Leaderboards.
CreateSpace Independent Publishing Platform.
[5]
Jasmin Decker, Henrik Wesseloh, and Matthias Schumann. 2015. Anforderun-
gen an mobile Micro Learning Anwendungen mit Gamication-Elementen in
Unternehmen. HMD Praxis der Wirtschaftsinformatik 52, 6 (2015), 851–865.
[6]
Hermann Ebbinghaus. 1885. Über das Gedächtnis: Untersuchungen zur experi-
mentellen Psychologie. Duncker & Humblot.
[7]
Gerhard Gassler, Theo Hug, and Christian Glahn. 2004. Integrated Micro Learning
— An outline of the basic method and rst results. Interactive Computer Aided
Learning 4 (2004).
[8]
Ron Houten and Ahmos Rolider. 1989. An analysis of several variables inuencing
the ecacy of ash card instruction. Journal of Applied Behavior Analysis 22, 1
(1989), 111–118.
[9]
Eric R Kandel. 2007. In search of memory: The emergence of a new science of mind.
WW Norton & Company.
[10]
Dejan Kovachev, Yiwei Cao, Ralf Klamma, and Matthias Jarke. 2011. Learn-as-
you-go: new ways of cloud-based micro-learning for the mobile web. Springer,
51–61.
[11]
Markus Krause, Marc Mogalle, Henning Pohl, and Joseph Jay Williams. 2015.
A Playful Game Changer: Fostering Student Retention in Online Education
with Social Gamication. In Proceedings of the Second (2015) ACM Conference on
Learning @ Scale (L@S ’15). ACM, New York, NY, USA, 95–102. https://doi.org/
10.1145/2724660.2724665
[12]
Sebastian Leitner. 1995. So lernt man lernen: angewandte Lernpsychologie–ein
Weg zum Erfolg. Herder.
[13]
Lara L MacQuarrie, James A Tucker, Matthew K Burns, and Brian Hartman. 2002.
Comparison of retention rates using traditional, drill sandwich, and incremental
rehearsal, ash card methods. School Psychology Review 31, 4 (2002), 584.
[14]
Jaap M. J. Murre and Joeri Dros. 2015. Replication and Analysis of EbbinghausâĂŹ
Forgetting Curve. PLOS ONE 10, 7 (07 2015), 1–23. https://doi.org/10.1371/journal.
pone.0120644
[15]
Marc Prensky. 2003. Digital game-based learning. Computers in Entertainment
(CIE) 1, 1 (2003), 21–21.
[16]
Josef Smolle and Peter A. Bruck. 2017. MicroLearning in Undergraduate Medical
Education. (oct 2017). http://bit.ly/2gAuh7e
[17]
Luis von Ahn. 2013. Duolingo: Learn a Language for Free While Helping to
Translate the Web. In Proceedings of the 2013 International Conference on Intelligent
User Interfaces (IUI ’13). ACM, New York, NY, USA, 1–2. https://doi.org/10.1145/
2449396.2449398