Content uploaded by Adam Palmquist
Author content
All content in this area was uploaded by Adam Palmquist on Jun 16, 2021
Content may be subject to copyright.
”I think it's quite subtle, so it doesn't disturb me”:
Employee Perceptions of Levels, Points and Badges in
Corporate Training
Adam Palmquist
1
, Izabella Jedel2
1 Applied IT, University of Gothenburg, Gothenburg, Sweden
adam.palmquist@ait.gu.se
2 Insert Coin, Gothenburg, Sweden
Izabella.jedel@insertcoin.se
Abstract. This exploratory study examines the white-collar worker perception
of the three most common game elements in learning Level,
Points and Badges applied in online training. Through surveys and interviews,
the study reveals that the perception of the gamified course design was engag-
ing. The game elements Levels and Badges were considered positive, while
Points was viewed as indifferent. The study also detects that respondents in
both the surveys and interviews had not noticed parts of the gamification de-
sign, making them negative towards the gamified course due to lack of coher-
ence in the design. The authors of the paper suggest that further studies should
address multimodal feedback, juiciness, and gamification to disclose which type
of feedback is paramount in various gamified situations.
Keywords: Gamification · Corporate training · Game elements · Design · Juici-
ness · Multimodal feedback · Learning · Learning Management System
1 Introduction
Due to the accelerated digitalisation and automation in industrialised and post-
industrial countries, labour market skills are changing rapidly. Employees are likely to
participate in lifelong learning to pursue various careers and develop competencies
necessary in their current workplace [1]. Concurrently, the labour markets median age
is rising, prognosticating later retiring, increasing the retraining demand on the exist-
ing workforce [2]. One essential solution for the issue is a corporate training, retrain-
ing, or upskilling [3]. A transformation from assembly line production to more auton-
omous work organizations in the production industries has led to a declining amount
of low-skill blue-collar jobs in the manufacturing industries. Meanwhile, the numbers
of high-skill white-collar occupations have generally been on the rise [3]. These oc-
cupations have traditionally demanded a more consistent strategy regarding renewing
and updating due to the skills acquired during formal education currently do not keep
pace with the rapidly digitalized work life. In 2017, it was estimated that the half-life
2
of white-collar skills was approximately five years. That is, a skill learned five years
ago is half as valuable as it was when it was learned [4]. In the software industry, the
half-life of employee skills is considered to be around ten months [5]. This is predict-
ed to go even faster in the upcoming year's making life-long learning not a luxury for
the few but a requirement for the many to stay employable in the labour market. The
situation has called for more scalable and quicker learning approaches for corporate
retraining. One of the approaches is gamification, which is discussed for its uses in
building habits through feedback. Gamification is often described as the application of
game design elements in a non-game context [6]. The present case study investigates
three of the most commonly applied game elements interface design patterns in learn-
ing, Level, Points and Badges [7][8] when implemented in a gamified employee train-
ing course at an international transport company to understand employee perception
of the game elements.
2 Related work
In the field of human resource management (HRM) gamification has been described
to offer an opportunity to enhance engagement and collaboration among employees
[9], and to aid HR-departments in influencing employees’ attitudes, individualizing
training programs, offering incentives, and aligning processes with strategy [10]. De-
spite this, limited studies have been conducted in the field [9].
Qualitative interview studies have showed higher reported motivation, recognition,
feedback, relationships building and enjoyment due to a gamified training program
[11] as well as a positive attitude toward gamified training [12]. Another case, includ-
ing comparative interviews, showed that gamification affected participants eagerness
to cooperate, created a more enjoyable training experience, and provided a more posi-
tive attitude towards the training [13]. Despite the positive attitude expressed by em-
ployees, gamification has shown limited effects on performance outcomes. In a lab
and field study showed that even though gamification resulted in higher satisfaction it
only led to marginally higher learning outcomes[14]. Similarly, that adding narrative
game-elements in a case of employee training had led to increased satisfaction but
equal or lesser training scores [15]. In a more long term field experiment, however,
gamification showed an increased internal motivation in employee training [16]. Also,
in an experiment design study with 22 participants regarding gamified AR-training
did not find a significant difference in performance and engagement between a gami-
fied and non-gamified condition [17]. The varied results expressed in previous studies
is noteworthy Suppose gamification has value in the upcoming reskilling revolution
[1] then there is a requirement for a more dependable understanding of different game
elements perform in corporate training and how adults perceive different them while
they undergo training. The following explorative research question investigated: How
do white-collar workers perceive the most common game elements Level, Points and
Badges in a corporate training course?
3
3 Methods
The investigated case is a self-paced digital corporate training online course on up-
coming technology trends in the production industry given in a Learning Management
System (LMS). Gamification was not part of the initial LMS designed but was added
by a gamification studio collaborating with the LMS provider. Eighteen respondents
participated in the course that lasted for three weeks. The course had a 68% comple-
tion rate, which was higher than the transportation company's average employee
courses. Structured interviews were conducted with four participants, and a survey
was sent out after the course to all participants. A survey was constructed to collect
valid information on how the employee perceives the game elements in their corporate
training. The survey consisted of 14 balanced equally between ordinal- and open-
ended questions. Ordinal questions used a seven-point Likert scale addressing ques-
tions such as "My experience of XX in the course was..." The ordinal questions were
followed with an open-ended question asking the respondent to elaborate on the pre-
vious question. The purpose of the chosen survey design correlates to the survey ob-
jective to gather comprehensive data from the respondents [18]
In the standardized open-ended interview, the participants were asked the same
questions in the same order. They were expected to contribute with extensive, open-
ended answers regarding their perspectives and experiences, providing a substantial
and in-depth narrative of the inquiry [19]. Standardized interviews were chosen to
facilitate an inductive content analysis [20] in the investigation, pursuing a deep un-
derstanding of how the employees perceive the game elements. A purposeful sampling
of the interview respondents was made by the corporation to address the departments
participating in corporate training.
Table 1. Respondents
Name in study
Role in company
Age
Course time
Interview Length
Respondent 1
Business Development
45
3 h
23 min
Respondent 2
Human Relation
41
10 h
31 min
Respondent 3
Business Development
55
4 h
25 min
Respondent 4
Human Relation
31
8-10 h
24 min
4 The gamification design flow
The corporate training was gamified through an API implemented in the corporation
LMS by a gamification studio. The API added a gamified visualisation of the employ-
ee progression in the course. Every time a task was completed, the employees gained
Points. When employees completed all tasks in a course block, they were rewarded
Points. These Points were needed to advance Levels, which were designed to show
employee progression in the course. The Badge element served as the backbone of the
gamification design. The Badge was outlined on a set of different goals and was visu-
alised by different icons. The Badge was designed to engage the employees in com-
pleting different course tasks in the LMS system. When a Badge was completed, the
box related to the task was checked. In the gamification design, there were three types
of Badges: Milestones: providing information on the employee progression in the
4
course; Social achieved by completing community task like peer-review course mates
assignments; Award attained thru being active in the LMS and thorough in the course
like complete all the course quizzes with 100%. The Badges outline the gamification
designer and the corporate learning manager co-designed to increase the employee's
probability of succeeding in the course (Figure 1).
Fig. 1 The gamified dashboard showing the game elements Level, Points and Badge
(called Achievement) in the gamification implementation.
5 Results
5.1 Survey responses
Of the survey respondents 8 out of 9 (88%) were positive toward Level 1 skipped the
question. 7 out of 9 (77%) were positive toward Badge, while the rest were neutral. 5
out of 9 (66%) were positive toward Points, while the rest were neutral (Figure 2).
Fig. 2 From left to right descriptive statistics showing perceptions of Badge, Level
and Points
The employees also gave their remark on the game elements. The employees ex-
pressed that the Level element provided them with visual progression as the level
5
meter is at the bottom and fills when they were progressing. One expressed concern
was how many levels there were in the course. There was also an expression that the
employees were not sure of the Level element’s purpose other than motivation indicat-
ing that the respondent was unsure of the Levels purpose or expressed an untapped
potential.
Employees expressed that the Badge’s purpose was vague, expressing a need for
clarification. Furthermore, there were opinions that the Badge elements should be
better connected with the actual course content. Also, there were comments regarding
the Badge element’s visibility in the interface, indicating a requirement for a more
inductive design approach.
The given opinions about the Points in the course indicated that their significance
was hard to understand. The employee expressed that they did not understand their
significance and how many points indicated a course’ pass’ ’a pass with distinction’,
etcetera. Also, the employees expressed that Points would have a more significant
impact on them if an indication of how many points should have been accumulated
arriving at a new theme in the course. Again, opinions appeared concerning
the Point’s visibility, as some of the employees had not percept them in the interface.
The game elements Level and Badges were perceived as more positive than
the Points in the course, which could depend on the circumstance that Points’ inten-
tion and purpose were unclear for the employees.
5.2 Interviews
The interviews were done after half of the course was completed. All the interviews
were conducted in a timespan of two days. The inductive content analysis addressing
the respondents’ opinions on gamification and their perception of the game elements
gave a diverse representation. In the interview, respondents 2 and 4 expressed being
positive toward the gamification set-up, supporting them to stay engaged in the
course.
” I think it's really good because you have an overview of what you have done so far
and what's your level is and so on.” (Respondent 4)
Also, there are notions that gamification within this company’s corporate training was
viewed as something novel and innovative.
“From my perspective, it's good that they have done something new and have experi-
mented with this section, taken it to the next step.” (Respondent 2)
Respondent 2 and 4 expressed that gamification gave a suitable outline of the
course, making it easy to follow and grasp if they completed the assignments and
were in the course they currently were situated. Also stressed that gamification was an
appreciated approach in the corporation´s learning strategy. However, these expres-
sions were not shared in all the interviews. Respondent 1 and 3 expressed that gamifi-
cation probably was helpful for some learners but not for themself.
“I believe it's a good thing. And I believe that it is the one thing you might need
thinking forward […] I don't feel this is a problem [...] like it is now. I might not be
the kind of person who is buying it (gamification), but it can provide a bit more moti-
vation for some.” (Respondent 1)
6
“That might probably motivate some people because we are all different […] I am
not personally motivated by it. I think it's quite subtle, so it doesn't disturb me” (Re-
spondent 3)
Respondent 1 and 3 indicated that integrating gamification in the course was legiti-
mate to aid less motivated course participants underlining the everybody is different,
having their drivers and barriers regarding company training. Both respondents also
mentioned that the gamification design did not distract them in its current state due to
its subtility and nonintrusive design.
However, in the interviews, there was also uncertainty about what was part of the
gamification design. Respondent 1-3 had not noticed the Level element, and Respond-
ents 1 and 3 had not noticed the Badge element. Not recognizing the entire gamifica-
tion in the interface affected the perception of the design. Respondent 3, who had
noticed Badge, but not the Level, commented on the lack of coherence that some
Badges were inaccurate due to indicated progress in the course.
” Now, when I started to think about it maybe I miss some of it. But I just noticed that
you know whenever I submitted an answer (on a course task) I would get a message
that I had reached blah blah blah. But you know it wasn't important to me, so I did
not pay much attention to it other than remembering it now since you're asking about
it.” (Respondent 3)
Respondent 3 noticed parts of the gamification design after two weeks after the
course starts indicating that it was due to not considering gamification. In contrast,
Respondent 4 had perceived both Badge and Level and were favourable toward gami-
fication and appreciated the gamification design.
“And yeah, I think there are quite a few good things here, the milestone is good be-
cause then you can grasp like: “I am finished this one. This is what I have left and so
on. And the course completion bar tells me what I have left to achieve.” (Respondent
4)
Respondent 4 had observed the Badge and the Level, and in the interview, she made
inferences of how the design flow could be optimised. Respondent 4 suggested that
the Badges could resemble the course tasks better and that it should be more apparent
how much effort it was needed to collect them. This indicated that she had compre-
hended the gamification design intention and asked the designer to optimise it further.
Regarding Points, respondents 1-3 had not recognised that they were earning points
when they completed different course assignments. Respondent 4 had noticed that she
was earning points when she finished something in the course but had not compre-
hended the Point 's purpose.
6 Discussion
The results concluded that the most common game elements in learning [8] are per-
ceived positive by the employees, especially the Badge and the Level. Points, howev-
er, was perceived as less positive by the employees than the Badge and the Level.
Points was perceived as hard to grasp and contained a low significance in the course,
which differs from the Badges and the Level. Badges and Level were commented as
more useful for the employee, giving them a visualisation of their progression. De-
scending deeper into the notions of the employee's interview, the respondents had a
7
mixed impression on the gamified design. While Respondent 1 and 3 expressed it as a
thought-provoking and novel take on the corporate course, they stated that it didn’t
affect them. Respondent 2 and 4, on the other hand, expressed that the added gamifi-
cation features in the LMS enhanced the learning experience making the course easy
to follow and made it straightforward to track their course progression.
One topic that occurred in both the surveys and the interviews was that some gami-
fication features had been designed too subtle in the LMS interface resulting in that
some employees had not noticed them. Not comprehending all the aspects of the gam-
ification designed seemed to harm the perception of gamification. Gamification de-
sign has been suggested is not sufficient, just adding game elements [21]. The game-
ful experience that game elements elicit is what motivates the user [22][23]. A good
gamification design should include juiciness, meaning that the user's feedback should
apply to several senses in the form of sounds, visuals, and animations [22][23]. In the
investigated case, the Points, Level or Badge may have provided sufficiently informa-
tional feedback, but their visual representation was too subtle and lacked "juici-
ness". A more multimodal feedback approach in the gamified course could have been
beneficial.
The inquiry suggests that the current implementation did not offer sufficient game-
ful and juicy aspects due to gamification's subtle nature, highlighting that visibly-
factor should be investigated more and discussed for inclusion in forthcoming studies.
As shown here, employees can be positive toward gamification without understanding
the purpose of the game elements, highlighting whether a positive attitude toward
gamification should be a useful performance indicator. As suggested in previous stud-
ies [14][15], engagement does not necessarily imply increased knowledge; this calls
for future research to investigate how game elements in employee training should be
designed to elicit more than motivation but also learning performance.
7 Conclusion & Further research
This exploratory inquiry aimed to exam some of the most used game elements in
gamified learning design to determine how white-collar workers perceived them in
their corporate training. Badges and Levels were perceived as positive by the study
respondents, while Points was considered indifferent. Both survey and interviews
revealed that the game elements had been too subtle, making them hard to grasp for
the user indicating a negative perception of the implementation. Therefore, further
research should do more investigations regarding game elements localisation in the
user interface regarding visibility and understandability for the user.
8 References
1. World Economic Forum: Towards a Reskilling Revolution | World Economic Forum. ,
Davos (2018).
2. Ritchie, H., Roser, M.: Age Structure. (2019).
3. OCDE: OECD Skills Strategy 2019, Skills to shape a better future. OECD Publishing
Paris (2019).
4. Kasriel, S.: Skill, re-skill and re-skill again. How to keep up with the future of work.
8
In: World Economic Forum (2017).
5. Merriam, S.B., Baumgartner, L.M.: Learning in adulthood: A comprehensive guide.
John Wiley & Sons (2020).
6. Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to
gamefulness: Defining “gamification.” (2011).
https://doi.org/10.1145/2181037.2181040.
7. Dichev, C., Dicheva, D.: Gamifying education: what is known, what is believed and
what remains uncertain: a critical review. Int. J. Educ. Technol. High. Educ. 14, 9
(2017). https://doi.org/10.1186/s41239-017-0042-5.
8. Cheong, C., Filippou, J., Cheong, F.: Towards the Gamification of Learning:
Investigating Student Perceptions of Game Elements.
9. Wanick, V., Bui, H.: Gamification in Management : analysis and research directions
Background. Int. J. Serious Games. 6, 57–74 (2019).
https://doi.org/http://dx.doi.org/10.17083/ijsg.v6i2.282.
10. Chen, E.T.: The Gamification as a Resourceful Tool to Improve Work Performance.
In: Gamification in Education and Business. pp. 473–488. Springer International
Publishing, Cham (2015). https://doi.org/10.1007/978-3-319-10208-5_24.
11. Copenhaver, K., Pritchard, L.: Digital badges for staff training: Motivate employees to
learn with micro-credentials. J. Electron. Resour. Librariansh. 29, 245–254 (2017).
https://doi.org/10.1080/1941126X.2017.1378543.
12. Rogach, O. V., Frolova, E. V., Demina, S. V., Ryabova, T.M.: Gamification use for
government authority employee training. Espacios. 39, (2018).
13. Fauziyah, U., Kaburuan, E.R., Wang, G., Aqsha: Gamification for Employee Training
Platform in Banking Industries. Proc. 2019 Int. Conf. Inf. Manag. Technol. ICIMTech
2019. 503–508 (2019). https://doi.org/10.1109/ICIMTech.2019.8843750.
14. Baxter, R.J., Holderness, D.K., Wood, D.A.: Applying basic gamification techniques
to it compliance training: Evidence from the lab and field. J. Inf. Syst. 30, 119–133
(2016). https://doi.org/10.2308/isys-51341.
15. Armstrong, M.B., Landers, R.N.: An Evaluation of Gamified Training: Using
Narrative to Improve Reactions and Learning. Simul. Gaming. 48, 513–538 (2017).
https://doi.org/10.1177/1046878117703749.
16. Silic, M., Lowry, P.B.: Using Design-Science Based Gamification to Improve
Organizational Security Training and Compliance. J. Manag. Inf. Syst. (2020).
https://doi.org/10.1080/07421222.2019.1705512.
17. Nguyen, D., Meixner, G.: Gamified augmented reality training for an assembly task: A
study about user engagement. In: Proceedings of the 2019 Federated Conference on
Computer Science and Information Systems, FedCSIS 2019. pp. 901–904 (2019).
https://doi.org/10.15439/2019F136.
18. Fink, A.: Survey research methods. Int. Encycl. Educ. 152–160 (2010).
https://doi.org/10.1016/B978-0-08-044894-7.00296-7.
19. Gall, M.D.: Educational research : an introduction. Boston : Pearson, Boston (2007).
20. Kyngäs, H.: Inductive content analysis. In: The application of content analysis in
nursing science research. pp. 13–21. Springer (2020).
21. Huotari, K., Hamari, J.: Defining gamification: a service marketing perspective. In:
Proceeding of the 16th international academic MindTrek conference. pp. 17–22. ACM
(2012).
22. Deterding, S.: The lens of intrinsic skill atoms: A method for gameful design, (2015).
https://doi.org/10.1080/07370024.2014.993471.
23. Hicks, K., Gerling, K., Richardson, G., Pike, T., Burman, O., Dickinson, P.:
Understanding the effects of gamification and juiciness on players. In: 2019 IEEE
Conference on Games (CoG). pp. 1–8. IEEE (2019).
https://doi.org/10.1109/mci.2018.2871703.
9
