Available via license: CC BY 4.0
Content may be subject to copyright.
Educ. Sci. 2023, 13, 16. https://doi.org/10.3390/educsci13010016 www.mdpi.com/journal/education
Article
Factors Influencing the Effectiveness of Serious Gaming in the
Field of Vocational Orientation
Christopher Keller
1,
*, Anna K. Döring
2
and Elena Makarova
1
1
Institute for Educational Sciences, University of Basel, 4132 Muttenz, Switzerland
2
Centre for Psychological Sciences, University of Westminster, London W1W 6UW, UK
* Correspondence: christopher.keller@unibas.ch
Abstract: This study investigates the effectiveness of the serious game like2be, which has been de-
veloped to support the individual career orientation process of adolescents by broadening their oc-
cupational horizon. In this paper, we present results from an intervention study with n = 809 ado-
lescents in Swiss schools at the lower secondary education level. To analyze the extent to which
cognitive, affective, and motivational factors are stimulated and what influence they have on ex-
panding knowledge about occupations (measured learning outcome), we applied confirmatory fac-
tor analysis, multiple linear regression, and a structural equation model. The results indicate that
the stimulation of cognitive processes through serious gaming has a statistically significant impact
on learning outcome, although such factors as enjoyment, flow experience, or self-perceived benefits
in playing like2be did not significantly impact gain in knowledge about occupations.
Keywords: serious game; game-based learning; like2be; career; orientation; career choice;
vocational orientation
1. Introduction
Technological development is advancing rapidly with considerable impact on edu-
cation worldwide. One such advance in the educational field is learning through the use
of serious games. Their implementation in the classroom not only enriches conventional
teaching methods but has also become a popular research topic [1]. Serious games contain
major pedagogical potential, as they have been explicitly developed for learning purposes
[2,3]. They have been used for a number of different educational purposes [4,5], including
in the field of vocational orientation [6–8].
Vocational orientation acquires special importance towards the end of compulsory
schooling as adolescents need to decide which career they want to pursue. Career choice
is a major biographical event for young people as it strongly relates to individual devel-
opment and life experience [9]. Furthermore, it is embedded in a personal and multi-lay-
ered career orientation process that begins in early childhood [10]. In a career choice pro-
cess, adolescents strive for a professional career that fits particularly well with their ca-
reer-related personal characteristics. In the context of career research, the level of fit be-
tween these characteristics and those of the professional environment (occupational fit)
indicates the probability of professional satisfaction, commitment and career stability [11].
However, occupational fit is a complex construct. It emerges and is transformed through
interaction with the environment [12] and makes career orientation a lifelong process.
From early childhood, young individuals begin to develop their own personality,
self-concept [13], and role in their social environment [14]. In the process, different inter-
ests, skills, competencies, and talents emerge, as well as values, strengths, and limitations.
These are important indicators for the identification of career-related personality traits
and for the formation of a career self-concept with which to find a fitting entry into the
occupational world [13,14]. However, children also learn about different professions at an
Citation: Keller, C.; Döring, A.K.;
Makarova, E. Factors Influencing the
Effectiveness of Serious Gaming in
the Field of Vocational Orientation.
Educ. Sci. 2023, 13, 16. https://
doi.org/10.3390/educsci13010016
Academic Editor: Huei Tse Hou
Received: 5 December 2022
Revised: 20 December 2022
Accepted: 20 December 2022
Published: 23 December 2022
Copyright: © 2022
by the authors . Li-
censee MDPI, Basel, Switzerland. This
article is an open access article distrib-
uted under the terms and conditions of
the Creative Commons Attribution
(CC BY) license (https://creativecom-
mons.org/licenses/by/4.0/).
Educ. Sci. 2023, 13, 16 2 of 20
early age. They recognize occupational activities and functions and learn to distinguish
between them. Moreover, in interaction with their social–cultural environment, they ob-
serve not only who is typically employed in certain professions, but also what prestige is
ascribed to certain professions. In consequence, the children develop their first occupa-
tional stereotypes (e.g., occupational gender roles), which influence their future career ori-
entation [15].
Because career choice is closely related to individual development and can be influ-
enced by a variety of factors, it is important that young learners engage in career explora-
tion. Research in the field of vocational education and training (VET) shows that their
transition into the professional world is particularly successful when they are intensively
engaged in in their own career exploration process [16–18]. For this reason, the serious
game like2be was developed so that adolescents can learn about professions in a playful
way and thus broaden their own career horizons [2].
Since serious gaming offers an innovative alternative to conventional learning meth-
ods, empirical research in this research field has been intensified in recent years [1]. To
date, however, there is a lack of empirical evidence on effectiveness of serious games for
career orientation purposes. To overcome this research gap, we conducted a quasi-exper-
imental intervention study in order to discover whether like2be effectively supports learn-
ing in the context of career choice. The data indicated that the integration of the serious
game like2be into school lessons can effectively support adolescents in acquiring
knowledge about professions, thereby broadening their career horizons [19].
In the present study, we applied a confirmatory factor analysis (CFA) and calculated
a structural equation model (SEM) to provide a differentiated answer to the following
research question: To what extent do cognitive, affective, and motivational factors influ-
ence learning with the serious game like2be?
1.1. Serious Games
The term serious game covers a wide range of attributes. First, a serious game is ba-
sically a game, and represents a series of voluntary and enjoyable activities in which play-
ers are involved. Furthermore, it is governed by constraints or rules, and it involves cer-
tain goals and possibilities for achieving these goals through moves or actions [20,21].
Second, serious games are usually developed by experts explicitly for a specific target
group and their abilities and needs, and since concrete learning objectives as well as me-
dia-didactic models are taken into account during development, they lead to an entertain-
ing mental contest and are able to stimulate learning processes effectively [3].
Although serious games do provide entertainment, their main purpose is learning
[22–24]. As such, they are primarily used for educational purposes [22], but are occasion-
ally used with the aim of supporting attitude and behavior change, e.g., to discourage
smoking or promote recycling [25,26]. The term Serious Game also includes the sub-cate-
gory Digital Game-based Learning (DGBL), which refers to an innovative approach to
learning skill acquisition and training through computer games (e.g., serious computer
games) and therefore has high educational value and potential [26,27]. However, it should
be noted that the two terms (Serious Computer Gaming, DGBL) are sometimes used syn-
onymously [28]. In this paper, the terms Serious Game and Serious Gaming are used to
refer exclusively to computer-based games.
1.2. Serious Gaming and Subject Learning
Serious Gaming has become a popular research topic. In particular, the past few years
have seen research into whether serious games benefit learning in specific subject areas.
Although studies have demonstrated that serious games can be used effectively for vari-
ous “exotic” educational purposes, such as to reduce school phobia [5], to raise students’
awareness of Internet dangers [29], for vocational training purposes [30–33], particularly
in commercially oriented businesses [34], or for addiction and disease prevention [35,36],
Educ. Sci. 2023, 13, 16 3 of 20
serious gaming has principally been analyzed for its effectiveness in the context of subject-
related learning.
For example, Byun and Joung [37] showed that serious games significantly benefited
the learning of mathematics among students in grades K-12, although the effect was con-
sidered small and the data analysis implied that there may be other options for learning
mathematics more effectively than with serious gaming. Likewise, Tokac and colleagues
[38] mentioned that serious gaming leads to small but significantly higher learning gains
for students in PreK-12th-grade compared to traditional instructional methods. Talan and
colleagues [39] similarly described the effect of serious gaming on mathematics learning
as moderate, but, unlike in previous studies, they included outcomes from preschool to
university in the analysis. Further, in a currently published review, Hussein and col-
leagues [40] also reported that serious gaming positively impacted mathematics learning
in K-12 levels. Although the studies reviewed focus on a wide range of mathematics-re-
lated topics, most studies were limited to the effect of serious gaming on learning arith-
metic operations.
The effectiveness of serious games for language learning has also been addressed in
various studies. Chiu and colleagues [41] found that serious gaming leads to a medium
positive effect size with regard to foreign language learning. They noted that drill and
practice games result in a small positive learning effect, while meaningful and engaging
games result in a large positive effect size. Talan and colleagues [39] also showed a large
effect of serious gaming on language learning. They therefore suggest that second lan-
guage learning is a promising area in which serious gaming could be more effective than
traditional media for students from preschool to university. Similarly, Chen and col-
leagues [42] mentioned large effects of serious gaming on language learning, especially
on vocabulary acquisition. In this regard, a gaming approach appears to be superior to
conventional learning methods. Similarly to a previous study, their results suggest that
the effects of serious gaming on language learning depend on the characteristics of the
game design. In particular, adventure games seem to be more challenging for players and
thus prove more effective for learning outcomes.
As in learning math and language, research has shown that serious games have a
positive impact on science learning achievements. For example, Tsai and Tsai [43] com-
pared science learning using conventional activities with learning supported by playing
serious science games. They report that compared to conventional instruction in science
classrooms, students learn significantly better with serious games in science class. Also,
Riopel and colleagues [44] showed that serious gaming is an effective alternative for sci-
ence learning. Especially in terms of knowledge acquisition, knowledge learning and
knowledge retention, was serious gaming shown to outperform learning with more con-
ventional methods under conditions of similar time investment or engagement. Likewise,
Hu and colleagues [45] describe science learning through serious gaming as particularly
effective. They also note that serious gaming had a positive effect on cognition, emotion,
motivation, and retention and thus led to learning effects. Finally, Lei and colleagues [46]
also examined the effects of learning with serious games compared with traditional teach-
ing methods on the academic achievement of students from elementary school through
university. They found that students learned substantially more with the use of serious
games than under traditional instruction. As possible moderation effects, the authors con-
sider an increase in student autonomy or motivation.
Overall, the state of research shows that serious gaming is used for different purposes
in the school context and supports learning, especially subject learning (e.g., mathematics,
language or science). Furthermore, it shows that serious gaming can be used to create
conditions conducive to learning.
1.3. Serious Gaming and Factors That Promote Learning
The cultural historian Johan Huizinga noted that the human species was once called
Homo Sapiens. When man himself realized that he was probably not as reasonable as
Educ. Sci. 2023, 13, 16 4 of 20
assumed, Homo Faber was created, man the maker. However, since humans have always
also been playful beings, the human species, in addition to Homo Sapiens and Homo Fa-
ber, can also justifiably be labeled Homo Ludens, man the player [47]. Archaeological re-
search has revealed that mankind engaged in play thousands of years ago. Mostly natural
materials such as stones, fruit, grains, wood, and later bones were used for different forms
of games. Excavations also confirm that the ball, the doll, and toy animals are among the
oldest objects of play. Furthermore, ancient games such as nine men’s morris or chess are
still played with great fascination today. [48]. Moreover, it is emphasized by historians,
e.g., Retter [49], that as early as approx. 5000 B.C. the peoples of present-day China, India,
Persia, Egypt and Greece began to think about playful activities and their importance in
education. Consequently, it can be assumed that humans have always had a marked ten-
dency to play games, which gives games and play a considerable potential.
However, the potential of serious games goes far beyond the historical perspective.
On the one hand, serious games fit in with trends of the current age, and on the other,
they are exclusively available on devices (e.g., smart devices) that have great influence on
people’s everyday lives. Concerning the professional development of serious games,
many studies have reported that serious gaming has a positive effect on specific learning
of subject knowledge among learners of different ages. Other studies have focused less on
the learning outcome (i.e., subject related knowledge), but instead on factors that can pro-
mote effective learning [4], such as an increase in cognitive performance or the stimulation
of flow experience, of emotions (e.g., enjoyment), as well as of self-perceived motivating
aspects (e.g., learning benefits through serious gaming).
1.3.1. Stimulation of Cognitive Processes with Serious Gaming
In a meta-analysis on serious gaming and its influence on cognitive processes and
motivation, Wouters and colleagues [50] found that serious games are more effective in
learning and retention, but no more motivating than conventional instruction methods.
However Mao and colleagues [51] mention that serious gaming is particularly effective
because it often focuses on problem-based learning, where players have to try different
strategies to solve problems in order to progress within the game. Additionally, Jong and
colleagues [52] found that serious gaming has a positive learning impact when players
perceive the game as a challenging task that also offers them the prospect of winning.
Furthermore, Sailer and Homner [53] find that gamified learning methods have significant
but small overall effects on cognitive, motivational, and behavioral outcomes. Although
they point out that gamified learning methods and serious gaming share the same focus
on learning value beyond entertainment, but they differ in nature. They also add that col-
laborative and competitive aspects in learning games are of great importance for cognitive
stimulation. Moreover, serious games have been effectively used to increase players’ cog-
nitive performance level in terms of fostering attention capacity [54], enhancing short-
term and visual memory performance [55], and reducing the fear of learning failure, thus
having a positive effect on learning motivation [56].
All studies showed that serious gaming stimulates cognitive processes and that this
has a positive effect on learning. Here, the focus is on increasing motivation. A similar
relationship exists between enjoyment and effective learning.
1.3.2. Enjoyment through Serious Gaming
Wouters and colleagues [50] mention that it is generally assumed that serious games have
a similar motivational appeal, such as high entertainment value, much as commercial com-
puter games have. Nevertheless, they add that serious games cannot compete with commer-
cial computer games in terms of gameplay, game content and game design and therefore have
a less motivating effect. This is especially evident in the sandbox game MineCraft. Although
it is not a serious game by definition, considerable effort has been put into the development of
an Educational Edition in recent years. This edition was developed by educators for educators
and includes innumerable possibilities to use MineCraft in a school context in a way that
Educ. Sci. 2023, 13, 16 5 of 20
enhances learning processes. Owing to the Educational Edition, MineCraft can be understood
as a kind of serious gaming. In various studies, MineCraft is said to have great educational
potential, which is related to its high level of enjoyment [57–59]. One indicator for this is its
open, creative gameplay which encourages exploration and learning—even requires it [60].
Moreover, self-confidence and self-efficacy are increased through serious gaming
[56,61], which is strongly related to the enjoyment of playing serious games, and which
contributes to motivational or behavioral learning outcomes [54,61]. Although Breien and
Wasson [62] consider the effect of serious games on learning outcomes to be positive in
their review, a narrative or story included in the gameplay was found to be a key aspect
of the effectiveness of serious gaming. In particular, they identified four narrative or story-
related possibilities that increase the positive impact of serious games on enjoyment, en-
gagement, and, consequently, learning: a virtual, quest-based game landscape that can be
explored in-game, changeable objects used to overcome challenges and achieve goals, and
game-relevant avatars whose own story makes an exciting contribution to the game story,
or the integration of real, significant events in human history into the game story.
According to Iten and Petko [29], however, although some scientific evidence sug-
gests that serious gaming may enhance motivation and positive emotions (especially en-
joyment), which are strong factors in positive learning outcomes, connections between fun
and learning have not yet been fully analyzed by empirical research. They found no clear
connection between fun and learning success in their intervention study evaluating the
effectiveness of a serious game in Swiss schools. Yet their data showed that greater enjoy-
ment of serious gaming led to greater interest in the learning content, and so they con-
cluded that serious gaming is an effective alternative for introducing a new topic and in-
creasing motivation during the learning process [29].
Thus, studies have shown that serious games can be fun and entertaining and that
they consequently promote learning. Nevertheless, it is assumed that they must provide
players with a very high level of enjoyment in order to be effective for learning. Another
factor related to effective learning is the experience of flow.
1.3.3. Stimulation of Flow Experience with Serious Gaming
Serious gaming can foster intellectual competition in an entertaining way [22–24],
and because it is an act of doing (i.e., learning by doing) it contributes to a flow experience.
If certain factors are present during the game (such as comprehensible game goals, rules
and gameplay, feasible challenges, player concentration, or feedback or assistance), play-
ers can fall into a flow state [63]. According to Csíkszentmihályi [64], this state is a situa-
tion of complete absorption or engagement in an activity and is of great importance to
learning processes. If learners additionally experience a particularly high level of enjoy-
ment while playing a serious game, they can fall into a GameFlow state [65]. Fu and col-
leagues [63] assume that players in a GameFlow state increase their motivation and en-
gagement, which positively stimulates their learning process. Finally, Wronowski and col-
leagues [66] reported that students who used a serious game (Deadly Distribution) for the
purpose of learning statistics were highly absorbed during gameplay and showed higher
levels of engagement and ultimately interest in the subject of statistics than those who
used conventional methods.
For those who are completely immersed in an activity, totally focused and exclu-
sively engaged, we talk about the mental sensation of flow. In the context of learning, this
creates an environment conducive to learning. Research has shown that serious games can
lead to such flow experiences. In addition to stimulating the flow experience, serious
games can have an impact on self-perceived benefit through the activity.
1.3.4. Self-Perceived Benefit through Serious Gaming
Although serious games effectively promote learning, the use of supplemental mate-
rials appears to further increase learning outcome. Wouters and colleagues [50] noted that
serious games were most effective for learning processes when supplemented with other
Educ. Sci. 2023, 13, 16 6 of 20
teaching materials rather than when used as the sole teaching method. In a learning pro-
cess, serious games in combination with specific didactic support (e.g., reflection, model-
ing, collaboration, modality, feedback, or personalization) led to well-structured prior
knowledge that helped learners to build on and continue to learn successfully [50,67].
Similarly, Chen and Law [68] mentioned the benefit of additional support. They analyzed
the effectiveness of in-game scaffolding in the serious gaming process and concluded that
with additional scaffolding, the positive effect of serious gaming on student motivation
and learning performance was significantly enhanced. Moreover, for Mao and colleagues
[51], additional support also represents an important aspect of effective learning with se-
rious games. If players receive beneficial feedback while gaming or afterwards, it helps
them to better understand and reflect on information use as well as on the decisions made
in the serious gaming process, which ultimately supports learning. In order to effectively
support learning processes and make learning effects visible, learning options such as se-
rious games must be methodologically and didactically embedded in the school context
[69].
In other words, serious games can support learning processes, but purposeful appli-
cation and the use of additional learning materials can strengthen the learning effect. It
appears to be important that additional deepening of the subject matter of the serious
games consolidates learning on the one hand, and on the other, the players come to rec-
ognize that they can learn successfully through serious gaming.
1.4. like2be—A Serious Game for Vocational Orientation
In school lessons on “Vocational Orientation”, adolescents at the Swiss secondary
level are specifically stimulated for an intensive and individual vocational exploration
phase, the aim of which is to help them succeed in making the transition to the world of
work [70]. An innovative possibility for exploration in the context of career choice is the
serious game like2be. [6,71] Like2be was developed as part of a research project supported
by the Swiss National Science Foundation (SNSF) with the involvement of various experts.
It is a web-based online game and can be played for free in German, French and Italian
(www.like2be.ch). In addition, this serious game was developed specifically for adoles-
cents in their career orientation process who will have to make an initial career choice
decision by the end of compulsory schooling [2].
With like2be, young people can expand their career choice horizons in a playful way.
It is a simple point-and-click game without narrative. As players, they take on the role of
a personnel agent who must place applicants in suitable jobs or training positions based
on their application folders and CVs within a specified time (see Figure 1). A suitable
placement is followed by promotion at the end of the game round and the level of diffi-
culty increases (i.e., more vacant jobs to choose from). In the case of unsuitable placements,
the player is threatened with dismissal from the virtual job agency [2].
Educ. Sci. 2023, 13, 16 7 of 20
Figure 1. The like2be Gameplay. (1) Applicant, (2) job or training position, (3) applicant’s CV, (4)
specified time (game round).
Like2be includes different jobs or training positions, a variety of cartoon-style game
characters (avatars), and various personal profiles (application folders and CVs). The jobs
or training positions integrated into like2be are taken from the official Swiss information
portal for career, study, and career guidance (www.berufsberatung.ch). The serious game
like2be does not contain a specific roster. Instead, a randomization mechanism ensures that
all application folders and CVs occurring in the game are randomly assigned to a game
character (avatar) each time the game is started. Consequently, players can thereby only
play successfully if they compare skills, abilities, and individual wishes of applicants with
the job jobs or training positions offer [2].
2. Study Design & Methodology
In 2021, we conducted a quasi-experimental intervention study with adolescents at
the secondary school level to analyze the extent to which like2be can broaden young adults’
career choice horizons. The total sample included n = 809 adolescents from German-speak-
ing Switzerland. Of these, 49.4% were female, 48.2% were male, 2% assigned themselves
to another gender. Their average age was 13.77 years (SD = 0.82). After incomplete data
were excluded, a final sample including 532 adolescents remained. Of these, n = 415 were
in the intervention groups and played like2be in the classroom; 48.7% were female, 50.1%
were male, and 1.2% assigned themselves to another gender. Their average age was 13.78
years (SD = 0.87). In the control group there were n = 117 adolescents who did not play
like2be, of whom 57.3% were female, 38.5% were male, and 4.3% assigned themselves to
another gender. Their average age was 13.62 years (SD = 0.61). The participants were sur-
veyed at two measurement points (T1, T2). Between T1 (November 2021) and T2 (December
2021), there was an intervention phase of four weeks. During the intervention phase, the
adolescents in the experimental groups played like2be twice for two entire lessons of 45
min each. During the two lessons, the teachers were present but did not offer any assis-
tance; except for technical problems. The adolescents in the control group did not play
like2be.
We found that the intervention with the serious game had a positive effect on ex-
panding knowledge about occupations [19]. In this respect, students in the intervention
groups could effectively expand their job-related knowledge with the serious game like2be
Educ. Sci. 2023, 13, 16 8 of 20
(e.g., knowledge about the job, requirements, or job benefits). Their post-test scores (i.e.,
knowledge about occupations) were significantly higher than those of the control group.
Since recent research has shown that serious gaming can effectively promote learning
on different levels, such as the acquisition of subject-specific learning content, but also the
stimulation of factors that promote learning, we focus on the latter. In the present study
we aim to describe more precisely to what extent certain factors (cognitive processes, flow,
enjoyment, and subjectively perceived benefit) influenced the learning outcome with the
serious game like2be. Therefore, we tested the following four hypotheses among those stu-
dents who played like2be during the intervention:
H1. The serious game stimulates the cognitive processes and thus supports the expansion of
knowledge about occupations.
H2. The serious game is highly enjoyable and thus supports the expansion of knowledge about oc-
cupations.
H3. The serious game stimulates the flow experience and thus supports the expansion of knowledge
about occupations.
H4. The serious game leads to a high level of self-perceived benefit and thus supports the expansion
of knowledge about occupations.
2.1. Operationalization of the Constructs
The study included two measurement points (T1, T2) in which the adolescents com-
pleted an online questionnaire. At both measurement points, we asked the participants
how they would rate their knowledge of each occupation from like2be on a 6-point Likert
Scale ranging from “nothing” to “a lot”. Since we found a significant group difference
showing that the experimental groups learned more than the control group [19], we used
the score from the post-test (T2) as the outcome variable for the data analysis.
Further, to evaluate the serious game like2be for its effectiveness, we created a new
questionnaire based on the “Evaluation of the learning game AWWWARE” questionnaire
[29] and the eGameFlow questionnaire [63]. We used items to measure eight of ten latent
factors from the scale “Evaluation of the learning game AWWWARE”, developed by Iten
and Petko [29] based on the EGameFlow Scale (Fu and colleagues [63]). Furthermore, we
added items to the scale to measure the latent factor “Challenge” because of its high im-
portance regarding the effectiveness of a serious game [63]. We used the three valid and
reliable items with the highest factor loadings from the EGameFlow Scale. Our new scale
“Effectiveness Scale for like2be” (see Table 1) consisted of nine subscales (factors): (1) Goal
clarity (three items): the game objectives should be clear from the beginning and through-
out the game. (2) Controlling the game (three items): game controls should be simple so
that players can quickly navigate through the game and focus on its content or tasks. (3)
Strategic approach (three items): the game should stimulate cognitive processes during
play so that successful game strategies are developed. (4) Use of prior knowledge (three
items): The game should be a game for everyone. In this respect, all players should be able
to play successfully, regardless of their prior knowledge of the subject matter. (5) Flow
during gameplay (three items): the game should lead the player into a state of immersion.
(6) Enjoyment of the game (three items): the game should be entertaining and fun. (7)
Challenge of the game (three items): the game should offer challenges that fit the player’s
level of skills and the difficulty of these challenges should change in accordance with the
increase in the player’s skill level. (8) Learning Outcome (four items): the game should
increase the level of knowledge or skills of the players while meeting the game objectives.
(9) Motivational Outcome (three items): the game should create motivation to explore a
certain topic.
Educ. Sci. 2023, 13, 16 9 of 20
Table 1. Effectiveness scale for like2be.
Factor Item
Content
Goal clarity (three
items)
“I understood the goal at the beginning of the
like2be
game.”
“I always had the goal in my head while playing the
like2be
game.”
“The goal was clear throughout the game.”
(e)
Controlling the
game (three
items)
“The game control of the
like2be
game was difficult.” (Reverse item)
(e)
“I had to be very skilled to control the
like2be
game.” (Reverse item)
(e)
“I learned to control the
like2be
game very quickly.”
(e)
Strategic ap-
proach (three
items)
“While playing the
like2be
game, I thought carefully about whether or
not I was placing job applicants in suitable positions.”
“While playing the
like2be
game, I didn
’
t bother with placing job ap-
plicants in suitable positions, I just tried everything.” (Reverse item)
(e)
At the end of the
like2be
game, I reflected on why the job
placements
were suitable
or not.
Use of prior
knowledge (three
items)
“To play the
like2be
game, it was important to know a lot about jobs.”
“To place job applicants in suitable
positions, I had to know a lot
about jobs.”
(e)
“To improve myself, I need to learn more about jobs.”
Flow during
gameplay (three
items)
“While playing, I only thought about the
like2be
game.”
“While I was playing the
like2be
game, I forgot everything else around
me.”
“While playing the
like2be
game, I did not
notice how time passed.”
Enjoyment of the
game (three
items)
“The
like2be
game was a lot of fun.”
“I want to play the
like2be
game again.”
“The
like2be
game was entertaining.”
Challenge of the
game (three
items)
“My skills in the
like2be
game improved as I mastered the challenges.”
“The
like2be
game offered new challenges with a reasonable pace of
play.”
“I enjoyed the
like2be
game without being bored or anxious.”
Learning Out-
come (four items)
“With the
like2be
Game I learned about new jobs.”
“With the
like2be
Game I gained knowledge about jobs.”.
“Because of the
like2be
Game, I’
ve been thinking about the career
choices of women and men.”
“Because of the
like2be
game, I’
ve been thinking about what jobs suit
me.”
Motivational Out-
come (three items)
“The
like2be
game enhanced my interest in the topic of career choice.”
“Because of playing the
like2be
game, I realized that I wanted to learn
more about jobs.”
“Because of the
like2be
game, I will think more about my career
choice.”
(e): Items excluded for the final adjusted model
2.2. Scale Design
To investigate the factorial structure of the Effectiveness scale for like2be, we com-
puted a confirmatory factor analysis (CFA) using the lavaan package in R. To determine
whether the data were multivariate normally distributed, we applied the Mardia test for
skewness and kurtosis [72]. Since both p-values were significant, we concluded that the
data did not have a multivariate normal distribution. To correct for the violation of the
multivariate normal distribution, we performed robust estimation of the model using the
Satorra–Bentler adjustments [73,74] for all calculations.
Educ. Sci. 2023, 13, 16 10 of 20
First, we analyzed the basic model (Effectiveness scale for like2be) for its model fit.
For this purpose, we evaluated the global fit, the local fit, and the parameter estimation.
According to Hu and Bentler [74] the fit indices (see Table 2) suggest that the model did
not fit the data.
Table 2. CFA fit indices of basic model 1.
Fit
Statistic
Robust
Basic Model
Chi2 (df) χ2 = 1.270 (314);
p
< 0.001
CFI 0.907
RMSEA (90% CI) 0.066 (0.060; 0.072);
p
= 0.002
SRMR 0.087
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation,
CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Second, we analyzed the basic model (Effectiveness scale for like2be) for its model fit
a second time, but this time we tested a one-factor model. Therefore, we added all 22 items
to one factor. However, according to Hu and Bentler [74] the fit indices (see Table 3) sug-
gest that the second model did not fit the data either.
Table 3. CFA fit indices of basic model 2.
Fit
Statistic
Robust
Basic Model
Chi
2
(df)
χ2
= 1.
306
(3
50
)
;
p
<
0
.0
01
CFI 0.801
RMSEA (90% CI) 0.092 (0.087; 0.098);
p
< 0.001
SRMR 0.076
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation,
CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Third, we reduced the basic model 1 (Effectiveness scale for like2be) to an adjusted
model with the four subscales (factors): (1) Cognitive learning process, (2) Enjoyment of
the game, (3) Flow during the game, and (4) Learning Outcome. The first subscale refers
to competitive, challenging, problem-oriented gameplay and other mental processes that
stimulate players’ cognition during their gameplay (e.g., goal clarity, gaming strategy, use
of prior knowledge, learning by doing). In the context of serious gaming research, the
stimulation of cognitive processes is considered to be particularly effective for learning
because it can improve the attention performance, short-term memory and visual memory
[51,53–55]. Since enjoyment of the game was attributed great importance [29,54,61,63], we
created a second subscale including all items related to fun, entertainment, enjoyment or
items that appeal to players on an emotional level. Although the flow experience was de-
scribed as a component of motivation [64,75], the state of immersion is considered to be a
very important component for the effectiveness of serious games [63] and we therefore
created a separate third subscale for flow. Finally, serious games should increase the level
of knowledge and skills of players while also meeting the games’ objectives to encourage
the players to keep playing [63,65]. Consequently, learning outcome as the fourth subscale
included all items related to self-perceived learning effect, achievement of game objec-
tives, or learning progress.
Since the adjusted model was improved, but the fit indices still pointed to an insuffi-
cient model fit (see Table 4), we identified and excluded certain items (see items with (e) in
Table 1) from the model based on the modification indices. Criteria for the exclusion of
items were: items that lead to a large improvement in the model and (a) are indistinguish-
able from other items or ask the same question in different words, (b) can be assigned to
more than one factor, or (c) correlate strongly with one or more error terms of other items.
We also omitted all items of the initial factor “Controlling the game” because like2be is a
Educ. Sci. 2023, 13, 16 11 of 20
point-and-click game. Hence, the game control did not present any obstacles, particularly
not for adolescents.
Table 4. CFA fit indices of adjusted model.
Fit Statistic Robust Basic Model
Chi2 (df) χ2 = 1.289 (344);
p
< 0.001
CFI
0
.873
RMSEA (90% CI) 0.074 (0.068; 0.080);
p
< 0.001
SRMR 0.069
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation,
CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Fourth, we created the final adjusted model, including the four subscales (factors):
(1) Cognitive learning process (seven items, Cronbach’s α = 0.81), (2) Enjoyment of the
game (four items, Cronbach’s α = 0.89), (3) Flow during the game (four items, Cronbach’s
α = 0.80), and (4) Learning outcome (seven items, Cronbach’s α = 0.89). The fit indices of
the final adjusted model (see Table 5) also showed that the model fit sufficiently. Thus,
the CFI was > 0.95, the RMSEA was < 0.06 and statistically not significant, and the SRMR
was < 0.08, which, according to Hu and Bentler [74], is a good model fit. Also, in terms of
parameter estimation, the model contained no negative variances, and the standardized
loadings between the items and the factors were high throughout the model; the loadings
within the factors were similarly high (see Table 6).
Table 5. CFA fit indices of final adjusted model.
Fit Statistic Final Adjusted Model
Chi2 (df)
p
= 1.311 (203);
p
< 0.001
CFI 0.959
RMSEA (90% CI) 0.044 (0.038; 0.050);
p
= 0.943
SRMR
0
.045
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation,
CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Educ. Sci. 2023, 13, 16 12 of 20
Table 6. Final adjusted model.
Factor Item
No. Item Content Mean, SD Coefficient
Standard Er-
ror z-Value p-value Factor
loading
Cognitive learning
process
(seven items, α =
0.81)
C1 “I understood the goal at the beginning of the
like2be
game.”
M
=
3.60,
SD
=
1.18 0.47 0.07 6.99 < 0.001 0.40
C2 “I always had the goal in my head while playing the
like2be
game.”
M
=
2.98,
SD
=
1.16 0.79 0.05 16.39 < 0.001 0.68
C3 “While playing the
like2be
game, I thought carefully about whether or not I was
placing job applicants in suitable positions.” M = 3.04, SD = 1.16 0.77 0.05 14.62 < 0.001 0.66
C4 At the end of the
like2be
game, I reflected on why the placements of jobs were
suitable or not. M = 2.66, SD = 1.18 0.85 0.05 18.88 < 0.001 0.72
C5 “To play the
like2be
game, it was important to know a lot about jobs.”
M
=
2.99,
SD
=
1.19 0.60 0.06 10.09 < 0.001 0.50
C6 “To improve myself, I need to learn more about jobs.”
M
=
2.92,
SD
=
1.18 0.69 0.06 12.26 < 0.001 0.59
C7 “My skills in the
like2be
game improved as I mastered the challenges.”
M
=
2.87,
SD
=
1.14 0.83 0.04 19.01 < 0.001 0.73
Enjoyment of the
game
(four items, α =
0.89)
E1 “The
like2be
game was a lot of fun.”
M
=
3.06,
SD
=
1.24 1.06 0.04 24.56 < 0.001 0.86
E2 “I want to play the
like2be
game again.”
M
=
2.78,
SD
=
1.34 1.08 0.05 23.24 < 0.001 0.81
E3 “The
like2be
game was entertaining.”
M
=
3.01,
SD
=
1.25 1.07 0.04 26.28 < 0.001 0.85
E4 “I enjoyed the
like2be
game without being bored or anxious.”
M
=
2.96,
SD
=
1.23 0.94 0.05 19.80 < 0.001 0.77
Flow during the
game
(four items, α =
0.80)
Fl1 “While playing, I only thought about the
like2be
game.”
M
=
2.70,
SD
=
1.26 0.89 0.05 18.00 < 0.001 0.71
Fl2 “While I was playing the
like2be
game, I forgot everything else around me.”
M
=
2.39,
SD
=
1.17 0.84 0.05 17.69 < 0.001 0.72
Fl3 “While playing the
like2be
game, I didn’t notice how time passed.”
M
=
2.79,
SD
=
1.26 0.92 0.05 19.01 < 0.001 0.73
Fl4 “The
like2be
game offered new challenges with a reasonable pace of play.”
M
=
2.77,
SD
=
1.15 0.78 0.05 15.41 < 0.001 0.68
Learning outcome
(seven items, α =
0.89)
L1 “With the
like2be
Game I learned about new jobs.”
M
=
3.17,
SD
=
1.24 0.78 0.05 14.49 < 0.001 0.63
L2 “With the
like2be
Game I gained knowledge about jobs.”.
M
=
2.80,
SD
=
1.21 0.84 0.05 17.58 < 0.001 0.70
L3 “Because of the
like2be
Game, I’ve been thinking about the career choices of
women and men.” M = 2.34, SD = 1.19 0.81 0.05 15.59 < 0.001 0.68
L4 “Because of the
like2be
game, I’ve been thinking about what jobs suit me.”
M
=
2.65,
SD
=
1.21 0.92 0.05 19.67 < 0.001 0.76
L5 “The
like2be
game enhanced my interest in the topic of career choice.”
M
=
2.99,
SD
=
1.19 0.94 0.05 20.66 < 0.001 0.79
L6 “Because playing the
like2be
game, I realized that I wanted to learn more about
jobs.” M = 2.63, SD = 1.21 0.97 0.04 23.73 < 0.001 0.81
L7 “Because of the
like2be
game, I will think more about my career choice.”
M
=
2.64,
SD
=
1.22 0.97 0.04 21.98 < 0.001 0.80
Educ. Sci. 2023, 13, 16 13 of 20
2.3. Data Analysis Methods
Again, using the lavaan package in R, we conducted a multiple linear regression and
structural equation modeling (SEM) to analyze how the four factors identified above (cog-
nitive learning process, enjoyment of the game, flow during the game, and learning out-
come) predicted knowledge gain. In this process, the latent (exogenous or non-observable)
factors are theoretical constructs that cannot be observed directly. Therefore, latent factors
include observable/manifest (endogenous or dependent) variables. In our model, the four
dimensions are latent factors (Cognitive learning process, Enjoyment of the game, Flow
during the game, Learning outcome) emerged based on the 22 observed variables (C1-L7)
(see Table 4).
With SEM, we performed a path analysis to (1) understand more precisely the corre-
lation patterns among factors in our model, and (2) to show how much of the variation in
the learning outcome of the intervention with like2be can be explained by the latent factors.
3. Results
Based on our CFA, we regarded clp (cognitive learning process), ejm (enjoyment of
the game), flw (flow during the game), and loc (self-perceived learning outcome) as latent
factors. To analyze to what extent the four latent factors influenced the increase in
knowledge about occupations through playing the serious game like2be, we defined
knowledge of occupations measured in the post-test as the outcome variable. To analyze
to what extent the four latent factors influenced the increase in knowledge about occupa-
tions due to playing like2be, we defined the knowledge about all occupations from the
serious game measured in the posttest as the outcome variable kaj (knowledge about jobs).
To determine the direction and strength of associations among the latent factors, we
performed Pearson’s correlation. The correlation matrix (see Table 7) shows positive high
and statistically significant correlations within the latent factors.
Table 7. Correlation matrix of latent factors.
clp ejm flw loc
clp 1.000
ejm 0.802 *** 1.000
flw 0.866 *** 0.821 *** 1.000
loc 0.880 *** 0.737 *** 0.821 *** 1.000
clp: Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the game, loc: Learn-
ing outcome, *** p < 0.001.
First, a multiple linear regression showed that the overall final adjusted model ex-
plained 17% of the variance in the outcome variable (F(4410) = 20.87, p < 0.001). Second,
we analyzed the impact of the latent factors using SEM. The initial model fitting indices
by using the Satorra–Bentler adjustments for the SEM were as follows: χ2 (df) = 1.284 (221),
CFI = 0.960, RMSEA (p-value) = 0.043 (p = 0.978), SRMR = 0.044. Overall, the SEM model
fitted the data well. In this context, the factor clp explained 14% of the outcome variable
kaj (β = 0.43, EST = 13.97, SE = 6.11, z = 2.29). Additionally, the influence of the latent factor
on the outcome variable was statistically significant (p = 0.022). However, the three re-
maining latent factors ejm (β = 0.12, EST = 1.78, SE = 1.94, z = 0.91, p = 0.361), flw (β = −0.23,
EST = −3.91, SE = 3.62, z = −1.08, p = 0.280), and loc (β = 0.10, EST = 1.93, SE = 2.91, z = 0.66,
p = 0.507) had lower factor loadings than clp, were not statistically significant, and there-
fore showed no significant effect on the outcome variable (see Figure 2).
Educ. Sci. 2023, 13, 16 14 of 20
Figure 2. SEM path model (final adjusted model). clp: kaj: Knowledge about jobs, Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the
game, loc: Learning outcome, *** p < 0.001.
Educ. Sci. 2023, 13, 16 15 of 20
4. Discussion
In this study, we emphasized the effectiveness of serious gaming for purposes of vo-
cational orientation. In particular, we examined the extent to which cognitive, affective,
and motivational factors have an impact on measured learning outcomes in the context of
an intervention with the serious game like2be.
First, we developed a scale to evaluate the effectiveness of like2be. The basic model
(Effectiveness Scale for like2be) included nine factors (see Table 1). Despite adopting all
items from reliable and valid scales (Iten and Pekto [29]; Fu and colleagues [63]), our basic
model was found to be inadequate in the course of the CFA. The fit indices did not indicate
a good model fit. Consequently, we optimized the model. For this, we reduced our model
to four dimensions based on current research. The first dimension included items in rela-
tion to measuring the extent of stimulation of cognitive processes (1st factor), which were
considered to be particularly effective for learning [51,53–55]. The second dimension re-
ferred to items that measured the extent of enjoyment of the game (2nd factor), which
were considered to be of great importance in terms of learning success [29,54,61,63]. The
third dimension contained items measuring the flow experience (3rd factor), as state of
immersion is considered to be a very important component in the effectiveness of serious
games [63]. Finally, the fourth dimension included items measuring self-perceived learn-
ing outcome (4th factor), which motivates gamers to continue playing [63,65]. After re-
ducing the basic model to four dimensions, we excluded a few items because they were
either indistinguishable from other items or severely disrupted the model. Despite the fact
that we could show with a CFA that our final adjusted model fits the data well, we should
note that it deviates considerably from the factor models in past studies. One possible
explanation for this could be that the serious games in the two studies we referred to were
not comparable to the serious game like2be in terms of gameplay, subject matter, content,
controls, etc. However, our final adjusted model is reliable and valid and is suitable for
further evaluations of the effectiveness of serious games, especially in the field of voca-
tional orientation.
Second, by applying regression analysis, we investigated the overall impact of the
four latent factors on increase in knowledge about occupations. Our results indicate a sta-
tistically significant impact explaining 17% of variance. Despite the proportion of ex-
plained variance being rather small, cognitive, affective, and motivational factors turned
out to positively influence knowledge acquisition and contribute to effective learning with
serious games.
Third, we conducted a SEM to analyze the extent to which the four latent factors
influence the measured increase in knowledge about occupations. Regarding H1, the re-
sults showed that stimulation of cognitive processes while playing like2be had a positive
and statistically significant effect on increased knowledge about occupations. Therefore,
we accept H1, that serious gaming stimulates cognitive processes and thus supports the
expansion of knowledge about occupations. Regarding the importance of problem-based
learning in gameplay [51], and game challenge, as well as the prospect of winning [52],
our data analysis showed that the challenge of achieving the game objectives, the game
strategies necessary to do so, and also prior knowledge, all had an impact on the expan-
sion of knowledge. According to Sailer and Homner [53], collaboration and competition
are important for cognitive stimulation. Although, like2be offers few possibilities for col-
laborative play, it includes competitive aspects because of the game time limit. Despite
not evaluating effectiveness on attention capacity [54], short-term and visual memory per-
formance [55], we conclude that the serious game like2be stimulates players’ cognition
with its competitive, challenging, and problem-oriented gameplay during the game and
therefore has a positive impact on knowledge enhancement.
Furthermore, the data analysis indicated that how much players enjoy playing like2be
has no significant effect on increasing knowledge about occupations. Thus, we reject H2,
that the serious game like2be is highly enjoyable and supports the expansion of knowledge
Educ. Sci. 2023, 13, 16 16 of 20
about occupations. In accordance with Iten and Petko [29], we did not find a clear connec-
tion between enjoyment and learning outcome (i.e., increased knowledge) either. In line
with Wouters and colleagues [50], we assume that like2be was not as entertaining as other
commercial computer games and was therefore less motivating. Additionally, the lack of
an exciting narrative or story in like2be with high exploration, challenge, or success aspects
may have led to a lower impact of this serious game on learning outcomes [62].
Moreover, the results of SEM indicated that the flow experience while playing like2be
had no significant effect on improving knowledge of occupations. Accordingly, we reject
H3, that the serious game like2be stimulates the flow experience. Although, the serious
game like2be provided intellectual competition in an entertaining way [22–24] and aspects
such as comprehensible game goals, rules and gameplay, plus feasible challenges were
given, it neither led to a GameFlow state [63,65] nor to a situation of complete absorption
in the gaming activity mentioned by Csíkszentmihályi [64]. A possible explanation is of-
fered by the study results relating to enjoyment. Thus, we assume that players did not
experience a particularly high level of enjoyment while playing like2be for several reasons.
In terms of H4, the data analysis showed that the extent of subjectively experienced
benefits through like2be had no significant effect on increasing knowledge of occupations.
Therefore, we reject H4, that the serious game like2be leads to high level of self-perceived
benefit. In this context, like2be did not contain in-game scaffolding [68] or did it provide
players with a way to ask for feedback [51]. In this respect, like2be does not include in-
game opportunities for additional reinforcement of the content, which would consolidate
learning and help players realize that they are learning successfully through serious gam-
ing. However, there is additional teaching material to deepen learning, following Wouters
and colleagues [50], who mention that serious games are most effective for learning when
supplemented with additional teaching materials. Furthermore, a set of analog activities
for the serious game like2be already exists and includes reflection, modeling, collaboration,
or personalization activities designed to support learners and help them continue to learn
successfully [50,67,69]. The materials have even been favorably evaluated but were unfor-
tunately not part of the current investigation.
5. Conclusions
Recent studies have shown that serious games can be used effectively for various
educational purposes, in particular to promote the acquisition of specific technical
knowledge, but also to enhance factors that can promote effective learning. In this study,
we focused on the impact of the serious game like2be on factors conducive to learning,
such as increasing cognitive performance and enjoyment of the game, stimulation of a
flow experience and self-perceived benefits through serious gaming.
As a result of our data analysis, we conclude that like2be stimulates cognitive pro-
cesses in players, which consequently has a positive effect on their expansion of
knowledge about occupations. However, we were not able to demonstrate that the impact
on enjoyment, flow experience, and self-perceived benefit through like2be promoted the
expansion of knowledge about occupations. Indeed, these findings are in accordance with
or can be explained by previous research. For example, enjoyment of a serious game was
found to be very important for learning outcome [63], but to provide enjoyment, serious
games must include an exciting narrative or story [62]. Additionally serious games must
be as highly enjoyable as commercial computer games in order to enhance motivation and
thus learning outcomes [50]. Since the fun factor of serious games is not considered com-
petitive with commercial computer games for several reasons [50], it is not surprising that
in their intervention study Iten and Pekto [29] found no impact of enjoyment on measured
learning outcomes. Considering that enjoyment of the game is a relevant indicator of im-
mersion in an activity [63–65], we assume that the perceived level of enjoyment while
playing like2be was too low and therefore failed to promote a Flow experience. Further,
studies have shown that when players had an in-game opportunity to request scaffolding
[68] or feedback [51], learning processes with serious games were effectively enhanced.
Educ. Sci. 2023, 13, 16 17 of 20
Due to the fact that like2be does not provide in-game scaffolding or feedback, it was not
surprising that the data showed no impact on expansion of knowledge about occupations.
Furthermore, Wouters and colleagues [50] have shown that additional instructional ma-
terials such as reflection, modeling, collaboration, or personalization [50,67,69] can en-
hance the impact of serious gaming. Although, such additional teaching materials have
been developed and are available for like2be, their impact was not considered in the con-
text of this study.
On the positive side, our scale (final adjusted model, see Table 6) can be used for
future evaluations of the effectiveness of serious games, as it was found to be reliable and
valid after testing with confirmatory factor analysis (CFA).
In conclusion, the serious game like2be stimulates cognitive processes through its
competitive, challenging, and problem-oriented gameplay, consequently promoting ef-
fective learning. Thus, it represents an effective tool and should be considered for use in
career choice classes.
For the future, we recommend intensifying research in the field of the effectiveness
of serious games, especially in the field of vocational orientation. Regarding like2be, the
effect of the additional teaching material on cognitive, affective and motivational factors
conducive to learning should be analyzed. Furthermore, our data analysis shows that the
game can be optimized in terms of enjoyment, flow experience, or self-perceived benefits.
In this respect, like2be should be evaluated with a view to identifying opportunities to add
more fun, flow experience indicators, or a scaffolding component to the gameplay. With
regard to an optimization of like2be as well as the development of other serious games, we
therefore recommend a stronger collaboration between game developers, educational ex-
perts and the target groups of players. We consider educational experts (e.g., teachers or
educational scientists) as ideal contacts for developing specific scaffolding as well as an
adequate and exciting narrative. Nevertheless, we suggest asking the target groups pf
players (e.g., students) about which aspects make computer games exciting and appealing
for them. The findings can be incorporated into the development of new or the optimiza-
tion of existing serious games and thus generate a high degree of enjoyment and flow
experience.
Author Contributions: Conceptualization, C.K., E.M. and A.K.D.; methodology, C.K., E.M. and
A.K.D.; validation, C.K.; formal analysis, C.K. and A.K.D.; investigation, C.K., E.M. and A.K.D.; re-
sources, C.K. and E.M.; data curation, C.K.; writing—original draft preparation, C.K.; writing—re-
view and editing, C.K., E.M. and A.K.D.; visualization, C.K.; project administration, C.K. and E.M.
All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding
Institutional Review Board Statement: This study was reviewed and approved by the Ethics Com-
mittee and the Data Protection Officer of the University of Basel (date of approval 24 August 2021).
Informed Consent Statement: Informed consent was obtained from all subjects involved in this
study.
Data Availability Statement: The data are not publicly available due to international data sharing
agreements.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Yang, K.-H.; Chu, H.-C.; Chiang, L.-Y.; Effects of a progressive prompting-based educational game on second graders’ mathe-
matics learning performance and behavioral patterns. J. Educ. Technol. Soc. 2018, 21, 322–334.
2. Makarova, E.; Lüthi, J.; Hofmann, M.; Innovative Wege einer gendersensiblen Berufsorientierung. In Instrumente der Berufsori-
entierung; Pädagogische Praxis im wissenschaftlichen Diskurs; Brüggemann, T., Driesel-Lange, K., Weyer, C., Eds.; Waxmann:
Münster, Germany, 2017; pp. 239–251.
Educ. Sci. 2023, 13, 16 18 of 20
3. Breitlauch, L. Conceptual design for serious games regarding didactical and playfully requirements. In Applied Playfulness, Pro-
ceedings of the Vienna Games Conference 2011: Future and Reality of Gaming, Vienna, Austria, 21–23 October 2017, Wimmer, J., Mit-
gutsch, K., Rosenstingl, H.; Eds.; New Academic Press: Vienna, Austria, 2012; pp. 91–97.
4. Keller, C.; Doering, A.K.; Makarova, E.; The Potential of Serious Games to Foster Learning among Children and Adolescents
with Disabilities: A Systematic Review. Digit. Cult. Educ. 2021, 13, 6–36.
5. Wastiau, P.; Kearney, C.; Van den Berghe, W.; Games in School—How Are Digital Games Used in Schools? Full report. European
Schoolnet: Brussels, Belgium, 2009.
6. Keller, C.; Makarova, E.; Döring, A.K.; Förderung der Exploration im Berufswahlprozess mit Serious Games am Beispiel von
like2be. In Entwicklungen und Perspektiven in der Berufsorientierung—Stand und Herausforderungen, Weyland, U., Ziegler, B.,
Driesel-Lange, K., Kruse, A., Eds.; Barbara Budrich: Opladen, Germany, 2021, pp. 135–155.
7. Spangenberger, P.; Kapp, F.; Kruse, L.; Hartmann, M.; Narciss, S. Can a Serious Game Attract Girls to Technology Professions?
Int. J. Gend. Sci. Technol. 2018, 10, 253–264.
8. Spangenberger, P.; Draeger, I.; Kapp, F.; Kruse, L.; Matthes, N. Gendersensible Berufsorientierung mit einem Serious Game:
Didaktische Einbettung des Spiels „Serena Supergreen“. In Digitale Lernwelten—Serious Games und Gamification: Didaktik, An-
wendungen und Erfahrungen in der Beruflichen Bildung; Becker, W., Metz, M., Eds.; Springer Fachmedien Wiesbaden: Wiesbaden,
Germany, 2022; pp. 127–146.
9. Brown, S.D.; Lent, R.W.; Career Development and Counseling: Putting Theory and Research to Work, 2nd ed.; John Wiley & Sons:
Hoboken, NJ, USA, 2013.
10. Herzog, W.; Makarova, E. Berufsorientierung als Copingprozess. In Berufsorientierung: Ein Lehr- und Arbeitsbuch; Brüggemann,
T., Rahn, S., Eds.; Waxmann: Münster, Germany, 2020; pp. 73–82.
11. Holland, J.L. Making Vocational Choices: A Theory of Vocational Personalities and Work Environments, 3rd ed.; Psychological Assess-
ment Resources: Odessa, Ukraine, 1997
12. Swanson, J.L.; Schneider, M. Minnesota Theory of Work Adjustment. In Career Development and Counseling: Putting Theory and
Research to Work, Brown, S.D., Lent, R.W., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2013; pp. 29–53.
13. Super, D.E. A life-span, life-space perspective on convergence. In Convergence in Theories of Career Choice and Development: Impli-
cations for Science and Practice; Savickas, M.L., Lent, R.W.; Eds.; Consulting Psychologists Press: Santa Clara, CA, USA, 1994; pp.
63–74.
14. Savickas, M.L. Career Construction Theory and Practice. In Career Development and Counseling: Putting Theory and Research to
Work; Brown, S.D., Lent, R.W., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2013; pp. 147–183.
15. Gottfredson, L.S. Applying Gottfredson’s Theory of Circumscription and Compromise in Career Guidance and Counseling. In
Career Development and Counseling: Putting Theory and Research to Work; Brown, S.D., Lent, R.W., Eds.; John Wiley & Sons: Hobo-
ken, NJ, USA, 2005; pp. 71–127.
16. Hartkopf, E. Berufswahlreife und Berufswahlkompetenz—Zwei Schlüsselbegriffe der Berufswahlforschung und der Berufsori-
entierungspraxis aus psychologischer und pädagogischer Perspektive. In Berufsorientierung. Ein Lehr-und Arbeitsbuch; Brügge-
mann, T., Rahn, S., Eds.; Waxmann: Münster, Germany, 2013; pp. 42–57.
17. Kracke, B.; Schmitt-Rodermund, E. Adolescents’ Career Exploration in the Context of Educational and Occupational. In Navi-
gating through Adolescence: European Perspectives; Nurmi, J.-E., Ed.; Routledge: New York, NY, USA, 2001; pp. 141–165.
18. Neuenschwander, M.P.; Gerber, M.; Frank, N.; Rottermann, B. Schule und Beruf. Wege in die Erwerbstätigkeit; VS Verlag: Wiesba-
den, Germany.
19. Keller, C. Using the serious game like2be to effectively support adolescents in vocational orientation: A quasi-experimental
intervention study (manuscript submitted for publication).
20. Caillois, R. Man, Play, and Games; Free Press of Glencoe: New York, NY, USA, 1961.
21. Dempsey, J.V.; Haynes, L.L.; Lucassen, B.A.; Casey, M.S. Forty Simple Computer Games and What They Could Mean to Edu-
cators. Simul. Gaming 2002, 33, 157–168.
22. Zyda, M. From visual simulation to virtual reality to games. Computer 2005, 38, 25–32.
23. Klopfer, E.; Osterweil, S.; Salen, K. Moving Learning Games Forward; The Education Arcade: Cambridge, MA, USA, 2009.
24. Gotterbarn, D. Serious games: Learning why professionalism matters can be fun. ACM Inroads 2013, 4, 26–28.
25. Bogost, I. Persuasive Games; MIT Press: Cambridge, MA, USA, 2007; Volume 5.
26. Boyle, E.; Connolly, T.M.; Hainey, T. The role of psychology in understanding the impact of computer games. Entertain. Comput.
2011, 2, 69–74.
27. Felicia, P. The Use of Computer Games in Education. In Handbook of Research on Improving Learning and Motivation through Edu-
cational Games; IGI Global: Hershey, PA, USA, 2011; pp. 29–50.
28. Corti, K. Games-based Learning; a serious business application. Inf. De PixelLearning 2006, 34, 1–20.
29. Iten, N.; Petko, D.; Learning with serious games: Is fun playing the game a predictor of learning success? Br. J. Educ. Technol.
2016, 47, 151–163.
30. Pourabdollahian, B.; Taisch, M.; Kerga, E. Serious games in manufacturing education: Evaluation of learners’ engagement. Pro-
cedia Comput. Sci. 2012, 15, 256–265.
Educ. Sci. 2023, 13, 16 19 of 20
31. Cain, J.; Piascik, P. Are serious games a good strategy for pharmacy education? Am. J. Pharm. Educ. 2015, 79, 47.
32. Le Compte, A.; Elizondo, D.; Watson, T. A renewed approach to serious games for cyber security. In Proceedings of the 7th
International Conference on Cyber Conflict: Architectures in Cyberspace, Tallinn, Estonia, 26–29 May 2015.
33. Wilson, D.; Calongne, C.; Henderson, B. Gamification challenges and a case study in online learning. J. Online Learn. Res. Pract.
2015, 4, 26910.
34. Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in education: A systematic mapping study. J. Educ. Technol. Soc.
2015, 18, 75–88.
35. Willmott, T.; Russell-Bennett, R.; Drennan, J.; Rundle-Thiele, S. The Impact of Serious Educational Gameplay on Adolescent
Binge Drinking Intentions: A Theoretically Grounded Empirical Examination. Health Educ. Behav. 2019, 46, 114–125.
36. Winskell, K.; Sabben, G.; Ondeng’e, K.; Odero, I.; Akelo, V.; Mudhune, V. A smartphone game to prevent HIV among young
Kenyans: Household dynamics of gameplay in a feasibility study. Health Educ. J. 2019, 78, 595–606.
37. Byun, J.; Joung, E. Digital game-based learning for K-12 mathematics education: A meta-analysis. Sch. Sci. Math. 2018, 118, 113–
126.
38. Tokac, U.; Novak, E.; Thompson, C.G. Effects of game-based learning on students’ mathematics achievement: A meta-analysis.
J. Comput. Assist. Learn. 2019, 35, 407–420.
39. Talan, T.; Dogan, Y.; Batdi, V.; Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-
thematic analysis. J. Res. Technol. Educ. 2020, 52, 474–514.
40. Hussein, M.H.; Ow, S.H.; Elaish, M.M.; Jensen, E.O. Digital game-based learning in K-12 mathematics education: A systematic
literature review. Educ. Inf. Technol. 2022, 27, 2859–2891.
41. Chiu, Y.H.; Kao, C.W.; Reynolds, B.L.; The relative effectiveness of digital game-based learning types in English as a foreign
language setting: A meta-analysis. Br. J. Educ. Technol. 2012, 43, E104–E107.
42. Chen, M.H.; Tseng, W.T.; Hsiao, T.Y. The effectiveness of digital game-based vocabulary learning: A framework-based view of
meta-analysis. Br. J. Educ. Technol. 2018, 49, 69–77.
43. Tsai, Y.L.; Tsai, C.C. A meta-analysis of research on digital game-based science learning. J. Comput. Assist. Learn. 2020, 36, 280–
294.
44. Riopel, M.; Nenciovici, L.; Potvin, P.; Chastenay, P.; Charland, P.; Sarrasin, J.B. Un Impact of serious games on science learning
achievement compared with more conventional instruction: An overview and a meta-analysis. Stud. Sci. Educ. 2020, 55, 169–
214.
45. Hu, Y.; Gallagher, T.; Wouters, P.; van der Schaaf, M.; Kester, L. Game-based learning has good chemistry with chemistry
education: A three-level meta-analysis. J. Res. Sci. Teach. 2022, 59, 1499–1543.
46. Lei, H.; Chiu, M.M.; Wang, D.; Wang, C.; Xie, T. Effects of Game-Based Learning on Students’ Achievement in Science: A Meta-
Analysis. J. Educ. Comput. Res. 2022, 60, 1373–1398.
47. Huizinga, J. Homo Ludens; Routledge: London, UK, 1949.
48. Heimlich, U.; Einführung in die Spielpädagogik, 3rd ed.; UTB: Bad Heilbrunn, Germany, 2015.
49. Retter, H. Spielzeug: Handbuch zur Geschichte und Pädagogik der Spielmittel; Beltz: Weinheim, Germany, 1979.
50. Wouters, P.; van Nimwegen, C.; van Oostendorp, H.; van der Spek, E.D. A Meta-Analysis of the Cognitive and Motivational
Effects of Serious Games. J. Educ. Psychol. 2013, 105, 249–265.
51. Mao, W.; Cui, Y.; Chiu, M.M.; Lei, H. Effects of Game-Based Learning on Students’ Critical Thinking: A Meta-Analysis. J. Educ.
Comput. Res. 2022, 59, 1682–1708.
52. Jong, B.-S.; Lai, C.-H.; Hsia, Y.-T.; Lin, T.-W.; Lu, C.-Y. Using Game-Based Cooperative Learning to Improve Learning Motiva-
tion: A Study of Online Game Use in an Operating Systems Course. IEEE Trans. Educ. 2013, 56, 183–190.
53. Sailer, M.; Homner, L. The Gamification of Learning: A Meta-analysis. Educ. Psychol. Rev. 2020, 32, 77–112.
54. Gelsomini, M.; Garzotto, F.; Montesano, D.; Occhiuto, D. Wildcard: A wearable virtual reality storytelling tool for children with
intellectual developmental disability. In Proceedings of the 38th Annual International Conference of the IEEE Engineering in
Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016.
55. El Kah, A.; Lakhouaja, A. Developing effective educative games for Arabic children primarily dyslexics. Educ. Inf. Technol. 2018,
23, 2911–2930.
56. Khamparia, A.; Pandey, B.; Mishra, B.P.; Effects of microworld game-based approach on neuromuscular disabled students
learning performance in elementary basic science courses. Educ. Inf. Technol. 2020, 25, 3881–3896.
57. Gunn, K.C.M.; Delafield-Butt, J.T. Teaching Children with Autism Spectrum Disorder with Restricted Interests: A Review of
Evidence for Best Practice. Rev. Educ. Res. 2016, 86, 408–430.
58. Schneier, J.; Taylor, N. Handcrafted Gameworlds: Space-Time Biases in Mobile Minecraft Play. New Media Soc. 2018, 20, 3420–
3436.
59. Nebel, S.; Schneider, S.; Beege, M.; Kolda, F.; Mackiewicz, V.; Rey, G.D. You cannot do this alone! Increasing task interdepend-
ence in cooperative educational videogames to encourage collaboration. Educ. Technol. Res. Dev. 2017, 65, 993–1014.
60. Elliott, D. Levelling the playing field: Engaging disadvantaged students through game-based pedagogy. Lit. Learn. : Middle
Years 2014, 22, 34–40.
Educ. Sci. 2023, 13, 16 20 of 20
61. Kosmas, P.; Ioannou, A.; Retalis, S. Moving Bodies to Moving Minds: A Study of the Use of Motion-Based Games in Special
Education. Techtrends 2018, 62, 594–601.
62. Breien, F.S.; Wasson, B. Narrative categorization in digital game-based learning: Engagement, motivation & learning. Br. J. Educ.
Technol. 2021, 52, 91–111.
63. Fu, F.-L.; Su, R.-C.; Yu, S.-C.; EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Comput. Educ. 2009, 52,
101–112.
64. Csikszentmihalyi, M. Flow. The Psychology of Optimal Experience; Harper Perennial: New York, NY, USA, 1990.
65. Sweetser, P.; Wyeth, P. GameFlow. Comput. Entertain. 2005, 3, 3–3.
66. Wronowski, M.; Urick, A.; Wilson, A.S.P.; Thompson, W.; Thomas, D.; Wilson, S.; Elizondo, F.J.; Ralston, R. Effect of a Serious
Educational Game on Academic and Affective Outcomes for Statistics Instruction. J. Educ. Comput. Res. 2019, 57, 2053–2084.
67. Wouters, P.; van Oostendorp, H. A meta-analytic review of the role of instructional support in game-based learning. Comput.
Educ. 2013, 60, 412–425.
68. Chen, C.H.; Law, V. Scaffolding individual and collaborative game-based learning in learning performance and intrinsic moti-
vation. Comput. Hum. Behav. 2016, 55, 1201–1212.
69. Reinmann, G.; Didaktisches Handeln. Die Beziehung zwischen Lerntheorien und Didaktischem Design. In L3T. Lehrbuch für
Lernen und Lehren mit Technologien; Ebner, M., Schön, S., Eds.; epubli: Berlin, Germany, 2013.
70. Deutschschweizer Erziehungsdirektoren-Konferenz (D-EDK). Berufliche Orientierung. In Lehrplan 21: Deutschschweizer Er-
ziehungsdirektoren-Konferenz (D-EDK): Luzern, Switzerland, 2016; pp. 502–511.
71. Makarova, E.; Keller, C. Berufswahl von Schülerinnen und Schülern mit Serious Games fördern. Das Spiel like2be und seine
Wirksamkeit. Berufsbildung 2019, 14–16.
72. Kankainen, A.; Taskinen, S.; Oja, H.; On Mardia’s Tests of Multinormality; Birkhäuser: Basel, Switzerland, 2004; pp. 153–164.
73. STATA. Satorra–Bentler Adjustments. 2022. Available online: https://www.stata.com/stata14/sem-satorra-bentler/ (accessed on
26 November 2022).
74. Hu, L.T.; Bentler, P.M.; Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alterna-
tives. Struct. Equ. Model. : A Multidiscip. J. 1999, 6, 1–55.
75. Urhahne, D. Sieben Arten der Lernmotivation. Psychol. Rundsch. 2008, 59, 150–166.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual au-
thor(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.