Available via license: CC BY 4.0
Content may be subject to copyright.
Citation: Baptista, R.; Coelho, A.; Vaz
de Carvalho, C. Training and
Certification of Competences through
Serious Games. Computers 2024,13,
201. https://doi.org/10.3390/
computers13080201
Academic Editor: Wenbing Zhao
Received: 8 July 2024
Revised: 6 August 2024
Accepted: 8 August 2024
Published: 15 August 2024
Copyright: © 2024 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
computers
Article
Training and Certification of Competences through Serious Games
Ricardo Baptista 1,2,3,* , António Coelho 1,2 and Carlos Vaz de Carvalho 4
1
Department of Informatics Engineering, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal;
acoelho@fe.up.pt
2INESC TEC—INESC Technology and Science, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
3
Instituto Politécnico da Maia, Avenida Carlos de Oliveira Campos—Castêlo da Maia, 4475-690 Maia, Portugal
4GILT (Games, Interaction, Learning Technologies), Instituto Superior de Engenharia do Porto,
4200-072 Porto, Portugal; cmc@isep.ipp.pt
*Correspondence: ricardjose@fe.up.pt
Abstract: The potential of digital games, when transformed into Serious Games (SGs), Games for
Learning (GLs), or game-based learning (GBL), is truly inspiring. These forms of games hold immense
potential as effective learning tools as they have a unique ability to provide challenges that align
with learning objectives and adapt to the learner’s level. This adaptability empowers educators to
create a flexible and customizable learning experience, crucial in acquiring knowledge, experience,
and professional skills. However, the lack of a standardised design methodology for challenges that
promote skill acquisition often hampers the effectiveness of games-based training. The four-step
Triadic Certification Method directly responds to this challenge, although implementing it may
require significant resources and expertise and adapting it to different training contexts may be
challenging. This method, built on a triadic of components: competencies, mechanics, and training
levels, offers a new approach for game designers to create games with embedded in-game assessment
towards the certification of competencies. The model combines the competencies defined for each
training plan with the challenges designed for the game on a matrix that aligns needs and levels,
ensuring a comprehensive and practical learning experience. The practicality of the model is evident
in its ability to balance the various components of a certification process. To validate this method, a
case study was developed in the context of learning how to drive, supported by a game coupled with
a realistic driving simulator. The real time collection of game and training data and its processing,
based on predefined settings, learning metrics (performance) and game elements (mechanics and
parameterisations), defined by both experts and game designers, makes it possible to visualise the
progression of learning and to give visual and auditory feedback to the student on their behaviour.
The results demonstrate that it is possible use the data generated by the player and his/her interaction
with the game to certify the competencies acquired.
Keywords: serious games; competencies and skills; “in-game” assessment
1. Introduction
The global computer games industry holds immense learning potential. Games, now
marketed to all ages and genders, are increasingly being recognised as powerful tools for
learning (GBLs). They have been shown to motivate players, enhance problem-solving,
and facilitate collaboration and competition. These benefits are not limited to a specific
discipline or education level but apply across the board, from formal classrooms to non-
formal learning processes. This recognition of the potential of game-based learning is a
cause for optimism about the future of education.
There are already many studies about the effectiveness of Serious Games, both in
training and other activities [
1
,
2
]. Sousa and other authors [
3
] consider measuring their
actual effects on learning as one of the biggest challenges for accepting Serious Games
Computers 2024,13, 201. https://doi.org/10.3390/computers13080201 https://www.mdpi.com/journal/computers
Computers 2024,13, 201 2 of 29
as an effective educational method. Mayer [
4
] highlighted that few existing evidence-
based approaches to assessing their contribution to learning are still available. Various
frameworks have been developed relating to theories of instruction and learning to the
mechanics and elements of games, one of two distinct domains: one with the design
and development of Serious Games on distinct approaches like describing game-based
learning scenarios [
5
], a scenario generation framework for mission-based virtual training
from both the trainer and trainee’s perspective [
6
], a conceptual framework to design
Serious Games that have empathy as part of the learning outcomes [
7
], considering the
flow framework that uses the dimensions of flow experience to analyse the quality of
educational games [
8
], a framework of an evaluation-driven design which offers guidance
in the evaluation process [
9
], a Serious Games design framework in cultural heritage with
steps to follow during the whole process [
10
], and a conceptual framework based upon a
systematic literature review of developments in student-centred digital learning [11]. The
other domain is evaluation, which uses other approaches, such as the serious game design
assessment framework, as a constructive structure to examine purpose-based games [
12
];
a holistic approach to serious game evaluation with four key areas: theoretical, technical,
empirical, and external [
13
]; an interpretive evaluation framework that can identify the
educational value in COTS games [
14
]; and the dimensionalisation of game- based -learning
and further decomposition into factor/sub-factors based on theoretical constructs [15].
Yet, the current state of Serious Games and game-based assessments does not translate
into valid certifications. Students still need to prove their knowledge through traditional
evaluations. Despite games supporting learning, assessing the knowledge and skills ac-
quired this way needs significant improvement. This research aims to develop a design
and development process for Serious Games, considering that they integrate training, eval-
uation, and skills certification. The focus is on the potential to incorporate the evaluation
process and the consequent certification of competencies within the game context, governed
by specific norms that systematise student performance measurement.
Four key questions guide this research, each exploring a crucial aspect of the intersec-
tion between gaming and skills training:
1.
To determine whether a significant relationship exists between game genres and the
training of specific skills. By analysing different game types, we aim to uncover which
genres are most effective in cultivating particular competencies, thereby enhancing
the educational potential of Serious Games. Another proposition is to identify prac-
tical elements of serious game design for evaluating learning and training player
skills. As such, the research also explores the possibility of a competency certification
method using Serious Games, with game design considering learning objectives and
certification performance metrics.
2.
To investigate the elements of serious game design that best support the evaluation
of learning and training players’ skills based on structured competency frameworks.
This proposal examines how game mechanics, narratives, and feedback systems can
be optimised to engage players and assess and develop their abilities.
3.
To identify a robust method for certifying competencies using games. This proposal
entails creating a game design that aligns with learning objectives and incorporates
performance metrics that can reliably measure and validate players’ skills. By estab-
lishing clear criteria and standards, we can ensure the certification process is rigorous
and credible, reassuring us about its reliability.
4.
To understand the integration of Learning Analytics into the in-game assessment
system to precisely measure player performance in skills training. This proposal
involves leveraging data analytics to track and analyse player behaviour and out-
comes, providing detailed insights into their learning progress. By doing so, we can
enhance the accuracy and effectiveness of skill assessments, ensuring that players
receive meaningful and actionable feedback.
This study was able to answer the various questions posed, the answers to which
allowed the four-step Triadic Certification Method to be structured and supported by
Computers 2024,13, 201 3 of 29
new tools. With the successful integration of the evaluation process into Serious Games,
the results obtained made it possible to check the players’ performance in the various
training contexts, attesting to the validity of the training design, as well as validating
the skills acquired by visualising the progression of skill acquisition through the Triadic
Certification Method. This component of the method is particularly comprehensive and
well-rounded, as it conventionally designs SGs for the acquisition of competencies based
on a balance between the three elements: essential competencies (skills), mechanics (game),
and reality (training).
This paper has four sections. The first section presents a literature review on digital
games, genre taxonomies of games, Serious Games and their taxonomies, game-based
learning, and learning design assessment. In the second section, we discuss the training
and certification of competencies. The subsequent section focuses on the Triadic Certifi-
cation Method, a comprehensive approach developed to address the identified problem,
integrating training, evaluation, and skills certification within the game context. The final
section describes a case study on the training and certification of competencies in teach-
ing automobile driving and the results obtained. It serves as a proof of concept for the
developed method, demonstrating its practical application and effectiveness.
2. Digital Games and Game Based-Learning (GBL)
In this part, we endeavour to elucidate the significance of games within educational
contexts and their consequential impact on human behaviour, drawing from constructivist
perspectives. We commence by delineating fundamental concepts such as “game” and
“play”. A game is a purposeful competition governed by rules wherein players strive for
victory. Conversely, play encompasses many intentional activities, often undertaken for
recreational or leisurely purposes.
Johan Huizinga’s conceptualisation of the “magic circle” accentuates the discrete space
wherein game-related activities manifest devoid of real-life repercussions, underscoring
the immersive nature intrinsic to gameplay. This concept has proved pivotal in elucidating
the symbiotic relationship between players and games, shaping discourse across digital
and traditional gaming paradigms [16].
Scholars including Huizinga [
16
], Caillois [
17
], Juul [
18
], and Salen and Zimmer-
man [
19
] underscore the inherent allure of games to players, influenced by variables such
as age, cognitive aptitude, and individual personality traits. Game designers leverage these
factors to augment player engagement by iteratively adjusting goals, rules, challenges, and
participant dynamics.
Contrary to prevalent misconceptions, games are structured environments imbued
with clearly delineated objectives, adversaries, and regulatory frameworks. They thereby
afford players opportunities for cognitive stimulation and skill refinement. Nonetheless,
unlike real-world scenarios, games have distinct consequences for successes and failures,
contributing to their intrinsic allure and divergence from reality [20].
Games epitomise rule-bound activities characterised by delimited beginnings, mid-
dles, and denouements. They present players with cognitive challenges necessitating
proactive engagement. While games ostensibly simulate real-world scenarios, they deviate
in outcome predictability and repercussions, thereby furnishing distinct pedagogical and
experiential paradigms for players [21].
2.1. Game Genre Taxonomies
The defining characteristic of computer games lies in the interactive pattern estab-
lished between the player and the game environment. Video games are categorised into
genres primarily based on their patterns of gameplay interaction rather than their visual
or narrative elements [
21
]. However, the taxonomy of game genres has been a contention,
with numerous proposals in the field [
22
,
23
]. These genre classifications organise games
into distinct categories defined by their underlying gameplay mechanics.
Computers 2024,13, 201 4 of 29
Understanding game genres is crucial for game designers as it enables them to align
additional content, such as new levels or characters, with established gameplay mechanics
like jumping or shooting in a platformer game. It also allows them to innovate existing
ones, for instance, by introducing a new gameplay mechanic like time manipulation in a
puzzle game [24].
Game genres are more than just categories—they are a unifying force. The primary
factor uniting games within a genre is the similarity in the interactions facilitated between
the player and the game environment. These interactions manifest through various game-
play mechanics, encompassing the actions of in-game objects and players throughout
gameplay [
25
]. It is these recurrent actions or challenges that ultimately define the genre of
a game, creating a sense of belonging and connection among players and designers alike.
While there is no universally standardised taxonomy of video game genres, the indus-
try’s recognition is a validation of their importance. The industry commonly recognises
several overarching categories. These typically include action, strategy, role-playing, sports,
management simulation, adventure, puzzle, and quiz genres, acknowledging the diversity
and significance of each [24,26–28] (Table 1).
Table 1. Taxonomy of game genres.
Game Genre Goals Sub-Genre
Action
To overcome mental or physical challenges against one or more
opponents by engaging in a series of actions (timing—reaction
speed, in which accuracy may be emphasised). Realism is not
relevant.
Beat-’em-ups Beat ’em ups, Shooter
games (1st and 3rd person), Platform
games
Strategy
Deploy tactics/strategies to overcome complex challenges against
one or more opponents by planning a superior series of actions
(physical challenges are not emphasised).
4X (eXplore, eXpand, eXploit, eXtermine),
Real-time strategy games, Real-time
tactics, Turn-based strategy, War games
Role-Playing
Victory is achieved through superior planning or out-thinking the
opponents (physical challenges and chance take a more minor
role). Distinct from action games, RPGs seldom test a player’s
physical skill (combat is more tactical than physical) and involve
other non-action gameplay (resource management).
RPGs, MMORPGs
Sports
Similar to action games, except for the realism of movements and
techniques, which are very important. Exergames, Sports/management games
Management
Simulation
To overcome economic challenges, a series of actions must be
planned. Direct action upon an opponent is not emphasised.
They are typically designed to be never-ending (no-win scenario).
Goal example: to build a collection of objects.
Racing games/Vehicle, Virtual
worlds/Pets, Life simulation/social
games, Business
Adventure
To use an avatar for the exploration of an interactive story and to
overcome challenges in isolation (puzzle adventure) by planning
a superior series of actions (physical challenges are
not emphasised).
Graphics Adventure, Puzzle adventure
Puzzle
To overcome mental challenges in isolation (not around a conflict
with another opponent) by planning a superior series of actions.
Games usually involve shapes, colours, or symbols that the player
must directly or indirectly manipulate into a specific pattern.
Action/Arcade puzzle (timed), Reveal
the picture game, Physics game.
Quiz Gamepad controlled, mouse keyboard, Wii balance board.
However, it is essential to note that this classification is not exhaustive, and numerous
hybrid genres exist that blend elements from multiple established categories. Moreover, the
continuous technological advancements within the gaming industry constantly give rise
to new genres, particularly with the introduction of novel platforms or input devices. For
instance, the advent of Nintendo’s Wii console spurred the emergence of “physical” games
like Wii Fit, exemplifying the industry’s ongoing evolution and diversification of genres.
Computers 2024,13, 201 5 of 29
2.2. Serious Games
Serious Games (SGs) are not just a trend in corporate settings and research communi-
ties; they are a transformative tool. This game definition harnesses the engaging features of
video games to make learning processes not just bearable but exciting [
17
,
18
]. With SGs,
players can learn while playing, a concept revolutionising education and training. These
games are designed to engage players with specific topics, effectively teaching educational
content or training workers to perform particular tasks. This transformative power of
Serious Games inspires educators and researchers in their work.
SGs refer to digital games used for purposes other than entertainment, such as training,
advertising, simulation, or education. Intentionally, these games help learning, skills
acquisition, and behaviour change through a game design process that focuses on achieving
learning outcomes through gameplay [
4
,
21
,
29
,
30
]. Clark C. Abt [
31
], a pioneer in the field,
introduced the concept of Serious Games in his book. This marked a shift in the perception
of games, extending their meaning beyond mere entertainment when used or embedded in
a specific context. Serious Games, as he defined them, are not primarily for amusement
but have an explicit and carefully thought-out educational purpose. Since the first Serious
Games initiative sponsored by Woodrow Wilson in 2002 and the Serious Games Summit
in 2004, there has been significant growth in game-based learning. Serious Games are
interactive computer applications with a challenging goal, are fun to play and engage,
incorporate some concept of scoring, and impart skills, knowledge, or attitudes that can be
applied in the real world.
The concept of Serious Games still lacks a precise definition, with some authors using
other terms like immersive learning simulations, digital game-based learning, gaming
simulations, and “games you have to play” [
32
,
33
]. The main goal of Serious Games is to
provide an interactive means for the transference of knowledge to the player. One main
goal of Serious Games is to provide an interactive means for transferring knowledge to
the player.
SGs educate or train the player, contain a direct means of assessing a skill or learning,
and employ a game interface that provides these features. They are considered a new tool
within the active learning paradigm. If game design focuses on learning outcomes, learning
becomes a natural consequence of playing.
Serious Games have many applications, from government and corporate training to
health, public policy, and strategic communication [
34
]. Despite their diverse uses, the
primary focus of SGs remains their educational purpose [
29
,
35
]. They are designed to
facilitate learning and training and to apply new pedagogies. Research has shown that SGs
can accelerate learning, increase motivation, and support the development of higher-order
cognitive thinking skills [
5
]. This diverse range of applications intrigues game developers
and professionals in the field.
The key to the success of these games is motivation. It is a psychological process that
stimulates an individual to act upon something to attain a desired effect or goal. Motivation
in learning can be affected by intrinsic motivation [
36
], extrinsic motivation [
37
,
38
], and
emotional stability. These motivational factors should be considered when designing and
developing Serious Games. However, due to the broad range of individuals’ emotional
stability, it may take much work to address it.
Serious Games (SGs) are not limited to a single field; they have distinct classifications
that cater to various purposes. These games are used in government, defence, education,
corporate, and industry settings, highlighting their versatility. They address aspects such
as occupational safety, skills, communications, and orientation, proving that Serious Games
can be applied in various scenarios. Examples of SGs include Alcoa SafeDock, Rosser
Surgery Skills w/Games, Shield of Freedom, America’s Army, Darfur is Dying, and Tactical
Language & Culture.
Several attempts have been made to classify Serious Games into genres or similar
typologies. The criteria used to classify the games vary greatly, with the most commonly
used being the educational content and field of application of Serious Games. Michael
Computers 2024,13, 201 6 of 29
and Chen [
29
] name military, government, educational, corporate, healthcare, political,
religious, and art games. This typology solely bases itself on the application areas of the
games, showcasing the diverse range of Serious Games. For instance, in the health games,
Susi et al. [
39
] list the subgroups of exergaming [
40
], health education [
41
], biofeedback [
42
]
and therapy [43].
Sawyer and Smith [
44
] suggest a higher-resolution taxonomy that crosses game and
learning types with application areas. They list advergames, games for work, or games
for health as game types. The core innovation of Sawyer and Smith was to separate the
designed purpose from actual application areas. Based on the promising work of Ratan
and Rittefeld [
45
], we would propose the following label categories, which allow for the
inclusion of specifically designed Serious Games as well as COTS games for “serious”
purposes, as the following Table 2presents:
Table 2. Categories for classifying serious games.
Label/Tag Category Exemplary Labels
1. Platform Personal Computer, Sony Play Station 3, Nintendo Wii, Mobile Phone
2. Subject Matter World War II, sustainable development, physics, Shakespeare’s works
3. Learning goals Language skills, historical facts, environmental awareness
4. Learning principles Rote memorisation, exploration, observational learning, trial and error, conditioning
5. Target audience High school children, nurses, law students, the general public, preschoolers, military recruits
6. Interaction mode(s) Multiplayer, co-tutoring, single-player, massively multiplayer, tutoring agents
7. Application area Academic education, private use, professional training
8. Controls/interfaces Gamepad-controlled mouse keyboard, Wii balance board.
9. Common gaming labels Puzzle, action, role-play, simulation, card game, quiz
2.3. Game Assessment
Game assessment is a critical component of the learning process. It involves comparing
the expected learning goals with the evidence obtained from the learning actor. In the con-
text of digital game-based learning (DGBL), this evaluation is typically conducted through
traditional methods such as questionnaires, interviews, log file analysis, or observation
of experience [
46
]. However, new technologies and media are emerging to support more
advanced evaluation tools, addressing the current gaps in evaluation.
Over the past decades, research has aimed to develop new approaches that support
the evaluation paradigm in game-based learning. This analysis is divided into two contexts:
measuring learning and incorporating the evaluation component in the game development.
2.3.1. Assessment of Digital Learning
The assessment of the game’s learning experience is primarily carried out outside
the game’s context, using external tools such as Acumen Team Skills Assessment and
Profiles Team Analysis [
47
]. These tools attest to the results obtained in higher education,
management, and personal and team skills development.
The literature refers to three robust theoretical frameworks that underpin the evalua-
tion of learning in Serious Games: RETAIN (Relevance, Embedding, Transfer, Adaptation,
Immersion, and Naturalization) [
48
], Kirkpatrick’s levels of evaluation [
49
], and the CRESST
learning model [
50
]. These frameworks provide a solid foundation for developing and
evaluating learning games, ensuring they effectively incorporate educational content. This
thorough evaluation process of Serious Games gives educators and researchers confidence
in the effectiveness of this tool.
First, the RETAIN framework supports learning games’ development and evaluates
how they contain and incorporate educational content. It aims to identify the best combina-
tion between the various game elements, associating them with the genre taxonomy. This
Computers 2024,13, 201 7 of 29
framework’s relevance to this work is related to the analysis of the conceptualisation of the
relationship between the components of games—the elements and genres of games—and
the objectives and levels of learning competencies. The educational potential of games
depends on the coherence between different elements using distinct levels of learning
conceptualisation and assessment. These models join the curriculum and motivational
aspects of their design. The models of learning conceptualisation include Bloom’s taxon-
omy, which corresponds to a taxonomy of educational objectives, dividing the learning
into three main objectives: to generate skills, to develop competencies, and to transfer
knowledge in three distinct domains: cognitive, behaviour, and aptitude; and Gagné’s
Events of Instruction [
33
,
51
], which propose motivational events as positively influencing
the achievement of the expected results. Involvement with experience does not derive
from a hierarchy of events but rather from the assumption that what goes on inside and
outside the game learning experience are lines of and many elements in a single event can
be combined or interconnected. This approach, in particular, is a structuring example in
terms of the design of the structure of events in the gaming experience for this study, which
focuses on skills learning. The closer it is to reality, the more involvement it brings to the
transfer of knowledge after learning. Finally, Keller’s ARCS Model [
52
] states that moti-
vation in student learning corresponds to a systematic process represented by four steps,
within which motivation can be achieved or promoted. These models aim to maximise the
potential of educational situations by choosing the most appropriate combination of factors
to incorporate into the game’s development.
The second model for conceptualising evaluation is Kirkpatrick’s four levels of as-
sessment: Reaction, Learning, Behaviour, and Results. This model presents a hierarchy of
levels for evaluating learning or training programmes. The transition between levels plays
a vital role in the evaluation process, adding value to the information collected. However,
the process becomes more complex and time-consuming. Each evaluation level produces
expected results, but this methodology must be applied for a correct analysis [
49
]. This
framework is widely used in training evaluation due to its structure, aligning with the
learning aptitudes and competencies cycle.
The CRESST framework by Baker and Mayer [50] comprises five fundamental cogni-
tive requirements for learning: content comprehension, problem-solving, self-regulation,
communication, and collaboration/teamwork. This model focuses on the actions consid-
ered for each group, which will be tested to validate the expected results.
In summary, the three frameworks presented reflect the domains of learning evaluation
but still need to improve their convergence with technologies. Evaluation in game-based
learning (GBL) must consider a hierarchical set of needs (Bloom Taxonomy), expected
results (the four levels of Kirkpatrick’s assessment), and cognitive requirements (CRESST
model). The learning environment must always be considered, as it must promote student-
friendly involvement. For successful learning, the experience should increase students’
motivation in the game process through tactics (Keller’s ARCS model) and/or events
(Gagné’s Nine Events of Instruction model), focusing on results.
2.3.2. Evaluation Design in Serious Games
The evaluation design in Serious Games involves several conceptual frameworks that
apply various evaluation models integrated into the game development process.
The first is the Evidence-Centred Assessment Design (ECD) framework [
53
–
56
], which
posits that evaluation results from evidence analysis functioning in a triangular interaction.
The keys to a balanced triangular interaction are cognition (theory and information about
how students learn), observation (student task performance can demonstrate their learning),
and interpretation (used to draw inferences from observations).
Another evaluation framework is the structure of De Freitas and Oliver’s four dimen-
sions [
4
,
57
,
58
]: context, student specification, modes of representation, and pedagogical
principles. These dimensions allow for the evaluation of game-based learning and simula-
tions and should be considered interactive, each containing key characteristics. Harteveld
Computers 2024,13, 201 8 of 29
and other authors [
59
–
61
] propose another model that contends that the design fundamen-
tals of any game have a serious purpose.
This model is in continuity with the structures presented previously, where the balance
between the forces exerted by the three intervening areas (pedagogy, game elements, and
reality) in the design or use of educational games is key. This model has three nuclear
pillars: Play Space, Meaning, and Reality. Another structure developed for designing
and developing educational games is the “Design, Play, and Experience Framework”
(DPE)
[62–64]
. Its main objective is to describe the relationship between the designer and
the player as a mediated experience to achieve the expected results through the game.
This model is supported by three pillars, each of which contributes to the game’s design
according to the phase/level of the game and the type of player.
When the “DPE Framework” is applied to Serious Games, it expands with another
set of layers related to the specificity of the design of these types of games (learning,
storytelling, gameplay, and user experience), which are transversal to the three components
of the structure (Design, Play, and Experience).
Two more models in the context of Serious Games design are the “Experimental
Gaming Model” and the EFM Model (effective learning environment, flow experience,
and motivation). These models emphasise the experience in the game context, focusing
on goals such as motivation, the player’s learning experience, and other emotional or
affective aspects.
According to several researchers [
65
–
67
], the “Experimental Gaming Model” presents
the learning process as circular, based on constructing cognitive schemes through activities
within the game environment. The direct interactions between players and their experiences
with the environment create a circular learning mechanism that includes all the necessary
steps to ensure the success and achievement of the objectives. The principal elements
of an educational game should be contained in the scenario that will define the learning
objectives. Feedback is crucial in providing insight into the acquired knowledge and
evaluating the player’s performance. According to Song and Zhang [
68
] and Hussein [
69
],
the EFM model suggests clever design practises to inspire motivation and help learners
genuinely learn from the game. It proposes ideas for developing games with effective
learning environments where students develop increased motivation during the experience
flow. An effective learning environment supports seven basic requirements by presenting
specific tasks with clear objectives and appropriate challenges while achieving a high
degree of interaction and feedback. The model includes two distinct levels: a group of
nine components of the flow of experience, subdivided into three categories (conditional,
experience, and results), and another group of strategies with four essential components
(relevance, trust, satisfaction, and attention) to stimulate motivation.
2.3.3. Final Considerations on the Design and Evaluation of Learning with Games
Despite the existing research and diverse approaches to applying evaluation in game
design and development, the use of assessment in game-based learning has yet to gain full
recognition for its role in the success of this learning approach.
The referenced models aim to associate the assessment process with students’ gains in
game-based learning. They highlight how various elements of games contribute to practical
evaluation within the games themselves. Key aspects of the game, such as the interface,
play environment, narrative, mechanics, and student motivation and involvement, are
crucial in this process.
Another critical aspect of the evaluation process is its stakeholders’ respective roles.
Evaluators face new challenges as they are expected to collaborate and assist in the evalua-
tion process and in understanding and identifying the entertainment elements of the game.
Evaluating knowledge acquisition and transfer through games focuses on the need
to hierarchise learning to achieve expected results. However, learning is not seen merely
as a sequence of goals achieved through a gaming experience. Though still performed
traditionally (i.e., summative tests), the evaluation now considers convergence models. Mo-
Computers 2024,13, 201 9 of 29
tivation, a key element in the decision to carry out activities and corresponding knowledge
acquisition, is highlighted in various models.
According to Conole [
70
,
71
] and Wills and others researchers [
72
], there is a need
for greater convergence between the role of technology and its impact on evaluation. In
game-based learning, new models must be developed that allow us to explore and take
advantage of the success of the gaming experience. These issues are very important for
future research, especially in the methodologies for designing Serious Games.
3. Training, Competences, and Certification
Training and education are the same in definition but slightly different in context.
Both are actions associated with acquiring competencies (knowledge, skills, and attitudes).
However, education plays a crucial role in workplace learning, where learning is developed,
preferably in the workplace, to improve the performance of employees. They gain practise
with tools, equipment, and other elements that can be used daily. Complementing this is
another concept: certification. An initial definition of this concept is the validation of the
skills that the individual has achieved after the training programme, and this issue will be
discussed later in this chapter.
Training is associated with many contexts and is a planned learning experience that
ensures permanent change in individual knowledge, attitudes, or skills. The meaning of
this learning corresponds to improving the individual’s performance to achieve a certain
level of knowledge or skill through the organised transmission of information and/or
guided instructions.
Author Michael Armstrong [
73
] reinforces the idea of performance associated with
training by stating that it is a systematic development of the knowledge, skills, and attitudes
required of an individual to perform a particular task or job. Similarly, Edwin Flippo [
74
]
states that training increases an employee’s knowledge and skills to perform a specific or
particular job. Finally, the author Aswathappa [
75
] defines the concept as improving skills
and attitudes, where training contributes to updating old skills and developing new ones.
Training, as a systematic process, must be directed in such a way as to achieve the
expected benefits. We can characterise a training system (programme) in four phases: (1)
assessment of training needs; (2) design of training programmes; (3) implementation of the
training programme; and (4) evaluation of the training programme.
We can conclude that skills training reflects a programme whose structured approach
corresponds to an individual’s training needs to achieve specific results. When the training
is completed, the evaluation is carried out on-site, at work, or in the context in which the
task is carried out to verify whether or not the acquisition of desired knowledge, skills, or
attitudes is necessary.
Building on this understanding, certification is a voluntary process that precedes the
on-site verification of competencies (assessment). It is a powerful tool for professionally
recognising knowledge, skills, and other practises [
76
]. According to the authors Byrne,
Valentine, and Carter [
77
], the act of certification, when aimed at validating a more advanced
level of knowledge and practise, is a formal procedure that allows an individual or an
accredited/authorised entity to assess, verify, and attest, in writing and by issuing a
certificate, to the attributes, characteristics, quality, and/or other aspects related to the
status of individuals or organisations, procedures, or pre-processes, which are following
established requirements or standards [78,79].
Acquiring a certification signifies that the individual’s competencies and attitudes
gained endure. This enduring nature, coupled with benefits such as personal development,
career progression, financial reward, professional recognition, and perceived empower-
ment [
80
], underscores the value and security of the investment in training and certification.
Finally, training is an organised activity that imparts information or instructions to
improve performance or help individuals attain the required knowledge and skills. In
training, it is essential to distinguish between competency and competence. The former
refers to an individual’s ability to make deliberate choices from a repertoire of behaviours
Computers 2024,13, 201 10 of 29
in specific professional contexts. The latter is context-dependent and involves integrating
knowledge, skills, judgement, and attitudes. Competences are “domain-specific cognitive
dispositions that are required to cope with certain situations or tasks successfully and
acquired by learning processes” [81].
Different types of competencies can be considered in organisations or specific fields.
First, we have personal competencies, representing the core knowledge, skills, and attitudes
each person should have for superior performance. Next, functional competencies are
related to specific technical knowledge within a particular area or profession. Finally, task
competencies are implicit and associated with specific role functions.
Regarding skills, “skill” is often preferred over “competence” in the training and
work environment. Hard skills encompass specific technical abilities or solid factual
knowledge required for a job, such as machine operation, programming languages, and
safety standards. These skills are typically trainable and easy to observe, quantify, and
measure. On the other hand, soft skills (also known as “people skills”) are more subjective
and are associated with personal attributes and character. Soft skills are essential for
applying technical skills in the workplace, including communication, teamwork, problem-
solving, and time management. These skills are more challenging to observe and quantify.
3.1. Reference Structures for Skills and Competences
This section aims to present various frameworks used to prepare and recognise the
skills that are being learned. The skills matrix is a tool for assessing the skills needed to
achieve maximum impact and locating where these skills can be found. In this way, the skills
framework is a structure that establishes and defines each skill (such as problem-solving or
people management) required of people who work in or are part of an organization.
This matrix/framework can take several approaches, most notably when recruiting
employees by aptitude standard and performance appraisal or identifying the aptitudes
required to perform an activity in any given role. A matrix can, therefore, be considered an
inventory of skills categorised by level, with a given required/chosen level of skills. This
matrix results in what can be learned (skills) and the quantification (points) required to
acquire and improve a skill.
3.1.1. European Qualifications Framework
The recognition of qualifications in Europe, more specifically in the European Union,
is carried out through the European Qualifications Framework (EQFs) (https://wwwcdn.
dges.gov.pt/sites/default/files/brochure_eqf_en.pdf (accessed on 7 July 2024)), which acts
as a standard reference system to link all national qualifications systems. For each of the
eight levels defined, there is a set of indicators that specify the expected learning outcomes
corresponding to the qualifications of that level in any qualifications system, covering
several education levels (primary, secondary and higher education, vocational training) [
82
]
as well as the processes of recognition, validation, and certification of competences obtained,
whether by non-formal or informal means [83].
This approach is based on learning outcomes, with eight reference levels defining
what is necessary and sufficient for each student to know, understand, and be able to
achieve after completing the learning process. These criteria are defined in terms of
knowledge as the result of assimilating information during learning; aptitude, as the ability
to apply knowledge and know-how to complete tasks and solve problems; and attitude,
as the proven ability to use knowledge; skills; and personal, social, and methodological
competences in a work or study context and professional and personal development [84].
3.1.2. Lominger ’s Competency Models and Education Competencies: A Comprehensive
Approach
A competency model is a comprehensive framework that outlines the behaviours
employees must exhibit to achieve success in their roles or perform specific tasks effectively.
Unlike job descriptions, which enumerate the tasks and responsibilities associated with a
Computers 2024,13, 201 11 of 29
particular position, competency models delve deeper by elucidating how employees should
perform their duties. While job descriptions provide a list of tasks and functions required
for a role, competency models identify the requisite behaviours, skills, and knowledge
essential for executing those tasks proficiently.
Lominger’s sixty-seven competencies have emerged as a universal standard for achiev-
ing task success. Known as the Leadership Architect Competencies, this assessment tool
enables us to compile a comprehensive list of competencies by combining existing models.
The goal is to encapsulate the essential skills for success in various contexts [
85
]. This
competency model represents “a collection of competencies associated with successful
performance” [
86
]. To apply it specifically to education and training, the same authors
collaborated with Microsoft to create a similar approach known as Education Competencies
or the Educational Competency Wheel [
87
]. This tool encompasses various attributes,
behaviours, knowledge areas, and abilities for effective job performance.
The competency table, as depicted in Table 3, comprises six core skill sets and per-
sonality characteristics. These include individual excellence (IE), organisational skills
(OrSs), courage (C), results (Rs), strategic skills (SSs), and operating skills (OpSs). While
these categories initially draw from Lominger’s standard set of 44 competencies, they
can be extended beyond education to other domains, such as competency training. The
competency wheel offers additional resources to identify core competencies critical for an
organization’s success [
88
]. These resources include clear definitions, proficiency levels,
sample interview questions, and activities aimed at skill development, all geared toward
helping organisations achieve their goals.
The six qualities or success factors can be categorised into two main types: hard and
soft skills. Hard skills are teachable abilities or skill sets that lend themselves to quantifica-
tion. In contrast, soft skills are more subjective and challenging to measure. Among the
core skill sets, we can consider individual excellence, courage, results, and strategic skills
to be soft skills. These enable effective collaboration, direct communication, goal-oriented
action, and pursuit of longer-term objectives. On the other hand, operating skills and
organisational skills fall into the hard skills category. They encompass the practical skills
for daily task management, relationship building, and effective communication across
diverse organizational contexts. With this restricted and adequate number of core compe-
tencies, the mapping aligns with the technical and personal competence needs based on
the expected results.
Comprising 37 competencies referred to as success factors, this set of categories, while
aligning with the Lominger matrix, is not limited to formal education. Its versatility
extends to areas like skills training and shares striking similarities with other performance
standards, such as the Baldrige Education Criteria for Performance Excellence, defined
by the International Society for Technology in Education and the National Standards for
Educational Technology [
89
]. Another important aspect is that this competency standard is
recognised by UNESCO, whose general competencies are ICT-related competencies [
84
,
90
]
and application competencies [88].
To conclude this section about the competency standards, it is essential to differentiate
between certification and qualification as they carry distinct meanings. According to the
European Qualifications Framework (EQFs), qualification represents the formal outcome
of an assessment. In contrast, certification involves a validation process conducted by
a competent body to determine whether an individual has achieved specific learning
outcomes according to a predefined standard.
Numerous international and national standards govern professional certification.
Notably, the ISO/IEC 17024 standard, developed by the International Organization for
Standardization (ISO) and the International Electrotechnical Commission (IEC), specifies
requirements for certification bodies [
91
]. These standards apply independently of any
specific area of expertise. The European Community has also adopted ISO/IEC 17024.
In the United States, the National Organization for Competency Assurance (NOCA) has
Computers 2024,13, 201 12 of 29
established standards and an accreditation process for certification programmes since the
late 1970s.
Table 3. Educational Competency Wheel [87].
Educational Success Factors
Individual
Excellence (IE)
Organisational
Skills (OrSs) Courage (C) Results (Rs) Strategic Skills
(SSs)
Operating Skills
(OpSs)
Building Effective
Teams (IE1)
Comfort Around
Authority (OrS1)
Assessing Talent
(C1)
Action Oriented
(R1) Creativity (SS1)
Developing Others
(OpS1)
Compassion (IE2)
Organisational)
Organisational
Agility (OrS2)
Conflict
Management (C2)
Drive For Results
(R2)
Dealing with
Ambiguity (SS2))
Directing Others
(OpS2)
Customer Focus
(IE3)
Presentation Skills
(OrS3)
Managerial
Courage (C3)
Decision Quality
and Problem
Solving (SS3)
Managing and
Measuring Work
(OpS3)
Humour (IE4)
Written
Communications
(OrS4)
Functional /
Technical Skills
(SS4)
Managing
Through Processes
Systems (OpS4)
Integrity and Trust
(IE5)
Intellectual
Acumen (SS5) Organising (OpS5)
Interpersonal Skills
(IE6)
Learning on the
Fly (SS6) Planning (OpS6)
Listening (IE7)
Strategic Agility
and Innovation
Management (SS7)
Priority Setting
(OpS7)
Managing
Relationships (IE8)
Technical Learning
(SS8)
Time Management
(OpS8)
Managing Vision
and Purpose (IE9)
Timely
Decision-Making
(OpS9)
Managing Vision &
Purpose (IE9)
Motivating Others
(IE10)
Negotiating (IE11)
Personal Learning
and Development
(IE12)
Valuing Diversity
(IE13)
A professional certification effort, a journey of empowerment, involves three relatively
independent dimensions. Firstly, the professional role characterisation includes defining
the specific professional role to be certified. Secondly, the list of required abilities and skills
to identify the abilities and skills necessary for professionals in that role. Finally, the de-
scription of the certification process outlines the certification process and its organizational
aspects, all designed to empower professionals in their respective roles.
ISO/IEC 17024 is a crucial standard for individual certification. It is a benchmark for
recognising certification bodies and their national and international certification schemes.
This standard plays a pivotal role in defining the certification process, encompassing all
activities through which a certification body establishes that a person fulfils specified
competence requirements.
Computers 2024,13, 201 13 of 29
The competence certification system, a beacon of recognition, is a powerful tool
that enables professionals working in the labour market to gain recognition based on
their qualifications. By achieving specific parameters, professionals demonstrate their
competency, empowering them to showcase their skills and knowledge. For instance,
the EQFs assesses whether an individual has acquired learning outcomes aligned with
relevant standards. The validation process involves four phases: identification, where
dialogue is used to identify an individual’s experiences; visibility, which consists of making
these experiences visible through documentation; formal assessment, which evaluates the
experiences formally; and recognition, which leads to certification for partial or complete
qualifications. This process focuses on assessing the skills and knowledge demonstrated by
learners in specific tasks, ensuring their competency applies to real-world scenarios.
4. Triadic Certification Method
Creating virtual environments conducive to learning through games represents a
significant milestone. These environments guarantee success in acquiring knowledge
and experience for players and students, driven by motivational and engaging elements.
Previous literature reviews have highlighted the characteristics and strategies contributing
to successful game learning.
While individuals can be trained to achieve expected results in various situations and
contexts, assessing skills remains challenging. Specifically, further progress is required
to certify the knowledge and skills acquired during the learning processes conducted
through SGs.
This research addresses the development of new methods to maximise the benefits of
successful learning through games. To achieve convergence, we integrate certification of
competency training. Unlike assessing learning, this approach validates the knowledge
and skills acquired for professional functions or activities.
The Triadic Certification Method (TCM) aims to incorporate certification into SG devel-
opment (from conception to design and implementation). This method involves four steps
that influence game design, ensuring elements necessary for certification success. Commu-
nication between key stakeholders—the trainer/instructor and the designer—guarantees
fundamental decisions regarding SG functionality.
The method, a testament to its versatility, is not restricted to game taxonomy and
applies universally regardless of game type; it applies to any training context and adapts
to diverse training scenarios. Skills acquired during training levels align with proficiency
levels, reflecting the learning state. Certification occurs only when all defined competencies
are successfully trained, reassuring professionals of its comprehensive applicability.
To address the research question about integrating certification into game development,
we must embed the context of training and competency certification within the game
design process. This inclusion necessitates rethinking the entire SG development chain.
Additionally, we introduce a new team member—the instructor/coach—who defines skills
and competencies. The instructor actively contributes to specifying contextual elements,
such as characteristics, missions, specific objectives, and expected learning outcomes.
4.1. Relationship between Game Taxonomy and Competencies Development Survey
This research significantly defines the correlation matrix between game taxonomy
and competencies. The effectiveness of training-based games, especially Serious Games,
hinges on their ability to provide challenges that facilitate the acquisition of knowledge,
experience, and professional aptitudes. However, there is no ideal design methodology to
support this process.
A critical piece of information for game designers aiming to adapt mechanics for
practical certification through SGs is the game genre. To address this, this study analyses
standard options used in challenges based on a set of competencies. The evaluation draws
from various game taxonomies, including those proposed by Adams and Dormans [
92
],
Adams and Rollings [
26
], ESA [
93
], Bateman and Boon [
94
], Stahl [
95
], and Wolf [
28
]. Since
Computers 2024,13, 201 14 of 29
no standard or universally accepted taxonomy exists, the researchers define their taxonomy
as consisting of 8 categories subdivided into 22 subcategories.
Table 4provides a quantitative overview of the analysed games, categorised by genre
and subgenre. The genre with the highest number of Serious Games analysed is simulation,
followed by puzzle and adventure games.
Table 4. Quantitative summary by genre and subgenre.
Genre Subgenre Subtotal Total
Action
“Beat-’em-ups” “Beat ’em ups” 2
8
1st/ 3rd person game 1
Platform games 5
Strategy
4X (eXplore, eXpand, eXamine, eXtermine) 1
8
Real-time strategy 1
Real-time tactics 1
Turn-based strategy 4
War games 1
Role-Playing Action RPGs 2
4
MMORPGs 2
Sport Exergames 2 3
Sports/management games 1
Management
simulation
Racing/vehicle games 6
41
Virtual worlds/fantasy/pets 24
Business 8
Social games and life simulations 3
Adventure
Graphics adventures 17 19
Puzzle adventures 2
Puzzle
Arcade/Action puzzle (timed) 14
22
Physics games 4
Hidden images games 2
Traditional games 2
Quiz 11 11
The chosen competency model, between the previous reference structures, is educa-
tional competencies, developed by Microsoft. This set of competencies aligns with current
references and is considered essential for future success in performing various functions.
The research analyses 116 Serious Games from different sites and open repositories
available in [
96
]. For each game, they collect information such as description, classifi-
cation, domain areas, game genre, topics, audience, and type of realism. By analysing
available data and, when possible, playing the games, the researchers identify the specific
competencies involved.
The study’s results provided a cross-reference of the genre categories with a set of
competencies in a matrix, allowing us to identify some areas with significant intersections
to achieve learning outcomes [97–99].
Many genres’ potential to support aptitude learning is significant. It contributes in
the same way to developing various game design strategies. The contribution of this
study results in the mixture/combination of genres or the reinforcement of challenges to
reach skills such as Decision Quality and Problem-Solving (SS3) and Technical Learning
(SS8); Organisation (OpS5) and Timely Decision-Making (OpS9); and both results category
Computers 2024,13, 201 15 of 29
competencies, Action-Oriented (R1) and Move for Results (R2), which can be synchronised
in different strategies to achieve better student performance.
In summary, this research bridges the gap between game design, competencies, and
certification, emphasising the importance of aligning game mechanics with desired learning
outcomes. Developers can create more effective Serious Games for training and certification
by understanding the interplay between game genres and competencies.
4.2. Design of Triadic Certification Method
The Triadic Certification Method (TCM) represents an important advancement in SGs,
particularly concerning competency and skills certification. The TCM enables performance
measurement during training missions by directly integrating training guidelines into
SG design.
The TCM architecture comprises four steps, each contributing to evaluating skills
acquisition and certification. Unlike traditional post-game questionnaires [
99
], the TCM
assesses player performance within the game itself. This approach provides clear guidelines
to the development team, especially designers, ensuring competencies are seamlessly
woven into the game construction alongside an evaluation map.
The following figure, Figure 1, shows the method design:
Computers 2024, 13, x FOR PEER REVIEW 16 of 30
Figure 1. Workflow of Triadic Certification with used tools and defined goals.
In the first step, the analysis/diagnosis of the training context involves collaboration
between the trainer/instructor and the development team. The training needs and compe-
tencies required for specific learning objectives are defined. Identifying situations, scenar-
ios, and learning outcomes allows for detailed training planning, focusing on knowledge,
skills, and aptitudes.
The TCM also leverages two methodologies: The Mission Essential Task List (METL)
[100,101], which hierarchically lists essential tasks and activities, and the CRAWL-WALK-
RUN Approach [102–104], which defines task sequences to promote progressive learning.
Constructing a reference table for training scenarios associates training stages with lists of
essential tasks, ensuring successful training by achieving expected performance levels. To
visualise the scenarios, a concept similar to the use case diagrams (through the Unified
Modelling Language—UML [105]) can be used to reference training scenario actors.
The second step of the method involves mapping educational competencies to align
with the list of tasks and activities determined in the preceding stage, with the trainer or
instructor remaining the central figure. During this phase, the focus shifts to defining the
fundamental skills required by each target group profile, which will be developed
through training. This definition is derived from aligning training competencies with ed-
ucational competencies [87]. This mapping exercise contextualises the specific training en-
vironment with a standardised reference point, ensuring that skills acquisition can be ap-
propriately assessed across different scenarios.
Initiating this stage of the method involves utilising a chosen reference matrix as a
competency model to identify the essential competencies necessary for performing or
training in a specific role across various contexts such as employment, occupation, organ-
isation, or industry. The aim is to construct a behavioural description representative of the
function to be performed based on the definition of competencies associated with each
occupational role, as suggested by Fogg [106].
Once the competencies for a specific task or position have been identified through
mapping from the reference matrix, the next step involves determining the most suitable
actions within the game to achieve the learning objectives, known as game mechanics.
Figure 1. Workflow of Triadic Certification with used tools and defined goals.
In the first step, the analysis/diagnosis of the training context involves collaboration
between the trainer/instructor and the development team. The training needs and compe-
tencies required for specific learning objectives are defined. Identifying situations, scenarios,
and learning outcomes allows for detailed training planning, focusing on knowledge, skills,
and aptitudes.
The TCM also leverages two methodologies: The Mission Essential Task List (METL) [
100
,
101
], which hierarchically lists essential tasks and activities, and the CRAWL-WALK-RUN
Approach [
102
–
104
], which defines task sequences to promote progressive learning. Con-
structing a reference table for training scenarios associates training stages with lists of
essential tasks, ensuring successful training by achieving expected performance levels. To
Computers 2024,13, 201 16 of 29
visualise the scenarios, a concept similar to the use case diagrams (through the Unified
Modelling Language—UML [105]) can be used to reference training scenario actors.
The second step of the method involves mapping educational competencies to align
with the list of tasks and activities determined in the preceding stage, with the trainer or
instructor remaining the central figure. During this phase, the focus shifts to defining the
fundamental skills required by each target group profile, which will be developed through
training. This definition is derived from aligning training competencies with educational
competencies [
87
]. This mapping exercise contextualises the specific training environment
with a standardised reference point, ensuring that skills acquisition can be appropriately
assessed across different scenarios.
Initiating this stage of the method involves utilising a chosen reference matrix as
a competency model to identify the essential competencies necessary for performing
or training in a specific role across various contexts such as employment, occupation,
organisation, or industry. The aim is to construct a behavioural description representative
of the function to be performed based on the definition of competencies associated with
each occupational role, as suggested by Fogg [106].
Once the competencies for a specific task or position have been identified through
mapping from the reference matrix, the next step involves determining the most suitable
actions within the game to achieve the learning objectives, known as game mechanics.
Upon completing this step and defining the basic skills profile, attention shifts to the
subsequent step: selecting the genre of the SG based on the correlation between game
mechanics and basic skills.
The third step entails choosing the SG genre that best aligns with the previously
defined basic skills profile. To accomplish this, extensive research on various Serious
Games is conducted to comprehend the contributions of different gaming genres towards
competency acquisition. While identifying the optimal game mechanics for skill acquisition
can be challenging, analysing game genres aims to uncover patterns of competencies that
specific game genres effectively encompass.
By conducting a high-level analysis of the mechanics in various Serious Games, de-
signers can determine the most suitable game mechanics for acquiring specific skills. This
methodology aspect falls under the designer’s responsibility, providing a guiding frame-
work for game design while allowing room for creativity.
In certain instances, the choice of genre may not be straightforward but rather a com-
bination of genres, where insights from the correlation matrix combine mechanics and
challenges from various genres to train the desired skills effectively. A key conclusion
drawn from this correlation matrix is the importance of leveraging past successful experi-
ences with Serious Games to inform future development, thereby facilitating correct and
efficient implementation.
The fourth and final step of the method involves integrating previous design contri-
butions into the new game. This module aims to adapt the serious game design for skills
training while maintaining autonomy in operation and configuration, contingent upon
receiving values/elements related to player performance within established mechanics and
challenges. Additionally, this step finalises the development of the Triadic Certification
Method (TCMd).
The TCM serves as a communication tool among stakeholders, aiming to standardise
game design for competency acquisition by balancing three components: identified skills
and competencies (basic skills), mechanics and challenges based on game type (mechanics),
and training levels (reality). Through the “in-game” certification method facilitated by
the TCM, design contributions are defined, certified, and validated, ensuring that games
effectively foster learning. Figure 2is the Certification Triadic Model for training local tour
guides, which comprises three reference axes: vertical, horizontal, and oblique. Each axis
assumes a specific function to achieve defined competencies [107].
Starting with the vertical axis, it encompasses competencies aligned with various
proficiency levels (basic, intermediate, advanced, expert), distributed across a mechan-
Computers 2024,13, 201 17 of 29
ics framework. The progression of skills throughout the training sessions is marked by
assigning the achieved proficiency level.
The horizontal axis represents the mechanics of skill acquisition, applied transversally
across competencies. Learning progresses linearly through the accumulation of successful
tasks, with higher competency levels indicating previous success.
Computers 2024, 13, x FOR PEER REVIEW 17 of 30
Upon completing this step and defining the basic skills profile, aention shifts to the sub-
sequent step: selecting the genre of the SG based on the correlation between game me-
chanics and basic skills.
The third step entails choosing the SG genre that best aligns with the previously de-
fined basic skills profile. To accomplish this, extensive research on various Serious Games
is conducted to comprehend the contributions of different gaming genres towards com-
petency acquisition. While identifying the optimal game mechanics for skill acquisition
can be challenging, analysing game genres aims to uncover paerns of competencies that
specific game genres effectively encompass.
By conducting a high-level analysis of the mechanics in various Serious Games, de-
signers can determine the most suitable game mechanics for acquiring specific skills. This
methodology aspect falls under the designer’s responsibility, providing a guiding frame-
work for game design while allowing room for creativity.
In certain instances, the choice of genre may not be straightforward but rather a com-
bination of genres, where insights from the correlation matrix combine mechanics and
challenges from various genres to train the desired skills effectively. A key conclusion
drawn from this correlation matrix is the importance of leveraging past successful expe-
riences with Serious Games to inform future development, thereby facilitating correct and
efficient implementation.
The fourth and final step of the method involves integrating previous design contri-
butions into the new game. This module aims to adapt the serious game design for skills
training while maintaining autonomy in operation and configuration, contingent upon
receiving values/elements related to player performance within established mechanics
and challenges. Additionally, this step finalises the development of the Triadic Certifica-
tion Method (TCMd).
The TCM serves as a communication tool among stakeholders, aiming to standardise
game design for competency acquisition by balancing three components: identified skills
and competencies (basic skills), mechanics and challenges based on game type (mechan-
ics), and training levels (reality). Through the “in-game” certification method facilitated
by the TCM, design contributions are defined, certified, and validated, ensuring that
games effectively foster learning. Figure 2 is the Certification Triadic Model for training
local tour guides, which comprises three reference axes: vertical, horizontal, and oblique.
Each axis assumes a specific function to achieve defined competencies [107].
Figure 2. Certification Triadic Model for training local tour guides.
Figure 2. Certification Triadic Model for training local tour guides.
The oblique axis refers to sessions or training levels, illustrating the progression of
competencies from basic to expert levels. Each subsequent level builds upon the results of
the previous level, fostering a cumulative advancement in proficiency profiles.
While the model emphasises progressive learning, it acknowledges that certain com-
petencies may depend on a single mechanic or that a single training level may incorporate
multiple mechanics. It also recognises the possibility of continuity or discontinuity in learn-
ing objectives between different training levels, with some outcomes remaining consistent
across multiple proficiency profiles.
After presenting the model’s guidelines, we propose demonstrating its application as
a customisable process since the starting point is always the context in which the skills are
trained (reality). The Triadic Certification Method shown in Figure 2refers to the training
of local tour guides. Starting from this specific reality, it is a priority to understand the
elements of certification for this activity, such as which tourist region and tourist resources
contribute to a variety of tourist experiences: cultural, gastronomic, and traditional points of
interest, whether it is one location or a group of locations. The certification of competencies
for this professional is based on a successful profile that requires mastery of various areas
of knowledge and skills, such as the geography of tourism, history, and cultural and
architectural heritage, as well as culture and traditions of the regions, various types of
communication (oral, written and active listening), group facilitator, mastery of several
languages, and planning and organising tourist routes and circuits [99].
Following the TCM, various steps were defined (training scenarios, identifying the
educational competencies to support certification, and finally, identifying the correlation
of the expected competencies with the game genre and its most appropriate mechanics).
Bearing in mind that scenarios for exploring the tourist region have been defined, the
following competencies are defined:
- Planning (OpS6), Organisation (OpS5), and Time Management (OpS8).
- Written Communications (OrS4) and Presentation Skills (OrS3).
- Action Orientated (R1).
Computers 2024,13, 201 18 of 29
-
Decision Quality and Problem Solving (SS3), Technical Learning (SS8), and Strategic
Agility and Innovation Management (SS7).
These competencies correspond to various tasks related to the acquisition and demon-
stration of knowledge in the following aspects:
- Monuments;
-
Cultural and architectural heritage (centuries-old and specific stories and traditions);
-
Understanding the people’s traditions as a key to organising and planning different
thematic and tourist itineraries for different target audiences.
Considering the third step, the most suitable game genre combines both adventure
genres: graphic and puzzle.
The game aims to use an avatar to explore an interactive story, which together takes on
mental challenges in mini-games (puzzles and challenges) about tourist resources. Three
training levels were defined to demonstrate the Triadic Certification Method’s construction:
navigation, knowledge, and recommendation about a tourist region. With more detail on
each training level, on the navigation level, the player will use a map of the tourist region,
monitored by a GPS device whose route taken between two points of interest (POIs) will
help validate route choices, time, and distance travelled. This level also ensures the physical
recognition of routes and their POIs. As for the second level, knowledge acquisition occurs
by identifying the POI and answering questionnaires and other challenges in various
on-site situations. The questions will be trivia about random locations, destinations, or
other more specific contexts relating to current or past events. The third and final level,
the recommendation, has a double meaning: the extra motivation to share opinions and
ratings on the spot of the various resources encountered and the collection of other helpful
information from other participants that will be fundamental to the planning component
of the thematic and other more specific itineraries that will have to be trained. The final
classification of the route planning depends on the knowledge already acquired of the POIs
included in the route, as well as the recommendations made to them.
However, we must bear in mind that the levels are cumulative, and it is necessary
to collect several navigation routes between crucial tourist spots and the respective POIs
found to ensure that all the other knowledge acquisition actions happen.
In conclusion, the Triadic Model offers a comprehensive and customisable framework
for understanding and implementing skills training through Serious Games in different
areas. It emphasises the integration of competencies, mechanics, and training levels to
facilitate effective learning outcomes. In summary, the TCM bridges game design, compe-
tencies, and certification, emphasising the importance of aligning game mechanics with
desired learning outcomes. By involving players in the design process, the TCM supports
effective Serious Games for training and certification purposes.
5. Case Study of Driver’s Licences—Comprehensive Training Method for Light
Vehicle Driving
To test our research hypothesis, we develop gaming applications, specifically proto-
types, to support the case study of competence training for obtaining a driver’s licence. This
case study focuses on road safety, which remains an enduring priority. Utilising games as a
learning tool offers scalability of results, cost reduction, and solid consolidation of learning.
Acquiring driving skills is considered complex and dynamic because it involves
various psychological processes on the driver’s part. This complexity can be broken down
into three acquisition stages: information gathering, information processing, and action.
New Serious Game-based learning tools can enhance the quality of driving skills acquisition
and training to promote safer and more responsible drivers.
5.1. Case Study Context and Implementation in DRIS
The motivations for studying the acquisition of driver’s licences are twofold. Firstly,
analysis of OECD (Young Drivers—The Road to Safety: Conference of Ministers of Trans-
port (ECMT): OCDE 2006; https://www.oecd-ilibrary.org/transport/young-drivers_97
Computers 2024,13, 201 19 of 29
89282113356-en (accessed on 7 July 2024)) data revealed that road accidents were a major
cause of death among individuals aged 15 to 24. Deaths and serious injuries resulting
from road accidents pose a significant public health issue, with young drivers being major
contributors. Seeking validation solutions for driving tasks could gradually reduce this
ongoing catastrophe.
Secondly, we tested this case study in the context of automotive driving learning
using the virtual automobile simulator (DriS) in the Traffic Analysis Laboratory of the Civil
Engineering Department at the Faculty of Engineering of the University of Porto [
108
,
109
].
The virtual simulator was configured with a training environment to monitor students’
real-time performance evolution and corresponding validation according to predefined
plans/assignments and learning objectives. The simulation room has an image projection
system with a projector and a screen. The driving position consists of a real vehicle
(customised Volvo 440 turbo). Figure 3shows images of the simulation room and its
driving position. The vehicle’s integrated instrumentation includes sensors for actuating
the pedals (clutch, brake, and accelerator). The car also has instrumentation for reading the
gear engaged, direction indicators, the position of the ignition key, and all the light controls.
Computers 2024, 13, x FOR PEER REVIEW 20 of 30
Figure 3. Picture of the simulation room and the driving position.
Utilising DriS, our objective was to test the application of the concept of validating
vehicle control and mastery skills (operational) and adaptation to constant changes in the
road environment (tactical) based on the learning support matrix of Driving: GDE—Goals
for Driver Education [110–112]. This matrix hierarchically defines the driving task, em-
phasising individual driver characteristics impacting driving, including experience, ai-
tudes, skills, motivations, decisions, and behaviours. This matrix allows for defining edu-
cational objectives and performance indicators in driver training as a tool for defining the
skills necessary to become a safe driver. Understanding learning guidelines is vital, as
they indicate that some areas must be learned before others may progress and that the
development of different components has varying timings.
Feedback is another crucial aspect of learning to drive, informing driving practises at
higher- and lower-order factors. In the first case, feedback acts as a regulator and behav-
ioural motivator, while in the second case, it actively engages the trainee throughout the
driving task, connecting to necessary automatisms and procedures. The amount of feed-
back perceived in the driving task at this lower level is greater, suggesting that low-level
skills are learned faster than high-level skills. A prototype was implemented for the train-
ing certification system module to validate the acquisition of a driving licence. Developed
for the Windows Operating System, the module aims to integrate with any serious game,
providing a set of metrics representing the necessary mechanics for real-time training. The
module’s coupling is intended to be generic and established through communication via
“sockets” in TCP/IP networks. The programming module is designed to be compatible
with other application cases through the definition of distinct projects associated with a
separate database file. This option ensures independence and portability for each applica-
tion case.
The tool allows each participant to independently interact with three distinct work
areas in the training process: trainer, who defines training competencies; game designer,
who defines game elements and mechanics associated with training; and certification, as
the training outcome component, which analyses student performance data and provides
feedback on the training plan. The feedback system is a critical component in this imple-
mentation. It enables monitoring of tasks performed by students during implementation
through visual and auditory means.
5.2. TCM Design
The implementation of the TCM commences with the analysis of training scenarios.
To conduct this competence diagnosis, it was imperative to consult the legal code of the
Figure 3. Picture of the simulation room and the driving position.
Utilising DriS, our objective was to test the application of the concept of validating
vehicle control and mastery skills (operational) and adaptation to constant changes in
the road environment (tactical) based on the learning support matrix of Driving: GDE—
Goals for Driver Education [
110
–
112
]. This matrix hierarchically defines the driving task,
emphasising individual driver characteristics impacting driving, including experience,
attitudes, skills, motivations, decisions, and behaviours. This matrix allows for defining
educational objectives and performance indicators in driver training as a tool for defining
the skills necessary to become a safe driver. Understanding learning guidelines is vital,
as they indicate that some areas must be learned before others may progress and that the
development of different components has varying timings.
Feedback is another crucial aspect of learning to drive, informing driving practises
at higher- and lower-order factors. In the first case, feedback acts as a regulator and
behavioural motivator, while in the second case, it actively engages the trainee throughout
the driving task, connecting to necessary automatisms and procedures. The amount of
feedback perceived in the driving task at this lower level is greater, suggesting that low-
level skills are learned faster than high-level skills. A prototype was implemented for
the training certification system module to validate the acquisition of a driving licence.
Developed for the Windows Operating System, the module aims to integrate with any
serious game, providing a set of metrics representing the necessary mechanics for real-
Computers 2024,13, 201 20 of 29
time training. The module’s coupling is intended to be generic and established through
communication via “sockets” in TCP/IP networks. The programming module is designed
to be compatible with other application cases through the definition of distinct projects
associated with a separate database file. This option ensures independence and portability
for each application case.
The tool allows each participant to independently interact with three distinct work
areas in the training process: trainer, who defines training competencies; game designer,
who defines game elements and mechanics associated with training; and certification, as
the training outcome component, which analyses student performance data and provides
feedback on the training plan. The feedback system is a critical component in this imple-
mentation. It enables monitoring of tasks performed by students during implementation
through visual and auditory means.
5.2. TCM Design
The implementation of the TCM commences with the analysis of training scenarios.
To conduct this competence diagnosis, it was imperative to consult the legal code of the
road and driving instructional manuals, which organise various theoretical themes of
road safety, traffic rules, and traffic signals. Additionally, technical files developed by
the IMT (Portuguese public entity regulating mobility and transport) and testimonies
from professional instructors were utilised to understand the procedures for initiating the
practise of driving a Category B motor vehicle.
Driving instruction in Portugal follows a matrix structure allowing for the organization
of various levels of learning for drivers. The legislative and regulatory responsibility for
driving education lies with the Portuguese State through the Institute of Mobility and
Transport (IMT, IP). Based on the gathered information about the practical learning of
driving tasks, frameworks of learning phases were developed, aligning with the Portuguese
driving education system.
In the initial step of this Triadic Method, we conduct a meticulous analysis of the tasks
and competencies essential for driving light vehicles. Our objective is to design distinct
training levels that ensure thorough mastery of these competencies, ultimately preparing
drivers for real-world scenarios.
We have identified three core competencies crucial for effective driver training: speed
adaptation and vehicle control, complete vehicle mastery, and traffic situation resolution.
Each competency is broken down into specific learning topics, which are then linked to
targeted tasks, ensuring a comprehensive learning experience.
Our training strategy is founded on the proven “Crawl-Walk-Run” methodology. In
the “Crawl” phase, trainees begin with fundamental tasks, building a solid foundation
of basic skills. In the “Walk” phase, the difficulty and realism of tasks gradually increase,
promoting steady progress. Finally, in the “Run” phase, trainees achieve high-level perfor-
mance through advanced practise, simulating real-world driving conditions.
Using the Mission Essential Task List (METL) methodology, we create a structured
hierarchy of tasks for each training mission. Training scenarios are designed to increase
in complexity progressively, allowing learners to build on their knowledge and skills
systematically. For this case study, we focus on three key competencies: speed control,
navigating crossings and intersections, and manoeuvring through wide curves.
To instil automatic responses in trainees, we include specific tasks such as safe vehicle
startup and stopping, speed changes emphasising the coordination of gearbox and ped-
als, light driving on straight and curved tracks, and defensive driving decision-making
based on road signs. A variety of routes are created to incorporate different driving
situations, connecting various activities through specific route signalling, ensuring compre-
hensive learning.
The routes (T1 to T5) include right and left curves with appropriate signalling. We
simulate real traffic conditions by placing other vehicles in the trainee’s path and at in-
tersections, enhancing the realism of the training. Driver performance is assessed based
Computers 2024,13, 201 21 of 29
on compliance with signage and successful task execution along the routes, covering all
competencies cumulatively.
The interconnected routes allow for continuous competency training. During evalua-
tion, these connections ensure that the training paths align seamlessly with the assessment
scenarios, promoting a cohesive learning experience. The sequence of training paths,
detailed in Figure 4, showcases the interconnection of routes and competencies:
Computers 2024, 13, x FOR PEER REVIEW 21 of 30
road and driving instructional manuals, which organise various theoretical themes of
road safety, traffic rules, and traffic signals. Additionally, technical files developed by the
IMT (Portuguese public entity regulating mobility and transport) and testimonies from
professional instructors were utilised to understand the procedures for initiating the prac-
tise of driving a Category B motor vehicle.
Driving instruction in Portugal follows a matrix structure allowing for the organiza-
tion of various levels of learning for drivers. The legislative and regulatory responsibility
for driving education lies with the Portuguese State through the Institute of Mobility and
Transport (IMT, IP). Based on the gathered information about the practical learning of
driving tasks, frameworks of learning phases were developed, aligning with the Portu-
guese driving education system.
In the initial step of this Triadic Method, we conduct a meticulous analysis of the
tasks and competencies essential for driving light vehicles. Our objective is to design dis-
tinct training levels that ensure thorough mastery of these competencies, ultimately pre-
paring drivers for real-world scenarios.
We have identified three core competencies crucial for effective driver training: speed
adaptation and vehicle control, complete vehicle mastery, and traffic situation resolution.
Each competency is broken down into specific learning topics, which are then linked to
targeted tasks, ensuring a comprehensive learning experience.
Our training strategy is founded on the proven “Crawl-Walk-Run” methodology. In
the “Crawl” phase, trainees begin with fundamental tasks, building a solid foundation of
basic skills. In the “Walk” phase, the difficulty and realism of tasks gradually increase,
promoting steady progress. Finally, in the “Run” phase, trainees achieve high-level per-
formance through advanced practise, simulating real-world driving conditions.
Using the Mission Essential Task List (METL) methodology, we create a structured
hierarchy of tasks for each training mission. Training scenarios are designed to increase
in complexity progressively, allowing learners to build on their knowledge and skills sys-
tematically. For this case study, we focus on three key competencies: speed control, navi-
gating crossings and intersections, and manoeuvring through wide curves.
To instil automatic responses in trainees, we include specific tasks such as safe vehicle
startup and stopping, speed changes emphasising the coordination of gearbox and pedals,
light driving on straight and curved tracks, and defensive driving decision-making based
on road signs. A variety of routes are created to incorporate different driving situations,
connecting various activities through specific route signalling, ensuring comprehensive
learning.
The routes (T1 to T5) include right and left curves with appropriate signalling. We
simulate real traffic conditions by placing other vehicles in the trainee’s path and at inter-
sections, enhancing the realism of the training. Driver performance is assessed based on
compliance with signage and successful task execution along the routes, covering all com-
petencies cumulatively.
The interconnected routes allow for continuous competency training. During evalu-
ation, these connections ensure that the training paths align seamlessly with the assess-
ment scenarios, promoting a cohesive learning experience. The sequence of training paths,
detailed in Figure 4, showcases the interconnection of routes and competencies:
Figure 4. Table of interconnection between routes and competences, where each competence has a
colour and corresponding training and assessment context.
Figure 4. Table of interconnection between routes and competences, where each competence has a
colour and corresponding training and assessment context.
This comprehensive training framework ensures drivers progressively acquire essen-
tial driving skills, leading to superior performance in real-world situations. Our meticulous
approach guarantees that every driver is thoroughly prepared, confident, and safe on the
road. Embrace this training method to master the art of driving light vehicles and transform
your driving experience today.
The second step in our Triadic Method involves competency mapping, focusing on
four specific competencies: speed limit control, in-road vehicle control, and approach to
crossings and junctions, distinguished by STOP signalling and signalling with and without
right of way. By aligning these competencies with our educational competency matrix, we
identified that they align best with strategic skills (SSs) and operational skills (OpSs).
From our reference matrix (Figure 5), the following competencies emerged as critical:
•
Strategic skills (SSs): Decision Quality and Problem Solving (SS3) and Functional/
Technical Skills (SS4)
•
Operational skills (OpSs): Planning (OpS6), Priority Setting (OpS7), and Timely
decision-making (OpS9)
•Results (Rs): Action Oriented (R1)
In the third step, we selected the game genre based on a correlation matrix between
game genres and competence benchmarks. This step ensures the skills identified are
seamlessly integrated into the game design, facilitating effective training. Our analysis
revealed that action, strategy, and simulation genres had the highest success rates. Thus,
our case study involves a hybrid of simulation games with vehicles (such as rally or heavy-
vehicle driving) combined with action and strategy elements. This blend allows us to
leverage individual mechanics effectively, with lower levels engaging in action mechanics
and higher levels incorporating strategic elements.
In the fourth step, we implemented the game using this combination of mechanics,
aligned with the chosen game taxonomy. This approach enabled us to pinpoint specific
challenges that correlate with the game’s objectives. For each selected genre, we identified
mechanics and actions crucial for training the anticipated competencies:
•
Spatial perception: enhancing the ability to navigate through the game environment
to develop a spatial relationship essential for reaching destinations.
•
Points: providing feedback on progress within the scenario, enhancing visualisation
and goal tracking.
•
Levels: introducing new sets of challenges in different scenarios to demonstrate
progression.
•
Detailed simulation actions: including acceleration, deceleration with pedals and
gearbox, braking, coordinating the vehicle within the lane, and stopping the car.
Computers 2024,13, 201 22 of 29
Figure 6illustrates the mapping of competencies within the training plan, where
each horizontal axis corresponds to a different path (e.g., Path 1, Path 2). The alignment
between skills and paths is established through the mechanics or challenges implemented
in the game.
Computers 2024, 13, x FOR PEER REVIEW 22 of 30
This comprehensive training framework ensures drivers progressively acquire essen-
tial driving skills, leading to superior performance in real-world situations. Our meticu-
lous approach guarantees that every driver is thoroughly prepared, confident, and safe
on the road. Embrace this training method to master the art of driving light vehicles and
transform your driving experience today.
The second step in our Triadic Method involves competency mapping, focusing on
four specific competencies: speed limit control, in-road vehicle control, and approach to
crossings and junctions, distinguished by STOP signalling and signalling with and with-
out right of way. By aligning these competencies with our educational competency matrix,
we identified that they align best with strategic skills (SSs) and operational skills (OpSs).
From our reference matrix (Figure 5), the following competencies emerged as critical:
• Strategic skills (SSs): Decision Quality and Problem Solving (SS3) and Func-
tional/Technical Skills (SS4)
• Operational skills (OpSs): Planning (OpS6), Priority Seing (OpS7), and Timely deci-
sion-making (OpS9)
• Results (Rs): Action Oriented (R1)
Figure 5. Summary grid highlighting the competencies identified for the training scenario.
In the third step, we selected the game genre based on a correlation matrix between
game genres and competence benchmarks. This step ensures the skills identified are seam-
lessly integrated into the game design, facilitating effective training. Our analysis revealed
that action, strategy, and simulation genres had the highest success rates. Thus, our case
study involves a hybrid of simulation games with vehicles (such as rally or heavy-vehicle
driving) combined with action and strategy elements. This blend allows us to leverage
individual mechanics effectively, with lower levels engaging in action mechanics and
higher levels incorporating strategic elements.
In the fourth step, we implemented the game using this combination of mechanics,
aligned with the chosen game taxonomy. This approach enabled us to pinpoint specific
challenges that correlate with the game’s objectives. For each selected genre, we identified
mechanics and actions crucial for training the anticipated competencies:
• Spatial perception: enhancing the ability to navigate through the game environment
to develop a spatial relationship essential for reaching destinations.
Figure 5. Summary grid highlighting the competencies identified for the training scenario.
Computers 2024, 13, x FOR PEER REVIEW 23 of 30
• Points: providing feedback on progress within the scenario, enhancing visualisation
and goal tracking.
• Levels: introducing new sets of challenges in different scenarios to demonstrate pro-
gression.
• Detailed simulation actions: including acceleration, deceleration with pedals and
gearbox, braking, coordinating the vehicle within the lane, and stopping the car.
Figure 6 illustrates the mapping of competencies within the training plan, where each
horizontal axis corresponds to a different path (e.g., Path 1, Path 2). The alignment be-
tween skills and paths is established through the mechanics or challenges implemented
in the game.
Each defined competency has a scalable profile (1—basic, 2—intermediate, 3—ad-
vanced, 4—expert) associated with it, classifying the use of mechanics in the task. This
profile classification serves a dual purpose: categorising all mechanics and competency
alignments with a learning level and contextualising the student during training tasks to
achieve learning outcomes.
Each training path incorporates one or more skills that can be learned or trained se-
quentially. This mapping allows us to track the evolution of competency along the scale,
contingent on the successful performance of the mechanics. This structure frames the
learning of competencies within each path, enabling students to accumulate evidence of
their learning by successfully performing tasks through mechanics. Post-training, the
learning outcome is determined by combining successful evidence from various mechan-
ics, creating a profile, and then assigning the achieved degree of competence.
Figure 6. Mapping of the training (paths) with the level of competence acquired through the me-
chanics identified by the game taxonomy.
By adopting this method, we ensure a structured, progressive, and practical ap-
proach to mastering light vehicle driving. This method prepares drivers comprehensively
and instils confidence and ensures safety on the road, ultimately transforming the driving
experience.
Mapping of competencies along the competency training plan allows for alignment
between skills and paths through mechanics or challenges implemented in the game. Each
training path incorporates one or more skills that can be learned or trained sequentially.
If the mechanics’ performance is successful, the evolution of competency along the
scale is verified. This structure enables framing the learning of competencies within each
path, accumulating evidence of learning through successful task performance. After
Figure 6. Mapping of the training (paths) with the level of competence acquired through the
mechanics identified by the game taxonomy.
Each defined competency has a scalable profile (1—basic, 2—intermediate, 3—advanced,
4—expert) associated with it, classifying the use of mechanics in the task. This profile
classification serves a dual purpose: categorising all mechanics and competency alignments
with a learning level and contextualising the student during training tasks to achieve
learning outcomes.
Computers 2024,13, 201 23 of 29
Each training path incorporates one or more skills that can be learned or trained
sequentially. This mapping allows us to track the evolution of competency along the scale,
contingent on the successful performance of the mechanics. This structure frames the
learning of competencies within each path, enabling students to accumulate evidence of
their learning by successfully performing tasks through mechanics. Post-training, the
learning outcome is determined by combining successful evidence from various mechanics,
creating a profile, and then assigning the achieved degree of competence.
By adopting this method, we ensure a structured, progressive, and practical approach to
mastering light vehicle driving. This method prepares drivers comprehensively and instils
confidence and ensures safety on the road, ultimately transforming the driving experience.
Mapping of competencies along the competency training plan allows for alignment
between skills and paths through mechanics or challenges implemented in the game. Each
training path incorporates one or more skills that can be learned or trained sequentially.
If the mechanics’ performance is successful, the evolution of competency along the
scale is verified. This structure enables framing the learning of competencies within
each path, accumulating evidence of learning through successful task performance. After
training, the learning result is achieved by combining successful evidence from various
mechanics, creating a profile, and assigning the achieved degree of competence.
This comprehensive methodology ensures a systematic approach to competency train-
ing in driving education, facilitating effective learning outcomes for drivers.
5.3. Analysis of Results
Following the completion of the tests, 50 volunteers participated, comprising 38 men
and 12 women aged 18 to 65. Unfortunately, four participants withdrew prematurely due
to simulation-induced nausea. The remaining 46 participants were categorised into two
groups based on whether they held a driver’s licence: 31 participants held licences, while
15 did not, as delineated in Table 5.
Table 5. Description of participants.
Category Groups Frequency Percentage
Age
18–23 26 56.5%
24–30 12 26.2%
31–40 4 8.7%
41–50 3 6.5%
52–65 1 2.2%
Gender
Female 11 23.9%
Male 35 76.1%
Driver’s Licence Yes 31 67.4%
No 15 32.6%
All participants underwent a comprehensive familiarisation process with the simulator
throughout the experiment. This involved understanding acceleration, deceleration, steer-
ing wheel control, and traffic signs. Even those without a driver’s licence were fully briefed
on traffic signs and their meanings. The data collected from the experiment provided a
meticulous analysis of each competency, showcasing the individual learning progression
under the well-structured training plan.
Although the study presented in this document has a more methodological focus,
the instrument validation must be addressed. To analyse and process the data collected
with the user tests, we considered various statistical instruments before and after the
descriptive analysis of the results, which we do not present here. Firstly, we check the
internal consistency of the data using Cronbach’s alpha test, with two separate analyses:
Computers 2024,13, 201 24 of 29
one for performance by competence and the other for overall performance. Secondly, the
sample distribution was validated using the frequency and standard distribution curve,
and lastly, the correlation between the variables through Pearson’s coefficient (r).
We considered qualitative variables such as gender, driving ability, and age to assess
participant performance during training (paths 1, 2, 3, and 5) and evaluation (paths 7 and 8).
Performance was analysed based on both components’ success in training and evaluation
and the transition from training failure to evaluation success. Graphical data analysis
allowed us to evaluate overall performance differences and individual progress.
The analysis of speed competence revealed high failure rates globally and in specific
segments. Only 18 out of 46 participants succeeded, possibly due to the training design and
the imposed 40 km/h speed limit. The repetitive nature of the training routes likely caused
disinterest and monotony, contributing to the low success rate. In contrast, the control of
the car within the lane showed a positive learning evolution, with a 10% improvement
despite a reduced growth margin. This positive trend was further reinforced by the fact
that most participants (37 out of 41) had completed training and evaluation.
There was an improvement in the competence of approaching crossings and junctions
with signalling. The success rates increased from 9 participants in training to 23 in evalua-
tion, marking a remarkable 255% improvement. However, the slight difference between
training and both moments (7 participants) indicated that many struggled with mandatory
STOP signalling.
The right-of-way competence also saw positive results, with 31 out of 46 participants
(67%) achieving success. The desired behaviour improved significantly, with 17 participants
showing positive evolution between training and evaluation. Globally, the performance
evaluation indicated that 86% of successful participants were qualified to drive, validating
our training plan. Only seven participants (six males and one female, with four under
23) successfully acquired all four competencies. The low overall success rate could be
attributed to the repetitive nature of the training routes and the 40 km/h speed limit, which
caused frustration and demotivation.
The CRAWL-WALK-RUN method, which involves sequential and repetitive task
performance, yielded limited success in skills acquisition. Despite this, the high success
rate among participants with driver’s licences (86%) confirms the validity of our training
method for designing and validating competencies.
In conclusion, despite some challenges, our comprehensive training method effectively
prepares participants for real-world driving scenarios. It ensures a structured, progressive,
and practical approach to mastering light vehicle driving. This method instils confidence,
ensures safety, and transforms the driving experience.
6. Conclusions
In recent years, Serious Games have emerged as a compelling alternative for acquiring,
training, and certifying skills because they provide a more engaging and meaningful learn-
ing experience. By incorporating rules, behavioural simulations, and feedback mechanisms,
Serious Games create an environment where learners can make mistakes without real-life
consequences and receive instant feedback. However, Serious Games must be designed us-
ing appropriate training validation and certification methodologies. The gameplay element
is crucial for progression and successful learning outcomes.
This research was conducted with a rigorous and innovative approach to