Content uploaded by Hans Stefan
Author content
All content in this area was uploaded by Hans Stefan on Jun 17, 2024
Content may be subject to copyright.
Journal of Safety Research 90 (2024) 48–61
Available online 15 June 2024
0022-4375/© 2024 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
How effective is virtual reality for electrical safety training? Evaluating
trainees’ reactions, learning, and training duration
Hans Stefan
a
,
*
, Michael Mortimer
a
, Ben Horan
a
, Scott McMillan
b
a
Deakin University, 75 Pigdons Rd, Waurn Ponds, VIC 3216, Australia
b
Melbourne Water Corporation, 990 La Trobe Street, Docklands VIC 3008, Australia
ABSTRACT
Introduction: The widespread use and inherent risks associated with low-voltage electrical equipment require all workers to understand its hazards and how to
manage them. Despite being the most commonly used method for raising safety awareness, lecture-based training often proves ineffective. Virtual reality (VR) allows
the user to be immersed in a virtual environment and actively participate in practical training while maintaining their safety, which can potentially result in engaging
and effective training. This paper investigates the effectiveness of using immersive VR for low-voltage (LV) electrical safety training to understand the potential
benets of VR technology for industrial safety training applications. Method: A within-group experimental design was employed with 18 participants undertaking the
LV VR training. The effectiveness of the training was evaluated by measuring participants’ reactions, learning, and training duration. Participants’ learning was
measured before, immediately after, and four weeks after the training, whereas reaction and suitability of training duration were measured after the training. Results:
Participants reported highly positive reactions to the LV VR training, particularly regarding their level of engagement, intention to use the system in the future, and
enjoyment. Knowledge test scores signicantly improved immediately after training with high effect sizes. Although scores decreased signicantly four weeks after
training, they remained signicantly higher compared to before training. The training duration, ranging from 29 to 44 minutes was considered suitable for the
training. Conclusions: LV VR training effectively elicited positive reactions from trainees and supporting trainees to acquire and retain safety information while
maintaining appropriate training duration. Practical Implications: VR emerges as a viable alternative training method worth exploring for organizations seeking to
improve their safety training programs. VR combines educational and entertainment values, facilitating enjoyable learning experiences.
1. Introduction
Electrical equipment is essential for the operations of workplaces
across various industries. The widespread use of electrical equipment in
workplaces exposes various types of workers to electrical hazards. Un-
derstanding electrical hazards is therefore a crucial aspect of safely
working with such equipment. This applies not only to electrical workers
directly responsible for installing, maintaining, and working on elec-
trical systems, but also to nonelectrical workers such as eld workers,
safety ofcers, and management (Neitzel, 2018). Despite not working
directly on electrical systems, nonelectrical eld workers often perform
their work in proximity to such systems or use electrically powered
equipment, which expose them to electrical hazards often resulting in
more electrical incidents compared to electrical workers (Anderson,
McGaw, & Parra, 2021; Mills & McAlhaney, 2013; Neitzel, 2018). Mills
and McAlhaney (2013) reported that approximately half of electrical
incidents involved nonelectrical eld workers and encouraged the need
for safety programs that extend beyond electrical workers. More
recently, Anderson et al. (2021) also reported similar problems,
suggesting that nonelectrical workers experienced many electrical in-
cidents and require basic electrical safety training. Safety ofcers and
managers have also been reported to lack the necessary knowledge of
electrical hazards, despite their responsibility in mitigating such hazards
through the implementation of safety programs and training (Bugaris &
Floyd, 2020). The authors further appealed for increased focus on
electrical issues in safety research.
Occupational health and safety (OHS) training is one of the most
common ways to reduce work-related injuries, illnesses, and fatalities,
which has been shown to support the improvement of workers’ safety
attitudes, beliefs, and knowledge (Ricci, Chiesi, Bisio, Panari, & Pelosi,
2016). Despite this, work-related accidents are still prevalent with the
Bureau of Labor Statistics (2023) reporting over 2.5 million nonfatal
injuries and illnesses, along with more than 5,000 fatalities in the United
States in 2021. These numbers were reported by employers and as such,
are likely to be substantially lower than the actual number of injuries
(Michaels & Barab, 2020). Exposure to electricity remains an important
cause of fatalities in the United States, contributing to 152 deaths in
2021. Similarly, in Australia, electrical injuries led to 1,100
* Corresponding author.
E-mail addresses: h.stefan@deakin.edu.au (H. Stefan), m.mortimer@deakin.edu.au (M. Mortimer), ben.horan@deakin.edu.au (B. Horan), Scott.McMillan@
melbournewater.com.au (S. McMillan).
Contents lists available at ScienceDirect
Journal of Safety Research
journal homepage: www.elsevier.com/locate/jsr
https://doi.org/10.1016/j.jsr.2024.06.002
Received 5 May 2023; Received in revised form 2 February 2024; Accepted 4 June 2024
Journal of Safety Research 90 (2024) 48–61
49
hospitalization and 55 fatalities between 2014 and 2016 (Tovell,
McKenna, & Harrison, 2018). Importantly, these statistics only account
for injuries requiring hospital admittance or resulting in death. As such,
the actual number of injuries is also likely undercounted, excluding
unreported injuries and those treated in non-hospital-based treatment
facilities, by general practitioners, or in emergency departments.
Lecture-based safety training is the most widely used traditional OHS
program, which has been reported to be ineffective in improving safety
knowledge, attitudes, beliefs, behavior, and health (Ricci et al., 2016).
To ensure effectiveness, safety training should be engaging, incorpo-
rating methods such as hands-on demonstrations, which encourage
active participation from trainees (Burke et al., 2006). Another highly
engaging training method is using simulation-based training (SBT),
which replicates certain elements of the real world environment and
allows trainees to practice in such an environment (Salas et al., 2009).
Bracco, Masini, Glowinski, Piccinno, and Schaerlaeken (2021) devel-
oped and evaluated an SBT for electrical distribution workers and re-
ported positive trainee reactions and effective self-reection
capabilities. Although highly engaging training such as hands-on
training and SBT are considered more effective, they are less
commonly used as they are typically more expensive and time-
consuming, and in the case of hands-on training, exposes trainees to
actual site risks (Li, Lu, Chan, & Skitmore, 2015; Li, Yi, Chi, Wang, &
Chan, 2018).
Virtual Reality (VR) offers a unique capability to immerse the user in
a constructed world, evoking a sense of presence—the feeling of being in
an environment instead of just perceiving pictures (Schubert, 2003). By
realistically replicating work environments, VR training enables trainees
to practice in a “real-world” environment without exposing them to real
hazards. This way, VR training allows users to actively participate in
practical tasks, which can be highly engaging and effective (Freina &
Ott, 2015; Gao, Gonzalez, & Yiu, 2019; Li et al., 2015; Sacks, Perlman, &
Barak, 2013; Stefan, Mortimer, Horan, & Kenny, 2023a). The freedom to
create a wide variety of work environments and scenarios in VR,
including accidents and failures, overcome the boundaries of SBT and
hands-on training which are limited to ideal, controlled, and accessible
environments. VR’s ability to allow trainees to experience failure safely
supports the recognized benets of learning from failure (Weinzimmer
& Esken, 2017), facilitating reection on the actions leading up to the
failure.
In addition to its potential effectiveness, the current generation of VR
technologies can alleviate some practical challenges associated with SBT
and hands-on training. Notably, VR hardware has become relatively
affordable, with certain devices costing less than smartphones. For
example, as of 2022, the Meta Quest 2, one of the most popular VR
headsets, is priced at USD399, whereas the best-selling smartphone of
that year, the Apple iPhone 13, costs USD699 (Rastogi, 2023). Although
the majority of the cost associated with implementing a custom-made
VR training is front-loaded in the application development, once
completed, the training can be scaled to use for as many trainees as
required. Scaling thus only requires acquiring additional headsets and
necessary hardware or software, depending on the complexity of the VR
training. In comparison, SBT, hands-on training, and even lecture-based
training require ongoing non-trivial logistical costs including the need
for instructor(s). Moreover, these traditional methods are often con-
ducted in groups, rendering them inexible in terms of training times
(Stefan et al., 2023a). These advantages make VR training potentially
cost-effective, particularly for organizations with large workforces or
high turnover rates, or for training topics that require frequent refresher
training. Alternatively, there have been an increasing number of com-
panies offering VR training as a service in exchange for subscriptions or
xed upfront fees, which provide options to organizations with differing
structures and needs.
Research into the usage and the benets of using VR for safety
training has been increasing, particularly in elds such as medicine
(Alaker, Wynn, & Arulampalam, 2016; Joda, Gallucci, Wismeijer, &
Zitzmann, 2019; Vaughan, Dubey, Wainwright, & Middleton, 2016),
construction (Gao et al., 2019; Li et al., 2018), and emergency pre-
paredness (Feng, Gonz´
alez, Amor, Lovreglio, & Cabrera-Guerrero, 2018;
Hsu et al., 2013). Although there is a considerable number of studies
utilizing VR for construction safety training (Stefan, Mortimer, & Horan,
2023b), studies focusing on electrical safety are limited. This is sur-
prising, considering that electrocution is part of the “fatal four” or “focus
four,” which is one of the leading causes of fatalities in construction
along with falls, caught-in or -between, and struck-by (Occupational
Safety and Health Administration, n.d.; Zhao, McCoy, Kleiner, & Smith-
Jackson, 2015). The Australian Institute of Health and Welfare also re-
ported that the construction sector experienced the highest number of
electrical injuries in Australia between 2014 and 2016 (Tovell et al.,
2018). Existing VR training addressing electrical safety has focused
solely on high-voltage (HV) power lines (García, Bobadilla, Figueroa,
Ramírez, & Rom´
an, 2016; Herrington & Tacy, 2020; Zhao & Lucas,
2015) and utilized non-immersive desktop VR (Barrett & Blackledge,
2012; García et al., 2016; Zhao & Lucas, 2015). To the best of the au-
thors’ knowledge, there is currently no study specically focused on the
use of VR for training in the safety of low-voltage electrical equipment.
Safety training for hazards associated with LV equipment is impor-
tant due to the widespread usage of such equipment across various in-
dustries and its often-underestimated danger. A piece of equipment is
considered low voltage when it operates between 50 and 1,000 V (AC) or
between 120 and 1,500 V (DC), encompassing items such as program-
mable logic controllers (PLCs), motor controllers, circuit breakers, dis-
tribution boards, and switches. In addition to the inherent danger of
electricity, including its invisibility which makes it more difcult to
identify (Zhao & Lucas, 2015), LV has been suggested to be seven times
more dangerous than HV (Australian Energy Council, 2019). This
increased danger is attributed to LV’s potential for arc ash events with
higher incident energy, resulting from higher fault currents and other
contributing factors (Australian Energy Council, 2019). Besides an arc
ash event, electric shock in LV can also be more dangerous than HV.
LV-induced electric shock causes muscles to contract, often leading to
the victim’s hands locking onto and not being able to let go of an elec-
trical conductor, further worsening the damage and resulting in death
(Energy Safe Victoria, 2018).
This paper investigates the effectiveness of using immersive VR for
LV electrical safety training, aiming to understand the potential benets
of VR technology for industrial safety training applications.
2. Methods
2.1. Population
A total of 18 participants underwent the evaluation. An a priori
power analysis was conducted using G*Power 3.1.9.7 software (Faul,
Erdfelder, Lang, & Buchner, 2007) and the sample size is deemed suit-
able (>15 for dependent t-test and >12 for repeated measures ANOVA)
based on commonly accepted parameters (type 1 error probability (
α
) of
5% and type 2 error probability (β) of 20%) and a large effect size
(Cohen’s d greater than 0.8 and Cohen’s f greater than 0.4) (Cohen,
1988). A post hoc power analysis yields actual power values of 0.89 and
0.93 for the dependent t-test and repeated measures ANOVA,
respectively.
Participants included physically t adults recruited via email from
the existing networks of Deakin University, Melbourne Water Corpora-
tion, and Innovative Manufacturing Cooperative Research Centre
(IMCRC). The email included a Plain Language Statement explaining
what participation will involve, a participation consent form to sign and
return for those willing to participate, and a withdrawal of consent form
to be used to withdraw at any time. Participation was voluntary, and no
monetary incentive was provided.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
50
2.2. Research design
This research employed a ‘within-group’ experimental design to
compare the effectiveness of LV VR training before, immediately after,
and four weeks after training. Fig. 1 presents the experimental proced-
ure of this study.
Firstly, participants completed a pre-survey involving demographics
information (i.e., name, age, gender, and job title), risk propensity, and a
baseline knowledge test related to LV safety. The baseline knowledge
test used the same questions as the knowledge gain test immediately
after the training and the knowledge retention test four weeks after the
training. The participant composition was predominantly males (N =
15, female N =3) with an average age of 43.7.
To measure participants’ VR experience, a 7-point Likert scale was
employed, ranging from 1–no experience to 7–high experience, which
includes three descriptions of what the labels stand for. The label “no
experience” can be thought of as “have never used VR before,” the label
“high experience” as “use VR on a regular basis,” and the midpoint of 4
as “use VR occasionally.” The average score was 3.28 (sd =1.49),
indicating a moderate to low experience with VR among participants.
The risk propensity scale by Meertens and Lion (2008) was used to
measure the risk propensity of participants, which is evaluated using a 7-
point Likert scale. A higher score signies a greater inclination toward
risk-taking. As observed from Table 1, a relatively low score of 2.75 with
a low standard deviation of 0.13 indicates that participants tend to be
risk-averse or less likely to take risks. None of the participants scored
above 3.72, which is below the midpoint of 4.
Table 2 presents different categories of employment and specic job
titles within participant group. OHS ofcers constitute 33.33%, opera-
tors or coordinators 27.78%, engineers 22.22%, electrical specialists
11.11%, and other specialists 5.56%.
After completing the pre-survey, the researcher provided an over-
view of the VR training experience followed by verbal instructions on
operating the VR system. Instructions included how to grab objects,
select options, teleport, and navigate the menu system. As the controls
were deemed relatively simple, with only four main interactions/but-
tons needed to complete the training, there was no specic tutorial
section within the VR experience.
Participants were then equipped with the VR head-mounted display
(HMD) where they were placed in an empty room with four different
options displayed on a virtual screen in front of them, as can be seen in
Fig. 2. These options represented the different virtual training envi-
ronments as part of the VR training experience, which are the theory
presentation room, safety preparation room, electrical room, and pump room,
each illustrated in Fig. 3. Before selecting any options, participants were
guided by the researcher to practice and become familiar with the
teleport function by teleporting around the empty room. Once
comfortable, they initiated the training by selecting the “Theory
Training” option on the virtual screen by pointing the controller and
pressing the select/grab button.
Upon completing each session/room, participants were transported
back to the empty room with the four options (Fig. 2). Participants were
instructed to select the room options in the correct order. Upon starting
the theory presentation session until the completion of the pump isola-
tion session, multiple events were logged with corresponding time-
stamps for measuring the duration of the training experience.
The VR training experience guides participants through the safe
practice of working with low-voltage equipment by asking participants
to perform electrical isolation for the removal of a pump motor.
After participants completed the LV VR training experience, they
were asked to complete a post-survey. The post-survey included multiple
Fig. 1. Experimental procedure.
Table 1
Baseline pre-survey results.
Demographic category Statistics
Gender, frequency male:female 15:3
Age, years (sd; range) 43.7 (13.0; 26 to 68)
VR experience, rating (sd) 3.28 (1.49)
Risk propensity (sd) 2.75 (0.13)
Table 2
Employment categories and job titles.
Category of
employment (N)
Job title
OHS Ofcer (6) Occupational Health and Safety (OHS) Performance
Specialist, OHS Advisor, Safety Advisor, Safety Manager,
Safety Health Environment and Quality (SHEQ) Service
Delivery Specialist, SHEQ Manager.
Operator/Coordinator
(5)
Operations Coordinator, Works Coordinator (2), Sewer
Transfer Operator, and Senior Water Transfer Operator.
Engineer (4) Geotechnical Engineer, Project Engineer, Lead Research
Engineer, Senior Engineer.
Electrical Specialists
(2)
Technical Specialist, High-voltage Specialist.
Other Specialists (1) Innovation and Sustainability Specialist.
Fig. 2. Empty room to select LV VR training session.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
51
evaluation measures such as reaction, knowledge gain, and training
duration perception. Additionally, the quality of the VR training system
in terms of its usability and its ability to evoke a sense of presence was
rated by each participant. Established questionnaires such as the System
Usability Scale (SUS) (Brooke, 1996) and the Igroup Presence Ques-
tionnaire (IPQ) (Schubert, Friedmann, & Regenbrecht, 2001) were used
for measuring usability and presence respectively.
The VR training yielded a mean SUS score of 79.22 (sd =9.43) and a
median of 81.25. Comparing this score to the guidelines provided by
Bangor, Kortum, and Miller (2008), the LV VR training falls within the
fourth quartile (>78.51) indicating acceptable (>70) and good (>72.75)
usability.
For the sense of presence, the VR training resulted in a mean
aggregate IPQ score of 4.21 (sd =0.63). The IPQ presence questionnaire
has a score range of 1 to 6, with 6 indicating the highest sense of pres-
ence. Based on the scale and in comparison with similar safety-relevant
VR training (Buttussi & Chittaro, 2018; Kwon, 2020; Leder, Horlitz,
Puschmann, Wittstock, & Schütz, 2019), the LV VR training demon-
strated an ability to evoke a high sense of presence.
Four weeks after the training, participants received an email
prompting them to complete the knowledge retention test.
2.3. Measures
The effectiveness of the LV VR training is evaluated using two pri-
mary measures, which are reaction and learning. Additionally, the ef-
ciency and suitability of the training duration are investigated. These
measures follow the evaluation measures summarized by Stefan et al.
(2023a) and can be seen in Fig. 4. Stefan et al. (2023a) based their
measures on the rst two levels of Kirkpatrick’s four-level model for
training evaluation (Kirkpatrick, 1976).
2.3.1. Reaction
Reaction evaluates the attitudes of participants toward the training
by considering both their direct and indirect responses (Stefan et al.,
2023a). Participants’ direct responses are measured by the perceived
training outcomes. Perceived training outcomes consist of three sub-
measures, which are perceived learning effectiveness, satisfaction, and
intention to use the system. Participants’ indirect responses are
measured by the psychological constructs. Psychological constructs
consist of ve sub-measures, which are motivation, enjoyment,
engagement, control and active learning, and perceived cognitive ben-
ets. The statement items used to measure each sub-measure are ob-
tained from Stefan et al. (2023a), which were summarized from
established questionnaires developed for various purposes. Each sub-
measure has been suggested to be an important indicator of effective
Fig. 3. Four training environments of the LV VR training. (a) theory presentation room, (b) safety preparation room, (c) electrical room, and (d) pump room.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
52
training as well as having a role in trainees’ learning. Table 3 provides a
description of each sub-measure along with the source of the question-
naires used.
Internal consistencies of the reaction measures and sub-measures are
assessed using Cronbach’s Alpha, as presented in Table 4. The Cron-
bach’s alpha for reaction (
α
=0.94), perceived training outcomes (
α
=
0.94), and psychological construct (
α
=0.84) are calculated using the
mean of the corresponding sub-measures. All sub-measures (i.e., moti-
vation, enjoyment, etc.) except for engagement (
α
=0.62) report alpha
values greater than 0.71 with an average of 0.82. The alpha value of
engagement is below the recommended minimum of 0.70 (Nunnaly,
1978). Engagement, measured by three-item statements, including “I
was engaged in the learning activity,” “the training was boring”
(reversed rating), and “I forgot the passing of time” exhibited improved
internal consistency (
α
=0.90) upon removing the third item. This
suggests that the third item does not correlate well with the other two
items and may not be an effective measure of engagement and as such, it
is excluded for analysis. As for the aggregate reaction measure, the
questionnaire reported satisfactory internal reliability with Cronbach’s
alpha (
α
) value of 0.94, average variance extracted (AVE) value of 0.55
and composite reliability value of 0.97.
2.3.2. Learning
Learning evaluates the effectiveness of VR training in acquiring and
retaining knowledge, measured using knowledge gain and knowledge
retention.
A knowledge gain test measures the level of knowledge gained by
participants after the training. This test was developed by the authors in
collaboration with Melbourne Water Corporation’s safety managers.
The test and the training materials are based on the Australian/New
Zealand Standards AS/NZS 4836:2011: Safe working on or near low-
voltage electrical installations and equipment (Standards Australia,
2011), and the Electrical Arc Flash Hazard Management Guideline by
the Australian Energy Council (2019). Comprising 15 short-answer
questions, the test includes ve questions related to declarative knowl-
edge delivered using a presentation-type format in VR (hereafter
referred to as DKPT), ve questions related to declarative knowledge
delivered in a realistic virtual setting (DKVR), and ve questions related
to procedural knowledge delivered in a realistic virtual setting (PKVR).
Each question is allocated one point, with a full score attainable through
either a single correct answer or multiple partial point answers. Each
knowledge category (i.e., DKPT, DKVR, PKVR) has a maximum score of
Fig. 4. Summary of evaluation measures reproduced from Stefan et al. (2023a).
Table 3
Descriptions of reaction sub-measures.
Reaction sub-
measures
Descriptions Questionnaire Source
Perceived
learning
effectiveness
The extent to which participants
consider the training method to be
effective in delivering the learning
content (Makransky & Lilleholt,
2018)
Lee, Wong, and Fung
(2010)
Satisfaction The degree to which participants feel
satised with the overall training
method (Makransky & Lilleholt,
2018)
Lee et al. (2010)
Intention to use
the system
Participants’ willingness and their
likelihood of using a similar method
of training in the future (Makransky
& Lilleholt, 2018)
Huang, Rauch, and
Liaw (2010)
Motivation The desire to undertake the training
for its inherent satisfaction (Ryan &
Deci, 2000)
McAuley, Duncan,
and Tammen (1989)
Enjoyment The extent to which the training is
perceived to be enjoyable in itself
irrespective of potential associated
positive outcomes (Davis, Bagozzi, &
Warshaw, 1992)
Venkatesh and Bala
(2008)
Engagement The extent of subjective involvement
in the training activity (
Csikszentmihalyi, 1988)
Buttussi and Chittaro
(2018)
Control and
active learning
The degree of autonomy and the
extent to which trainees can control
their own learning experience (
Makransky & Lilleholt, 2018)
Lee et al. (2010)
Perceived
cognitive
benets
The perception of improved
understanding, application, and
overall view of the learning content (
Lee et al., 2010)
Lee et al. (2010)
Table 4
Internal consistencies for reaction.
Measures Cronbach’s alpha (
α
)
Reaction 0.94
Perceived training outcomes 0.94
−Perceived learning effectiveness 0.91
−Satisfaction 0.90
−Intention to use the system 0.71
Psychological construct 0.84
−Motivation 0.75
−Enjoyment 0.82
−Engagement 0.62*
−Control and active learning 0.93
−Perceived cognitive benets 0.93
Note: *
α
<0.70.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
53
5 points. The questions are presented to each participant in a random-
ized order and the same questions are used both before and after (and
four weeks after training for the knowledge retention test) the training
to allow for direct comparisons of results. Signicant improvement in
the test score indicates the effectiveness of LV VR training in acquiring
new safety knowledge. A list of the knowledge gain test questions is
available in Table 5.
Knowledge retention was measured by utilizing the same knowledge
gain test (Table 5) four weeks after the training. The questions are
presented to each participant in a randomized order and the same
questions are used before (pre-survey), after (post-survey), and four
weeks after (follow-up-survey) training to allow for direct comparison of
results. The score difference between the post-test and the follow-up-test
are compared to determine whether knowledge gained after the training
can be retained effectively.
2.3.3. Training duration
The training duration is measured for each participant and in
conjunction with the learning outcomes, helps determine if the LV VR
training is efcient. In addition, a questionnaire, outlined in Table 6,
was used to gauge the perception of participants regarding the duration,
with an open-ended eld also provided to allow for comments.
Analyzing both the perception and actual duration of training will
provide valuable insights on the optimal duration for similar VR in-
dustrial safety training.
2.4. Materials
The VR trainer operated on a Meta Quest 2 HMD tted with an Elite
strap with battery. The Meta Quest 2 HMD ran on an Android-based
operating system with a Qualcomm Snapdragon XR2 platform and 6
GB RAM. The HMD allowed 6-Degree-of-Freedom inside-out tracking to
track both the HMD itself and the two Touch controllers used for in-
teractions in the VR training experience. A ber-optic cable (i.e., the
Link cable from Meta Platforms Inc) was connected to the HMD to cast
the image seen by the participant onto a computer screen, allowing the
researcher to provide assistance if participants experienced difculties
navigating the VR training experience. Although the HMD is capable of
wireless casting, wired casting was chosen to reduce the processing
demand of the HMD, thus maintaining appropriate and consistent per-
formance. The VR training software was developed using the Unity
gaming engine.
The VR trainer comprises four sessions, each featuring distinct vir-
tual environments. Upon wearing the VR HMD, participants would nd
themselves in an empty room with a virtual screen displaying the four
sessions (see Fig. 2). This empty room functioned as a “lobby” to which
participants would be transported after the end of each session. Struc-
turing the training into four distinct sessions aimed to prevent partici-
pants from having to restart the entire training should any unidentied
error be experienced in one of the sessions.
2.4.1. Theory presentation
The theory presentation session explains the primary hazards asso-
ciated with LV equipment, namely electric shock and arc ash. This
explanation is delivered in a manner similar to traditional presentations,
as shown in Fig. 3a. Slides appear on the virtual screen accompanied by
a voiceover explaining the content on the slides. Alongside passively
listening, interactive components are integrated into this session.
The rst interactive component involves questions within VR
(hereafter referred to as in-VR question) and in this session, participants
are prompted to guess the voltage range of LV equipment. Regardless of
whether participants answer the in-VR question correctly or incorrectly,
the correct answer is presented afterwards with a slightly different
script—afrmation for correct answers and vice versa. Each VR expe-
rience session/room includes one in-VR question, totaling four in-VR
questions (Fig. 5), designed to ensure participants pay attention to the
presented information.
The second interactive component prompts participants to ask a
question by selecting a displayed text in front of them, which is then
answered by the voiceover. The question relates to the distance re-
quirements of electrical work, and the participants are informed about
the importance of maintaining a safe distance, including the exclusion
zone for non-competent persons, and the meaning of working on or near
an electrical installation.
The theory presentation session delivers information related to the
ve questions as part of the DKPT knowledge category (see Section 2.3.2
Learning; Table 5, Questions 1–5). Information related to the remaining
10 questions as part of the DKVR and PKVR knowledge categories (see
Section 2.3.2 Learning; Table 5, Questions 6–15) is delivered in the
subsequent sessions—safety preparation, electrical isolation, and pump
isolation session.
Upon completion of the theory presentation session, participants are
transported back to the empty room (Fig. 2).
2.4.2. Safety preparation
The safety preparation session introduces the work that participants
must complete and asks them to select the appropriate personal pro-
tective equipment (PPE) and tools before going to the site and
Table 5
Knowledge baseline (pre-survey), gain (post-survey), and retention (follow-up-
survey) test questions.
Questions
1 What are the two main events that could occur when working with low-voltage
electrical equipment? (1 Point; DKPT1)
2 What are the four factors affecting the extent of injury when an electric current
passes through a body? (1 Point; DKPT2)
3 Why can low-voltage equipment sometimes be more dangerous than high-
voltage equipment? (1 Point; DKPT3)
4 How far is the exclusion zone from energized exposed conductors or parts for
someone other than competent persons? And what is the maximum distance to
be considered working ’on or near’ an energized exposed conductors or parts?
(1 Point; DKPT4)
5 What is the voltage range of equipment to be considered low voltage in both AC
and DC? (1 Point; DKPT5)
6 What are the four important pieces of personal protective equipment (PPE) to
be worn when isolating low-voltage electrical equipment other than arc-rated
long-sleeve shirts and pants and leather work shoes? (1 Point; DKVR1)
7 How shall all electrical conductors be treated until they have been proven to be
de-energised? How do you prove that an electrical conductor is de-energised?
(1 Point; DKVR2)
8 Excluding category 0, how many PPE categories are there depending on the
severity of the incident energy related to low-voltage electrical equipment?
What is the maximum incident energy that can be protected using the highest
PPE category? (1 Point; DKVR3)
9 While doing your work, what should you immediately do if you encounter a
new hazard? (1 Point; DKVR4)
10 Where should you position yourself when opening a switchboard? Why? (1
Point; DKVR5)
11 After wearing the appropriate PPE and taking the required tools, what are the
two important things that need to be identied before isolating a piece of
electrical equipment? (1 Point; PKVR1)
12 What is the next step after the electrical source has been isolated and proved de-
energised? (1 Point; PKVR2)
13 What are the steps in voltage testing? (1 Point; PKVR3)
14 When visually inspecting test equipment such as a multimeter, what are the
four necessary checks to look for? (1 Point; PKVR4)
15 When inspecting the mechanical isolation of a pump, what are the three
important checks to look for before proceeding to electrical isolation? (1 Point;
PKVR5)
Table 6
Duration perception for LV VR training.
Statements/Questions Answer eld
I think the duration of the training is Too short (1) – Perfect (4) – Too long (7)
Comments on the duration: (blank)
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
54
performing the work. A virtual avatar serves as the trainer who provides
explanations, guides participants on what needs to be performed and
explains the reasons behind it. The avatar, depicted in Fig. 3b, remains
available from the start of this session through subsequent sessions until
the completion of training.
There are three different stations in the safety preparation room,
which are the PPE station, Lock-out-tag-out (LOTO) station, and tools
station. Firstly, participants are guided to the PPE station by the avatar
to retrieve the work order. Once grabbed, the work order disappears, but
participants can access it using the menu system as shown in Fig. 6. The
work order provides all the necessary information to complete the work
and is adapted from the work order used by Melbourne Water Corpo-
ration. After the work order is obtained, it remains accessible in the
menu system, including in subsequent sessions (i.e., electrical isolation
and pump isolation sessions).
Participants are tasked to perform electrical isolation on a pump for
replacement, with the replacement process excluded from the training.
After reading the work order, the avatar explains the different PPE
categories required depending on the maximum incident energy asso-
ciated with the LV equipment. Participants are able to listen to expla-
nations while looking at a relevant poster above the PPE station. Then,
participants must answer an in-VR question regarding the appropriate
PPE category for the task (Fig. 5b). Similar to the in-VR question in the
theory presentation session, the correct answer is revealed after partic-
ipants make their selections.
Participants are then asked to visually inspect the available PPE on
the PPE station for defects, such as broken components. Defective PPE is
to be placed in a bin next to the station. After removing all defective PPE,
participants are instructed to select the required PPE based on the pre-
viously identied PPE category by following the PPE category poster.
Once selected, all selected items are visually available in the PPE tab in
the menu system. Participants are then instructed to teleport to the
LOTO station.
At the LOTO station, participants are briefed on the purpose of the
LOTO process and the specic functions of each colored lock. This verbal
explanation is also aided by a poster above the LOTO station. Partici-
pants are asked to pick up the different colored locks and the lock box
one by one, with each picked item visually available in the lock-out-tag-
Fig. 5. In-VR questions of each LV VR training session. (a) theory presentation, (b) safety preparation, (c) electrical isolation, and (d) pump isolation
Fig. 6. Example work order in LV VR training.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
55
out tab in the menu system. Participants are then instructed to teleport
to the tools station.
At the tools station, participants are required to collect a multi-meter
and voltage tester proving unit for voltage testing, along with rescue
equipment. The avatar provides information on necessary visual checks
before using a multi-meter, assisted by a poster above the tools station.
As with PPE and LOTO, each selected item becomes visually available in
the tools tab in the menu system. After collecting all measuring and
rescue equipment, participants are teleported back to the empty room
with four available sessions on the virtual screen (Fig. 2). Here, they
need to select the next session to continue.
2.4.3. Electrical isolation
The electrical isolation session trains participants on the procedures
for electrically isolating an asset (i.e., pump) from the power source.
Initially, participants must identify the correct asset by consulting the
work order, exploring the room, and touching the cabinet door of both
the asset switchboard and the main isolator. Performing asset identi-
cation by matching the asset number in the work order with the envi-
ronment is aimed at realistically replicating real-world practice.
Realistic simulation is anticipated to assist trainees in practicing the
process and fostering the desired behavior.
After asset identication, participants proceed to hazard identica-
tion by removing slip, trip, and fall hazards, such as chairs and brooms,
as shown in Fig. 3c. The avatar also reminds participants the importance
of preparing for potential future hazards and maintaining a safe distance
while working with LV equipment. Participants then encounter the in-
VR question for this session as shown in Fig. 5c, covering the exclu-
sion zone previously taught during the theory presentation. Participants
are then instructed to set up barriers and warning signs by grabbing
them from the corner of the room.
Following this, participants pick up the “isolate here during an
emergency” sign and place it on the main isolator lever. Participants are
then instructed to turn off the asset switchboard lever and open the
cabinet door, ensuring they stand next to the hinges to avoid exposure to
a potential arc ash event. Once the cabinet is open, a multi-meter and
voltage tester proong unit appear next to the cabinet, prompting par-
ticipants to perform voltage testing.
To complete voltage testing, participants follow four steps: rst,
testing the voltage tester on a known voltage source (i.e., voltage tester
proong unit) to ensure correct operation; second, testing between all
conductors and a known earth; third, testing between all conductors and
each other; and nally, retesting the voltage tester on a known voltage
source. Step-by-step verbal instructions and visual guidance with blue
highlights on testing points are provided, as seen in Fig. 7.
Upon completing voltage testing, participants close the cabinet door
and secure the isolation by attaching a lock to the lever. Once locked, the
avatar instructs participants to place the key inside the lock box and
conrm this by checking the Lock-out-tag-out tab on the menu system,
where it will be visually represented as shown in Fig. 8. Following this,
participants are teleported back to the initial room, where they must
select the next session to continue (Fig. 2).
2.4.4. Pump isolation
The pump isolation session guides participants through the nal
checks necessary before the isolation is considered complete, allowing
the removal and replacement of the pump. Similar to the electrical
isolation session, the rst step before commencing work is to identify the
correct asset. Participants are instructed to consult the work order,
explore the room, and physically touch the designated pump. Repeating
this process is aimed at reinforcing the importance of asset identica-
tion, which encourages safe behavior.
Following the identication of the pump, participants are required to
verify that the pump has been mechanically isolated. The process of
mechanical isolation itself is out of the scope of the training as this is
sometimes performed by a different (authorized) personnel. Instead,
participants are tasked with conrming the correct execution of the
mechanical isolation by checking that the pressure gauge reads zero and
that the valve is locked and tagged. The participants then need to
respond with a “Yes” or “No” on whether mechanical isolation has been
performed, constituting the in-VR question for the pump session (see
Fig. 5d).
Subsequently, participants are asked to perform voltage testing on
the pump itself by opening the wiring cover atop the pump as depicted in
Fig. 3d. The process is identical to the one in the electrical isolation
session. Once completed, participants must secure the wiring cover by
attaching a lock and conrming that the key has been placed inside the
lock box by accessing the menu system. Finally, participants lock the
lock box and following this, they are teleported back to the initial room,
marking the conclusion of the LV VR training experience.
Fig. 7. Voltage testing in LV VR training.
Fig. 8. Visual representation of key inside LOTO box in LV VR training.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
56
3. Results and discussions
3.1. Reaction
Table 7 presents the reaction results of the LV VR training. The re-
action measures use a scale ranging from one to seven, with seven
indicating the most positive reaction. The results show a highly positive
overall reaction with an average of 6.22 and a median of 6.31. Higher
median scores compared to mean scores are observed for all sub-
measures, indicating a highly negative skew. Both the direct evalua-
tion in perceived training outcomes (mean =6.24) and psychological
constructs (mean =6.20) also show highly positive scores. Sub-
measures with the highest scores include engagement (mean =6.69)
followed by intention to use the system (mean =6.44) and enjoyment
(mean =6.32). Overall, the reaction scores indicate that LV VR training
can elicit positive reactions from trainees.
It is important to highlight that all sub-measures exhibit a high
negative skew, with 80.9% of total responses being six or seven
(reversed item adjusted). While this is not uncommon in similar Likert-
scale questionnaires and may suggest a genuinely positive reaction, it
could also indicate response biases such as acquiescence, social desir-
ability, or positivity bias. Despite efforts to mitigate these biases, such as
incorporating reverse-coded items and ensuring condentiality and non-
identiable data, complete elimination and measurement of these in-
uences presents challenges. Nevertheless, even with the potential
presence of these biases, the results offer valuable insights into the
promising benets of VR training in evoking positive reactions from
trainees.
The nature of the relationship between reaction and learning is a
subject of considerable debate. One popular perspective, as proposed by
Kirkpatrick (1976) in his four-level model of training evaluation, sug-
gests a direct relationship between reaction and learning. According to
this model, learning is not expected to increase if the training is not well
received by trainees, suggesting hierarchical levels, where positive re-
action serves as a prerequisite for improvements in learning (Salas,
Tannenbaum, Kraiger, & Smith-Jentsch, 2012). Some consider positive
reactions to have a less direct but mediating effect toward learning
(Makransky & Petersen, 2019; Pekrun, 2006). As such, positive emo-
tions can inuence achievements and facilitate learning through
increased focus on the task (Pekrun, 2006; Pekrun, Muis, Frenzel, &
Goetz, 2017; Plass & Kaplan, 2016). Others have also reported no cor-
relations between reaction and learning (Alliger & Janak, 1989), or
observed positive reactions without improved learning (Makransky,
Andreasen, Baceviciute, & Mayer, 2021; Stefan et al., 2023a).
Regardless of the mechanisms and the extent to which reaction
inuences learning, it is generally accepted that when exists, the rela-
tionship between the two is positive. Moreover, even in cases where
immediate improvement in learning are not observed, the positive re-
action can foster continued engagement with the training, developing
individual interest and potentially enhancing maximum learning po-
tential in the long run (Renninger & Hidi, 2016). This potential is
especially relevant for the LV VR training, which reported high levels of
engagement, intention to use the system, and enjoyment.
3.2. Learning
Table 8 presents the average knowledge test scores for different
knowledge categories before (pre-) and immediately after (post-)
training. Dependent t-tests were performed to compare the pre- and
post-training results of each knowledge category, with signicant dif-
ferences found for all categories. The LV VR training signicantly im-
proves participants’ knowledge, with a mean increase of 179.5% for
DKPT, 51.7% for DKVR, 148.7% for PKVR, and 105.8% for total
knowledge test score. Large effect sizes were found for all knowledge
categories and the total knowledge score, with a Cohen’s d value greater
than 0.80. Analyzing the knowledge categories, the highest effect size is
identied for PKVR, representing procedural knowledge delivered when
performing tasks in VR. This suggests that VR is suitable for learning
procedures where the tasks themselves can be visualized and practically
performed. This nding supports the notion that VR’s ability to recreate
a working environment facilitates trainees to learn (Gao et al., 2019).
Regarding the acquisition of declarative knowledge, signicant im-
provements were found for both DKPT and DKVR, but a larger effect size
was observed for DKPT. This indicates that declarative knowledge can
be more effectively delivered using presentation-type training compared
to performing practical tasks. This is possibly because visualizations and
actions are less important for understanding declarative knowledge. As
such, visual inputs and animations in virtual environments may not
necessarily be helpful and could even be distracting for understanding
declarative knowledge (Moreno & Mayer, 2002; Parong & Mayer,
2018).
A total of 16 participants completed the online knowledge retention
test, with two participants returning invalid responses, equating to an
11.11% drop-out rate. Due to the inability to enforce survey completion
on the same day, overall, participants took between 28 to 46 days after
the training to complete the survey (mean =33.123; sd =6.702).
Table 9 presents the average knowledge test scores for different
knowledge categories before (pre-), immediately after (post-), and four
weeks after (follow-up-) training. Repeated measures ANOVA was per-
formed to compare the knowledge tests results at different time points,
revealing signicant differences for all knowledge categories, including
the total knowledge. Pairwise comparisons were performed to identify
which comparisons were signicant, and the results are presented in
Table 10. Signicant differences were found for all knowledge cate-
gories and at all different time points. The mean knowledge score
signicantly increased after training and decreased signicantly after
four weeks, however, the scores remained signicantly higher than
before training. This trend is consistent across all knowledge categories
and is illustrated in Fig. 9.
Despite signicant decreases after four weeks, these decreases are
relatively low compared to the increase resulting from the training. The
knowledge score decreases by 34.21% for DKPT, 13.05% for DKVR,
18.60% for PKVR, and 21.35% for total knowledge compared to the
higher increases of 196.09%, 56.80%, 142.83%, and 111.15% respec-
tively. Indeed, after the four weeks, participants gained and retained
signicant portions of the knowledge, as evidenced by aggregate in-
creases in knowledge test scores of 94.85% for DKPT, 36.36% for DKVR,
97.65% for PKVR, and 66.08% for the total. This indicates that the LV
VR training is effective in assisting trainees to gain and retain safety
knowledge.
Regarding the retention of each knowledge category, the highest
Table 7
Reaction results.
Measures Mean Std.
Dev.
Median Skewness Kurtosis
Reaction 6.22 0.834 6.31 −1.65 2.30
Perceived training
outcomes
6.24 0.950 6.63 −1.53 1.62
Perceived learning
effectiveness
6.19 0.819 6.25 −1.45 2.83
Satisfaction 6.08 1.240 6.63 −1.49 1.16
Intention to use the
system
6.44 0.906 7.00 −1.42 0.65
Psychological constructs 6.20 0.747 6.497 −1.65 2.48
Motivation 5.94 0.653 6.000 −1.45 3.75
Enjoyment 6.32 0.736 6.333 −1.14 1.56
Engagement* 6.69 0.645 7.000 −2.69 7.91
Control and active
learning
5.91 1.512 6.667 −1.49 1.56
Perceived cognitive
benets
6.14 0.900 6.250 −1.63 3.45
*
Engagement uses two-item questions with the third item removed due to
limited internal consistency.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
57
decrease occurred in DKPT with 34.21%, followed by nearly half of the
decrease in PKVR with 18.60%, and the lowest decrease in DKVR with
13.05%. It is important to note that while DKVR had the lowest decrease
after four weeks, it also had the lowest increase after training (i.e.,
56.80% compared to 196.09% for DKPT and 142.83% for DKVR) and
the highest average score for all time points compared to the other two
categories, as shown in Fig. 9. On the other hand, the mean scores for
DKPT and PKVR were comparatively similar before and after training
but dropped at different rates after four weeks, with a weaker reduction
in PKVR compared to DKPT. This suggests that despite the higher initial
increase for DKPT (196.09%) compared to PKVR (142.86%), partici-
pants seemed to be able to retain the PKVR knowledge better. Overall,
the nding suggests that learning in a realistic VR environment by
completing objectives, performing tasks, and interacting with virtual
objects (DKVR and PKVR) can perhaps benet trainees in remembering
the training materials compared to presenting the information using
presentation-style delivery (DKPT).
3.3. Training duration
Table 11 presents the total duration of the LV VR training, as well as
the duration of each session. Overall, the training took approximately
35 min to complete, with the fastest participant completing the training
in approximately 29 min and the longest taking just over 44 min. Par-
ticipants spent the longest time during the electrical isolation session,
with an average of 11 min and 44 s, followed by the safety preparation
session, with an average of 11 min and 8 s. The pump isolation session
took an average of 5 min and 58 s, and the theory presentation session is
the fastest, with an average of 5 min and 15 s.
The theory presentation session is designed with minimal in-
teractions, with most of the training being delivered through voiceover
and slide presentation. This design minimizes variations between par-
ticipants, as evident from the low standard deviation of 6.4 s and a
narrow range of 18 s between the fastest and the slowest session. In
contrast, the electrical room has the most interactions, some of which
are relatively more complex than others, allowing greater variation in
the experience among participants. This is reected in the high standard
deviation of 141.2 s and a wide range of 479 s between the fastest and
the slowest session.
The duration of the electrical isolation session is also highly posi-
tively skewed, indicating that only a few participants were particularly
slower than the rest. These are perhaps because these slower partici-
pants were having difculties with the interactions or the instructions or
because they were exploring the environment.
In addition to evaluating the actual duration of the training, partic-
ipants were asked to share their opinion on the training duration (see
Table 4). The training duration rating uses a scale from one to seven,
where one is labelled “too short,” seven is labelled “too long,” and four is
labelled “perfect.” As depicted in Fig. 10, 14 participants, or 77.8% of
the participants, considered the training duration as perfect. Addition-
ally, three participants (16.7%) perceived the training as slightly too
short, while one participant (5.6%) found it slightly too long.
Interestingly, the three participants who considered the training
slightly too short took an average of 36 min and 32 s to complete the
overall training, all exceeding the average duration. Conversely, the
participant who considered the training slightly too long took 32 min
and 2 s to complete the training, which was shorter than the average
duration. This observation suggests that individual preferences for
training duration vary subjectively. Nonetheless, the current training
duration, ranging from 29 to 44 min, appears to be satisfactory for LV VR
training.
Table 8
Knowledge gain results.
Measure Time Point Mean Std. Dev. Mean Diff. % Diff. Sig. (1-tailed) Effect Size*
Declarative Knowledge Presentation-Type (DKPT) Pre- 1.222 0.878 −2.194 179.5 <.001 −1.884
Post- 3.417 1.188
Declarative Knowledge VR (DKVR) Pre- 2.820 1.277 −1.458 51.7 <.001 −1.323
Post- 4.278 0.575
Procedural Knowledge VR (PKVR) Pre- 1.455 1.113 −2.163 148.7 <.001 −2.167
Post- 3.618 0.626
Knowledge Total Pre- 5.497 2.784 −5.816 105.8 <.001 −2.441
Post- 11.312 1.944
Note: *Cohen’s d, sample standard deviation of the mean difference.
Table 9
Repeated measures ANOVA of learning between time points.
Measure Time
Point
Mean Std.
Dev.
Sig.
Declarative Knowledge Presentation-
Type (DKPT)
Pre- 1.203 0.905 <.001
Post- 3.563 1.149
Follow-
up-
2.344 1.091
Declarative Knowledge VR (DKVR) Pre- 2.750 1.342 <.001
Post- 4.313 0.574
Follow-
up-
3.750 0.871
Procedural Knowledge VR (PKVR) Pre- 1.492 1.165 <.001
Post- 3.623 0.648
Follow-
up-
2.949 . 789
Knowledge Total Pre- 5.445 2.936 <.001
Post- 11.498 1.966
Follow-
up-
9.043 2.330
Note: N =16.
Table 10
Pairwise comparisons of learning between time points.
Measure Time Point Mean
Diff.
% Diff. Sig.
b
Declarative Knowledge
Presentation-Type (DKPT)
Pre – Post −2.359 −196.09 <.001
Pre –
Follow-up
−1.141 −94.85 <.001
Post –
Follow-up
1.219 34.21 <.001
Declarative Knowledge VR
(DKVR)
Pre – Post −1.562 −56.80 <.001
Pre –
Follow-up
−1.000 −36.36 .002
Post –
Follow-up
0.563 13.05 .027
Procedural Knowledge VR
(PKVR)
Pre – Post −2.131 −142.83 <.001
Pre –
Follow-up
−1.457 −97.65 <.001
Post –
Follow-up
0.674 18.60 <.001
Knowledge Total Pre – Post −6.052 −111.15 <.001
Pre –
Follow-up
−3.598 −66.08 <.001
Post –
Follow-up
2.455 21.35 <.001
Note:
b
Adjustment for multiple comparisons: Bonferroni.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
58
After rating the suitability of the training duration, participants were
allowed to provide additional comments to elaborate on their rating.
Out of the 18 participants, 11 provided comments, and the majority
agreed that the duration was appropriate. These included comments
such as “the duration of the training is very good,” “it was a good length
as I could undertake it at my own pace,” and “time seemed to pass
quickly (and that) content seemed to be the right length.”.
Few participants considered the training duration to be appropriate
for them but recognized that it may not be for others. For example, one
participant mentioned feeling “able to operate in VR environment well
so (they) felt no time pressures or constraints, but can imagine how
others may feel overwhelmed, and the training may appear to be too
short.” Another participant suggested that “different people may nd it
too long if they struggle to stand for long periods.” Additionally, one
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Pre- Post- Follow-up-
Mean Score
Declarative Knowledge Presentation-Type (DKPT)
Declarative Knowledge VR (DKVR)
Procedural Knowledge VR (PKVR)
Fig. 9. Mean knowledge scores of LV experiment on different time points.
Table 11
Training duration results in seconds.
Session Min Max Mean Std.
Dev.
Skewness Kurtosis
Theory
Presentation
306 324 314.6 6.4 0.238 −1.410
Safety
Preparation
529 838 668.2 99.6 0.596 −1.237
Electrical
Isolation
546 1025 703.7 141.2 1.228 0.593
Pump Isolation 277 534 357.9 65.7 1.095 1.554
Overall Training 1745 2650 2079.5 269.1 0.913 0.012
0
2
4
6
8
10
12
14
16
1234567
Frequency
Duration rating (Too short–Too long)
Fig. 10. Training duration ratings of the LV VR training.
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
59
participant suggested that the training “would have (a) bigger impact” if
repeated a few times.
One participant suggested reduction in VR usage time, possibly
replaced with video or slides for situations where interactions are un-
necessary. Another participant considered the duration to be suitable
but because the “display was a bit blurry,” they “might have gotten dizzy
if it went for much longer.” Finally, one participant proposed the need
for a period of “getting used to the VR equipment prior to the launch of
the training,” indicating that the current familiarization period (i.e.,
teleport across an empty room) is insufcient.
4. General discussions, limitations, and future works
This study aims to investigate the effectiveness of LV VR training
with a focus on reaction, learning, and training duration. The ndings
suggest that LV VR training effectively supports trainees in acquiring
and retaining important safety knowledge across various categories,
demonstrating signicant enhancements after training. Although there
is a decline in knowledge levels four weeks post-training, the scores
remain signicantly higher than pre- training baseline (evidenced by a
substantial aggregate total knowledge increase of 66.08%). This sug-
gests successful retention of a signicant portion of safety information
provided during training.
The VR training also elicits highly positive reactions from trainees,
particularly in terms of engagement, intention to use the system, and
enjoyment. The trainees express satisfaction with the average training
duration of 35 min. The combination of an appropriate training duration
and highly positive reactions, particularly in terms of enjoyment and
intention to use the system can foster continued practice. The ability of
VR training to improve safety knowledge immediately after and four
weeks after training, when reinforced with practice, may lead to even
more effective learning. This suggests that VR can serve as an effective
training tool for LV scenarios and warrants further exploration for other
similar safety training scenarios.
As this paper evaluates a specic VR training program for LV sce-
narios, it is important to exercise caution when extending the ndings of
this study to other VR safety training programs. The results may nd
greater applicability among VR safety training programs that share
similar features with the LV VR training. Distinctive features of the LV
VR training include the presence of a virtual instructor, a menu system,
interactive questions during presentations, gamied step-by-step in-
structions, training delivery segmented into multiple sections, and the
presentation of context-relevant visuals accompanied by audio expla-
nations (following the Modality Principle in the Multimedia Learning
Theory; Moreno & Mayer, 1999). Despite the effectiveness observed
with the combination of these features, along with adequate usability
and immersion, this research does not delve into the specic identi-
cation of features that positively or negatively contribute to training
effectiveness. Future studies could benet from isolating and evaluating
the benets and drawbacks associated with the aforementioned
features.
One of the limitations of the present study is the absence of a control
group. As such, it is not possible to draw denitive conclusions about the
effectiveness of LV VR training compared to other methods of training.
Future work could benet from performing a before-and-after study
similar to the current training, while also including a control group
taught using traditional, lecture-based training. Additionally, expanding
the sample size in future work would contribute to greater statistical
robustness and more conclusive ndings.
The wide range of effect sizes observed across different knowledge
categories (Table 8) suggests that, despite the potential benets of VR
for safety training, it may not serve as a universally optimal solution for
all types of training. Instead, traditional lecture-based training might
prove more effective for certain types of training, as could hands-on
training or simulation-based training (SBT). Future studies should
examine the unique strengths of each training method and seek to
optimize their combined effectiveness, potentially through a blended
approach. The authors believe that each training method has its role in
safety training, and only through the appropriate selection and inte-
gration of these methods will a training achieve optimal effectiveness.
Another limitation pertains to the scope of measures used in the
current study, which focuses on reaction, learning, and duration.
Consequently, the ndings are limited to these measures and do not offer
insights into whether LV VR training effectively inuences safer work
behavior or contributes to a reduction in safety incidents. Although
improvements in safety knowledge are a crucial element in workers’
understanding of hazards and their control measures, whether this un-
derstanding is transferred to their work behavior remains unexplored.
Future studies could benet from evaluating workers’ immediate and
long-term behaviors following VR safety training, thereby building ev-
idence regarding the potential benets of VR safety training.
5. Practical Implications
The study demonstrates the potential of VR as an alternative training
method for industrial safety training, particularly for low-voltage elec-
trical safety. The premise of the technology is its ability to simulate
dangerous environments for practical and context-based learning, which
provides a more realistic and engaging training experience safely. The
ndings show that trainees react positively toward VR training and show
interest in the potential usage of the training method. This highly
receptive attitude is supported by the effectiveness of VR training to
support learning. This suggests that VR possesses both educational and
entertainment values and if combined appropriately, facilitates enjoy-
able learning experiences.
VR is a promising alternative training method worth exploring for
organizations seeking to enhance their current safety training programs.
As effectiveness levels are dependent on the types of knowledge deliv-
ered, careful consideration should be given when deciding which safety
training should be delivered in VR. From the preliminary ndings,
procedural safety knowledge where realistic environments and sce-
narios are important should be prioritized in VR, whereas declarative
knowledge can be delivered via lecture-based training. The suitability of
VR training duration may also improve an organization’s training cul-
ture as trainees perceived as if their time is being used effectively. It is
important to also evaluate the effectiveness of the VR training once
implemented, which helps determine whether the current ndings can
also be observed across different training applications. The positive
ndings encourage continued research into innovative VR safety
training solutions and further exploration of its effectiveness.
6. Conclusions
This paper evaluates the effectiveness of LV VR training across
various parameters, including trainees’ reactions, learning outcomes,
and training duration. The ndings demonstrate the overall effective-
ness of LV VR training in all measures. The LV VR training generates
highly positive reactions from trainees particularly in their perceived
engagement, enjoyment, and their intention to use the system in the
future. The training signicantly improves trainees’ safety knowledge
immediately after training, and sustains the acquired knowledge four
weeks after training, showcasing a notable 66.08% overall score in-
crease. The training duration, spanning 29 to 44 min, aligns with the
preferences of 77.8% of participants, suggesting its appropriateness for
similar VR safety training programs. These ndings support the poten-
tial of VR training in industrial Occupational Health and Safety (OHS)
contexts, warranting further exploration and application.
Declaration of competing interest
The authors declare that they have no known competing nancial
interests or personal relationships that could have appeared to inuence
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
60
the work reported in this paper.
Acknowledgements
This study was co-funded by the Department of Industry, Science,
Energy and Resources (Innovative Manufacturing CRC Ltd) and Mel-
bourne Water Corporation (IMCRC/MWC/131020).
References
Alaker, M., Wynn, G. R., & Arulampalam, T. (2016). Virtual reality training in
laparoscopic surgery: A systematic review & meta-analysis. International Journal of
Surgery, 29, 85–94. https://doi.org/10.1016/j.ijsu.2016.03.034
Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick’s level of training criteria: Thirty years
later. Personnel Psychology, 42(2), 331–342. https://doi.org/10.1111/j.1744-
6570.1989.tb00661.x
Anderson, R., McGaw, S., & Parra, G. (2021). Electrical Safety in Industrial Construction:
An Analysis of 10 Years of Incidents in the Global Engineering, Procurement, and
Construction Industry. IEEE Industry Applications Magazine, 27(3), 31–41. https://
doi.org/10.1109/MIAS.2020.3024452
Standards Australia. (2011). Safe Working on or Near Low-voltage Electrical Installations
and Equipment. (AS/NZS 4836:2011).
Australian Energy Council. (2019). Electrical Arc Flash Hazard Management Guideline.
Retrieved from https://www.energycouncil.com.au/media/15808/eafhm_guideline
_30_25-03-2019_web.pdf.
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An Empirical Evaluation of the System
Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574–594.
https://doi.org/10.1080/10447310802205776
Barrett, M., & Blackledge, J. (2012). Evaluation of a prototype desktop virtual reality
model developed to enhance electrical safety and design in the built environment.
ISAST Trans. on Computing and Intelligent Systems, 3, 3.
Bracco, F., Masini, M., Glowinski, D., Piccinno, T., & Schaerlaeken, S. (2021). Simulation
as a Training Method for Electricity Workers’ Safety. Int J Environ Res Public Health,
18(4). https://doi.org/10.3390/ijerph18041591
Brooke, J. (1996). SUS: A “quick and dirty” usability scale. In P. W. Jordan, B. Thomas,
B. A. Weerdmeester, & A. L. McClelland (Eds.), Usability evaluation in industry (1st
ed., pp. 189–195). London: CRC Press.
Bugaris, R., & Floyd, H. L. (2020, 2-6 March 2020). Best Practices for Engaging Safety
Professionals on Electrical Topics. Paper presented at the 2020 IEEE IAS Electrical
Safety Workshop (ESW).
Bureau of Labor Statistics. (2023). Injuries, Illnesses, and Fatalities. https://www.bls.
gov/iif/latest-numbers.htm. Accessed on 3 May 2023.
Burke, M. J., Sarpy, S. A., Smith-Crowe, K., Chan-Seran, S., Salvador, R. O., & Islam, G.
(2006). Relative effectiveness of worker safety and health training methods.
American journal of public health, 96(2), 315–324. https://doi.org/10.2105/
AJPH.2004.059840
Buttussi, F., & Chittaro, L. (2018). Effects of different types of virtual reality display on
presence and learning in a safety training scenario. IEEE transactions on visualization
and computer graphics, 24(2), 1063–1076. https://doi.org/10.1109/
TVCG.2017.2653117
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York,
NY, USA: Lawrence Erlbaum Associates.
Csikszentmihalyi, M. (1988). The ow experience and its signicance for human
psychology. In I. S. Csikszentmihalyi, & M. Csikszentmihalyi (Eds.), Optimal
experience: Psychological studies of ow in consciousness (pp. 15–35). Cambridge,
United Kingdom: Cambridge Univ. Press.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to
use computers in the workplace. Journal of Applied Social Psychology, 22(14),
1111–1132. https://doi.org/10.1111/j.1559-1816.1992.tb00945.x
Energy Safe Victoria. (2018). Electricity hazards safety guide for emergency services workers.
Retrieved from https://www.esv.vic.gov.au/sites/default/les/2022-12/Electricity
HazardsSafetyGuide.pdf.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* Power 3: A exible
statistical power analysis program for the social, behavioral, and biomedical
sciences. Behavior research methods, 39(2), 175–191. https://doi.org/10.3758/
BF03193146
Feng, Z., Gonz´
alez, V. A., Amor, R., Lovreglio, R., & Cabrera-Guerrero, G. (2018).
Immersive virtual reality serious games for evacuation training and research: A
systematic literature review. Computers & Education, 127, 252–266. https://doi.org/
10.1016/j.compedu.2018.09.002
Freina, L., & Ott, M. (2015). A literature review on immersive virtual reality in
education: state of the art and perspectives. In Proceedings of the 11th International
Scientic Conference eLearning and Software for Education (Vol. 1, pp. 133–141).
Romania: Carol I National Defence University Publishing House.
Gao, Y., Gonzalez, V. A., & Yiu, T. W. (2019). The effectiveness of traditional tools and
computer-aided technologies for health and safety training in the construction
sector: A systematic review. Computers & Education, 138, 101–115. https://doi.org/
10.1016/j.compedu.2019.05.003
García, A. A., Bobadilla, I. G., Figueroa, G. A., Ramírez, M. P., & Rom´
an, J. M. (2016).
Virtual reality training system for maintenance and operation of high-voltage
overhead power lines. Virtual Reality, 20(1), 27–40. https://doi.org/10.1007/
s10055-015-0280-6
Herrington, A., & Tacy, J. (2020). Crossing the Power Line: Using Virtual Simulation to
Prepare the First Responders of Utility Linemen. Informatics, 7(3). https://doi.org/
10.3390/INFORMATICS7030026
Hsu, E. B., Li, Y., Bayram, J. D., Levinson, D., Yang, S., & Monahan, C. (2013). State of
virtual reality based disaster preparedness and response training. PLoS Curr., 5.
https://doi.org/10.1371/currents.dis.1ea2b2e71237d5337fa53982a38b2aff
Huang, H.-M., Rauch, U., & Liaw, S.-S. (2010). Investigating learners’ attitudes toward
virtual reality learning environments: Based on a constructivist approach. Computers
& Education, 55(3), 1171–1182.
Joda, T., Gallucci, G. O., Wismeijer, D., & Zitzmann, N. U. (2019). Augmented and virtual
reality in dental medicine: A systematic review. Computers in Biology and Medicine,
108, 93–100. https://doi.org/10.1016/j.compbiomed.2019.03.012
Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and
Development Handbook. New York, NY, USA: McGraw-Hill.
Kwon, C. (2020). The Effect of the Degree of Anxiety of Learners during the Use of VR on
the Flow and Learning Effect. Applied Sciences, 10(14). https://doi.org/10.3390/
app10144932
Leder, J., Horlitz, T., Puschmann, P., Wittstock, V., & Schütz, A. (2019). Comparing
immersive virtual reality and powerpoint as methods for delivering safety training:
Impacts on risk perception, learning, and decision making. Safety science, 111,
271–286. https://doi.org/10.1016/j.ssci.2018.07.021
Lee, E.-A.-L., Wong, K. W., & Fung, C. C. (2010). How does desktop virtual reality
enhance learning outcomes? A structural equation modeling approach. Computers &
Education, 55(4), 1424–1442. https://doi.org/10.1016/j.compedu.2010.06.006Get
Li, H., Lu, M., Chan, G., & Skitmore, M. (2015). Proactive training system for safe and
efcient precast installation. Automation in Construction, 49, 163–174. https://doi.
org/10.1016/j.autcon.2014.10.010
Li, X., Yi, W., Chi, H.-L., Wang, X., & Chan, A. P. (2018). A critical review of virtual and
augmented reality (VR/AR) applications in construction safety. Automation in
Construction, 86, 150–162. https://doi.org/10.1016/j.autcon.2017.11.003
Makransky, G., Andreasen, N. K., Baceviciute, S., & Mayer, R. E. (2021). Immersive
virtual reality increases liking but not learning with a science simulation and
generative learning strategies promote learning in immersive virtual reality. Journal
of educational psychology, 113, 719–735. https://doi.org/10.1037/edu0000473
Makransky, G., & Lilleholt, L. (2018). A structural equation modeling investigation of the
emotional value of immersive virtual reality in education. Educational Technology
Research and Development, 66(5), 1141–1164. https://doi.org/10.1007/s11423-018-
9581-2
Makransky, G., & Petersen, G. B. (2019). Investigating the process of learning with
desktop virtual reality: A structural equation modeling approach. Computers &
Education, 134, 15–30. https://doi.org/10.1016/j.compedu.2019.02.002
McAuley, E., Duncan, T., & Tammen, V. V. (1989). Psychometric properties of the
Intrinsic Motivation Inventory in a competitive sport setting: A conrmatory factor
analysis. Research quarterly for exercise and sport, 60(1), 48–58. https://doi.org/
10.1080/02701367.1989.10607413
Meertens, R. M., & Lion, R. (2008). Measuring an Individual’s Tendency to Take Risks:
The Risk Propensity Scale. Journal of Applied Social Psychology, 38(6), 1506–1520.
https://doi.org/10.1111/j.1559-1816.2008.00357.x
Michaels, D., & Barab, J. (2020). The Occupational Safety and Health Administration at
50: Protecting Workers in a Changing Economy. Am J Public Health, 110(5),
631–635. https://doi.org/10.2105/ajph.2020.305597
Mills, T. D., & McAlhaney, J. H. (2013, 11-15 March 2013). Integrating the non-electrical
worker into the electrical safety program. Paper presented at the 2013 IEEE IAS
Electrical Safety Workshop.
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role
of modality and contiguity. J. Educ. Psychol., 91(2), 358–368. https://doi.org/
10.1037/0022-0663.91.2.358
Moreno, R., & Mayer, R. E. (2002). Learning science in virtual reality multimedia
environments: Role of methods and media. J. Educ. Psychol., 94(3), 598–610.
https://doi.org/10.1037/0022-0663.94.3.598
Neitzel, D. K. (2018, 17-22 June 2018). Electrical Safety Training Requirements for all
Personnel. Paper presented at the 2018 IEEE IAS Pulp, Paper and Forest Industries
Conference (PPFIC).
Nunnaly, J. C. (1978). Psychometric methods (2nd ed.). New York, NY, USA: McGraw-Hill.
Occupational Safety and Health Administration (n.d.). Construction Focus Four Training,
https://www.osha.gov/training/outreach/construction/focus-four. Accessed on 3
May 2023.
Parong, J., & Mayer, R. E. (2018). Learning science in immersive virtual reality. Journal
of educational psychology, 110, 785–797. https://doi.org/10.1037/edu0000241
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions,
corollaries, and implications for educational research and practice. Educational
psychology review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9
Pekrun, R., Muis, K. R., Frenzel, A. C., & Goetz, T. (2017). Emotions at school. New York,
NY, USA: Routledge.
Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In
S. Y. Tettegah, & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp.
131–161). San Diego, CA, USA: Academic Press.
Rastogi, H. (2023). Apple rst to capture 8 spots in list for global top 10 smartphones, htt
ps://www.counterpointresearch.com/top-smartphones-global-2022/#:~:text
=Apple’s%20iPhone%2013%20was%20the,expected%20to%20increase%20in%
202023. Accessed on 3 May 2023.
Renninger, K. A., & Hidi, S. E. (2016). The power of interest for motivation and engagement.
New York, NY, US: Routledge/Taylor & Francis Group.
Ricci, F., Chiesi, A., Bisio, C., Panari, C., & Pelosi, A. (2016). Effectiveness of occupational
health and safety training : A systematic review with meta-analysis. Journal of
Workplace Learning, 28(6), 355–377. https://doi.org/10.1108/JWL-11-2015-0087
H. Stefan et al.
Journal of Safety Research 90 (2024) 48–61
61
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic denitions
and new directions. Contemporary educational psychology, 25(1), 54–67. https://doi.
org/10.1006/ceps.1999.1020
Sacks, R., Perlman, A., & Barak, R. (2013). Construction safety training using immersive
virtual reality. Construction Management and Economics, 31(9), 1005–1017. https://
doi.org/10.1080/01446193.2013.828844
Salas, E., Rosen, M. A., Held, J. D., & Weissmuller, J. J. (2009). Performance
Measurement in Simulation-Based Training: A Review and Best Practices. Simulation
& Gaming, 40(3), 328–376. https://doi.org/10.1177/1046878108326734
Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The science of
training and development in organizations: What matters in practice. Psychological
science in the public interest, 13(2), 74–101. https://doi.org/10.1177/
1529100612436661
Schubert, T. (2003). The sense of presence in virtual environments. Zeitschrift für
Medienpsychologie, 15(2), 69–71. https://doi.org/10.1026//1617-6383.15.2.69
Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The experience of presence:
Factor analytic insights. Presence: Teleoperators & Virtual Environments, 10(3),
266–281. https://doi.org/10.1162/105474601300343603
Tovell, A., McKenna, K., & Harrison, J. (2018). Electrical injuries: hospitalisations and
deaths, 2014–15 and 2015–16. Retrieved from Canberra: https://www.aihw.gov.au/
getmedia/2c37bd48-ee08-43d5-be11-f6929f671b83/aihw-injcat-197.pdf.aspx?inli
ne=true.
Stefan, H., Mortimer, M., & Horan, B. (2023b). Evaluating the effectiveness of virtual
reality for safety-relevant training: A systematic review. Virtual Reality, 27,
2839–2869. https://doi.org/10.1007/s10055-023-00843-7
Stefan, H., Mortimer, M., Horan, B., & Kenny, G. (2023a). Evaluating the preliminary
effectiveness of industrial virtual reality safety training for ozone generator isolation
procedure. Safety science, 163, Article 106125. https://doi.org/10.1016/j.
ssci.2023.106125
Vaughan, N., Dubey, V. N., Wainwright, T. W., & Middleton, R. G. (2016). A review of
virtual reality based training simulators for orthopaedic surgery. Medical Engineering
& Physics, 38(2), 59–71. https://doi.org/10.1016/j.medengphy.2015.11.021
Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda
on interventions. Decision sciences, 39(2), 273–315. https://doi.org/10.1111/j.1540-
5915.2008.00192.x
Weinzimmer, L. G., & Esken, C. A. (2017). Learning From Mistakes: How Mistake
Tolerance Positively Affects Organizational Learning and Performance. The Journal
of Applied Behavioral Science, 53(3), 322–348. https://doi.org/10.1177/
0021886316688658
Zhao, D., & Lucas, J. (2015). Virtual reality simulation for construction safety promotion.
Int. J. Injury Control and Saf. Promotion, 22(1), 57–67. https://doi.org/10.1080/
17457300.2013.861853
Zhao, D., McCoy, A. P., Kleiner, B. M., & Smith-Jackson, T. L. (2015). Control measures
of electrical hazards: An analysis of construction industry. Safety science, 77,
143–151. https://doi.org/10.1016/j.ssci.2015.04.001
Hans Stefan is a PhD student and research assistant at the CADET Virtual Reality Training
and Simulation Research Lab at Deakin University. His research focuses on virtual reality,
safety training, and evaluation methodology. Hans received his B.Sc. (Civil Systems) from
The University of Melbourne in 2019 and his M.Eng. (Research) from Deakin University in
2023.
Michael Mortimer is a Research Fellow at the CADET Virtual Reality Training and
Simulation Research Lab at Deakin University. He specialises in the eld of immersive
reality and related advanced technologies for simulation, training and education. Michael
received his B.Eng. (Hons.) in 2013 and his PhD. (Eng.) in 2017 both from Deakin Uni-
versity, he was also awarded one of Australia’s most innovative Engineers by Engineers
Australia in 2020.
Ben Horan received the B.Eng. (Hons.) and PhD degrees in engineering from Deakin
University, Australia, in 2005 and 2009, respectively. He is currently the Head of School
for the School of Engineering at Deakin University. His current interests include mecha-
tronics, virtual reality, industrial electronics, and renewable energy. He has received the
Endeavour Fellowship and the Australian Academy of Science Early Career Researcher
Fellowship and was awarded one of Australia’s most innovative Engineers by Engineers
Australia.
Scott McMillan is an Honorary Fellow at Deakin University for his industry leadership
with the School of Engineering by using immersive technologies to improve industrial
safety training. Scott has a trade background as an Electrician and Instrumentation
Technician which has taken him to remote locations like Antarctica and the Cocos Islands.
Scott works at Melbourne Water as the Manager for Safety Innovation and Technology
where he focuses on developing more contemporary safety solutions for eld-based
employees.
H. Stefan et al.
