ArticlePDF Available

Using virtual reality simulation for training practical skills in musculoskeletal wrist X-ray - A pilot study

Authors:

Abstract and Figures

Objectives Using virtual reality (VR), students of radiography can practice acquisition and positioning of musculoskeletal radiographs and get immediate feedback on their performance within the simulator. The purpose of this study was to assess usability of a newly developed VR simulator and to explore self-perceived clinical readiness (SPCR) of radiography students before and after training acquisition of wrist radiographs in the VR simulator. Material and Methods A prospective methodology was applied where the students ( n = 10) estimated their own SPCR in regard to acquisition of wrist radiographs pre- and post-VR training. A questionnaire on usability, realism, and educational value of the simulator was answered post-VR training. Usability and SPCR scores were calculated. The student’s paired t -test was applied to explore the impact of VR training on SPCR. Results The students (90%) reported that the simulator was realistic and they thought that it could contribute to learning. The pre- and post-SPCR scores were 75 (95% confidence interval [CI]: 54–96) and 77 (95% CI: 59–95), respectively. There was no significant difference ( P = 0.4574) between the pre- and post-SPCR scores. Conclusion Results indicated that the concept of training acquisition and positioning of wrist radiographs in a VR simulator is feasible with positive feedback from the students. The SPCR scores improved slightly, although not statistically significant, after completion of the training session.
Journal of Clinical Imaging Science • 2023 • 13(20) | 1
is is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others
to remix, transform, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.
©2023 Published by Scientic Scholar on behalf of Journal of Clinical Imaging Science
Original Research Musculoskeletal Imaging
Using virtual reality simulation for training practical
skills in musculoskeletal wrist X-ray-A pilot study
Janni Jensen1, Ole Graumann2, Rune Overgaard Jensen2, Signe K. K. Gade2, Maria Grabau ielsen2, Winnie Most3,
Pia Iben Pietersen2
1Department of Radiology, Odense University Hospital, 2Research and Innovation Unit of Radiology, Department of Clinical Research, University of Southern
Denmark, 3Department of Radiography, University College UCL, Odense, Denmark.
*Corresponding author:
Janni Jensen,
Department of Radiology,
Odense University Hospital,
Odense, Denmark.
janni.jensen@rsyd.dk
Received : 05May2023
Accepted : 13June 2023
Published : 11 July 2023
DOI
10.25259/JCIS_45_2023
Quick Response Code:
INTRODUCTION
Distal radius fractures (DRFs) are one of the most commonly occurring fractures and account
for 15–20% of all fractures. e DRF is typically diagnosed on radiographs taken in the acute
setting.[1] Treatment of a DRF is, in part, based on quantication of fracture displacement as
measured in radiographs.[2,3] Improper patient positioning during the radiographic procedure can
aect the measured values of fracture displacement; in other words, correct patient positioning
is directly linked to diagnostic value.[4,5] Correct radiographic positioning is, however, a skill that
requires practice and knowledge.[6]
Until recently, radiographic positioning has typically been taught in the classroom or trained
using phantoms. With the advent of new technologies such as virtual reality (VR), students
ABSTRACT
Objectives: Using virtual reality (VR), students of radiography can practice acquisition and positioning of
musculoskeletal radiographs and get immediate feedback on their performance within the simulator. e purpose
of this study was to assess usability of a newly developed VR simulator and to explore self-perceived clinical
readiness (SPCR) of radiography students before and aer training acquisition of wrist radiographs in the VR
simulator.
Material and Methods: Aprospective methodology was applied where the students (n = 10) estimated their
own SPCR in regard to acquisition of wrist radiographs pre-and post-VR training. Aquestionnaire on usability,
realism, and educational value of the simulator was answered post-VR training. Usability and SPCR scores were
calculated. e student’s paired t-test was applied to explore the impact of VR training on SPCR.
Results: e students (90%) reported that the simulator was realistic and they thought that it could contribute to
learning. e pre-and post-SPCR scores were 75(95% condence interval [CI]: 54–96) and 77(95% CI: 59–95),
respectively. ere was no signicant dierence (P = 0.4574) between the pre-and post-SPCR scores.
Conclusion: Results indicated that the concept of training acquisition and positioning of wrist radiographs in a
VR simulator is feasible with positive feedback from the students. e SPCR scores improved slightly, although
not statistically signicant, aer completion of the training session.
Keywords: Virtual reality, Wrist radiography, Patient positioning
www.clinicalimagingscience.org
Journal of Clinical Imaging Science
Journal of Clinical Imaging Science • 2023 • 13(20) | 2
Jensen, et al.: Virtual reality and wrist X-rays
of radiography are given the opportunity to practice and
simulate musculoskeletal radiography acquisition skills using
a VR headset and controllers while in a safe and limitless
environment with neither radiation nor involvement of
patients.[7] e risk of radiation makes it challenging for
radiographers to train radiographic positioning involving
patients; this risk is eliminated using VR. Another challenge
is that it may be time consuming to get real-time feedback
on the radiographs from an expert. Arecent study involving
1styear students of radiography showed positive correlation
between VR simulation and their subsequent clinical
performance across a range of parameters.[8]
e virtual three-dimensional VR universe has also been
used in educational contexts within other health domains.[9-12]
Studies have already explored the eect of VR learning in the
areas of ultrasound, anatomy, and surgery and have found
signicant positive eects although further research is
suggested in previous studies.[11,13]
e use of VR has also been seen within the eld of X-ray
training where students of radiography reported a positive
attitude toward training procedures in VR. e biggest
challenge they faced was lack of tactile feedback when
palpating anatomical landmarks during positioning.
Moreover, a general lack of interaction and communication
with the patient was missing as well as feedback on
positioning from the system. e students mentioned
that they would like feedback on their radiographs so they
could learn how to improve their positioning and improve
diagnostic quality of the images.[7]
ere is, to the best of our knowledge, a relatively limited
amount of literature on the topic - especially on students
training in a virtual simulation where the focus is positioning
(rotation, deviation, and exion/extension) of the wrist
involving direct feedback and information based on their
own radiographs as taken in the VR simulator. Accordingly,
the primary aim of this study was to conduct a pilot study
and evaluate the eect of an immersive VR simulator for
training musculoskeletal X-ray of the wrist. e objectives of
this study were to explore: (i) Self-perceived clinical readiness
(SPCR) before and aer training in the VR simulator and (ii)
VR system usability.
MATERIAL AND METHODS
Study design
A single group study design was used to assess feasibility
of training acquisition of wrist radiographs using a VR-
based simulation. e study was approved by the Local
Committee for Ethics in Research (RA2008003). All students
of radiography, currently attending University College, i.e.,
not in clinical placement were invited to participate. Ten
students of radiography at varying stages of their education
were recruited voluntarily. All participants were given a
unique ID and neither name nor any other person-sensitive
data were collected. e tests took place over a 3-day period
in April 2022. e participants were informed that they
could withdraw from the study at any time without having
to provide a reason. ey were also informed that they
could take a break during the simulation if needed. An oral
introduction was prepared to ensure that all participants
received the same information.
Data collection
Data were collected using moderated usability testing,
a real-time test with the presence of a trained facilitator
with an in-depth knowledge of the VR system.[14] Each
test had a duration of approximately 1 h and consisted
of three questionnaires, VR introduction videos, and the
acquisition of wrist radiographs in the VR simulator. All
tests were performed at a classroom at University College of
Radiography.
Questionnaires
Before the VR simulation, the participants answered two
questionnaires - an initial questionnaire on demographics,
VR-and/or gaming experience, and with wrist X-rays and
next, a questionnaire on SPCR in relation to obtaining wrist
radiographs was answered. e SPCR questionnaire was
answered twice, a pre-SPCR before entering the VR simulator
and a post-SPCR aer completing the VR simulation.
e SPCR questionnaire contained ten statements on the
participant’s self-perceived ability when obtaining wrist
X-rays using a ve-point Likert scale (strongly disagree,
disagree, neutral, agree, and strongly agree) with statements
such as “I feel insecure on positioning the wrist of the patient
when performing musculoskeletal X-ray” [Appendix 1]. e
same SPCR questionnaire was answered aer completion
of the VR simulation. Finally, a modied questionnaire
on system usability, system usability scale (SUS),[15,16] was
answered by all participants containing statements such
as “I felt very condent using the system.” e SUS was
expanded with 2 statements (1) “I think the X-ray room was
realistic” and (2) “I think the simulator can contribute with
learning to the education of radiography,” totaling 2 questions
[Appendix 2].
e SUS scores were calculated using the method suggested
by Alathas[16] by adding the scores (from 1 = strongly disagree
to 5 = strongly agree) of all the odd-numbered questions and
subtracting 5 from the total to achieve a scoreodd. e total of
all even-numbered questions was subtracted from 25 and this
sum was multiplied by 2.5 to achieve the scoreeven. e sum
of scoreodd and scoreeven was the nal total SUS score. us,
a SUS score from 0 to 100 could be achieved, with higher
Journal of Clinical Imaging Science • 2023 • 13(20) | 3
Jensen, et al.: Virtual reality and wrist X-rays
SUS scores indicating higher usability.[16] e SPCR score was
calculated using the same method as described for the SUS
scores, therefore the higher the SPCR-score the better, with
100 being the maximum score obtainable.
VR simulator
e VR simulation was developed in collaboration with a
company specializing in VR soware for medical education
(VitaSim ApS, Odense, Denmark). In the VR simulator, a
short immersive VR tutorial was given to the participants,
before positioning the patient and taking posterior-anterior
(PA) and lateral wrist radiographs. e tutorial consisted of
an oral introduction to the VR simulator and the basic VR
interactions. Hereaer, the students were led into the X-ray
simulation room where an avatar introduced the tasks in the
room, i.e., positioning the virtual patient, taking the wrist
radiographs and how they could get feedback on radiographic
positioning. In Figure1, the test ow is presented.
e virtual patient had the following anatomical movement
of the shoulder (abduction, adduction, and rotation), elbow
(exion and extension), and wrist (supination, pronation,
exion, extension, radial, and ulnar deviation). Moreover,
the participants had the opportunity to remove the skin to
expose the bones. Figures 2 and 3, respectively, depict the
X-ray room as seen in the VR simulator and the VR patient
with the bones exposed.
Aer positioning the patient and collimating the image,
PA and lateral wrist radiographs were obtained of the VR
patient, the participants were shown the corresponding
radiographs and given immediate feedback on positioning,
i.e., deviation, exion, extension, and rotation. e feedback
was visually depicted on the wall behind the patient. e
user could choose between feedback following each single
radiograph or aer acquisition of both PA and lateral
radiographs. An avatar appeared in the X-ray room and gave
further information on how to interpret the feedback as well
as theoretical background knowledge on positioning criteria
of PA and lateral wrist radiographs. Following the feedback,
the participants could reposition the patient and obtain as
many retakes as needed. Updated feedback was available
following each new wrist radiograph.
Technical VR setup
An Acer Predator Helios 300 with RTX 3070 GPU and 16
GB RAM (Compal Electronics Inc., Taiwan) was used for
Figure 1: Flowchart of the virtual reality
simulation test.
Figure 2: e virtual X-ray room with the patient seated and
positioned for a wrist radiograph.
Figure 3: e patient is positioned for a posterior-anterior wrist
radiograph and the bones have been exposed to help with correct
positioning for a retake.
Journal of Clinical Imaging Science • 2023 • 13(20) | 4
Jensen, et al.: Virtual reality and wrist X-rays
developing and running the VR simulator tethered. An
Oculus Quest headset (Meta Technologies, Meta Inc., Menlo
Park, CA, USA) was used for immersive VR.
Data analysis
Demographics of participants were presented descriptively
with mean and range. e SUS scores were presented by
mean and standard deviation (SD). e modied questions
on simulator realism and educational usability were reported
in percentage.
e SPCR score was presented by mean and SD with
corresponding 95% condence intervals (CIs). Assuming
normality of data, a student’s paired t-test was applied to
assess dierences between pre- and post-SPCR scores.
Results were visually depicted in a box plot. Stata version16
(StataCorp.2019, TX) was used for statistical analyses.
RESULTS
Ten participants (6females and 4 males) with a mean age
of 23.5 years (range: 20–27) completed the VR simulation.
e students were at dierent stages of their education
(1stsemester n = 1; 3rdsemester n = 5; 5thsemester n = 4). e
3rd and 5th semester students had educational and practical
experience with musculoskeletal radiographs of the wrist.
Demographics on experience with video games, VR, and
wrist X-rays are shown in Figure4.
Based on subjective assessment of how condent the
participants felt obtaining a radiograph of the wrist, the mean
pre-SPCR score was 75(95% CI: 54–96) and SD: 29.8. Average
post-SPCR score was 77(95% CI: 59–95), SD (25) [Figure5].
us, no statistically signicant dierences were shown
between the pre-and post-SPCR score, P = 0.4574 [Figure6].
e average SUS score was 77.5 (SD: 19.1) out of 100. Most
of the participants, 90% (n = 9), strongly agreed that the
X-ray room in the VR simulator was realistic whereas one
participant was neutral on this question. Regarding learning
potential, 90% (n = 9) of the participants strongly agreed
that the VR simulator could contribute to learning at the
radiographer education.
DISCUSSION
is study piloted feasibility of using a VR simulator as an
educational tool when training correct positioning of wrist
radiographs with specic objectives of exploring: (i) SPCR
before and aer training in the VR simulator and (ii) system
usability. In the simulator, feedback was oered based on
patient positioning along with educational pointers on wrist
radiographs and positioning regarding anatomical landmarks
and how to assess anatomical positioning of the wrist and
hand in a radiograph.
Despite positive feedback on the use of the simulator, the
SPCR score was improved only by an average of 2 points
Figure4: Demographics. n = 10(6female; 4male).
Figure6: Box plot with medians, quartiles, and ranges comparing
the pre-and post-self-perceived clinical readiness scores.
Figure5: Depicting individual pre-and post-self-perceived clinical
readiness (SPCR) scores (n = 10). Cases with only one red dot signify
equal pre-and post-SPCR scores, as seen in participants 1, 5, and 10.
Journal of Clinical Imaging Science • 2023 • 13(20) | 5
Jensen, et al.: Virtual reality and wrist X-rays
aer the completion of the VR sessions. is nding could
indicate what psychologists Kruger and Dunning described
in 1999 – the Dunning–Kruger eect. ey described
aer four experiments that students who objectively
performed well oen subjectively underestimated their own
performances.[17] On the other hand, students that performed
objectively poor overestimated their own performances.[17]
Considering that all participants in the current study were
students, they had a rather high SPCR regarding obtaining
wrist radiographs. is may in part be because of the
relatively low esteem of musculoskeletal radiography.
However, while the radiography skills and competencies
have not altered overtime, the importance of accuracy and
recognition of the skill may have become downgraded by
some.[6] Aforementioned Dunning–Kruger[17] eect may also
explain the student’s trust in own competences, or perhaps,
more likely, a combination of the two.
As technologies and imaging modalities have developed,
musculoskeletal radiography has been increasingly seen
as less complex compared to magnetic resonance imaging
and computed tomography. [6] A hypothesis could be that
the students overestimated their own ability to obtain wrist
radiographs assuming that it is a common and therefore
simple procedure. In a recent rejection analysis study, it was,
however, uncovered that improper positioning was one of the
most common reasons for retake of digital musculoskeletal
radiographs.[18] Hence, in a profession such as radiography, it
is critical that clinical competency assessments are part of the
curriculum both as formative assessments for identifying the
strengths and weaknesses and for highlighting deciencies,
or as summative pass/fail tests, e.g., for high-risk or high-
stake procedures. By implementing tests, trainees, in this
case, radiography students can train the procedure repeatedly
until sucient prociency. Some students might need more
training, guidance, and supervision than others due to, e.g.,
dierent previous experiences and learning paces. is
principle is called mastery learning and is a frequent and well-
described method for health-care education.[19] VR supports
the principles of mastery learning, because it is possible to
incorporate assessment tools within the virtual simulation.
It is possible to make the training and learning period more
exible to the students and decrease the number of instructors
needed compared to, e.g., classroom-based education and
training. Mastery learning, however, requires clear learning
objectives, a competence level to meet or exceed, and the
opportunity for continuous feedback. To our knowledge, no
studies have been published presenting VR tests for assessing
radiography positioning skills but solid validity evidence have
been published for VR tests in ultrasound and for preparing
students of radiography for clinical practice.[8,13]
Particularly, regarding positioning and musculoskeletal
radiography, even small degrees of malpositioning can
inuence the diagnostic value derived from the radiograph.
In the case of wrist radiographs, forearm rotation away from
the mid-prone position can depict the apparent angulation
of the articular surface of the radius dierently as seen in the
lateral image. Studies have shown that forearm supination
can decrease the radiographically measured value of dorsal
angulation and conversely, pronation may increase the
apparent dorsal angulation.[5,20,21] Patient positioning is also
important in other anatomical regions, such as the pelvis,
where pelvic rotation during the radiographic procedure will
impact the appearance of radiographic signs of acetabular
retroversion such as the crossover sign.[22] Hence, the value of
VR training and positioning in musculoskeletal radiography
is important in, but not limited to, wrist radiographs.
Limitations
An inherent limitation in the current study is the low
number of participants (n = 10) combined with the fact that
the participants were at various stages of their education.
A larger and more homogenous cohort of participants
might have changed the results. e path from educational
research to clinical outcomes is long with multiple factors
and variables aecting the pathway. e rst step could be
to investigate the clinical eect of the VR training, e.g.,
exploring the transfer to a clinical setting by comparing the
image quality (positioning) of radiographers or students who
have trained using VR compared to those who have trained
using normal methods. An alternative approach could be to
gather validity evidence for a VR-based simulation test in
radiography competences such as the wrist X-ray. Moreover,
this VR simulator allows for training in patient positioning
only, and thus, factors such as radiation protection, radiation
dose, and subsequent technical image quality are not assessed
in this study.
CONCLUSION
is pilot study showed no dierence in SPCR before and
aer the use of VR. e feasibility of using VR simulation
for training correct positioning of wrist radiographs
was identied though with positive feedback from the
participants. Additional studies are recommended to further
elaborate on this initial research.
Declaration of patient consent
e Institutional Review Board (IRB) permission was
obtained for the study.
Financial support and sponsorship
is research was funded by “Innovationspuljen” at Odense
University Hospital. e funding body had no role in study
Journal of Clinical Imaging Science • 2023 • 13(20) | 6
Jensen, et al.: Virtual reality and wrist X-rays
design; data collection, analyses, or interpretation; in the
writing of the manuscript; or in the decision to publish the
results.
Conicts of interest
R.O.J. is a co-founder and co-owner of VitaSim. R.O.J.
advised the authors M.G.T. and S.K.K.G. on technical
solutions to developing the X-ray training module in virtual
reality. R.O.J. had no inuence on data collection, data
analysis, interpretation, or publishing decisions.
REFERENCES
1. Nellans KW, Kowalski E, Chung KC. e epidemiology of
distal radius fractures. Hand Clin 2012;28:113-25.
2. Lalone EA, Grewal R, King GJ, MacDermid JC. Astructured
review addressing the use of radiographic measures of
alignment and the denition of acceptability in patients with
distal radius fractures. Hand 2015;10:621-38.
3. Lichtman DM, Bindra RR, Boyer MI, Putnam MD, Ring D,
Slutsky DJ, et al. American Academy of Orthopaedic Surgeons
clinical practice guideline on: e treatment of distal radius
fractures. JBone Joint Surg Am 2011;93:775-8.
4. Jensen J, Tromborg HB, Rasmussen BS, Gerke O, Torng T,
Precht H, et al. e eect of forearm rotation on radiographic
measurements of the wrist: An experimental study using
radiostereometric analyses on cadavers. Eur Radiol Exp
2021;5:15.
5. Jensen J, Tromborg HB, Rasmussen BS, Gerke O, Torng T,
Precht H, et al. Dorsal tilt of the distal radius fracture changes
with forearm rotation when measured on radiographs. JHand
Surg Glob Online 2021;3:182-9.
6. Mussmann BR, Hardy M, Jensen J. ere’s nothing plain about
projection radiography! A discussion paper. Radiography
2021;27:1227-30.
7. O’Connor M, Stowe J, Potocnik J, Giannotti N, Murphy S,
Rainford L. 3D virtual reality simulation in radiography
education: e students’ experience. Radiography (Lond)
2021;27:208-14.
8. O’Connor M, Rainford L. e impact of 3D virtual reality
radiography practice on student performance in clinical
practice. Radiography (Lond) 2022;29:159-64.
9. Larsen JD, Jensen RO, Pietersen PI, Jacobsen N, Falster C,
Nielsen AB, et al. Education in focused lung ultrasound using
gamied immersive virtual reality: A randomized controlled
study. Ultrasound Med Biol 2023;49:841-52.
10. Jacobsen N, Larsen JD, Falster C, Nolsøe CP, Konge L,
Graumann O, et al. Using immersive virtual reality simulation
to ensure competence in contrast-enhanced ultrasound.
Ultrasound Med Biol 2022;48:912-23.
11. Andersen NL, Jensen RO, Konge L, Laursen CB, Falster C,
Jacobsen N, et al. Immersive virtual reality in basic point-
of-care ultrasound training: A randomized controlled trial.
Ultrasound Med Biol 2023;49:178-85.
12. Nielsen MR, Kristensen EQ, Jensen RO, Mollerup AM,
Pfeier T, Graumann O. Clinical ultrasound education
for medical students: Virtual reality versus e-learning, a
randomized controlled pilot trial. Ultrasound Q 2021;37:292-6.
13. Nielsen MS, Clausen JH, Homann-Petersen J, Konge L,
Nielsen AB. Can virtual-reality simulation ensure transthoracic
echocardiography skills before trainees examine patients? Int J
Med Educ 2022;13:267-73.
14. Justinmind. Moderated vs Unmoderated Usability Testing:
Which Do You Need? Available from: https://www.justinmind.
com/blog/moderated-vs-unmoderated-usability-testing-
which-do-you-need [Last accessed on 2022Dec18].
15. Brooke J. SUS: Aretrospective. JUsability Stud 2013;8:29-40.
16. Alathas H. How to Measure Product Usability with the System
Usability Scale (SUS) Score; 2008. Available from: https://
uxplanet.org/how-to-measure-product-usability-with-the-
system-usability-scale-sus-score-69f3875b858f [Last accessed
on 2022Dec18].
17. Kruger J, Dunning D. Unskilled and unaware of it: How
diculties in recognizing ones own incompetence lead to
inated self-assessments. JPers Soc Psychol 1999;88:1121-34.
18. Rastegar S, Beigi J, Saeidi E, Dezhkam A, Mobaderi T,
Ghaari H, et al. Reject analysis in digital radiography: Alocal
study on radiographers and students’ attitude in Iran. Med J
Islam Repub Iran 2019;33:49.
19. McGaghie WC. Mastery learning: It is time for medical
education to join the 21stcentury. Acad Med 2015;90:1438-41.
20. Pennock AT, Phillips CS, Matzon JL, Daley E. e eects
of forearm rotation on three wrist measurements: Radial
inclination, radial height and palmar tilt. Hand Surg
2005;10:17-22.
21. Zanetti M, Gilula LA, Jacob HA, Hodler J. Palmar tilt of
the distal radius: Inuence of o-lateral projection initial
observations. Radiology 2001;220:594-600.
22. Siebenrock KA, Kalbermatten DF, Ganz R. Eect of pelvic tilt
on acetabular retroversion: A study of pelves from cadavers.
Clin Orthop Relat Res 2003;407:241-8.
How to cite this article: Jensen J, Graumann O, Jensen RO, Gade SK,
ielsen MG, Most W, et al. Using virtual reality simulation for training
practical skills in musculoskeletal wrist X-ray - A pilot study. J Clin
Imaging Sci 2023;13:20.
Journal of Clinical Imaging Science • 2023 • 13(20) | 7
Jensen, et al.: Virtual reality and wrist X-rays
Appendix 1: SPCR.
S. No. SPCR
1. I feel theoretically condent about how to perform
musculoskeletal X-rays of the wrist
2. I feel condent in being able to carry out
musculoskeletal X-rays of wrists in the clinic
3. I feel uncertain about how to use an X-ray machine
4. I feel uncertain about positioning the patient’s wrist
when performing a musculoskeletal X-ray
5. I feel condent about how to position for the good PA
wrist X-ray
6 I feel condent about how to position for the good
lateral wrist X-ray
7. I feel uncertain about assessing whether an x-ray of a
wrist is diagnostically useful
8. I feel condent in what anatomical structures to look for
to assess whether a PA wrist X-ray is appropriate
9. I feel condent in what anatomical structures to look for
to assess whether a lateral wrist X-ray is appropriate
10. I feel condent in how to ne tune the wrist position
based on the X-ray.
All questions were answered on a scale from 1 to 5.(1: Strongly disagree,
2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree). SPCR: Self-perceived
clinical readiness, PA: Posterior-anterior
Appendix 2: System usability scale.
1. I think that I would like to use this VR simulator frequently
2. I found the VR simulator unnecessarily complex
3. I thought that the VR simulator was easy to use
4. I think that I would need the support of a technical person to
be able to use this VR simulator
5. I found that the various functions in this VR simulator were
well integrated
6. I thought that there was too much inconsistency in this VR
simulator
7. I would imagine that most people would learn to use this VR
simulator very quickly
8. I found the VR simulator very cumbersome to use
9. I felt very condent using the VR simulator
10. I needed to learn a lot of things before I could get going with
this VR simulator
11. I think the X-ray room was realistic
12. I think the VR simulator can contribute with learning to the
education of radiography.
All questions were answered on a scale from 1 to 5.(1: Strongly disagree,
2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree). VR: Virtual reality
APPENDICES
... The studies used a wide range of VR software and hardware. Some of the studies used 3D simulation software packages displayed on 2D desktop computers [22,24,25,36], whereas others used headsets for an immersive VR environment [15,23,26,35,37]. The most used VR teaching software were the CETSOL VR Clinic software [33,35], Virtual Medical Coaching VR software [15,30,32], Projection VR (Shaderware) software [36], SieVRt VR system (Luxsonic Technologies) [37], medical imaging training immersive environment software [23], VR CT Sim software [25], VitaSim ApS software [26], VR X-Ray (Skilitics and Virtual Medical Coaching) software [27], and radiation dosimetry VR software (Virtual Medical Coaching Ltd) [31]. ...
... Some of the studies used 3D simulation software packages displayed on 2D desktop computers [22,24,25,36], whereas others used headsets for an immersive VR environment [15,23,26,35,37]. The most used VR teaching software were the CETSOL VR Clinic software [33,35], Virtual Medical Coaching VR software [15,30,32], Projection VR (Shaderware) software [36], SieVRt VR system (Luxsonic Technologies) [37], medical imaging training immersive environment software [23], VR CT Sim software [25], VitaSim ApS software [26], VR X-Ray (Skilitics and Virtual Medical Coaching) software [27], and radiation dosimetry VR software (Virtual Medical Coaching Ltd) [31]. ...
... The findings from the study by Gunn et al [25] revealed that 68% of students agreed or strongly agreed that VR simulation was significantly helpful in learning about computed tomography (CT) scanning. In another study by Jensen et al [26], 90% of the students strongly agreed that VR simulators could contribute to learning radiography, with 90% reporting that the x-ray equipment in the VR simulation was realistic. In the study by Wu et al [37], most of the students (55.6%) agreed or somewhat agreed that VR use was useful in radiology education. ...
Article
Full-text available
Background: In recent years, virtual reality (VR) has gained significant importance in medical education. Radiology education also has seen the induction of VR technology. However, there is no comprehensive review in this specific area. This review aims to fill this knowledge gap. Objective: This systematic literature review aims to explore the scope of VR use in radiology education. Methods: A literature search was carried out using PubMed, Scopus, ScienceDirect, and Google Scholar for articles relating to the use of VR in radiology education, published from database inception to September 1, 2023. The identified articles were then subjected to a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)–defined study selection process. Results: The database search identified 2503 nonduplicate articles. After PRISMA screening, 17 were included in the review for analysis, of which 3 (18%) were randomized controlled trials, 7 (41%) were randomized experimental trials, and 7 (41%) were cross-sectional studies. Of the 10 randomized trials, 3 (30%) had a low risk of bias, 5 (50%) showed some concerns, and 2 (20%) had a high risk of bias. Among the 7 cross-sectional studies, 2 (29%) scored “good” in the overall quality and the remaining 5 (71%) scored “fair.” VR was found to be significantly more effective than traditional methods of teaching in improving the radiographic and radiologic skills of students. The use of VR systems was found to improve the students’ skills in overall proficiency, patient positioning, equipment knowledge, equipment handling, and radiographic techniques. Student feedback was also reported in the included studies. The students generally provided positive feedback about the utility, ease of use, and satisfaction of VR systems, as well as their perceived positive impact on skill and knowledge acquisition. Conclusions: The evidence from this review shows that the use of VR had significant benefit for students in various aspects of radiology education. However, the variable nature of the studies included in the review reduces the scope for a comprehensive recommendation of VR use in radiology education
... This may be attributed to ChatGPT's ability to identify learning difficulties in real-time and provide guidance, reducing cognitive load and facilitating active learning, as described in Lee's study [6]. The provision of tangible simulation training with templates and digital imaging can also enhance understanding and concretize abstract concepts [7,8]. For skill scores, the ChatGPT and template groups performed similarly, outperforming the traditional group by 11% post-training and 5% at the 3-month follow-up. ...
Article
Full-text available
Background Traditional puncture skills training for refresher doctors faces limitations in effectiveness and efficiency. This study explored the application of generative AI (ChatGPT), templates, and digital imaging to enhance puncture skills training. Methods 90 refresher doctors were enrolled sequentially into 3 groups: traditional training; template and digital imaging training; and ChatGPT, template and digital imaging training. Outcomes included theoretical knowledge, technical skills, and trainee satisfaction measured at baseline, post-training, and 3-month follow-up. Results The ChatGPT group increased theoretical knowledge scores by 17–21% over traditional training at post-training (81.6 ± 4.56 vs. 69.6 ± 4.58, p < 0.001) and follow-up (86.5 ± 4.08 vs. 71.3 ± 4.83, p < 0.001). It also outperformed template training by 4–5% at post-training (81.6 ± 4.56 vs. 78.5 ± 4.65, p = 0.032) and follow-up (86.5 ± 4.08 vs. 82.7 ± 4.68, p = 0.004). For technical skills, the ChatGPT (4.0 ± 0.32) and template (4.0 ± 0.18) groups showed similar scores at post-training, outperforming traditional training (3.6 ± 0.50) by 11% (p < 0.001). At follow-up, ChatGPT (4.0 ± 0.18) and template (4.0 ± 0.32) still exceeded traditional training (3.8 ± 0.43) by 5% (p = 0.071, p = 0.026). Learning curve analysis revealed fastest knowledge (slope 13.02) and skill (slope 0.62) acquisition for ChatGPT group over template (slope 11.28, 0.38) and traditional (slope 5.17, 0.53). ChatGPT responses showed 100% relevance, 50% completeness, 60% accuracy, with 15.9 s response time. For training satisfaction, ChatGPT group had highest scores (4.2 ± 0.73), over template (3.8 ± 0.68) and traditional groups (2.6 ± 0.94) (p < 0.01). Conclusion Integrating AI, templates and digital imaging significantly improved puncture knowledge and skills over traditional training. Combining technological innovations and AI shows promise for streamlining complex medical competency mastery.
Article
Virtual reality (VR) simulation is a technology that empowers students and radiographers to practice radiography in a virtual environment that resembles real-life clinical scenarios. The purpose of this randomised study was to examine the relationship between clinical specialty and the ability to assess and obtain a lateral wrist radiograph using a VR simulator.
Article
Full-text available
This study was aimed at comparing the learning efficacy of a traditional instructor-led lesson with that of a completely virtual, self-directed lesson in immersive virtual reality (IVR) in teaching basic point-of-care ultrasound (PoCUS) skills. We conducted a blinded, non-inferiority, parallel-group, randomized controlled trial in which final-year medical students were randomized to an instructor-led (n = 53) or IVR (n = 51) lesson. Participants’ learning efficacy was evaluated by blinded assessors, who rated each participant's performance using the Objective Structured Assessment of Ultrasound Skills (OSAUS) assessment tool.The mean total scores for participants were 11.0 points (95% confidence interval: 9.8–12.2) for the instructor-led lesson and 10.3 points (95% confidence interval: 9.0–11.5) for the IVR lesson. No significant differences were observed between the groups with respect to total score (p = 0.36) or subgroup objectives of the OSAUS score (p = 0.34 for familiarity, p = 0.45 for image optimization, p = 0.96 for systematic approach and p = 0.07 for interpretation). Maintenance costs for both courses were estimated at 400 euros each. Startup costs for the instructor-led course were estimated 16 times higher than those for the IVR course. The learning efficacy of an instructor-led lesson on basic US did not differ significantly from that of a self-directed lesson in IVR, as assessed using the OSAUS. The results suggest that IVR could be an equivalent alternative to instructor-led lessons in future basic US courses, but further research is warranted to clarify the role of IVR in PoCUS courses.
Article
Full-text available
Objectives: This study aimed to develop and gather the validity evidence for a standardised simulation-based skills test in transthoracic echocardiography and to establish a credible pass/fail score. Methods: Experts developed a virtual-reality simulator test in cardiology, medical education and simulation-based education. Thirty-six physicians with different experiences in transthoracic echocardiography completed the test at Odense University Hospital, Denmark. The performances of novice, intermediate and experienced participants were compared using the Bonferroni post hoc test. Cronbach's alpha was used to determine the internal consistency reliability of the test. The consistency of performance was analysed using the intraclass correlation coefficient. A pass/fail score was established using the contrasting groups' standard-setting method. Results: We developed a test with high consistent reliability (Alpha = .81), 95% CI [.69, .89]. In both cases, the performers' level was consistent, fitting others at the same level of experience (intraclass correlation r(35)=.81, p<.001). A pass/fail score of 48/50 points was established based on the mean test score of novice and experienced physicians. Conclusions: We developed a standardised virtual-reality simulation-based test of echocardiography skills with the ability to distinguish between participants with different levels of transthoracic echocardiography experience. This test could direct a mastery learning training program where trainees practise until they reach the pre-defined level and secure a higher level of competency to ensure quality and safety for patients.
Article
Full-text available
Objectives: Unlike the technological advances in cross-sectional imaging, the adoption of CR and DR has been relatively overlooked in terms of the additional radiographer skills and competences required for optimal practice. Furthermore, projection radiography is often referred to as basic, plain or other words suggesting simplicity or entry-level skill requirements. Radiographers' professional identity is connected with the discourse expressed via the language used in daily practice and consequently, if the perception of projection radiography is regarded as simple practice not requiring much reflection or complex decision-making, apathy and carelessness may arise. The purpose of this narrative review was to raise projection radiography from its longstanding lowly place and re-position it as a specialist imaging field. Key findings: Danish pre-registration radiography curricula contain little mention of projection radiography and a low proportion (n = 17/144; 11.8%) of Danish radiography students chose to focus on projection radiography within publicly available BSc. theses between 2016 and 2020 as compared to topics related to CT and MRI (n = 60/144; 41.7%). Conclusion: By changing how we as the profession perceive the role and position of projection radiography, we can start to rebuild its lost prestige and demand a greater, more detailed and clinically relevant educational offering from academic partners. For this to commence, the language and terminology we use to describe ourselves and tasks undertaken must reflect the complexity of the profession. Implications for practice: Regardless of imaging modality, every patient should be assured that a radiographer with expertise in acquiring images of diagnostic quality undertakes their examination. Reclaiming the prestige of projection radiography may lead students and radiographers to recognize projection radiography as a demanding specialist field for the benefit of the patients.
Article
Full-text available
Purpose This study examined the impact of pronation and supination on the reliability of the radiographically measured values of dorsal tilt, radial inclination (RI), and ulnar variance (UV) in cadaveric forearms with artificially created distal radius fractures. Methods We prepared 21 human cadaveric forearms (11 right and 10 left) for radiostereometric analysis (RSA) by insertion of tantalum markers. Distal radius fractures were created midway between the marker segments. Radiographs and RSA images were taken at different degrees of supination and pronation. The precise degree of forearm rotation was calculated using RSA software. Two observers (H.B.T. and T.T.) independently measured tilt, RI, and UV on all radiographs in a blinded and randomized fashion. Univariate linear regression analyses were used to determine the relationship between forearm rotation and the measured radiographic values. Results The radiographically measured value of tilt was significantly impacted by forearm rotation. Supinating or pronating the forearm by 10° decreased and increased, respectively, the radiographic value of dorsal tilt by approximately 3°. Conclusions This study showed that the positioning of the fractured forearm during the radiographic procedure significantly impacted subsequent radiographic measurements of tilt. Dorsal tilt measurements increased (ie, fracture displacement measured more dorsal) with pronation and decreased (ie, fracture displacement measured more toward neutral, with less dorsal tilt) with supination of the forearm. However, measurements of RI (p = 0.12 and p = 0.55 for observer 1 and 2) and UV (p = 0.34 and p = 0.17, observer 1 and 2) were not significantly impacted by rotation. Clinical relevance Treatment of a distal radius fracture is, at least to some extent, based on radiographic quantification of fracture deformity. Therefore, unreliable measurements may adversely influence clinical decision making.
Article
Full-text available
Background Although dorsal/palmar tilt, radial inclination (RI), and ulnar variance (UV) are measurements commonly performed in wrist radiographs, the impact of forearm rotation on those measurements during the radiographic procedure is uncertain. Our aim was to determine the impact of supination and pronation on the reliability of measurements of tilt, RI, and UV. Methods Tantalum markers were inserted into the distal radius of 21 unfractured cadaver forearms. The forearms were radiographed in different degrees of supination and pronation. The exact degree of rotation was calculated with radiostereometric analyses. Tilt, RI, and UV were measured by two independent readers in a random and anonymised fashion. Association between forearm rotation and radiographic measurements was examined using linear regression. Results Forearm rotation significantly impacted the radiographically measured tilt. One degree of supination and pronation respectively increased and decreased palmar tilt with 0.68° and 0.44°, observers 1 and 2, respectively. As opposed to observer 1, observer 2 found that RI was significantly impacted by rotation with a slope of 0.08. Ulnar variance was not significantly impacted by rotation with linear regression slopes of 0.01° (95% confidence interval [CI] − 0.02–0.05, p = 0.490) and 0.02° (95% CI − 0.02–0.07; p = 0.288), observer 1 and observer 2, respectively. Conclusion In unfractured forearms, the radiographically measured tilt was significantly affected by rotation. Palmar tilt increased with supination and decreased with pronation. Rotation significantly affected radial inclination, although of a magnitude that is probably not clinically relevant. No significant impact on UV was found.
Article
Focused lung ultrasound (FLUS) has high diagnostic accuracy in many common conditions seen in a variety of emergency settings. Competencies are essential for diagnostic success and patient safety but can be challenging to acquire in clinical environments. Immersive virtual reality (IVR) offers an interactive risk-free learning environment and is progressing as an educational tool. First, this study explored the educational impact of novice FLUS users participating in a gamified or non-gamified IVR training module in FLUS by comparing test scores using a test with proven validity evidence. Second, the learning effect was assessed by comparing scores of each group with known test scores of novices, intermediates and experienced users in FLUS. A total of 48 participants were included: 24 received gamified and 24 received non-gamified IVR training. No significant difference was found between gamified (mean = 15.5 points) and non-gamified (mean = 15.2 points), indicating that chosen gamification elements for our setup did not affect learning outcome (p = 0.66). The mean scores of both groups did not significantly differ from those of known intermediate users in FLUS (gamified p = 0.63, non-gamified p = 0.24), indicating that both IVR modules could be used as unsupervised out-of-hospital training for novice trainees in FLUS.
Article
Introduction Simulation-based learning plays an integral role in preparing students for clinical practice. This study investigated the impact of immersive three-dimensional (3D) virtual reality (VR) simulation-based learning on first-year radiography students’ performance in the clinical setting. Methods A retrospective analysis of first-year radiography clinical assessments was carried out to compare performance pre-and post-introduction of VR. The stage one cohort with no VR education was considered the control group (n = 93). The VR group (n = 98) had seven hours of practice in the immersive VR suite (Virtual Medical Coaching). Experienced clinical tutors assessed first-year students performing an extremity radiographic examination in the clinical setting. Assessment criteria were ranked on a 5-point Likert scale from poor to excellent. Mann Whitney U Tests were applied to compare performance across cohorts. Results Students trained with VR performed better across 20 of the 22 assessment criteria. VR-trained students performed significantly better (more ranked as ‘very good’ or ‘excellent’) than the control group in the following criteria; positioning patients for X-rays (19% difference) (U = 3525, z = −2.66, p < 0.05), selecting exposure factors (12% difference) (U = 3680, z = −3.13, p < 0.05), image appraisal of patient positioning (27% difference) (U = 3448, z = −2.9, p < 0.05) and image appraisal of image quality (18% difference) (U = 3514, z = −2.6, p < 0.05). Their comprehension of clinical indications, equipment set up and explanation of the procedure was also significantly better (p < 0.05). Conclusion This is the first study to investigate the translation of VR learning into radiography clinical practice. VR learning had a positive impact on the performance of first-year students in their clinical assessment, especially with respect to patient positioning, exposure parameter selection and image appraisal. Implications for practice VR is a valuable educational tool in preparing novice radiography students for clinical practice. It is particularly useful to enhance student knowledge in the areas of patient positioning, exposure factor selection and radiographic image appraisal.
Article
Contrast-enhanced ultrasound (CEUS) is used in various medical specialties as a diagnostic imaging tool and for procedural guidance. Experience in the procedure is currently attained via supervised clinical practice that is challenged by patient availability and risks. Prior simulation-based training and subsequent assessment could improve and ensure competence before performance on patients, but no simulator currently exists. Immersive virtual reality (IVR) is a new promising simulation tool that can replicate complex interactions and environments that are unfeasible to achieve by traditional simulators. This study was aimed at developing an IVR simulation-based test for core CEUS competencies and gathering validity evidence for the test in accordance with Messick's framework. The test was developed by IVR software specialists and clinical experts in CEUS and medical education and imitated a CEUS examination of a patient with a focal liver lesion with emphasis on the pre-contrast preparations. Twenty-five medical doctors with varying CEUS experience were recruited as test participants, and their results were used to analyze test quality and to establish a pass/fail standard. The final test of 23 test items had good internal reliability (Cronbach's α = 0.85) and discriminatory abilities. The risks of false positives and negatives (9.1% and 23.6%, respectively) were acceptable for the test to be used as a certification tool prior to supervised clinical training in CEUS.
Article
The primary aim was to evaluate the effect of immersive virtual reality learning for training medical students in basic clinical ultrasound. Secondary outcomes were to explore if virtual reality learning had an effect on hand-eye coordination skills and if the medical students wanted more virtual reality learning. This pilot study was a double-blind, parallel-group, block-randomized, controlled trial. Participants (n = 20) were blinded and randomized to virtual reality or e-learning for basic ultrasound education. Medical students with no previous ultrasound education were recruited voluntarily from the University of Southern Denmark. Data were collected during introductory courses on ultrasound from March to May 2019. Participants were assessed with Objective Structured Assessment on Ultrasound Skills. Assessing supervisors were blinded. The virtual reality group (n = 11) scored a significantly higher Objective Structured Assessment on Ultrasound Skills score (143 [95% confidence interval {CI}, 135 to 151]) compared with the e-learning group (n = 9; 126 [95% CI, 113 to 138]; mean difference, 17 points [95% CI, 4 to 30]; P < 0.01). No significant effect on the hand-eye score was found (mean difference, 3 points [95 % CI, −3 to 9]; P = 0.32). Ninety-one percent of the virtual reality group wanted more virtual reality learning. Immersive virtual reality learning improved medical students’ ultrasound skills significantly compared with e-learning. The hand-eye score was higher in the virtual reality group, although not at a significant level. Students wanted more virtual reality learning. Further research is needed to clarify immersive virtual reality's educational role in the future.