Available via license: CC BY-NC-SA 4.0
Content may be subject to copyright.
Journal of Clinical Imaging Science • 2023 • 13(20) | 1
is is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others
to remix, transform, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.
©2023 Published by Scientic Scholar on behalf of Journal of Clinical Imaging Science
Original Research Musculoskeletal Imaging
Using virtual reality simulation for training practical
skills in musculoskeletal wrist X-ray-A pilot study
Janni Jensen1, Ole Graumann2, Rune Overgaard Jensen2, Signe K. K. Gade2, Maria Grabau ielsen2, Winnie Most3,
Pia Iben Pietersen2
1Department of Radiology, Odense University Hospital, 2Research and Innovation Unit of Radiology, Department of Clinical Research, University of Southern
Denmark, 3Department of Radiography, University College UCL, Odense, Denmark.
*Corresponding author:
Janni Jensen,
Department of Radiology,
Odense University Hospital,
Odense, Denmark.
janni.jensen@rsyd.dk
Received : 05May2023
Accepted : 13June 2023
Published : 11 July 2023
DOI
10.25259/JCIS_45_2023
Quick Response Code:
INTRODUCTION
Distal radius fractures (DRFs) are one of the most commonly occurring fractures and account
for 15–20% of all fractures. e DRF is typically diagnosed on radiographs taken in the acute
setting.[1] Treatment of a DRF is, in part, based on quantication of fracture displacement as
measured in radiographs.[2,3] Improper patient positioning during the radiographic procedure can
aect the measured values of fracture displacement; in other words, correct patient positioning
is directly linked to diagnostic value.[4,5] Correct radiographic positioning is, however, a skill that
requires practice and knowledge.[6]
Until recently, radiographic positioning has typically been taught in the classroom or trained
using phantoms. With the advent of new technologies such as virtual reality (VR), students
ABSTRACT
Objectives: Using virtual reality (VR), students of radiography can practice acquisition and positioning of
musculoskeletal radiographs and get immediate feedback on their performance within the simulator. e purpose
of this study was to assess usability of a newly developed VR simulator and to explore self-perceived clinical
readiness (SPCR) of radiography students before and aer training acquisition of wrist radiographs in the VR
simulator.
Material and Methods: Aprospective methodology was applied where the students (n = 10) estimated their
own SPCR in regard to acquisition of wrist radiographs pre-and post-VR training. Aquestionnaire on usability,
realism, and educational value of the simulator was answered post-VR training. Usability and SPCR scores were
calculated. e student’s paired t-test was applied to explore the impact of VR training on SPCR.
Results: e students (90%) reported that the simulator was realistic and they thought that it could contribute to
learning. e pre-and post-SPCR scores were 75(95% condence interval [CI]: 54–96) and 77(95% CI: 59–95),
respectively. ere was no signicant dierence (P = 0.4574) between the pre-and post-SPCR scores.
Conclusion: Results indicated that the concept of training acquisition and positioning of wrist radiographs in a
VR simulator is feasible with positive feedback from the students. e SPCR scores improved slightly, although
not statistically signicant, aer completion of the training session.
Keywords: Virtual reality, Wrist radiography, Patient positioning
www.clinicalimagingscience.org
Journal of Clinical Imaging Science
Journal of Clinical Imaging Science • 2023 • 13(20) | 2
Jensen, et al.: Virtual reality and wrist X-rays
of radiography are given the opportunity to practice and
simulate musculoskeletal radiography acquisition skills using
a VR headset and controllers while in a safe and limitless
environment with neither radiation nor involvement of
patients.[7] e risk of radiation makes it challenging for
radiographers to train radiographic positioning involving
patients; this risk is eliminated using VR. Another challenge
is that it may be time consuming to get real-time feedback
on the radiographs from an expert. Arecent study involving
1styear students of radiography showed positive correlation
between VR simulation and their subsequent clinical
performance across a range of parameters.[8]
e virtual three-dimensional VR universe has also been
used in educational contexts within other health domains.[9-12]
Studies have already explored the eect of VR learning in the
areas of ultrasound, anatomy, and surgery and have found
signicant positive eects although further research is
suggested in previous studies.[11,13]
e use of VR has also been seen within the eld of X-ray
training where students of radiography reported a positive
attitude toward training procedures in VR. e biggest
challenge they faced was lack of tactile feedback when
palpating anatomical landmarks during positioning.
Moreover, a general lack of interaction and communication
with the patient was missing as well as feedback on
positioning from the system. e students mentioned
that they would like feedback on their radiographs so they
could learn how to improve their positioning and improve
diagnostic quality of the images.[7]
ere is, to the best of our knowledge, a relatively limited
amount of literature on the topic - especially on students
training in a virtual simulation where the focus is positioning
(rotation, deviation, and exion/extension) of the wrist
involving direct feedback and information based on their
own radiographs as taken in the VR simulator. Accordingly,
the primary aim of this study was to conduct a pilot study
and evaluate the eect of an immersive VR simulator for
training musculoskeletal X-ray of the wrist. e objectives of
this study were to explore: (i) Self-perceived clinical readiness
(SPCR) before and aer training in the VR simulator and (ii)
VR system usability.
MATERIAL AND METHODS
Study design
A single group study design was used to assess feasibility
of training acquisition of wrist radiographs using a VR-
based simulation. e study was approved by the Local
Committee for Ethics in Research (RA2008003). All students
of radiography, currently attending University College, i.e.,
not in clinical placement were invited to participate. Ten
students of radiography at varying stages of their education
were recruited voluntarily. All participants were given a
unique ID and neither name nor any other person-sensitive
data were collected. e tests took place over a 3-day period
in April 2022. e participants were informed that they
could withdraw from the study at any time without having
to provide a reason. ey were also informed that they
could take a break during the simulation if needed. An oral
introduction was prepared to ensure that all participants
received the same information.
Data collection
Data were collected using moderated usability testing,
a real-time test with the presence of a trained facilitator
with an in-depth knowledge of the VR system.[14] Each
test had a duration of approximately 1 h and consisted
of three questionnaires, VR introduction videos, and the
acquisition of wrist radiographs in the VR simulator. All
tests were performed at a classroom at University College of
Radiography.
Questionnaires
Before the VR simulation, the participants answered two
questionnaires - an initial questionnaire on demographics,
VR-and/or gaming experience, and with wrist X-rays and
next, a questionnaire on SPCR in relation to obtaining wrist
radiographs was answered. e SPCR questionnaire was
answered twice, a pre-SPCR before entering the VR simulator
and a post-SPCR aer completing the VR simulation.
e SPCR questionnaire contained ten statements on the
participant’s self-perceived ability when obtaining wrist
X-rays using a ve-point Likert scale (strongly disagree,
disagree, neutral, agree, and strongly agree) with statements
such as “I feel insecure on positioning the wrist of the patient
when performing musculoskeletal X-ray” [Appendix 1]. e
same SPCR questionnaire was answered aer completion
of the VR simulation. Finally, a modied questionnaire
on system usability, system usability scale (SUS),[15,16] was
answered by all participants containing statements such
as “I felt very condent using the system.” e SUS was
expanded with 2 statements (1) “I think the X-ray room was
realistic” and (2) “I think the simulator can contribute with
learning to the education of radiography,” totaling 2 questions
[Appendix 2].
e SUS scores were calculated using the method suggested
by Alathas[16] by adding the scores (from 1 = strongly disagree
to 5 = strongly agree) of all the odd-numbered questions and
subtracting 5 from the total to achieve a scoreodd. e total of
all even-numbered questions was subtracted from 25 and this
sum was multiplied by 2.5 to achieve the scoreeven. e sum
of scoreodd and scoreeven was the nal total SUS score. us,
a SUS score from 0 to 100 could be achieved, with higher
Journal of Clinical Imaging Science • 2023 • 13(20) | 3
Jensen, et al.: Virtual reality and wrist X-rays
SUS scores indicating higher usability.[16] e SPCR score was
calculated using the same method as described for the SUS
scores, therefore the higher the SPCR-score the better, with
100 being the maximum score obtainable.
VR simulator
e VR simulation was developed in collaboration with a
company specializing in VR soware for medical education
(VitaSim ApS, Odense, Denmark). In the VR simulator, a
short immersive VR tutorial was given to the participants,
before positioning the patient and taking posterior-anterior
(PA) and lateral wrist radiographs. e tutorial consisted of
an oral introduction to the VR simulator and the basic VR
interactions. Hereaer, the students were led into the X-ray
simulation room where an avatar introduced the tasks in the
room, i.e., positioning the virtual patient, taking the wrist
radiographs and how they could get feedback on radiographic
positioning. In Figure1, the test ow is presented.
e virtual patient had the following anatomical movement
of the shoulder (abduction, adduction, and rotation), elbow
(exion and extension), and wrist (supination, pronation,
exion, extension, radial, and ulnar deviation). Moreover,
the participants had the opportunity to remove the skin to
expose the bones. Figures 2 and 3, respectively, depict the
X-ray room as seen in the VR simulator and the VR patient
with the bones exposed.
Aer positioning the patient and collimating the image,
PA and lateral wrist radiographs were obtained of the VR
patient, the participants were shown the corresponding
radiographs and given immediate feedback on positioning,
i.e., deviation, exion, extension, and rotation. e feedback
was visually depicted on the wall behind the patient. e
user could choose between feedback following each single
radiograph or aer acquisition of both PA and lateral
radiographs. An avatar appeared in the X-ray room and gave
further information on how to interpret the feedback as well
as theoretical background knowledge on positioning criteria
of PA and lateral wrist radiographs. Following the feedback,
the participants could reposition the patient and obtain as
many retakes as needed. Updated feedback was available
following each new wrist radiograph.
Technical VR setup
An Acer Predator Helios 300 with RTX 3070 GPU and 16
GB RAM (Compal Electronics Inc., Taiwan) was used for
Figure 1: Flowchart of the virtual reality
simulation test.
Figure 2: e virtual X-ray room with the patient seated and
positioned for a wrist radiograph.
Figure 3: e patient is positioned for a posterior-anterior wrist
radiograph and the bones have been exposed to help with correct
positioning for a retake.
Journal of Clinical Imaging Science • 2023 • 13(20) | 4
Jensen, et al.: Virtual reality and wrist X-rays
developing and running the VR simulator tethered. An
Oculus Quest headset (Meta Technologies, Meta Inc., Menlo
Park, CA, USA) was used for immersive VR.
Data analysis
Demographics of participants were presented descriptively
with mean and range. e SUS scores were presented by
mean and standard deviation (SD). e modied questions
on simulator realism and educational usability were reported
in percentage.
e SPCR score was presented by mean and SD with
corresponding 95% condence intervals (CIs). Assuming
normality of data, a student’s paired t-test was applied to
assess dierences between pre- and post-SPCR scores.
Results were visually depicted in a box plot. Stata version16
(StataCorp.2019, TX) was used for statistical analyses.
RESULTS
Ten participants (6females and 4 males) with a mean age
of 23.5 years (range: 20–27) completed the VR simulation.
e students were at dierent stages of their education
(1stsemester n = 1; 3rdsemester n = 5; 5thsemester n = 4). e
3rd and 5th semester students had educational and practical
experience with musculoskeletal radiographs of the wrist.
Demographics on experience with video games, VR, and
wrist X-rays are shown in Figure4.
Based on subjective assessment of how condent the
participants felt obtaining a radiograph of the wrist, the mean
pre-SPCR score was 75(95% CI: 54–96) and SD: 29.8. Average
post-SPCR score was 77(95% CI: 59–95), SD (25) [Figure5].
us, no statistically signicant dierences were shown
between the pre-and post-SPCR score, P = 0.4574 [Figure6].
e average SUS score was 77.5 (SD: 19.1) out of 100. Most
of the participants, 90% (n = 9), strongly agreed that the
X-ray room in the VR simulator was realistic whereas one
participant was neutral on this question. Regarding learning
potential, 90% (n = 9) of the participants strongly agreed
that the VR simulator could contribute to learning at the
radiographer education.
DISCUSSION
is study piloted feasibility of using a VR simulator as an
educational tool when training correct positioning of wrist
radiographs with specic objectives of exploring: (i) SPCR
before and aer training in the VR simulator and (ii) system
usability. In the simulator, feedback was oered based on
patient positioning along with educational pointers on wrist
radiographs and positioning regarding anatomical landmarks
and how to assess anatomical positioning of the wrist and
hand in a radiograph.
Despite positive feedback on the use of the simulator, the
SPCR score was improved only by an average of 2 points
Figure4: Demographics. n = 10(6female; 4male).
Figure6: Box plot with medians, quartiles, and ranges comparing
the pre-and post-self-perceived clinical readiness scores.
Figure5: Depicting individual pre-and post-self-perceived clinical
readiness (SPCR) scores (n = 10). Cases with only one red dot signify
equal pre-and post-SPCR scores, as seen in participants 1, 5, and 10.
Journal of Clinical Imaging Science • 2023 • 13(20) | 5
Jensen, et al.: Virtual reality and wrist X-rays
aer the completion of the VR sessions. is nding could
indicate what psychologists Kruger and Dunning described
in 1999 – the Dunning–Kruger eect. ey described
aer four experiments that students who objectively
performed well oen subjectively underestimated their own
performances.[17] On the other hand, students that performed
objectively poor overestimated their own performances.[17]
Considering that all participants in the current study were
students, they had a rather high SPCR regarding obtaining
wrist radiographs. is may in part be because of the
relatively low esteem of musculoskeletal radiography.
However, while the radiography skills and competencies
have not altered overtime, the importance of accuracy and
recognition of the skill may have become downgraded by
some.[6] Aforementioned Dunning–Kruger[17] eect may also
explain the student’s trust in own competences, or perhaps,
more likely, a combination of the two.
As technologies and imaging modalities have developed,
musculoskeletal radiography has been increasingly seen
as less complex compared to magnetic resonance imaging
and computed tomography. [6] A hypothesis could be that
the students overestimated their own ability to obtain wrist
radiographs assuming that it is a common and therefore
simple procedure. In a recent rejection analysis study, it was,
however, uncovered that improper positioning was one of the
most common reasons for retake of digital musculoskeletal
radiographs.[18] Hence, in a profession such as radiography, it
is critical that clinical competency assessments are part of the
curriculum both as formative assessments for identifying the
strengths and weaknesses and for highlighting deciencies,
or as summative pass/fail tests, e.g., for high-risk or high-
stake procedures. By implementing tests, trainees, in this
case, radiography students can train the procedure repeatedly
until sucient prociency. Some students might need more
training, guidance, and supervision than others due to, e.g.,
dierent previous experiences and learning paces. is
principle is called mastery learning and is a frequent and well-
described method for health-care education.[19] VR supports
the principles of mastery learning, because it is possible to
incorporate assessment tools within the virtual simulation.
It is possible to make the training and learning period more
exible to the students and decrease the number of instructors
needed compared to, e.g., classroom-based education and
training. Mastery learning, however, requires clear learning
objectives, a competence level to meet or exceed, and the
opportunity for continuous feedback. To our knowledge, no
studies have been published presenting VR tests for assessing
radiography positioning skills but solid validity evidence have
been published for VR tests in ultrasound and for preparing
students of radiography for clinical practice.[8,13]
Particularly, regarding positioning and musculoskeletal
radiography, even small degrees of malpositioning can
inuence the diagnostic value derived from the radiograph.
In the case of wrist radiographs, forearm rotation away from
the mid-prone position can depict the apparent angulation
of the articular surface of the radius dierently as seen in the
lateral image. Studies have shown that forearm supination
can decrease the radiographically measured value of dorsal
angulation and conversely, pronation may increase the
apparent dorsal angulation.[5,20,21] Patient positioning is also
important in other anatomical regions, such as the pelvis,
where pelvic rotation during the radiographic procedure will
impact the appearance of radiographic signs of acetabular
retroversion such as the crossover sign.[22] Hence, the value of
VR training and positioning in musculoskeletal radiography
is important in, but not limited to, wrist radiographs.
Limitations
An inherent limitation in the current study is the low
number of participants (n = 10) combined with the fact that
the participants were at various stages of their education.
A larger and more homogenous cohort of participants
might have changed the results. e path from educational
research to clinical outcomes is long with multiple factors
and variables aecting the pathway. e rst step could be
to investigate the clinical eect of the VR training, e.g.,
exploring the transfer to a clinical setting by comparing the
image quality (positioning) of radiographers or students who
have trained using VR compared to those who have trained
using normal methods. An alternative approach could be to
gather validity evidence for a VR-based simulation test in
radiography competences such as the wrist X-ray. Moreover,
this VR simulator allows for training in patient positioning
only, and thus, factors such as radiation protection, radiation
dose, and subsequent technical image quality are not assessed
in this study.
CONCLUSION
is pilot study showed no dierence in SPCR before and
aer the use of VR. e feasibility of using VR simulation
for training correct positioning of wrist radiographs
was identied though with positive feedback from the
participants. Additional studies are recommended to further
elaborate on this initial research.
Declaration of patient consent
e Institutional Review Board (IRB) permission was
obtained for the study.
Financial support and sponsorship
is research was funded by “Innovationspuljen” at Odense
University Hospital. e funding body had no role in study
Journal of Clinical Imaging Science • 2023 • 13(20) | 6
Jensen, et al.: Virtual reality and wrist X-rays
design; data collection, analyses, or interpretation; in the
writing of the manuscript; or in the decision to publish the
results.
Conicts of interest
R.O.J. is a co-founder and co-owner of VitaSim. R.O.J.
advised the authors M.G.T. and S.K.K.G. on technical
solutions to developing the X-ray training module in virtual
reality. R.O.J. had no inuence on data collection, data
analysis, interpretation, or publishing decisions.
REFERENCES
1. Nellans KW, Kowalski E, Chung KC. e epidemiology of
distal radius fractures. Hand Clin 2012;28:113-25.
2. Lalone EA, Grewal R, King GJ, MacDermid JC. Astructured
review addressing the use of radiographic measures of
alignment and the denition of acceptability in patients with
distal radius fractures. Hand 2015;10:621-38.
3. Lichtman DM, Bindra RR, Boyer MI, Putnam MD, Ring D,
Slutsky DJ, et al. American Academy of Orthopaedic Surgeons
clinical practice guideline on: e treatment of distal radius
fractures. JBone Joint Surg Am 2011;93:775-8.
4. Jensen J, Tromborg HB, Rasmussen BS, Gerke O, Torng T,
Precht H, et al. e eect of forearm rotation on radiographic
measurements of the wrist: An experimental study using
radiostereometric analyses on cadavers. Eur Radiol Exp
2021;5:15.
5. Jensen J, Tromborg HB, Rasmussen BS, Gerke O, Torng T,
Precht H, et al. Dorsal tilt of the distal radius fracture changes
with forearm rotation when measured on radiographs. JHand
Surg Glob Online 2021;3:182-9.
6. Mussmann BR, Hardy M, Jensen J. ere’s nothing plain about
projection radiography! A discussion paper. Radiography
2021;27:1227-30.
7. O’Connor M, Stowe J, Potocnik J, Giannotti N, Murphy S,
Rainford L. 3D virtual reality simulation in radiography
education: e students’ experience. Radiography (Lond)
2021;27:208-14.
8. O’Connor M, Rainford L. e impact of 3D virtual reality
radiography practice on student performance in clinical
practice. Radiography (Lond) 2022;29:159-64.
9. Larsen JD, Jensen RO, Pietersen PI, Jacobsen N, Falster C,
Nielsen AB, et al. Education in focused lung ultrasound using
gamied immersive virtual reality: A randomized controlled
study. Ultrasound Med Biol 2023;49:841-52.
10. Jacobsen N, Larsen JD, Falster C, Nolsøe CP, Konge L,
Graumann O, et al. Using immersive virtual reality simulation
to ensure competence in contrast-enhanced ultrasound.
Ultrasound Med Biol 2022;48:912-23.
11. Andersen NL, Jensen RO, Konge L, Laursen CB, Falster C,
Jacobsen N, et al. Immersive virtual reality in basic point-
of-care ultrasound training: A randomized controlled trial.
Ultrasound Med Biol 2023;49:178-85.
12. Nielsen MR, Kristensen EQ, Jensen RO, Mollerup AM,
Pfeier T, Graumann O. Clinical ultrasound education
for medical students: Virtual reality versus e-learning, a
randomized controlled pilot trial. Ultrasound Q 2021;37:292-6.
13. Nielsen MS, Clausen JH, Homann-Petersen J, Konge L,
Nielsen AB. Can virtual-reality simulation ensure transthoracic
echocardiography skills before trainees examine patients? Int J
Med Educ 2022;13:267-73.
14. Justinmind. Moderated vs Unmoderated Usability Testing:
Which Do You Need? Available from: https://www.justinmind.
com/blog/moderated-vs-unmoderated-usability-testing-
which-do-you-need [Last accessed on 2022Dec18].
15. Brooke J. SUS: Aretrospective. JUsability Stud 2013;8:29-40.
16. Alathas H. How to Measure Product Usability with the System
Usability Scale (SUS) Score; 2008. Available from: https://
uxplanet.org/how-to-measure-product-usability-with-the-
system-usability-scale-sus-score-69f3875b858f [Last accessed
on 2022Dec18].
17. Kruger J, Dunning D. Unskilled and unaware of it: How
diculties in recognizing ones own incompetence lead to
inated self-assessments. JPers Soc Psychol 1999;88:1121-34.
18. Rastegar S, Beigi J, Saeidi E, Dezhkam A, Mobaderi T,
Ghaari H, et al. Reject analysis in digital radiography: Alocal
study on radiographers and students’ attitude in Iran. Med J
Islam Repub Iran 2019;33:49.
19. McGaghie WC. Mastery learning: It is time for medical
education to join the 21stcentury. Acad Med 2015;90:1438-41.
20. Pennock AT, Phillips CS, Matzon JL, Daley E. e eects
of forearm rotation on three wrist measurements: Radial
inclination, radial height and palmar tilt. Hand Surg
2005;10:17-22.
21. Zanetti M, Gilula LA, Jacob HA, Hodler J. Palmar tilt of
the distal radius: Inuence of o-lateral projection initial
observations. Radiology 2001;220:594-600.
22. Siebenrock KA, Kalbermatten DF, Ganz R. Eect of pelvic tilt
on acetabular retroversion: A study of pelves from cadavers.
Clin Orthop Relat Res 2003;407:241-8.
How to cite this article: Jensen J, Graumann O, Jensen RO, Gade SK,
ielsen MG, Most W, et al. Using virtual reality simulation for training
practical skills in musculoskeletal wrist X-ray - A pilot study. J Clin
Imaging Sci 2023;13:20.
Journal of Clinical Imaging Science • 2023 • 13(20) | 7
Jensen, et al.: Virtual reality and wrist X-rays
Appendix 1: SPCR.
S. No. SPCR
1. I feel theoretically condent about how to perform
musculoskeletal X-rays of the wrist
2. I feel condent in being able to carry out
musculoskeletal X-rays of wrists in the clinic
3. I feel uncertain about how to use an X-ray machine
4. I feel uncertain about positioning the patient’s wrist
when performing a musculoskeletal X-ray
5. I feel condent about how to position for the good PA
wrist X-ray
6 I feel condent about how to position for the good
lateral wrist X-ray
7. I feel uncertain about assessing whether an x-ray of a
wrist is diagnostically useful
8. I feel condent in what anatomical structures to look for
to assess whether a PA wrist X-ray is appropriate
9. I feel condent in what anatomical structures to look for
to assess whether a lateral wrist X-ray is appropriate
10. I feel condent in how to ne tune the wrist position
based on the X-ray.
All questions were answered on a scale from 1 to 5.(1: Strongly disagree,
2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree). SPCR: Self-perceived
clinical readiness, PA: Posterior-anterior
Appendix 2: System usability scale.
1. I think that I would like to use this VR simulator frequently
2. I found the VR simulator unnecessarily complex
3. I thought that the VR simulator was easy to use
4. I think that I would need the support of a technical person to
be able to use this VR simulator
5. I found that the various functions in this VR simulator were
well integrated
6. I thought that there was too much inconsistency in this VR
simulator
7. I would imagine that most people would learn to use this VR
simulator very quickly
8. I found the VR simulator very cumbersome to use
9. I felt very condent using the VR simulator
10. I needed to learn a lot of things before I could get going with
this VR simulator
11. I think the X-ray room was realistic
12. I think the VR simulator can contribute with learning to the
education of radiography.
All questions were answered on a scale from 1 to 5.(1: Strongly disagree,
2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree). VR: Virtual reality
APPENDICES