Cliffhanger-VR

Full text

Cliffhanger-VR
Marcel Tiator*
University of Applied Sciences D¨
usseldorf
Ben Fischer†
University of Applied Sciences D¨
usseldorf
Laurin Gerhardt‡
University of Applied Sciences D¨
usseldorf
David Nowottnik§
University of Applied Sciences D¨
usseldorf
Hendrik Preu¶
University of Applied Sciences D¨
usseldorf
Christian Geiger ||
University of Applied Sciences D¨
usseldorf
ABSTRACT
The performance and security in outdoor climbing sports can depend
on anxieties. These appear in frightening situations where, e.g. a
deep fall is risked. Deep falls can lead to serious injuries or even to
fatal accidents. Such situations can be trained in order to be mentally
resistant to them and thereby to make climbing safer. However,
drawbacks have to be taken into account. The trainee has to bring
himself in a possible hazardous situation and nature is not directly
reachable for every person. Thus, we present a system, where a user
can climb at low heights in reality and simultaneously on a high
cliff in
VR
. In this contribution, we describe the system architecture
and future possibilities to safely train stressful outdoor climbing
situations indoors.
Index Terms:
Virtual Reality-Climbing—Mental Training—Body-
Tracking—Finger-Tracking;
1 INTRODUCTION
There exist many different forms of climbing, such as bouldering,
top rope or lead, which can be done indoors and outdoors. Climbing
outdoors is very different in comparison to climbing indoors. On
the one hand, more things have to be managed, e.g., a climbing
route guidebook has to be obtained, the rock has to be found in
nature and enough food has to be taken along for the trip. On the
other hand, there are a lot of benefits: it is fun, it is an adventure,
mastering stressful situations together can cement a friendship, one
can also experience extraordinary views on its own, make journeys
or spend holidays in outdoor climbing [9]. However, injuries have
to be taken into account when falling [4], such that fear of falling
can be increased [13]. Furthermore, getting overwhelmed by fear
or panic while climbing can reduce the performance [3] and may
lead to fatal accidents [1]. It is possible to train and get mentally
resistant to fears [10], but drawbacks have to be taken into account.
Mental training in a gym may not have the same effect as mental
training outdoors. If this training is done outdoors, fatal accidents
can happen. Moreover, it is not always possible to reach a rock with-
out doing a day trip since most people live far from nature areas with
climbing spots. Therefore, we want to offer the possibility to train
frightening outdoor climbing situations indoors with a
VR
simulator.
Accordingly, we present Cliffhanger-VR: a system incorporating
a Head Mounted Display (
HMD
), where a user can climb at low
heights in reality while climbing on high cliffs in VR.
*e-mail: marcel.tiator@study.hs-duessledorf.de
†e-mail: Ben.Fischer.st@gmx.de
‡e-mail: laurinjoel@googlemail.com
§e-mail: david@nowottnik.com
¶e-mail: preu@tresarts.com
||e-mail: geiger@hs-duesseldorf.de
Figure 1: User climbs in VR on real climbing tower.
2 RE LATE D WORK
Most research in the field of climbing focuses on performance mea-
surements and enhancement incorporating
AR
or wearables. The
movement acquisition via wearable Inertial Measurement Units
(
IMU
s) or accelerometers [6, 8] could be advantageous, because
wearables could potentially support a huge tracking space and thus
rope climbing too. With the optional use of a rope, lead scenarios
could be supported as well. In contrast to optical tracking methods,
they are robust against occlusions. In order to visually offer the user
an outdoor climbing scenario, augmentation of an artificial climbing
wall with a projection could be considered [5, 7, 15]. However, we
think that it is easier to offer the perception of heights when one
climbs closer to the ground with
VR
. Moreover, the projection of
nature on an artificial climbing wall could be disturbed by shadows
thrown by the climber and the holds.
There are also few examples of climbing in
VR
. Commercial
games offer the perception of alpine or outdoor climbing experiences,
where the user interacts with controllers in order to imitate climbing
movements [2,14]. In these applications, climbing movements are
simulated to a certain degree, but natural interactions in terms of
using hands and feet are not supported and the user does not feel
the forces which appear when climbing in reality. In contrast to that,
Qualcomm reported from a student project, where climbing with the
whole body and a
HMD
is possible [12]. They attached the PS Move
controllers and a Leap Motion controller to the
HMD
. With this
method, a climber can see only his hands in reality and
VR
while
climbing, but noisy edges from the Leap Motion are visible, which
can decrease the perceived immersion. As opposed to existing work,
our system enables users to climb in reality and see only virtual
objects (body parts and environment). Besides that, we acquire data
of the climbing movements of hands, feet, head and fingers whereby
a climbing performance measurement would be possible.
3 SYSTEM ARCHITECTURE AND APPLICATION
In order to give motion feedback and support natural climbing move-
ments in
VR
, the system consists of the HTC-Vive-system and a
self-developed glove. Four Vive trackers, which are fixed on 3D

printed mounts, are attached to user’s hands and feet by velcros.
Placed on the two gloves, bending sensors measure flexion and ex-
tension movements of every finger. The bending sensors of each
hand are linked with a microcontroller, which sends the bending data
via 433 MHz module to one host receiver module. Furthermore, the
data communication is solved via ping-pong principle. This means
the host receiver queries data from the senders and waits for data
packets.
For the purpose of supporting a haptic feeling through visual
matching of the walls and stones of the tower, we reconstructed them
in 3D. The tower was easily measured by hand and reconstructed
with a 3D modeling tool. Since the shape of the stones is more
complicated than the shape of the tower, we scanned them with a
3D depth sensor. The mesh, created from the 3D point cloud of the
scanning process, was simplified in several ways, e.g., by merging
vertices to get less number of polygons for real-time purposes. 3D
reconstruction of the tower and stones alone is not enough to provide
correct position and orientation of these elements in the virtual scene.
The position and orientation of the tower are measured by a fixed
Vive tracker, which is statically attached to the tower. Like in reality,
the virtualized stones can only be attached to the holes in the wall.
Moreover, in our model, they are subordinate under the walls of the
tower and can only be rotated in the roll direction. Therefore, a user
chooses a certain stone and hole, such that the roll rotation can be
applied by pressing a button on the Vive controller and turning the
arm in the roll direction. This is done till the real and virtual roll of
the stone are approximately equal. To avoid stone placement after
each session, the placements are saved in a .json file.
Lastly, the user has to be calibrated in three steps after putting
on the trackers and the gloves. First, the range of motion of the
finger flexion is measured to map the bending data of the sensors
onto finger animations. Hence, we measure the values of the sensors
with an open and closed hand. Second, the virtual transformation
of the hands (position and scale) is not matching the real one. This
deviation is corrected by putting the user’s hands on predefined
positions one over the other. After that, the scale is corrected via
the distance of trackers. Third, the same correction is done with the
feet. In contrast to the hands, the user’s scale of his feet is corrected
by specifying the shoe size. After these calibration steps, the user
is able to precisely climb on the walls in
VR
. Among other things,
we think that a more immersive system increases the possibility
of anxiety perception due to the increased possibility of presence
perception (sense of being there) [11]. Finally, the perception of fear
is important to train calmness while climbing.
In accordance with that, wind machines in an optional combina-
tion with a wind pitch modulation played via headphones are used.
The use of headphones may increase perceived immersion through
auralization of virtual elements, e.g., birds that fly around. When
using the headphones it is possible to talk via microphone to the
operators to avoid the full isolation of the user, especially when he
feels threatened. In order to enhance the immersion, the user walks
over planks, which can give a greater sense of the perceived height.
The system is experimentally implemented in a scenario
1
, where the
user has to collect spheres and has to put his hand on certain spots
at the wall. The scenario takes place in a mountain environment,
where the real tower is placed in the scene (Fig. 1).
4 CONCLUSION
With this simulator, it is possible to climb at a very low height in
reality while climbing in
VR
on a high cliff, such that frightening
outdoor situations can be simulated. In order to prove that such a
simulator induces the fear of falling and provides an appropriate per-
ception of presence, such that training calmness is possible, several
investigations have to be done. Moreover, the visualization in
VR
1
Video of the demo-application:
https://www.vimeo.com/
248452265
should provide the illusion of a natural cliff, to realistically simulate
the outside climbing scenario. The whole system works well from
our perspective, but there is still room for technical improvements.
Since we are using an optical tracking, trackers or the
HMD
could
be occluded leading to visualization errors. This could be a critical
point, where a user might feel like losing control. Additionally,
the tracking space of the HTC Vive is limited, which restricts the
climbing area.
This problem could be reduced by using a multi-camera setup
for tracking, or by using an inside-out tracking
HMD
combined
with wearable
IMU
’s, such that rope climbing could be supported.
Another consideration is to use a treadmill climbing wall, where
a fixed tracking area would be appropriate. Hence, the feeling
of climbing higher can be supported, while the climber is still at
the same low height. To decrease the disturbance of perceived
immersion due to the imprecise finger tracking, we want to constrain
the visualized finger bending animation to appropriately grab objects.
After finalizing the development, we presented our system at an in-
house exhibition and received informal feedback. Most users were
able to climb the route with
VR
while belayed by a rope. Some of
them slipped and fell down, but continued to climb the route to the
end, because the motivation to finish the experience was high. The
combination of
VR
, hand-tracking and sport-climbing aroused their
curiosity for participating on the experience.
ACKNOWLEDGMENTS
We wish to thank the wood workshop of the Peter Behrens School
of Arts for the great help while building the climbing tower.
REFERENCES
[1]
S. Backe, L. Ericson, S. Janson, and T. Timpka. Rock climbing injury
rates and associated risk factors in a general climbing population. Scan-
dinavian Journal of Medicine & Science in Sports, 19(6):850–856, dec
2009. doi: 10.1111/j.1600-0838.2008.00851.x
[2] Crytek GmbH. The Climb: Official Site - Home page.
[3]
N. Draper, G. A. Jones, S. Fryer, C. I. Hodgson, and G. Blackwell.
Physiological and psychological responses to lead and top rope climb-
ing for intermediate rock climbers. European Journal of Sport Science,
10(1):13–20, 2010. doi: 10.1080/17461390903108125
[4]
G. Jones, A. Asghar, and D. J. Llewellyn. The epidemiology of rock-
climbing injuries. British Journal of Sports Medicine, 42(9):773–778,
2008. doi: 10.1136/bjsm.2007.037978
[5]
R. Kajastila. The Augmented Climbing Wall : High-Exertion Proximity
Interaction on a Wall-Sized Interactive Surface. 2016.
[6]
F. Kosmalla and F. Daiber. ClimbSense - Automatic Climbing Route
Recognition using Wrist-worn Inertia Measurement Units. pp. 2033–
2042, 2015.
[7]
F. Kosmalla, F. Daiber, and F. Wiehr. ClimbVis — Investigating In-situ
Visualizations for Understanding Climbing Movements by Demonstra-
tion. pp. 270–279, 2017.
[8]
C. Ladha, N. Y. Hammerla, P. Olivier, and T. Pl. ClimbAX : Skill
Assessment for Climbing Enthusiasts. pp. 235–244, 2013.
[9] A. Laurence. Why People Climb Mountains.
[10]
M. Lloyd. The Mind Game: How to Overcome Fear - Climbing
Magazine, 2014.
[11]
M. Price and P. Anderson. The role of presence in virtual reality
exposure therapy. 21(5):742–751, 2007. doi:
10.1016/j.janxdis.2006.11.
002.The
[12]
Qualcomm. Qualcomm Institute Convenes Conference on Future of
Virtual Reality.
[13] A. Rainey. Climbing: Dealing With the Shock of Real Rock.
[14] Solfar Studios. Everest VR.
[15]
F. Wiehr, F. Kosmalla, F. Daiber, and A. Kr
¨
uger. betaCube – Enhancing
Training for Climbing by a Self-Calibrating Camera-Projection Unit.
pp. 1998–2004, 2016.