Conference PaperPDF Available

Cliffhanger-VR

Authors:

Figures

Content may be subject to copyright.
Cliffhanger-VR
Marcel Tiator*
University of Applied Sciences D¨
usseldorf
Ben Fischer
University of Applied Sciences D¨
usseldorf
Laurin Gerhardt
University of Applied Sciences D¨
usseldorf
David Nowottnik§
University of Applied Sciences D¨
usseldorf
Hendrik Preu
University of Applied Sciences D¨
usseldorf
Christian Geiger ||
University of Applied Sciences D¨
usseldorf
ABSTRACT
The performance and security in outdoor climbing sports can depend
on anxieties. These appear in frightening situations where, e.g. a
deep fall is risked. Deep falls can lead to serious injuries or even to
fatal accidents. Such situations can be trained in order to be mentally
resistant to them and thereby to make climbing safer. However,
drawbacks have to be taken into account. The trainee has to bring
himself in a possible hazardous situation and nature is not directly
reachable for every person. Thus, we present a system, where a user
can climb at low heights in reality and simultaneously on a high
cliff in
VR
. In this contribution, we describe the system architecture
and future possibilities to safely train stressful outdoor climbing
situations indoors.
Index Terms:
Virtual Reality-Climbing—Mental Training—Body-
Tracking—Finger-Tracking;
1 INTRODUCTION
There exist many different forms of climbing, such as bouldering,
top rope or lead, which can be done indoors and outdoors. Climbing
outdoors is very different in comparison to climbing indoors. On
the one hand, more things have to be managed, e.g., a climbing
route guidebook has to be obtained, the rock has to be found in
nature and enough food has to be taken along for the trip. On the
other hand, there are a lot of benefits: it is fun, it is an adventure,
mastering stressful situations together can cement a friendship, one
can also experience extraordinary views on its own, make journeys
or spend holidays in outdoor climbing [9]. However, injuries have
to be taken into account when falling [4], such that fear of falling
can be increased [13]. Furthermore, getting overwhelmed by fear
or panic while climbing can reduce the performance [3] and may
lead to fatal accidents [1]. It is possible to train and get mentally
resistant to fears [10], but drawbacks have to be taken into account.
Mental training in a gym may not have the same effect as mental
training outdoors. If this training is done outdoors, fatal accidents
can happen. Moreover, it is not always possible to reach a rock with-
out doing a day trip since most people live far from nature areas with
climbing spots. Therefore, we want to offer the possibility to train
frightening outdoor climbing situations indoors with a
VR
simulator.
Accordingly, we present Cliffhanger-VR: a system incorporating
a Head Mounted Display (
HMD
), where a user can climb at low
heights in reality while climbing on high cliffs in VR.
*e-mail: marcel.tiator@study.hs-duessledorf.de
e-mail: Ben.Fischer.st@gmx.de
e-mail: laurinjoel@googlemail.com
§e-mail: david@nowottnik.com
e-mail: preu@tresarts.com
||e-mail: geiger@hs-duesseldorf.de
Figure 1: User climbs in VR on real climbing tower.
2 RE LATE D WORK
Most research in the field of climbing focuses on performance mea-
surements and enhancement incorporating
AR
or wearables. The
movement acquisition via wearable Inertial Measurement Units
(
IMU
s) or accelerometers [6, 8] could be advantageous, because
wearables could potentially support a huge tracking space and thus
rope climbing too. With the optional use of a rope, lead scenarios
could be supported as well. In contrast to optical tracking methods,
they are robust against occlusions. In order to visually offer the user
an outdoor climbing scenario, augmentation of an artificial climbing
wall with a projection could be considered [5, 7, 15]. However, we
think that it is easier to offer the perception of heights when one
climbs closer to the ground with
VR
. Moreover, the projection of
nature on an artificial climbing wall could be disturbed by shadows
thrown by the climber and the holds.
There are also few examples of climbing in
VR
. Commercial
games offer the perception of alpine or outdoor climbing experiences,
where the user interacts with controllers in order to imitate climbing
movements [2,14]. In these applications, climbing movements are
simulated to a certain degree, but natural interactions in terms of
using hands and feet are not supported and the user does not feel
the forces which appear when climbing in reality. In contrast to that,
Qualcomm reported from a student project, where climbing with the
whole body and a
HMD
is possible [12]. They attached the PS Move
controllers and a Leap Motion controller to the
HMD
. With this
method, a climber can see only his hands in reality and
VR
while
climbing, but noisy edges from the Leap Motion are visible, which
can decrease the perceived immersion. As opposed to existing work,
our system enables users to climb in reality and see only virtual
objects (body parts and environment). Besides that, we acquire data
of the climbing movements of hands, feet, head and fingers whereby
a climbing performance measurement would be possible.
3 SYSTEM ARCHITECTURE AND APPLICATION
In order to give motion feedback and support natural climbing move-
ments in
VR
, the system consists of the HTC-Vive-system and a
self-developed glove. Four Vive trackers, which are fixed on 3D
printed mounts, are attached to user’s hands and feet by velcros.
Placed on the two gloves, bending sensors measure flexion and ex-
tension movements of every finger. The bending sensors of each
hand are linked with a microcontroller, which sends the bending data
via 433 MHz module to one host receiver module. Furthermore, the
data communication is solved via ping-pong principle. This means
the host receiver queries data from the senders and waits for data
packets.
For the purpose of supporting a haptic feeling through visual
matching of the walls and stones of the tower, we reconstructed them
in 3D. The tower was easily measured by hand and reconstructed
with a 3D modeling tool. Since the shape of the stones is more
complicated than the shape of the tower, we scanned them with a
3D depth sensor. The mesh, created from the 3D point cloud of the
scanning process, was simplified in several ways, e.g., by merging
vertices to get less number of polygons for real-time purposes. 3D
reconstruction of the tower and stones alone is not enough to provide
correct position and orientation of these elements in the virtual scene.
The position and orientation of the tower are measured by a fixed
Vive tracker, which is statically attached to the tower. Like in reality,
the virtualized stones can only be attached to the holes in the wall.
Moreover, in our model, they are subordinate under the walls of the
tower and can only be rotated in the roll direction. Therefore, a user
chooses a certain stone and hole, such that the roll rotation can be
applied by pressing a button on the Vive controller and turning the
arm in the roll direction. This is done till the real and virtual roll of
the stone are approximately equal. To avoid stone placement after
each session, the placements are saved in a .json file.
Lastly, the user has to be calibrated in three steps after putting
on the trackers and the gloves. First, the range of motion of the
finger flexion is measured to map the bending data of the sensors
onto finger animations. Hence, we measure the values of the sensors
with an open and closed hand. Second, the virtual transformation
of the hands (position and scale) is not matching the real one. This
deviation is corrected by putting the user’s hands on predefined
positions one over the other. After that, the scale is corrected via
the distance of trackers. Third, the same correction is done with the
feet. In contrast to the hands, the user’s scale of his feet is corrected
by specifying the shoe size. After these calibration steps, the user
is able to precisely climb on the walls in
VR
. Among other things,
we think that a more immersive system increases the possibility
of anxiety perception due to the increased possibility of presence
perception (sense of being there) [11]. Finally, the perception of fear
is important to train calmness while climbing.
In accordance with that, wind machines in an optional combina-
tion with a wind pitch modulation played via headphones are used.
The use of headphones may increase perceived immersion through
auralization of virtual elements, e.g., birds that fly around. When
using the headphones it is possible to talk via microphone to the
operators to avoid the full isolation of the user, especially when he
feels threatened. In order to enhance the immersion, the user walks
over planks, which can give a greater sense of the perceived height.
The system is experimentally implemented in a scenario
1
, where the
user has to collect spheres and has to put his hand on certain spots
at the wall. The scenario takes place in a mountain environment,
where the real tower is placed in the scene (Fig. 1).
4 CONCLUSION
With this simulator, it is possible to climb at a very low height in
reality while climbing in
VR
on a high cliff, such that frightening
outdoor situations can be simulated. In order to prove that such a
simulator induces the fear of falling and provides an appropriate per-
ception of presence, such that training calmness is possible, several
investigations have to be done. Moreover, the visualization in
VR
1
Video of the demo-application:
https://www.vimeo.com/
248452265
should provide the illusion of a natural cliff, to realistically simulate
the outside climbing scenario. The whole system works well from
our perspective, but there is still room for technical improvements.
Since we are using an optical tracking, trackers or the
HMD
could
be occluded leading to visualization errors. This could be a critical
point, where a user might feel like losing control. Additionally,
the tracking space of the HTC Vive is limited, which restricts the
climbing area.
This problem could be reduced by using a multi-camera setup
for tracking, or by using an inside-out tracking
HMD
combined
with wearable
IMU
’s, such that rope climbing could be supported.
Another consideration is to use a treadmill climbing wall, where
a fixed tracking area would be appropriate. Hence, the feeling
of climbing higher can be supported, while the climber is still at
the same low height. To decrease the disturbance of perceived
immersion due to the imprecise finger tracking, we want to constrain
the visualized finger bending animation to appropriately grab objects.
After finalizing the development, we presented our system at an in-
house exhibition and received informal feedback. Most users were
able to climb the route with
VR
while belayed by a rope. Some of
them slipped and fell down, but continued to climb the route to the
end, because the motivation to finish the experience was high. The
combination of
VR
, hand-tracking and sport-climbing aroused their
curiosity for participating on the experience.
ACKNOWLEDGMENTS
We wish to thank the wood workshop of the Peter Behrens School
of Arts for the great help while building the climbing tower.
REFERENCES
[1]
S. Backe, L. Ericson, S. Janson, and T. Timpka. Rock climbing injury
rates and associated risk factors in a general climbing population. Scan-
dinavian Journal of Medicine & Science in Sports, 19(6):850–856, dec
2009. doi: 10.1111/j.1600-0838.2008.00851.x
[2] Crytek GmbH. The Climb: Official Site - Home page.
[3]
N. Draper, G. A. Jones, S. Fryer, C. I. Hodgson, and G. Blackwell.
Physiological and psychological responses to lead and top rope climb-
ing for intermediate rock climbers. European Journal of Sport Science,
10(1):13–20, 2010. doi: 10.1080/17461390903108125
[4]
G. Jones, A. Asghar, and D. J. Llewellyn. The epidemiology of rock-
climbing injuries. British Journal of Sports Medicine, 42(9):773–778,
2008. doi: 10.1136/bjsm.2007.037978
[5]
R. Kajastila. The Augmented Climbing Wall : High-Exertion Proximity
Interaction on a Wall-Sized Interactive Surface. 2016.
[6]
F. Kosmalla and F. Daiber. ClimbSense - Automatic Climbing Route
Recognition using Wrist-worn Inertia Measurement Units. pp. 2033–
2042, 2015.
[7]
F. Kosmalla, F. Daiber, and F. Wiehr. ClimbVis — Investigating In-situ
Visualizations for Understanding Climbing Movements by Demonstra-
tion. pp. 270–279, 2017.
[8]
C. Ladha, N. Y. Hammerla, P. Olivier, and T. Pl. ClimbAX : Skill
Assessment for Climbing Enthusiasts. pp. 235–244, 2013.
[9] A. Laurence. Why People Climb Mountains.
[10]
M. Lloyd. The Mind Game: How to Overcome Fear - Climbing
Magazine, 2014.
[11]
M. Price and P. Anderson. The role of presence in virtual reality
exposure therapy. 21(5):742–751, 2007. doi:
10.1016/j.janxdis.2006.11.
002.The
[12]
Qualcomm. Qualcomm Institute Convenes Conference on Future of
Virtual Reality.
[13] A. Rainey. Climbing: Dealing With the Shock of Real Rock.
[14] Solfar Studios. Everest VR.
[15]
F. Wiehr, F. Kosmalla, F. Daiber, and A. Kr
¨
uger. betaCube – Enhancing
Training for Climbing by a Self-Calibrating Camera-Projection Unit.
pp. 1998–2004, 2016.
Conference Paper
The perception of the movement of the own fingers is important in VR. Also the sense of touch of the hands provide crucial information about an object that is touched. This is especially noticeable when climbing in VR. While prototyping a VR climbing application, we developed the finger tracking glove g 1. Glove g 1 enables the perception of the finger movements but limits the sense of touch of the hand. Hence, we developed the proposed glove g 2 that enables to feel objects with the skin of the palm and the finger tips. In this paper we describe the iterative design process of g 2. Furthermore, pros and cons of g 2 compared to g 1 are discussed. Finally, an outlook of a future study about the measurement of the presence and the body ownership is given.
Conference Paper
Full-text available
In rock climbing, discussing climbing techniques with others to master a specific route and getting practical advice from more experienced climbers is an inherent part of the culture and tradition of the sport. Spatial information, such as the position of holds, as well as learning complex body postures plays a major role in this process. A typical problem that occurs during advising is an alignment effect when trying to picture orientation-specific knowledge, e.g. explaining how to perform a certain self-climbed move to others. We propose betaCube, a self-calibrating camera-projection unit that features 3D tracking and distortion-free projection. The system enables a life-sized video replay and climbing route creation using augmented reality. We contribute an interface for automatic setup of mobile distortion-free projection, blob detection for climbing holds, as well as an automatic method for extracting planar trackables from artificial climbing walls.
Conference Paper
Full-text available
In recent years the sport of climbing has seen consistent in- crease in popularity. Climbing requires a complex skill set for successful and safe exercising. While elite climbers re- ceive intensive expert coaching to refine this skill set, this progression approach is not viable for the amateur popu- lation. We have developed ClimbAX – a climbing perfor- mance analysis system that aims for replicating expert as- sessments and thus represents a first step towards an auto- matic coaching system for climbing enthusiasts. Through an accelerometer based wearable sensing platform, climber’s movements are captured. An automatic analysis procedure detects climbing sessions and moves, which form the ba- sis for subsequent performance assessment. The assessment parameters are derived from sports science literature and in- clude: power, control, stability, speed. ClimbAX was evalu- ated in a large case study with 47 climbers under competition settings. We report a strong correlation between predicted scores and official competition results, which demonstrate the effectiveness of our automatic skill assesment system.
Article
Full-text available
Rock climbing is a popular adventure sport with an increasing research base. Early studies in the field did not make comparisons of ascents using different styles of climbing. More recently, differences in the physiological responses for an on-sight lead climb and subsequent lead climb have been reported. The purpose of the present study was to examine the effect of style of climb (lead climb or top rope climb) on the physiological and psychological responses to rock climbing. Nine intermediate climbers volunteered for, and completed, two randomly assigned climbing trials and a maximum oxygen uptake (VO2max) test on a separate occasion. The climbers ascended the same 6a (sport grade) climb for both trials. Before climbing, heart rate, perception of anxiety (Revised Competitive State Anxiety Inventory-2), and blood lactate concentration were measured. Climb time, heart rate, VO2, lactate concentrations, and task load (National Aeronautics and Space Administration Task Load Index) in response to each trial were also recorded. Results indicated significant differences (P
Conference Paper
We present the design and evaluation of the Augmented Climbing Wall (ACW). The system combines computer vision and interactive projected graphics for motivating and instructing indoor wall climbing. We have installed the system in a commercial climbing center, where it has been successfully used by hundreds of climbers, including both children and adults. Our primary contribution is a novel movement-based game system that can inform the design of future games and augmented sports. We evaluate ACW based on three user studies (N=50, N=10, N=10) and further observations and interviews. We highlight three central themes of how digital augmentation can contribute to a sport: increasing diversity of movement and challenges, enabling user-created content in an otherwise risky environment, and enabling procedurally generated content. We further discuss how ACW represents an underexplored class of interactive systems, i.e., proximity interaction on wall-sized interactive surfaces, which presents novel human-computer interaction challenges.
Article
The objective was to examine injury rates and associated risk factors in a representative sample of climbers. A random sample (n=606) of the Swedish Climbing Association members was sent a postal survey, with an effective response rate of 63%. Self-reported data regarding climbing history, safety practices and retrospective accounts of injury events (recall period 1.5 years) were obtained. Descriptive statistical methods were used to calculate injury incidences, and a two-step method including zero-inflated Poisson's regression analysis of re-injuries was used to determine the combination of risk factors that best explained individual injury rates. Overall, 4.2 injuries per 1000 climbing hours were reported, overuse injuries accounting for 93% of all injuries. Inflammatory tissue damages to fingers and wrists were the most common injury types. The multivariate analysis showed that overweight and practicing bouldering generally implied an increased primary injury risk, while there was a higher re-injury risk among male climbers and a lower risk among the older climbers. The high percentage of overuse injuries implies that climbing hours and loads should be gradually and systematically increased, and climbers regularly controlled for signs and symptoms of overuse. Further study of the association between body mass index and climbing injury is warranted.
Article
A growing body of literature suggests that virtual reality is a successful tool for exposure therapy in the treatment of anxiety disorders. Virtual reality (VR) researchers posit the construct of presence, defined as the interpretation of an artificial stimulus as if it were real, to be a presumed factor that enables anxiety to be felt during virtual reality exposure therapy (VRE). However, a handful of empirical studies on the relation between presence and anxiety in VRE have yielded mixed findings. The current study tested the following hypotheses about the relation between presence and anxiety in VRE with a clinical sample of fearful flyers: (1) presence is related to in-session anxiety; (2) presence mediates the extent that pre-existing (pre-treatment) anxiety is experienced during exposure with VR; (3) presence is positively related to the amount of phobic elements included within the virtual environment; (4) presence is related to treatment outcome. Results supported presence as a factor that contributes to the experience of anxiety in the virtual environment as well as a relation between presence and the phobic elements, but did not support a relation between presence and treatment outcome. The study suggests that presence may be a necessary but insufficient requirement for successful VRE.
Article
To determine the prevalence and nature of rock-climbing injuries, and the factors associated with these injuries. A retrospective cross-sectional study. Rock climbers were recruited at five outdoor and six indoor climbing venues in the UK. 201 active rock climbers (163 male, 38 female climbers) aged 16-62 years. Rock climbing behaviours and key demographics. Injuries requiring medical attention or withdrawal from participation for > or = 1 day. Around 50% of climbers had sustained > or = 1 injury in the past 12 months, causing a total of 275 distinct anatomical injuries. 21 climbers (10%) had sustained acute climbing injuries as a result of a fall, 67 (33%) had chronic overuse injuries, and 57 (28%) had acute injuries caused by strenuous climbing moves. Dedicated climbers participating in different forms of rock climbing more often and at a higher level of technical difficulty may be more prone to injury, particularly overuse injuries of the finger and shoulder. The principal sources of treatment or advice sought by climbers were physiotherapists (18%), other climbers (14%) and doctors (11%). Climbing frequency and technical difficulty are associated with climbing injuries occurring at both indoor and outdoor venues, particularly cumulative trauma to the upper extremities.
ClimbSense -Automatic Climbing Route Recognition using Wrist-worn Inertia Measurement Units
  • F Kosmalla
  • F Daiber
F. Kosmalla and F. Daiber. ClimbSense -Automatic Climbing Route Recognition using Wrist-worn Inertia Measurement Units. pp. 2033-2042, 2015.
The Mind Game: How to Overcome Fear -Climbing Magazine
  • M Lloyd
M. Lloyd. The Mind Game: How to Overcome Fear -Climbing Magazine, 2014.
Qualcomm Institute Convenes Conference on Future of Virtual Reality
  • Qualcomm
Qualcomm. Qualcomm Institute Convenes Conference on Future of Virtual Reality.