Integration of the HTC Vive into the Medical Platform MeVisLab
Jan Egger, Markus Gall, Jürgen Wallner, Pedro Boechat, Alexander Hann, Xing Li, Xiaojun Chen, Dieter Schmalstieg
Graz University of Technology, Institute for Computer Graphics and Vision, Graz, Austria; BioTechMed-Graz, Graz, Austria
Medical University of Graz, Department of Maxillofacial Surgery, Graz, Austria
Ulm University, Department of Internal Medicine I, Ulm, Germany
Shanghai Jiao Tong University, School of Mechanical Engineering, Shanghai, China
1. Chen, X. et al. “Development of a Surgical Navigation System based
on Augmented Reality using an Optical see-through Head-mounted
Display,” J Biomed Inform. 2015 Jun;55:124-31 (2015).
2. Egger, J. et al. “Integration of the OpenIGTlink network protocol for
image guided therapy with the medical platform MeVisLab,” The
international Journal of medical Robotics and Computer assisted
Surgery, 8(3):282-390 (2012).
The work received funding from BioTechMed-Graz in Austria (“Hardware accelerated intelligent medical imaging”), the 6th
Call of the Initial Funding Program from the Research & Technology
House (F&T-Haus) at the Graz University of Technology (PI: Jan Egger, Ph.D., Ph.D.) and ClinicIMPPACT (610886). Dr. Xiaojun Chen receives support by the Natural Science Foundation of
China (Grant No.: 81511130089) and the Foundation of Science and Technology Commission of Shanghai Municipality (Grants No.: 14441901002, 15510722200 and 16441908400). A video
demonstrating the integration of the HTC Vive into the medical platform MeVisLab is available under the following YouTube channel: https://www.youtube.com/c/JanEgger/videos
Virtual Reality (VR) is an immersive technology that replicates
an environment via computer-simulated reality. VR gets a lot
of attention in computer games but has also great potential in
other areas, like the medical domain. Examples are planning,
simulations and training of medical interventions, like for facial
surgeries where an aesthetic outcome is important. However,
importing medical data into VR devices is not trivial, especially
when a direct connection and visualization from your own
application is needed. Furthermore, most researcher don’t
build their medical applications from scratch, rather they use
platforms, like MeVisLab, Slicer or MITK. The platforms have
in common that they integrate and build upon on libraries like
ITK and VTK, further providing a more convenient graphical
interface to them for the user.
In this contribution, we demonstrate the usage of a VR
device for medical data under MeVisLab. Therefore, we
integrated the OpenVR library into MeVisLab as an own
module. This enables the direct and uncomplicated usage of
head mounted displays, like the HTC Vive under MeVisLab.
Summarized, medical data from other MeVisLab modules can
directly be connected per drag-and-drop to our VR module
and will be rendered inside the HTC Vive for an immersive
inspection (Figure 1).
Data –As datasets for testing and evaluating the integration
we used several high-resolution Computed Tomography (CT)
acquisitions from the clinical routine.
Workflow –A high level workflow diagram showing the
communication and interaction between MeVisLab and the
HTC Vive via OpenVR is presented in Figure 2.
Network –The overall MeVisLab network with our HTCVive
module is presented in Figure 3. In this network, the medical
data is loaded via a WEMLoad module (named DataLoad) and
is directly passed to the HTCVive module (rectangle input at
the bottom of the HTC Vive module).
We developed a new module for the medical
prototyping platform MeVisLab that provides an
interface via the OpenVR library to head mounted
devices, enabling the direct and uncomplicated
usage of the HTC Vive under MeVisLab.
The OpenVR API provides a way to connect and
interact with Virtual Reality displays without relying
on a specific hardware vendor’s SDK. Thus, our
module could also communicate with other VR
devices, like the Oculus Rift.
Overall, the goal of this contribution was to investigate the
feasibility of using the HTC Vive under the medical prototyping
1. The successful integration of OpenVR with MeVisLab
has been demonstrated;
2. The developed solution allows MeVisLab programs
to connect to virtual reality headset devices;
3. Real-time visualization of medical data in VR is now
possible under MeVisLab;
4. For proof of concept, the integration has been tested
with the HTC Vive device;
5. The HTC Vive module can be used in new MeVisLab
networks or added to existing ones.
Fig. 2 High level workflow diagram showing the
communication / interaction between MeVisLab and the HTC
Vive via OpenVR.
Fig. 1 HTC Vive
The integration could be successfully achieved
under Microsoft Windows 8.1 with the MeVisLab 2.8.1
(21-06-2016) Version for Windows Visual Studio 2015
X64 and OpenVR SDK 1.0.2 (Figure 4).
There are several areas for future work, like the
evaluation of our integration with a greater amount of
medical data formats.
Fig. 4 Demonstration of the integration of the HTC Vive
into the medical platform MeVisLab.
Fig. 3 The overall MeVisLab network with the HTCVive
module and its interface, and parameters on the left side.