Poster

Multi-Threaded Integration of HTC-Vive and MeVisLab

Abstract

Virtual Reality (VR) is a promising future. It sets up a virtual environment providing an (audio)visual user experience by creating an artificial world where a user is directly placed into. Extensible by haptic systems, it allows to simulate lifelike experiences to any user. Over the last years, VR hardware has more and more turned into something that is affordable and thus available to common end-users in daily life. At the point of time only limited by physical restrictions such as the walls of a room or the cables connecting the hardware devices to a computer, a user is able to move around inside that VR, i.e. physical movement is directly applied onto the artificially generated world. To experience VR, one is usually equipped with a so-called Head Mounted Device (HMD). Containing a screen in front of each eye, this device renders whatever wanted in front of the users eyes. Today’s VR products are used in different fields such as training simulations, gaming and even videos. This work presents how VR is integrated into medical applications by developing a VR plugin for a medical image processing framework called MeVisLab. The thread-based plugin has been developed using OpenVR, a VR library that can be used for developing vendor and platform independent VR applications and tested with the HTC Vive.
Multi-Threaded Integration of HTC-Vive and MeVisLab
Simon Gunacker, Markus Gall, Dieter Schmalstieg, Jan Egger
Graz University of Technology, Institute for Computer Graphics and Vision, Graz, Austria; BioTechMed-Graz, Graz, Austria
Medical University of Graz, Department of Maxillofacial Surgery, Graz, Austria; Computer Algorithms for Medicine (Cafe) Laboratory, Graz, Austria
AIT Austrian Institute of Technology, Department of Safety & Security, Vienna, Austria
METHODSINTRODUCTION RESULTS
1. Gall M., et al. “Integration of the HTC Vive into the medical platform
MeVisLab,” SPIE Medical Imaging (2017).
2. Gall, M., Li, X., Chen, X., Schmalstieg, D. & Egger, J. “Cranial Defect
Datasets,” ResearchGate (2016).
3. Egger, J. et al. “HTC Vive MeVisLab integration via OpenVR for
medical applications,” PLoS ONE 12(3): e0173972 (2017).
CONCLUSIONS
REFERENCES
The work received funding from BioTechMed-Graz in Austria (“Hardware accelerated intelligent medical imaging”), the 6th Call of the Initial Funding Program from the Research & Technology
House (F&T-Haus) at the Graz University of Technology (PI: Dr. Dr. habil. Jan Egger). The corresponding Macro module and source code is freely available under:
https://github.com/simon-gunacker/vive
A video demonstrating thread-based Implementation is available under the following YouTube channel: https://www.youtube.com/c/JanEgger/videos
Virtual Reality (VR) is a promising future. It sets up a virtual
environment providing an (audio)visual user experience by
creating an artificial world where a user is directly placed into.
Extensible by haptic systems, it allows to simulate lifelike
experiences to any user. Over the last years, VR hardware
has more and more turned into something that is affordable
and thus available to common end-users in daily life. At the
point of time only limited by physical restrictions such as the
walls of a room or the cables connecting the hardware devices
to a computer, a user is able to move around inside that VR,
i.e. physical movement is directly applied onto the artificially
generated world. To experience VR, one is usually equipped
with a so-called Head Mounted Device (HMD). Containing a
screen in front of each eye, this device renders whatever
wanted in front of the users eyes. Today’s VR products are
used in different fields such as training simulations, gaming
and even videos. This work presents how VR is integrated into
medical applications by developing a VR plugin for a medical
image processing framework called MeVisLab. The thread-
based plugin has been developed using OpenVR, a VR library
that can be used for developing vendor and platform
independent VR applications and tested with the HTC Vive.
Data Several high-resolution Computed Tomography (CT)
scans from real patients have been used to develop, test and
evaluate the VR plugin.
Workflow The overall goal of the HTC-Vive MeVisLab plugin
is to connect MeVisLab to Virtual Reality. It does so by
establishing a connection between those two (Figure 2).
Implementation The plugin consists of three parts (Figure 3).
The first part wraps the functionality of the actual application
into a module that can be used by MeVisLab. The second part
renders the medical data. The third part forwards the rendered
scene to Virtual Reality using OpenVR.
This work presents how VR is integrated into medical
applications by developing a VR plugin for a medical
image processing framework called MeVisLab. The
thread-based plugin is tested using the HTC Vive
(Figure 1).
Using OpenVR allows the plugin to work with
different VR systems. The plugin can be integrated
into any network created by MeVisLab and can thus
be a good visual support whenever it comes to the
inspection of processed 3D medical data.
The goal of this work was to show the feasibility of an
enhanced thread-based integration of Virtual Reality into
MeVisLab.
1. OpenVR has been integrated into MeVisLab allowing
MeVisLab to visualize data in Virtual Reality;
2. A threaded solution enables the simultaneous use of
MeVisLab while a user explores the interior of a real
human body in Virtual Reality;
3. The integration has successfully been tested with the
HTC Vive device;
4. The plugin can be used in different clinical
applications and provides a good visual support
when inspecting 3D medical data.
Fig. 2 High level workflow diagram showing the
communication and interaction between MeVisLab and the
HTC Vive via OpenVR capsuled by an own thread.
Fig. 1 An examination of medical data in VR.
The plugin has been developed and tested under
Microsoft Windows 8.1 with the MeVisLab 2.8.1 (21-
06-2016) Version for Windows Visual Studio 2015 X64
and OpenVR SDK 1.0.2 (Figure 4).
Future work includes to allow modifications done in
MeVisLab to be rendered to VR in real-time as well as
the interaction with the 3D data in VR.
Fig. 4 A MeVisLab sample network using the plugin to
render the given 3D medical data in VR.
Fig. 3 Diagram showing the three parts of our module
implementation and their interaction with each other.
Article
Purpose - Advanced visualization of medical imaging has been a motive for research due to its value for disease analysis, surgical planning and academical training. More recently, attention has been turning towards mixed reality as a means to deliver more interactive and realistic medical experiences. However, there are still many limitations to the use of virtual reality for specific scenarios. Our intent is to study the current usage of this technology and assess the potential of related development tools for clinical contexts. Methods - This paper focuses on virtual reality as an alternative to today's majority of slice-based medical analysis workstations, bringing more immersive three-dimensional experiences that could help in cross-slice analysis. We determine the key features a virtual reality software should support and present today's software tools and frameworks for researchers that intend to work on immersive medical imaging visualization. Such solutions are assessed to understand their ability to address existing challenges of the field. Results - It was understood that most development frameworks rely on well established toolkits specialized for healthcare and standard data formats such as DICOM. Also, game engines prove to be adequate means of combining software modules for improved results. Virtual reality seems to remain a promising technology for medical analysis but has not yet achieved its true potential. Conclusions - Our results suggest that pre-requisites such as real time performance and minimum latency pose the greatest limitations for clinical adoption and need to be addressed. There is also a need for further research comparing mixed realities and currently used technologies.
ResearchGate has not been able to resolve any references for this publication.