BrainVis: A cloud-connected 3D exploration and visualization tool for multi-modal neuroimaging data

Poster (PDF Available) · April 2017with 44 Reads
DOI: 10.13140/RG.2.2.17341.97761
ISMRM 2017: 25th Annual Meeting of the International Society for Magnetic Resonance in Medicine, DOI:10.13140/RG.2.2.17341.97761
Abstract
Diffusion MRI tractography is central to the study of complex brain circuitry since it is the only non-invasive technique capable of measuring the brain’s wiring. Data visualization gets increasingly complex as tractography is combined with morphometric results in structural connectomes, and even more so when coupled with functional information from fMRI techniques. Such complex imagery, aggregating multi-modal information, 3D meshes, and statistical maps is often visualized using research tools with complicated user interfaces (UIs) and cluttered visualizations. BrainVis is a sophisticated, free 3D neuroimaging visualization tool with a simple UI aimed for e􀃗ective exploration of neuroimaging data towards demonstrating neurobiological findings.
2/25/2018 indexsmart.mirasmart.com/ISMRM2017/PDFfiles/2403.html
http://indexsmart.mirasmart.com/ISMRM2017/PDFfiles/2403.html 1/2
2403
BrainVis: A cloud-connected 3D exploration and visualization tool for multi-modal
neuroimaging data
Vesna Prčkovska , Tim Peeters , David Moreno-Dominguez , and Paulo Rodrigues
Mint Labs, Barcelona, Spain
Synopsis
Diusion MRI tractography is central to the study of complex brain circuitry since it is the only non-invasive technique capable of measuring the brain’s
wiring. Data visualization gets increasingly complex as tractography is combined with morphometric results in structural connectomes, and even more so
when coupled with functional information from fMRI techniques. Such complex imagery, aggregating multi-modal information, 3D meshes, and statistical
maps is often visualized using research tools with complicated user interfaces (UIs) and cluttered visualizations.
BrainVis is a sophisticated, free 3D neuroimaging visualization tool with a simple UI aimed for eective exploration of neuroimaging data towards
demonstrating neurobiological ndings.
Purpose
Data visualization is an important aspect of neuroimaging: paramount for correct and intuitive interpretation of results and key for neurosurgical
planning. However, there is a lack of standardization in the current viewers, with various le types and data structures. Many viewers lack truly interactive
3D view and consistent or specialized development support.
We developed BrainVis to alleviate these problems: it is a fully interactive 3D viewer, where multiple modalities can be shown simultaneously. Advanced
tools are provided such as surface rendering and interactive exploration of tractography streamlines. Seamless integration with a cloud-system provides
transparent fetching and processing of data. For user interface (UI) simplicity, it features adaptive menus that only show the options applicable to the
current data. It can be coupled with Leap Motion hardware for intuitive touchless interaction.
Results
Neuroimaging Analysis
Various tools exist for computing dierent aspects of neuroimaging data, from structural morphometric maps (brain tissue segmentation, cortical
volumes, cortical thickness), to structural connectivity (e.g., dMRI tractography), and structural connectomes or resting state functional connectomes. In
parallel, we developed a cloud-based platform providing a standardized place for neuroimaging data management and image processing pipelines. All
computations can be run in the cloud environment, and the generated result les are kept in a managed data warehouse. All les (input and output) can
be retrieved through an application programming interface (API).
Desktop Application Implementation
BrainVis is a freely available desktop application developed to display the processed data in 3D for Windows, Linux and MacOS. With an active Internet
connection, BrainVis can search and download the processed neuroimaging studies (e.g., brain tissue segmentation, tractography, structural
connectomes) and visualize the respective data. Data les can also be downloaded from the cloud platform through the web browser. The cross-platform
components for data handling and graphics are written in C# and Javascript within Unity3D engine.
Features and User Interface
BrainVis is seamlessly integrated with the cloud infrastructure. The user simply selects which analysis to download and visualize, and all necessary les
are automatically retrieved without manual selection of the dierent les and parameters. BrainVis has three main components:
1. Adaptive options menu: The available options are automatically enabled if the corresponding data is available. For example, the 3D region-of-interest
(ROI) selection for tractography is only visible if tractography maps are loaded. This way a simple and sleek UI is ensured.
2. Interactive 3D view: The application is focused on maximizing the 3D view and exploration. It allows multi-modal visualization of structural data, as well
as overlays of segmentation analysis, lesion masks, tumor masks, FA maps, and tractography maps. It also provides 3D surface mesh generation for
cortical region visualization and advanced ber selection options.
3. 2D slices view: also interactive allowing users to click on a plane and the respective 2D position sets the 3D cursor in the view.
Multi-modal data is seamlessly enabled, the cursor position is checked against cortical label map, and the corresponding brain region is highlighted and
thus can be used to select ber-tracts.
There are several options for Interactive Fiber Selection: 2D plane ber selection; Spherical ROI with AND/OR ltering options (Figure 1a); Highlighting
bundles in ROI (Figure 1b); Filtering by cortical regions (Figure 1c). Exporting ber selections to external tools and applications (e.g., export to .stl format
for 3D printing as in Figure 2).
All these capabilities make BrainVis a exible tool suited for multiple scenarios. Several use-case examples follow: Figure 3 illustrates the visualization of
morphometric analysis pipeline applied to a healthy subject; Figure 4 shows a tumor case where the bers are automatically generated by seeding from
an fMRI motor task and in Figure 5 the lesions of a patient with multiple sclerosis have been projected onto tractography streamlines helping elucidate
what connections might be more severely aected by the disease.
Touchless interaction
In order to provide a better solution for the physicians and specically neurosurgeons for the data visualization, we oer a fully touchless interface
(Figure 1d). We use Leap Motion tracking device that detects and tracks hands, ngers, and nger-like tools. This touchless interface provides an intuitive
and natural way to explore 3D brain maps, especially towards precise moving of the ROI in 3D space, and has a direct application potential in
neurosurgery.
Discussion
1 1 1 1
1
1
2
2/25/2018 indexsmart.mirasmart.com/ISMRM2017/PDFfiles/2403.html
http://indexsmart.mirasmart.com/ISMRM2017/PDFfiles/2403.html 2/2
BrainVis oers a well-rounded option for various neuroimaging visualization needs. It oers convenient access to the detailed 3D-neuroimaging data and
the ability to perform data-mining and visualization of neuroanatomy in an integrated manner. This should greatly increase the power and benet of
neuroimaging informatics for the researchers in this eld.
Acknowledgements
No acknowledgement found.
References
1. http://www.mint-labs.com/latest
2. https://www.leapmotion.com
Figures
Figure 1: Dierent ber selection modes: a) ber selection by ROI located in the left caudal middle frontal region b) white matter bundles highlighted as
bers cross the selected ROIs c) ltering the selected cortical regions: precentral, postcentral and brainstem d) interactive ROI placement within the 3D
space of the ber tracts of the brain in a touchless manner simply by moving the ngers.
Figure 2: 3D print case exported from BrainVis. The tumor is in the frontal lobe surrounded by the pushed white matter tracts. The right hemisphere is
printed as a hollow surface from the segmented cortical mesh.
Figure 3: The cortical brain structures can be seen overlayed over the skull-stripped structural T1 image. We can also observe the 3D surface generated
for precentral and postcentral regions. The spatial position highlights the precuneus region and its coordinates (see box with text down right).
QMENTA_BrainVis_Poster.pdf
2.36 MB
This research hasn't been cited in any other publications.
This research doesn't cite any other publications.