Conference Paper

RegistrationShop: An Interactive 3D Medical Volume Registration System

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

In medical imaging, registration is used to combine images containing information from different modalities or to track treatment effects over time in individual patients. Most registration software packages do not provide an easy-to-use interface that facilitates the use of registration. 2D visualization techniques are often used for visualizing 3D datasets. RegistrationShop was developed to improve and ease the process of volume registration using 3D visualizations and intuitive interactive tools. It supports several basic visualizations of 3D volumetric data. Interactive rigid and non-rigid transformation tools can be used to manipulate the volumes and immediate visual feedback for all rigid transformation tools allows the user to examine the current result in real-time. In this context, we introduce 3D comparative visualization techniques, as well as a way of placing landmarks in 3D volumes. Finally, we evaluated our approach with domain experts, who underlined the potential and usefulness of RegistrationShop.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

Supplementary resources (11)

... In contrast to this, we use star glyphs to present an abstract version of multiparametric spatial data on top of spatial data. Smit et al. [24] presented a method to spatially query data by placing a sphere in a 3D view, and interaction techniques to effectively place spheres in volume renderings [25]. Bruckner et al. [4] introduced a probing tool for enabling visual queries. ...
Article
Full-text available
Multiparametric imaging in cancer has been shown to be useful for tumor detection and may also depict functional tumor characteristics relevant for clinical phenotypes. However, when confronted with datasets consisting of multiple values per voxel, traditional reading of the imaging series fails to capture complicated patterns. These patterns of potentially important imaging properties of the parameter space may be critical for the analysis, but standard approaches do not deliver sufficient details. Therefore, in this paper, we present an approach that aims to enable the exploration and analysis of such multiparametric studies using an interactive visual analysis application to remedy the trade-offs between details in the value domain and in spatial resolution. This may aid in the discrimination between healthy and cancerous tissue and potentially highlight metastases that evolved from the primary tumor. We conducted an evaluation with eleven domain experts from different fields of research to confirm the utility of our approach.
... The required pre-processing MRI segmentation of the major organs was performed semi-automatically in AMIRA and MITK [51]. We used RegistrationShop [42] and Elastix [21] to perform the rigid and deformable B-spline registration respectively. After this, we reconstructed surfaces for the segmented organs and risk zones using DeVIDE [3]. ...
Article
Full-text available
Due to the intricate relationship between the pelvic organs and vital structures, such as vessels and nerves, pelvic anatomy is often considered to be complex to comprehend. In oncological pelvic surgery, a trade-off has to be made between complete tumor resection and preserving function by preventing damage to the nerves. Damage to the autonomic nerves causes undesirable post-operative side-effects such as fecal and urinal incontinence, as well as sexual dysfunction in up to 80 percent of the cases. Since these autonomic nerves are not visible in pre-operative MRI scans or during surgery, avoiding nerve damage during such a surgical procedure becomes challenging. In this work, we present visualization methods to represent context, target, and risk structures for surgical planning. We employ distance-based and occlusion management techniques in an atlas-based surgical planning tool for oncological pelvic surgery. Patient-specific pre-operative MRI scans are registered to an atlas model that includes nerve information. Through several interactive linked views, the spatial relationships and distances between the organs, tumor and risk zones are visualized to improve understanding, while avoiding occlusion. In this way, the surgeon can examine surgically relevant structures and plan the procedure before going into the operating theater, thus raising awareness of the autonomic nerve zone regions and potentially reducing post-operative complications. Furthermore, we present the results of a domain expert evaluation with surgical oncologists that demonstrates the advantages of our approach.
... red/green) and use color blending of the slices to emphasize intensity differences [21], [22]. In [23] orange and blue colors are used as complementary colors. Adding them results in a shade of gray whenever both are of the same intensity, and in a shade of blue or orange if intensities differ. ...
Article
Deformable image registration (DIR) has the potential to improve modern radiotherapy in many aspects, including volume definition, treatment planning and image-guided adaptive radiotherapy. Studies have shown its possible clinical benefits. However, measuring DIR accuracy is difficult without known ground truth, but necessary before integration in the radiotherapy workflow. Visual assessment is an important step towards clinical acceptance. We propose a visualization framework which supports the exploration and the assessment of DIR accuracy. It offers different interaction and visualization features for exploration of candidate regions to simplify the process of visual assessment. The visualization is based on voxel-wise comparison of local image patches for which dissimilarity measures are computed and visualized to indicate locally the registration results. We performed an evaluation with three radiation oncologists to demonstrate the viability of our approach. In the evaluation, lung regions were rated by the participants with regards to their visual accuracy and compared to the registration error measured with expert defined landmarks. Regions rated as "accepted" had an average registration error of 1.8 mm, with the highest single landmark error being 3.3 mm. Additionally, survey results show that the proposed visualizations support a fast and intuitive investigation of DIR accuracy, and are suitable for finding even small errors.
... In practice we observe that at most 344 three surfaces are shown superimposed using alpha blending 345 and contrasting colors, see e.g. RegistrationShop[79]. Super-346 imposition is also used to assess results of pairwise registra-347 tion of surfaces or images. ...
Article
The combination of long established techniques in morphometrics with novel shape modeling approaches in geometry processing has opened new ways of visualizations of shape variability in different application areas like biology, medicine, epidemiology and agriculture. For the first time highly resolved 3D representations became accessible for statistical analysis as well as visualizations. In order to reveal causes for shape variability targeted statistical analysis correlating shape features against external and internal factors is necessary but due to the complexity of the problem often not feasible in an automated way. Therefore, visual analytics methods found their way into the field of morphometrics. This led to numerous publications in recent years that might be subsumed under the novel term visual shape analytics. In this paper we try to put these works into the context of visual analytics, outline the basic principles underlying these approaches and review the current state of the art. Finally, future challenges and possibilities in visual shape analytics are identified.
... We also see potential applications of our work beyond shape analysis. For instance, image registration algorithms could be monitored during runtime using our fast image warping method, enabling debug visualizations for parameter tuning in the spirit of Registra-tionShop [38], adding diffeomorphisms to its class of transformations. Concerning 3D modeling it would be interesting to see if the easy and efficient modeling of weighted local rotations, used in our reformation technique, does provide benefits for interactive editing of 3D volumes [14,8] and meshes [45]. ...
Article
Large image deformations pose a challenging problem for the visualization and statistical analysis of 3D image ensembles which have a multitude of applications in biology and medicine. Simple linear interpolation in the tangent space of the ensemble introduces artifactual anatomical structures that hamper the application of targeted visual shape analysis techniques. In this work we make use of the theory of stationary velocity fields to facilitate interactive non-linear image interpolation and plausible extrapolation for high quality rendering of large deformations and devise an efficient image warping method on the GPU. This does not only improve quality of existing visualization techniques, but opens up a field of novel interactive methods for shape ensemble analysis. Taking advantage of the efficient non-linear 3D image warping, we showcase four visualizations: 1) browsing on-the-fly computed group mean shapes to learn about shape differences between specific classes, 2) interactive reformation to investigate complex morphologies in a single view, 3) likelihood volumes to gain a concise overview of variability and 4) streamline visualization to show variation in detail, specifically uncovering its component tangential to a reference surface. Evaluation on a real world dataset shows that the presented method outperforms the state-of-the-art in terms of visual quality while retaining interactive frame rates. A case study with a domain expert was performed in which the novel analysis and visualization methods are applied on standard model structures, namely skull and mandible of different rodents, to investigate and compare influence of phylogeny, diet and geography on shape. The visualizations enable for instance to distinguish (population-)normal and pathological morphology, assist in uncovering correlation to extrinsic factors and potentially support assessment of model quality.
Article
A variety of visualization techniques can be utilized to compare multiple Spatial 3D or time-varying Spatial 3D data instances (e.g., comparing pre- versus post-treatment volumetric medical images). However, despite the fact that comparative visualization is frequently needed – scientists, engineers, and even humanists must routinely compare such data – visualization users and practitioners suffer from a lack of adequate Spatial 3D comparative visualization tools and guidelines. Here we survey the field and present a taxonomy for classifying existing and new comparison visualization techniques for such data into four fundamental approaches: Juxtaposition, Superimposition, Interchangeable, and Explicit Encoding. The results clarify the key design decisions and tradeoffs that designers must make to create an effective comparative Spatial 3D data visualization and suggest the potential of emerging hybrid approaches, ones creatively combining aspects of the four fundamental approaches.
Thesis
This thesis deals with visualizing anatomical data for medical education and surgical planning purposes. To this end, we have developed a detailed virtual atlas, the Virtual Surgical Pelvis (VSP), which unifies surgically relevant knowledge on pelvic anatomy. We provide methods to share the knowledge contained in the VSP for educational purposes, and to visualize the VSP in the context of individual patients for pre-operative planning purposes.
Article
Full-text available
Nonrigid image registration is an important, but time-consuming task in medical image analysis. In typical neuroimaging studies, multiple image registrations are performed, i.e., for atlas-based segmentation or template construction. Faster image registration routines would therefore be beneficial. In this paper we explore acceleration of the image registration package elastix by a combination of several techniques: (i) parallelization on the CPU, to speed up the cost function derivative calculation; (ii) parallelization on the GPU building on and extending the OpenCL framework from ITKv4, to speed up the Gaussian pyramid computation and the image resampling step; (iii) exploitation of certain properties of the B-spline transformation model; (iv) further software optimizations. The accelerated registration tool is employed in a study on diagnostic classification of Alzheimer's disease and cognitively normal controls based on T1-weighted MRI. We selected 299 participants from the publicly available Alzheimer's Disease Neuroimaging Initiative database. Classification is performed with a support vector machine based on gray matter volumes as a marker for atrophy. We evaluated two types of strategies (voxel-wise and region-wise) that heavily rely on nonrigid image registration. Parallelization and optimization resulted in an acceleration factor of 4-5x on an 8-core machine. Using OpenCL a speedup factor of 2 was realized for computation of the Gaussian pyramids, and 15-60 for the resampling step, for larger images. The voxel-wise and the region-wise classification methods had an area under the receiver operator characteristic curve of 88 and 90%, respectively, both for standard and accelerated registration. We conclude that the image registration package elastix was substantially accelerated, with nearly identical results to the non-optimized version. The new functionality will become available in the next release of elastix as open source under the BSD license.
Article
Full-text available
The aim of the Medical Imaging Interaction Toolkit (MITK) is to facilitate the creation of clinically usable image-based software. Clinically usable software for image-guided procedures and image analysis require a high degree of interaction to verify and, if necessary, correct results from (semi-)automatic algorithms. MITK is a class library basing on and extending the Insight Toolkit (ITK) and the Visualization Toolkit (VTK). ITK provides leading-edge registration and segmentation algorithms and forms the algorithmic basis. VTK has powerful visualization capabilities, but only low-level support for interaction (like picking methods, rotation, movement and scaling of objects). MITK adds support for high level interactions with data like, for example, the interactive construction and modification of data objects. This includes concepts for interactions with multiple states as well as undo-capabilities. Furthermore, VTK is designed to create one kind of view on the data (either one 2D visualization or a 3D visualization). MITK facilitates the realization of multiple, different views on the same data (like multiple, multiplanar reconstructions and a 3D rendering). Hierarchically structured combinations of any number and type of data objects (image, surface, vessels, etc.) are possible. MITK can handle 3D+t data, which are required for several important medical applications, whereas VTK alone supports only 2D and 3D data. The benefit of MITK is that it supplements those features to ITK and VTK that are required for convenient to use, interactive and by that clinically usable image-based software, and that are outside the scope of both. MITK will be made open-source (http://www.mitk.org).
Article
Full-text available
Extending previous work using Graphical Processing Units (GPUs) for medical image registration purposes, a medical image registration package has been developed capable of performing 3D non-rigid (free-form) registrations in less than a minute for typical MRI images from OASIS dataset (an average of 15s on a desktop with NVIDIA GeForce GTX 580 graphics card with 512 processing cores and an average of 45s on a laptop with NVIDIA GeForce 445M graphics card with 144 processing cores). Registration is done by minimizing an objective cost function (negative Mutual Information) subject to smoothness constraints. Typically 106 – 107 parameters are found using one control point per voxel. The program can be used in both diffeomorphic and non-diffeomorphic (faster) modes. The program has been written in C++ and NVIDIA CUDA programming languages performing expensive operations on the GPU. In order to test the validity of the results, the IBSR human brain database was used: pairs of segmented images were registered to each other and a median overlap ratio of segmented ROIs were calculated giving a value of not less than 0.61 in for skull stripped brains. This value has not been exceeded by any of image registration packages tested by Klein et al. 2009 in an equivalent calculation. A fully automatic atlas construction algorithm was built on top of the image registration engine and was tested on both human brains from OASIS and IBSR datasets and murine images obtained from Wolfson Brain Imaging Centre. A graphical user interface was written using Nokia QT framework to make the package easily usable.
Article
Full-text available
Purpose Whole-body MRI is seeing increasing use in the study and diagnosis of disease progression. In this, a central task is the visual assessment of the progressive changes that occur between two whole-body MRI datasets, taken at baseline and follow-up. Current radiological workflow for this consists in manual search of each organ of interest on both scans, usually on multiple data channels, for further visual comparison. Large size of datasets, significant posture differences, and changes in patient anatomy turn manual matching in an extremely labor-intensive task that requires from radiologists high concentration for long period of time. This strongly limits the productivity and increases risk of underdiagnosis. Materials and methods We present a novel approach to the comparative visual analysis of whole-body MRI follow-up data. Our method is based on interactive derivation of locally rigid transforms from a pre-computed whole-body deformable registration. Using this approach, baseline and follow-up slices can be interactively matched with a single mouse click in the anatomical region of interest. In addition to the synchronized side-by-side baseline and matched follow-up slices, we have integrated four techniques to further facilitate the visual comparison of the two datasets: the “deformation sphere”, the color fusion view, the magic lens, and a set of uncertainty iso-contours around the current region of interest. Results We have applied our method to the study of cancerous bone lesions over time in patients with Kahler’s disease. During these studies, the radiologist carefully visually examines a large number of anatomical sites for changes. Our interactive locally rigid matching approach was found helpful in localization of cancerous lesions and visual assessment of changes between different scans. Furthermore, each of the features integrated in our software was separately evaluated by the experts. Conclusion We demonstrated how our method significantly facilitates examination of whole-body MR datasets in follow-up studies by enabling the rapid interactive matching of regions of interest and by the explicit visualization of change.
Article
Full-text available
Finding the relationship between two coordinate systems using pairs of measurements of the coordinates of a number of points in both systems is a classic photogrammetric task. It finds applications in stereophotogrammetry and in robotics. I present here a closed-form solution to the least-squares problem for three or more points. Currently various empirical, graphical, and numerical iterative methods are in use. Derivation of the solution is simplified by use of unit quaternions to represent rotation. I emphasize a symmetry property that a solution to this problem ought to possess. The best translational offset is the difference between the centroid of the coordinates in one system and the rotated and scaled centroid of the coordinates in the other system. The best scale is equal to the ratio of the root-mean-square deviations of the coordinates in the two systems from their respective centroids. These exact results are to be preferred to approximate methods based on measurements of a few selected points. The unit quaternion representing the best rotation is the eigenvector associated with the most positive eigenvalue of a symmetric 4X4 matrix. The elements of this matrix are combinations of sums of products of corresponding coordinates of the points.
Article
Full-text available
AMILab is a free software for image analysis, processing and visualization. It provides convenient visualization tools for 2D and 3D images and it is highly extensible through its own scripting language. We describe the main visualization features and the scripting language of AMILab. The software includes an automatic C++ wrapping system which permits fast development of new visualization tools and image processing algorithms.
Article
Full-text available
It has long been recognized that transfer function setup for Direct Volume Rendering (DVR) is crucial to its usability. However, the task of finding an appropriate transfer function is complex and time-consuming even for experts. Thus, in many practical applications simpler techniques which do not rely on complex transfer functions are employed. One common example is Maximum Intensity Projection (MIP) which depicts the maximum value along each viewing ray. In this paper, we introduce Maximum Intensity Difference Accumulation (MIDA), a new approach which combines the advantages of DVR and MIP. Like MIP, MIDA exploits common data characteristics and hence does not require complex transfer functions to generate good visualization results. It does, however, feature occlusion and shape cues similar to DVR. Furthermore, we show that MIDA - in addition to being a useful technique in its own right - can be used to smoothly transition between DVR and MIP in an intuitive manner. MIDA can be easily implemented using volume raycasting and achieves real-time performance on current graphics hardware.
Article
Full-text available
The most essential technique to visualize 3D scalar data is direct volume rendering. For many applications it is necessary that two or more 3D data are visualized simultaneously. We present an overview of data intermixing techniques for visualization with the direct volume rendering technique ray casting. The techniques are Classification Level Intermixing, Accumulation Level Intermixing and Image Level Intermixing. The algorithms are implemented on the Graphics Processing Unit (GPU) in order to be able to interact with the visualization in real-time. We use the new CUDA technology from Nvidia for that. We compare performance by measuring frames per seconds (FPS) and analyzing image quality with criteria contrast and depth effect. Depth effect is determined by a small user study. In most cases Accumulation Level Intermixing is the best choice.
Article
Full-text available
Amide's a Medical Image Data Examiner (AMIDE) has been developed as a user-friendly, open-source software tool for displaying and analyzing multimodality volumetric medical images. Central to the package's abilities to simultaneously display multiple data sets (e.g., PET, CT, MRI) and regions of interest is the on-demand data reslicing implemented within the program. Data sets can be freely shifted, rotated, viewed, and analyzed with the program automatically handling interpolation as needed from the original data. Validation has been performed by comparing the output of AMIDE with that of several existing software packages. AMIDE runs on UNIX, Macintosh OS X, and Microsoft Windows platforms, and it is freely available with source code under the terms of the GNU General Public License.
Article
Full-text available
We present the detailed planning and execution of the Insight Toolkit (ITK), an application programmers interface (API) for the segmentation and registration of medical image data. This public resource has been developed through the NLM Visible Human Project, and is in beta test as an open-source software offering under cost-free licensing. The toolkit concentrates on 3D medical data segmentation and registration algorithms, multimodal and multiresolution capabilities, and portable platform independent support for Windows, Linux/Unix systems. This toolkit was built using current practices in software engineering. Specifically, we embraced the concept of generic programming during the development of these tools, working extensively with C++ templates and the freedom and flexibility they allow. Software development tools for distributed consortium-based code development have been created and are also publicly available. We discuss our assumptions, design decisions, and some lessons learned.
Article
We have modified the current VTK volume rendering on GPU to allow simultaneous rendering of two volumes, each of them with its own color and opacity transfer functions. These changes have led to the creation of two new C++ classes and several GLSL shaders. We explain the modifications made to the original classes and shaders and we discuss possible additional improvements. A C++ demo code shows how to use the new classes.
Article
Amide's a Medical Image Data Examiner (AMIDE) has been developed as a user-friendly, open-source software tool for displaying and analyzing multimodality volumetric medical images. Central to the package's abilities to simultaneously display multiple data sets (e.g., PET, CT, MRI) and regions of interest is the on-demand data reslicing implemented within the program. Data sets can be freely shifted, rotated, viewed, and analyzed with the program automatically handling interpolation as needed from the original data. Validation has been performed by comparing the output of AMIDE with that of several existing software packages. AMIDE runs on UNIX, Macintosh OS X, and Microsoft Windows platforms, and it is freely available with source code under the terms of the GNU General Public License.
Article
By splitting a complex ray-casting process into different tasks performed on different processors, Voreen provides a lot of flexibility because users can intervene at different points during ray casting. Voreen's object-oriented design lets users easily create customized processor classes that cooperate seamlessly with existing classes. A user-friendly GUI supports rapid prototyping of visualization ideas. We've implemented several applications based on our library. In the future, we'd like to further extend Voreen's capabilities to make visualization prototyping even easier on all abstraction levels. Thus, we plan to realize a set of dedicated processor skeletons, which are solely configured through shader programs and can thus be modified at runtime.
Conference Paper
This paper proposes a new interactive hybrid non-rigid registration framework that combines any intensity-based algorithm with a feature-based component, using an iterative dual energy minimization. The resulting transformation combines both intensity-based and feature-based deformation fields. The feature matching exploits user-placed landmark pairs, and based on saliency and similarity measures, optimizes the correspondences in the neighborhood of each landmark. A dense feature-based deformation field is then generated using a thin-plate spline interpolation. Additionally, the framework allows user interactivity for live guidance of the algorithm in case of errors or inaccuracies. We present three experimental results of our hybrid approach on lung, pelvis and brain datasets, and show that in each case, the registration benefited from the hybrid approach as opposed to its intensity component alone
Article
Multimodal visualization aims at fusing different data sets so that the resulting combination provides more information and understanding to the user. To achieve this aim, we propose a new information-theoretic approach that automatically selects the most informative voxels from two volume data sets. Our fusion criteria are based on the information channel created between the two input data sets that permits us to quantify the information associated with each intensity value. This specific information is obtained from three different ways of decomposing the mutual information of the channel. In addition, an assessment criterion based on the information content of the fused data set can be used to analyze and modify the initial selection of the voxels by weighting the contribution of each data set to the final result. The proposed approach has been integrated in a general framework that allows for the exploration of volumetric data models and the interactive change of some parameters of the fused data set. The proposed approach has been evaluated on different medical data sets with very promising results.
Article
Medical image registration is an important task in medical image processing. It refers to the process of aligning data sets, possibly from different modalities (e.g., magnetic resonance and computed tomography), different time points (e.g., follow-up scans), and/or different subjects (in case of population studies). A large number of methods for image registration are described in the literature. Unfortunately, there is not one method that works for all applications. We have therefore developed elastix, a publicly available computer program for intensity-based medical image registration. The software consists of a collection of algorithms that are commonly used to solve medical image registration problems. The modular design of elastix allows the user to quickly configure, test, and compare different registration methods for a specific application. The command-line interface enables automated processing of large numbers of data sets, by means of scripting. The usage of elastix for comparing different registration methods is illustrated with three example experiments, in which individual components of the registration method are varied.
Article
In this paper, we introduce a novel and efficient approach to dense image registration, which does not require a derivative of the employed cost function. In such a context, the registration problem is formulated using a discrete Markov random field objective function. First, towards dimensionality reduction on the variables we assume that the dense deformation field can be expressed using a small number of control points (registration grid) and an interpolation strategy. Then, the registration cost is expressed using a discrete sum over image costs (using an arbitrary similarity measure) projected on the control points, and a smoothness term that penalizes local deviations on the deformation field according to a neighborhood system on the grid. Towards a discrete approach, the search space is quantized resulting in a fully discrete model. In order to account for large deformations and produce results on a high resolution level, a multi-scale incremental approach is considered where the optimal solution is iteratively updated. This is done through successive morphings of the source towards the target image. Efficient linear programming using the primal dual principles is considered to recover the lowest potential of the cost function. Very promising results using synthetic data with known deformations and real data demonstrate the potentials of our approach.
Quo vadis, atlas-based segmentation? In Handbook of Biomedical Image Analysis
  • Rohlfing T Brandt R
  • Menzel R
  • B Rus-Sakoff D
  • Jr C R Maurer
[RBM * 05] ROHLFING T., BRANDT R., MENZEL R., RUS- SAKOFF D. B., MAURER JR C. R.: Quo vadis, atlas-based segmentation? In Handbook of Biomedical Image Analysis. Springer, 2005, pp. 435–486. 1
Information-based transfer functions for multimodal visualization
  • M Haidacher
  • S Bruckner
  • Kanitsar A
  • M E Grãűller
HAIDACHER M., BRUCKNER S., KANITSAR A., GRÃŰLLER M. E.: Information-based transfer functions for multimodal visualization. In Proceedings of the First Eurographics Conference on Visual Computing for Biomedicine (2008), EG VCBM'08, Eurographics Association, pp. 101-108. 8
  • Pieper S
  • M Halle
  • Kikinis R
PIEPER S., HALLE M., KIKINIS R.: 3d slicer. In Biomedical Imaging: Nano to Macro, 2004. IEEE International Symposium on (2004), IEEE, pp. 632-635. 2
Case Study Research: Design and Methods
  • K Yin R
YIN R. K.: Case Study Research: Design and Methods. Thousand Oaks, CA: Sage, 1994. 2, 6