PosterPDF Available

Lower Jawbone Data Generation for Deep Learning Tools under MeVisLab

Authors:

Abstract

Segmentation is an important branch in medical image processing and the basis for further detailed investigations on computed tomography (CT), magnetic resonance imaging (MRI), X-ray, ultrasound (US) or nuclear images. Through segmentation, an image is divided into various connected areas that correspond to certain tissue types. A common aim is to delineate healthy and pathologic tissues. A frequent example in medicine is the identification of a tumor or pathological lesion and its volume to evaluate treatment planning and outcome. In the clinical routine, segmentation is necessary for the planning of specific treatment tasks, that are for example used in the radiation therapy or for the creation of three-dimensional (3D) visualizations and models to simulate a surgical procedure. Segmentation can be classified into several families of techniques, such as thresholding, region growing, watershed, edge-based approaches, active contours and model-based algorithms. Recently, deep learning using neural networks is becoming important for automatic segmentation applications.
Lower Jawbone Data Generation for Deep Learning Tools under
MeVisLab
B. Pfarrkirchner, C. Gsaxner, L. Lindner, N. Jakse, J. Wallner, D. Schmalstieg, J. Egger
Graz University of Technology, Institute for Computer Graphics and Vision, Graz, Austria; BioTechMed-Graz, Graz, Austria
Medical University of Graz, Department of Maxillofacial Surgery, Graz, Austria; Computer Algorithms for Medicine (Cafe) Laboratory, Graz, Austria
METHODSINTRODUCTION RESULTS
1. Zhou, K. et al. “Deep Learning for Medical Image Analysis,” Elsevier,
pp. 1-458 (2017).
2. Egger, J. et al. “Integration of the OpenIGTlink network protocol for
image guided therapy with the medical platform MeVisLab,” The
international Journal of medical Robotics and Computer assisted
Surgery, 8(3):282-390 (2012).
3. Egger, J. et al. “HTC Vive MeVisLab integration via OpenVR for
medical applications,” PLoS ONE 12(3): e0173972 (2017).
CONCLUSIONS
REFERENCES
The work received funding from BioTechMed-Graz in Austria (“Hardware accelerated intelligent medical imaging”), the 6th Call of the Initial Funding Program from the Research & Technology
House (F&T-Haus) at the Graz University of Technology (PI: Dr. Dr. habil. Jan Egger). The corresponding Macro module and Python source code is freely available under:
https://github.com/birgitPf/Data_Generation
Segmentation is an important branch in medical image
processing and the basis for further detailed investigations on
computed tomography (CT), magnetic resonance imaging
(MRI), X-ray, ultrasound (US) or nuclear images. Through
segmentation, an image is divided into various connected
areas that correspond to certain tissue types. A common aim
is to delineate healthy and pathologic tissues. A frequent
example in medicine is the identification of a tumor or
pathological lesion and its volume to evaluate treatment
planning and outcome. In the clinical routine, segmentation is
necessary for the planning of specific treatment tasks, that are
for example used in the radiation therapy or for the creation of
three-dimensional (3D) visualizations and models to simulate
a surgical procedure. Segmentation can be classified into
several families of techniques, such as thresholding, region
growing, watershed, edge-based approaches, active contours
and model-based algorithms. Recently, deep learning using
neural networks is becoming important for automatic
segmentation applications.
All implementations of this work were accomplished with the
MeVisLab platform. MeVisLab is a medical image processing
software with a graphical user interface. Besides, it provides
built-in modules for basic image processing operations, such
as low-pass filtering. These modules can be connected to form
image processing networks.
We developed a MeVisLab module network that converts
segmentation contours into ground truth images and a
depiction of the patients' CT images. In addition, a MeVisLab
macro module was created, which saves image data as
separate and, optionally, modified slices.
In addition, we developed a SaveAsSingleSlices macro-
module, which allows storing all slices of one image data stack
as separate TIFF or PNG files automatically. In addition, our
macro-module is able to export selected slices and to
augment the dataset with geometric transformations and
noise. Geometric transformations may use any combination of
rotation, scaling and mirroring, while noise can be of the
uniform, Gaussian or salt-and-pepper variety. All parameters
can be interactive specified in a custom user interface panel.
The number of exportable images relies on the definition of
the storage parameters by the user. If the default values for
transformation and adding noise are applied on a single slice,
it is possible to export eleven slices (the original slice and ten
artificially generated slices). We chose rotation angles of
±8°, and a scale of 1±0.04 in x- and y-direction. The
amplitude of uniform noise has a value of 800 gray values, the
Gaussian noise has a mean value of zero and a standard
deviation of 300 gray values. The salt-and-pepper amplitudes
are set to ±2000 gray values, and the density is set to a value
of 0.05.
Fig. 3 Examples of exported CT slices in the lower
jawbone area. The left image shows the original acquired
CT slice of a patient. In the middle, is a flipped version of
the left image visible. The right CT slice displays an image
with added salt-and-pepper noise (amplitude ±2000 and
density 0.05).
Neural networks are constructed of neurons that are
organized into input layers, output layers, and hidden
layers, which are located between the input and
output layers. Neural networks with a large number
of layers are known as deep networks. The neurons
are connected via weights, which can be trained with
a training dataset to solve specific problems. For
efficient training, neural networks require large
amounts of training data.
In this work, a MeVisLab network and a macro-module have
been developed to provide a convenient way to handle data
preparation and augmentation for the segmentation of the
lower jawbones with deep learning networks. The macro-
module provides a convenient interface for tuning the training
data set by selecting slices and applying freely configurable
data augmentation. This approach makes it easy to
systematically vary the data set before training.
Fig. 2 Various representations created with a View2D
modules. No. 1 shows the original CT, No. 2 the CT with
an overlaid contour, No. 3 the binary mask and No. 4 the
CT with the overlaid binary mask.
To train deep learning networks for lower jawbone (mandible)
segmentation, we used anonymized CT datasets from ten
patients of the complete head and neck region. All CT data
acquisitions have been performed in the clinical routine for
diagnostic reasons at a maxillofacial surgery department at
our disposal. Each mandible of these CT images was
segmented by two specialized doctors manually (slice-by-slice
in axial direction) to generate the ground truth contours for the
segmentation task.
... The ground truth data and segmentations of this contribution can be used for the training procedures of deep learning networks, for a fully-automatic segmentation of pathological lesions such as tumors in CTs or PET/CTs or for a further comparison of CE-marked or ISO certified software packages such as Brainlab, Materialise Lead Project (Mechanics, Modeling and Simulation of Aortic Dissection). Some of the present data of this collection have partly been used in already published works [37,60,[66][67][68] . In these publications the data was used as testing data for single algorithms or deep learning networks. ...
... In these publications the data was used as testing data for single algorithms or deep learning networks. Some CT datasets were used to test a deep learning network [67] in the mandible for full automatic networkbased segmentation and to test the accuracy of the single opensource algorithm GrowCut ( www.growcut.com ) in the mandible [37] . ...
... (standard triangle language) file format as a surface model without segmentation to evaluate a software tool for computer-aided positioning planning of miniplates for oral & maxillofacial surgery [60] . Further, a collection of the compared manual segmented ground truth models was made available for end users out of reproducibility reasons [67] . ...
Article
Background and objectives: Computer-assisted technologies, such as image-based segmentation, play an important role in the diagnosis and treatment support in cranio-maxillofacial surgery. However, although many segmentation software packages exist, their clinical in-house use is often challenging due to constrained technical, human or financial resources. Especially technological solutions or systematic evaluations of open-source based segmentation approaches are lacking. The aim of this contribution is to assess and review the segmentation quality and the potential clinical use of multiple commonly available and license-free segmentation methods on different medical platforms. Methods: In this contribution, the quality and accuracy of open-source segmentation methods was assessed on different platforms using patient-specific clinical CT-data and reviewed with the literature. The image-based segmentation algorithms GrowCut, Robust Statistics Segmenter, Region Growing 3D, Otsu & Picking, Canny Segmentation and Geodesic Segmenter were investigated in the mandible on the platforms 3D Slicer, MITK and MeVisLab. Comparisons were made between the segmentation algorithms and the ground truth segmentations of the same anatomy performed by two clinical experts (n = 20). Assessment parameters were the Dice Score Coefficient (DSC), the Hausdorff Distance (HD), and Pearsons correlation coefficient (r). Results: The segmentation accuracy was highest with the GrowCut (DSC 85.6%, HD 33.5 voxel) and the Canny (DSC 82.1%, HD 8.5 voxel) algorithm. Statistical differences between the assessment parameters were not significant (p < 0.05) and correlation coefficients were close to the value one (r > 0.94) for any of the comparison made between the segmentation methods and the ground truth schemes. Functionally stable and time-saving segmentations were observed. Conclusion: High quality image-based semi-automatic segmentation was provided by the GrowCut and the Canny segmentation method. In the cranio-maxillofacial complex, these segmentation methods provide algorithmic alternatives for image-based segmentation in the clinical practice for e.g. surgical planning or visualization of treatment results and offer advantages through their open-source availability. This is the first systematic multi-platform comparison that evaluates multiple license-free, open-source segmentation methods based on clinical data for the improvement of algorithms and a potential clinical use in patient-individualized medicine. The results presented are reproducible by others and can be used for clinical and research purposes.
Article
PurposeMedical image segmentation is the most widely used technique in diagnostic and clinical research. However, accurate segmentation of target organs from blurred border regions and low-contrast adjacent organs in Computed tomography (CT) imaging is crucial for clinical diagnosis and treatment.Methods In this article, we propose a Multi-Scale Feature Pyramid Fusion Network (MS-Net) based on the codec structure formed by the combination of Multi-Scale Attention Module (MSAM) and Stacked Feature Pyramid Module (SFPM). Among them, MSAM is used to skip connections, which aims to extract different levels of context details by dynamically adjusting the receptive fields under different network depths; the SFPM including multi-scale strategies and multi-layer Feature Perception Module (FPM) is nested in the network at the deepest point, which aims to better focus the network's attention on the target organ by adaptively increasing the weight of the features of interest.ResultsExperiments demonstrate that the proposed MS-Net significantly improved the Dice score from 91.74% to 94.54% on CHAOS, from 97.59% to 98.59% on Lung, and from 82.55% to 86.06% on ISIC 2018, compared with U-Net. Additionally, comparisons with other six state-of-the-art codec structures also show the presented network has great advantages on evaluation indicators such as Miou, Dice, ACC and AUC.Conclusion The experimental results show that both the MSAM and SFPM techniques proposed in this paper can assist the network to improve the segmentation effect, so that the proposed MS-Net method achieves better results in the CHAOS, Lung and ISIC 2018 segmentation tasks.
ResearchGate has not been able to resolve any references for this publication.