No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Qualitative Criteria for Feasible Cranial Implant Designs
Abstract
As part of the 2021 MICCAI AutoImplant Challenge, CT scans from 11 patients who had undergone cranioplasty using artificial implants were collected. Images of the reconstructed defective skulls before cranioplasty for these patients were shared with participating teams. Three teams submitted cranial implant designs. An experienced neurosurgeon evaluated the submissions to judge the feasibility of the implant designs for use in cranioplasty procedures. None of the submitted cranial implant designs were deemed feasible for use in cranioplasty procedures without modifications. While many implants adequately restored the skull shape by covering the defect area, most contained excess material outside of the defect, fit poorly within the defect and were too thick. Future research should move beyond solely restoring the skull shape and focus on designing implants that contain smooth transitions between skull and implant, cover the entire defect, contain no material outside of the defect, have minimal thickness, and are implantable.
... Existing deep learning-based methods usually train a deep neural net on hundreds of skull images with either synthetic defects [25][26][27][28] or real-world clinical defects [29], which are however often not publicly available or large enough for training deep models. These approaches are data-and computation-intensive, and most importantly, the reconstruction quality for large and complex defects remains inadequate for clinical use [30,31]. The failure can largely be attributed to distribution shift of the defects: the synthetic defects in the training set have different distribution from that of the test set. ...
... Augmenting the training set intensively has proven to be an effective solution to the distribution adaptation problem [32]. However, current development in data augmentation-enhanced deep learning is still evaluated as substandard by experienced neurosurgeons [30,32]. A method that is insensitive to the defects may potentially avoid the distribution shift problem in cranial defect reconstruction. ...
... This section presents the implant generation results on the craniotomy skulls from the MUG500+ dataset [37]. Figure 4 shows the 3D Slicer-based manual processing procedure of an implant ( Fig. 4(A)) generated by subtracting the input defective skull from the skull reconstructed by SSM (30). First, a median smoothing filter is applied to the subtraction result to partially disconnect the implant from the noise (Fig. 4B). ...
Designing implants for large and complex cranial defects is a challenging task, even for professional designers. Current efforts on automating the design process focused mainly on convolutional neural networks (CNN), which have produced state-of-the-art results on reconstructing synthetic defects. However, existing CNN-based methods have been difficult to translate to clinical practice in cranioplasty, as their performance on large and complex cranial defects remains unsatisfactory. In this paper, we present a statistical shape model (SSM) built directly on the segmentation masks of the skulls represented as binary voxel occupancy grids and evaluate it on several cranial implant design datasets. Results show that, while CNN-based approaches outperform the SSM on synthetic defects, they are inferior to SSM when it comes to large, complex and real-world defects. Experienced neurosurgeons evaluate the implants generated by the SSM to be feasible for clinical use after minor manual corrections. Datasets and the SSM model are publicly available at https://github.com/Jianningli/ssm.
... Another aspect to consider is the fact that fully data-driven approaches trade automatization for a lack of manual intervention, which might be needed in cases where the method is not able to produce the desired result. For example, Ellis et al. [16] states that almost all of the evaluated implants exceeded the ideal thickness of 50%, which in turn made them unsuitable for cranioplasty [16]. Furthermore, many data-driven approaches do not generalize well to real-world defects out of the box, as they are usually trained on skulls with artificially induced defects [13]. ...
... Another aspect to consider is the fact that fully data-driven approaches trade automatization for a lack of manual intervention, which might be needed in cases where the method is not able to produce the desired result. For example, Ellis et al. [16] states that almost all of the evaluated implants exceeded the ideal thickness of 50%, which in turn made them unsuitable for cranioplasty [16]. Furthermore, many data-driven approaches do not generalize well to real-world defects out of the box, as they are usually trained on skulls with artificially induced defects [13]. ...
... Despite some of the challenges mentioned, data-driven methods, as proposed in [9,12,13], do produce state-of-theart results nevertheless. In the evaluation performed as part of the AutoImplant 2021 Challenge [16], experts assessed four submissions based on false positive area, completeness, fit, and overall feasibility [16]. Similarly to our results, all submissions of the study contain 'minimal' to 'gross' amounts of false positive geometry. ...
In traumatic medical emergencies, the patients heavily depend on cranioplasty-the craft of neurocranial repair using cranial implants. Despite the improvements made in recent years, the design of a patient-specific implant (PSI) is among the most complex, expensive, and least automated tasks in cranioplasty. Further research in this area is needed. Therefore , we created a prototype application with a graphical user interface (UI) specifically tailored for semi-automatic implant generation, where the users only need to perform high-level actions. A general outline of the proposed implant generation process involves setting an area of interest, aligning the templates, and then creating the implant in voxel space. Furthermore, we show that the alignment can be improved significantly, by only considering clipped geometry in the vicinity of the defect border. The software prototype will be open-sourced at https://github.com/3Descape/Cranial_Implant_Design
... Existing deep learning-based methods usually train a deep neural net on hundreds of skull images with either synthetic defects [25][26][27][28] or clinical defects [29], depending on the availability of the clinical images. These approaches are data-and computation-intensive, and most importantly, the quality of their reconstructions for large and complex defects, which are common in cranioplasty, remains inadequate for clinical use [30,31]. The failure can largely be attributed to domain shift: the synthetic defects in the training set have different distribution to that of the test set. ...
... Augmenting the training set intensively is a potentially practical and effective solution to the problem [32]. However, current development in data augmentation-enhanced deep learning is still evaluated as substandard by clinical experts [30,32]. ...
... Figure 2 gives an example of the results obtained using shape warping. We can see thatS (30) (Figure 2 B) shows no noticeable difference on the cranium compared 6 [36,27] did not report bDSC and HD95. We calculate the two metrics in this paper based on the prediction files (implants) from [36,27]. ...
Designing implants for large and complex cranial defects is a challenging task, even for professional designers. Current efforts on automating the design process focused mainly on convolutional neural networks (CNN), which have produced state-of-the-art results on reconstructing synthetic defects. However, existing CNN-based methods have been difficult to translate to clinical practice in cranioplasty, as their performance on complex and irregular cranial defects remains unsatisfactory. In this paper, a statistical shape model (SSM) built directly on the segmentation masks of the skulls is presented. We evaluate the SSM on several cranial implant design tasks, and the results show that, while the SSM performs suboptimally on synthetic defects compared to CNN-based approaches, it is capable of reconstructing large and complex defects with only minor manual corrections. The quality of the resulting implants is examined and assured by experienced neurosurgeons. In contrast, CNN-based approaches, even with massive data augmentation, fail or produce less-than-satisfactory implants for these cases. Codes are publicly available at https://github.com/Jianningli/ssm
... Existing deep learning-based methods usually train a deep neural net on hundreds of skull images with either synthetic defects [25][26][27][28] or clinical defects [29], depending on the availability of the clinical images. These approaches are data-and computation-intensive, and most importantly, the quality of their reconstructions for large and complex defects, which are common in cranioplasty, remains inadequate for clinical use [30,31]. The failure can largely be attributed to domain shift: the synthetic defects in the training set have different distribution to that of the test set. ...
... Augmenting the training set intensively is a potentially practical and effective solution to the problem [32]. However, current development in data augmentation-enhanced deep learning is still evaluated as substandard by clinical experts [30,32]. ...
... Figure 2 gives an example of the results obtained using shape warping. We can see thatS (30) (Figure 2 B) shows no noticeable difference on the cranium compared 6 [36,27] did not report bDSC and HD95. We calculate the two metrics in this paper based on the prediction files (implants) from [36,27]. ...
Designing implants for large and complex cranial defects is a challenging task, even for professional designers. Current efforts on automating the design process focused mainly on convolutional neural networks (CNN), which have produced state-of-the-art results on reconstructing synthetic defects. However, existing CNN-based methods have been difficult to translate to clinical practice in cranioplasty, as their performance on complex and irregular cranial defects remains unsatisfactory. In this paper, a statistical shape model (SSM) built directly on the segmentation masks of the skulls is presented. We evaluate the SSM on several cranial implant design tasks, and the results show that, while the SSM performs suboptimally on synthetic defects compared to CNN-based approaches, it is capable of reconstructing large and complex defects with only minor manual corrections. The quality of the resulting implants is examined and assured by experienced neurosurgeons. In contrast, CNN-based approaches, even with massive data augmentation, fail or produce less-than-satisfactory implants for these cases. Codes are publicly available at https://github.com/Jianningli/ssm
... To conduct the evaluation, we utilize 3D Slicer software [34] to compare the 3D shape of predicted and generated implants. We use 5 qualitative criteria adapted from the prior research [38] to evaluate the predicted implant including complete, no false positive area, restored skull shape, smooth transition with skull, and minimal thickness ( Table 2). The first row of Figure 7 presents the graphical examples for each criterion. ...
Automatic cranial implant design aims to design a patient-specific implant where various machine learning-based skull reconstruction techniques have been introduced to predict the implant. Despite the significant progress made in the previous research, the existing techniques often struggle to generalize to diverse clinical cases and may not fully leverage the latest advancements in deep learning architectures. Moreover, the limited availability of large-scale clinical datasets hinders the development of the models. In this paper, we represent a novel skull reconstruction model, CraNeXt, which utilizes a ConvNeXt backbone to achieve a 5.8x reduction in size when compared to 3DUNetCNN, without sacrificing reconstruction quality. In addition, we introduce a novel method, skull categorization, to classify unlabeled skulls and determine the location of defects and the distribution of skull areas. We expand the training dataset by incorporating a larger collection of 328 in-house clinical cases, enabling the model to better capture the diversity of real-world cranial defects. CraNeXt demonstrates superior results with the skull categorization technique, achieving a dice score of 0.7969±0.13 on both public and in-house data. We perform a qualitative assessment of the predicted implants and discuss potential improvements to the skull reconstruction toward clinical use cases.
... To make our framework applicable to such specific use cases, we integrated the step of choosing application-specific metrics in the main workflow (Fig. 2). Examples of such application-specific metrics can be found in related work 36,37 . ...
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.
... Figure 11 demonstrates this thickness-modification procedure using the defective cranial model described in Figure 9 with a 2 mm thickness for the new implant. Additionally, Ellis and coauthors in (Ellis et al., 2021) emphasized the importance of creating implants with smooth transitions and complete defect coverage without excess material. This example reveals that the updated design still fulfills these requirements, despite the change in thickness. ...
Creating a personalized implant for cranioplasty can be costly and aesthetically challenging, particularly for comminuted fractures that affect a wide area. Despite significant advances in deep learning techniques for 2D image completion, generating a 3D shape inpainting remains challenging due to the higher dimensionality and computational demands for 3D skull models. Here, we present a practical deep-learning approach to generate implant geometry from defective 3D skull models created from CT scans. Our proposed 3D reconstruction system comprises two neural networks that produce high-quality implant models suitable for clinical use while reducing training time. The first network repairs low-resolution defective models, while the second network enhances the volumetric resolution of the repaired model. We have tested our method in simulations and real-life surgical practices, producing implants that fit naturally and precisely match defect boundaries, particularly for skull defects above the Frankfort horizontal plane.
... This can help to disseminate and address remaining challenges in patient-specific and fully-automatic craniofacial implant design. An example is the usage of automatically designed implants in cranioplasty procedures without major modifications [24]. Another aspect that has to be addressed by the research community is the implant thickness, which should be thinner than the skull bone. ...
... This can help to disseminate and address remaining challenges in patient-individual and fully-automatic cranial implant design. An example is the usage of automatic designed implants in cranioplasty procedures without major modifications [29]. Another aspect that has to be addressed by the research community is the implant thickness, which should be thinner than the skull bone. ...
We present a deep learning-based approach for skull reconstruction for MONAI, which has been pre-trained on the MUG500+ skull dataset. The implementation follows the MONAI contribution guidelines, hence, it can be easily tried out and used, and extended by MONAI users. The primary goal of this paper lies in the investigation of open-sourcing codes and pre-trained deep learning models under the MONAI framework. Nowadays, open-sourcing software, especially (pre-trained) deep learning models, has become increasingly important. Over the years, medical image analysis experienced a tremendous transformation. Over a decade ago, algorithms had to be implemented and optimized with low-level programming languages, like C or C++, to run in a reasonable time on a desktop PC, which was not as powerful as today's computers. Nowadays, users have high-level scripting languages like Python, and frameworks like PyTorch and TensorFlow, along with a sea of public code repositories at hand. As a result, implementations that had thousands of lines of C or C++ code in the past, can now be scripted with a few lines and in addition executed in a fraction of the time. To put this even on a higher level, the Medical Open Network for Artificial Intelligence (MONAI) framework tailors medical imaging research to an even more convenient process, which can boost and push the whole field. The MONAI framework is a freely available, community-supported, open-source and PyTorch-based framework, that also enables to provide research contributions with pre-trained models to others. Codes and pre-trained weights for skull reconstruction are publicly available at: https://github.com/Project-MONAI/research-contributions/tree/master/SkullRec
Cranial implants are commonly used for surgical repair of craniectomy-induced skull defects. These implants are usually generated offline and may require days to weeks to be available. An automated implant design process combined with onsite manufacturing facilities can guarantee immediate implant availability and avoid secondary intervention. To address this need, the AutoImplant II challenge was organized in conjunction with MICCAI 2021, catering for the unmet clinical and computational requirements of automatic cranial implant design. The first edition of AutoImplant (AutoImplant I, 2020) demonstrated the general capabilities and effectiveness of data-driven approaches, including deep learning, for a skull shape completion task on synthetic defects. The second AutoImplant challenge (i.e., AutoImplant II, 2021) built upon the first by adding real clinical craniectomy cases as well as additional synthetic imaging data. The AutoImplant II challenge consisted of three tracks. Tracks 1 and 3 used skull images with synthetic defects to evaluate the ability of submitted approaches to generate implants that recreate the original skull shape. Track 3 consisted of the data from the first challenge (i.e., 100 cases for training, and 110 for evaluation), and Track 1 provided 570 training and 100 validation cases aimed at evaluating skull shape completion algorithms at diverse defect patterns. Track 2 also made progress over the first challenge by providing 11 clinically defective skulls and evaluating the submitted implant designs on these clinical cases. The submitted designs were evaluated quantitatively against imaging data from post-craniectomy as well as by an experienced neurosurgeon. Submissions to these challenge tasks made substantial progress in addressing issues such as generalizability, computational efficiency, data augmentation, and implant refinement. This paper serves as a comprehensive summary and comparison of the submissions to the AutoImplant II challenge. Codes and models are available at https://github.com/Jianningli/Autoimplant_II
Background and objective:
This article presents a robust, fast, and fully automatic method for personalized cranial defect reconstruction and implant modeling.
Methods:
We propose a two-step deep learning-based method using a modified U-Net architecture to perform the defect reconstruction, and a dedicated iterative procedure to improve the implant geometry, followed by an automatic generation of models ready for 3-D printing. We propose a cross-case augmentation based on imperfect image registration combining cases from different datasets. Additional ablation studies compare different augmentation strategies and other state-of-the-art methods.
Results:
We evaluate the method on three datasets introduced during the AutoImplant 2021 challenge, organized jointly with the MICCAI conference. We perform the quantitative evaluation using the Dice and boundary Dice coefficients, and the Hausdorff distance. The Dice coefficient, boundary Dice coefficient, and the 95th percentile of Hausdorff distance averaged across all test sets, are 0.91, 0.94, and 1.53 mm respectively. We perform an additional qualitative evaluation by 3-D printing and visualization in mixed reality to confirm the implant's usefulness.
Conclusion:
The article proposes a complete pipeline that enables one to create the cranial implant model ready for 3-D printing. The described method is a greatly extended version of the method that scored 1st place in all AutoImplant 2021 challenge tasks. We freely release the source code, which together with the open datasets, makes the results fully reproducible. The automatic reconstruction of cranial defects may enable manufacturing personalized implants in a significantly shorter time, possibly allowing one to perform the 3-D printing process directly during a given intervention. Moreover, we show the usability of the defect reconstruction in a mixed reality that may further reduce the surgery time.
The article introduces two complementary datasets intended for the development of data-driven solutions for cranial implant design, which remains to be a time-consuming and laborious task in current clinical routine of cranioplasty. The two datasets, referred to as the SkullBreak and SkullFix in this article, are both adapted from a public head CT collection CQ500 (http://headctstudy.qure.ai/dataset) with CC BY-NC-SA 4.0 license. The SkullBreak contains 114 and 20 complete skulls, each accompanied by five defective skulls and the corresponding cranial implants, for training and evaluation respectively. The SkullFix contains 100 triplets (complete skull, defective skull and the implant) for training and 110 triplets for evaluation. The SkullFix dataset was first used in the MICCAI 2020 AutoImplant Challenge (https://autoimplant.grand-challenge.org/) and the ground truth, i.e., the complete skulls and the implants in the evaluation set are held private by the organizers. The two datasets are not overlapping and differ regarding data selection and synthetic defect creation and each serves as a complement to the other. Besides cranial implant design, the datasets can be used for the evaluation of volumetric shape learning algorithms, such as volumetric shape completion. This article gives a description of the two datasets in detail.
Automatic cranial implant design can save clinicians time and resources by computing the implant shape and size from a single image of a defective skull. We aimed to improve upon previously proposed deep learning methods by augmenting the training data set using transformations that warped the images into different shapes and orientations. The transformations were computed by non-linearly registering the complete skull images between the 100 subjects in the training data set. The transformations were then applied to warp each of the defective and complete skull images so that the shape and orientation resembled that of a different subject in the training set. One hundred ninety-seven of the registrations failed, resulting in an augmented training set of 9,803 defective and complete skull image pairs. The augmented training set was used to train an ensemble of four U-Net models to predict the complete skull shape from the defective skulls using cross-validation. The ensemble of models performed very well and predicted the implant shapes with a mean dice similarity coefficient of 0.942 and a mean Hausdorff distance of 3.598 mm for all 110 test cases. Our solution ranked first among all participants of the AutoImplant 2020 challenge. The code for this project is available at https://github.com/ellisdg/3DUnetCNN.
We provide examples and highlights of Advanced Normalization Tools (ANTS) that address practical problems in real data.
Correct virtual reconstruction of a defective skull is a prerequisite for successful cranioplasty and its automatization has the potential for accelerating and standardizing the clinical workflow. This work provides a deep learning-based method for the reconstruction of a skull shape and cranial implant design on clinical data of patients indicated for cranioplasty. The method is based on a cascade of multi-branch volumetric CNNs that enables simultaneous training on two different types of cranioplasty ground-truth data: the skull patch, which represents the exact shape of the missing part of the original skull, and which can be easily created artificially from healthy skulls, and expert-designed cranial implant shapes that are much harder to acquire. The proposed method reaches an average surface distance of the reconstructed skull patches of 0.67 mm on a clinical test set of 75 defective skulls. It also achieves a 12% reduction of a newly proposed defect border Gaussian curvature error metric, compared to a baseline model trained on synthetic data only. Additionally, it produces directly 3D printable cranial implant shapes with a Dice coefficient 0.88 and a surface error of 0.65 mm. The outputs of the proposed skull reconstruction method reach good quality and can be considered for use in semi- or fully automatic clinical cranial implant design workflows.
A fast and fully automatic design of 3D printed patient-specific cranial implants is highly desired in cranioplasty - the process to restore a defect on the skull. We formulate skull defect restoration as a 3D volumetric shape completion task, where a partial skull volume is completed automatically. The difference between the completed skull and the partial skull is the restored defect; in other words, the implant that can be used in cranioplasty. To fulfill the task of volumetric shape completion, a fully data-driven approach is proposed. Supervised skull shape learning is performed on a database containing 167 high-resolution healthy skulls. In these skulls, synthetic defects are injected to create training and evaluation data pairs. We propose a patch-based training scheme tailored for dealing with high-resolution and spatially sparse data, which overcomes the disadvantages of conventional patch-based training methods in high-resolution volumetric shape completion tasks. In particular, the conventional patch-based training is applied to images of high resolution and proves to be effective in tasks such as segmentation. However, we demonstrate the limitations of conventional patch-based training for shape completion tasks, where the overall shape distribution of the target has to be learnt, since it cannot be captured efficiently by a sub-volume cropped from the target. Additionally, the standard dense implementation of a convolutional neural network tends to perform poorly on sparse data, such as the skull, which has a low voxel occupancy rate. Our proposed training scheme encourages a convolutional neural network to learn from the high-resolution and spatially sparse data. In our study, we show that our deep learning models, trained on healthy skulls with synthetic defects, can be transferred directly to craniotomy skulls with real defects of greater irregularity, and the results show promise for clinical use. Project page: https://github.com/Jianningli/MIA.
The aim of this paper is to provide a comprehensive overview of the MICCAI 2020 AutoImplant Challenge1. The approaches and publications submitted and accepted within the challenge will be summarized and reported, highlighting common algorithmic trends and algorithmic diversity. Furthermore, the evaluation results will be presented, compared and discussed in regard to the challenge aim: seeking for low cost, fast and fully automated solutions for cranial implant design. Based on feedback from collaborating neurosurgeons, this paper concludes by stating open issues and post-challenge requirements for intra-operative use. The codes can be found at https://github.com/Jianningli/tmi.
The AutoImplant Cranial Implant Design Challenge (AutoImplant 2020: https://
autoimplant.grand-challenge.org/) was initialized jointly by the Graz University of
Technology (TU Graz) and the Medical University of Graz (MedUni Graz), Austria,
through an interdisciplinary project “Clinical Additive Manufacturing for Medical
Applications” (CAMed: https://www.medunigraz.at/camed/) between the two institutions.
The project aims to provide more affordable, faster, and patient-friendly solutions
to the design and manufacturing of medical implants, including cranial implants, which
is needed in order to repair a defective skull from a brain tumor surgery or trauma.
3D Slicer is an open-source platform for the analysis and display of information derived from medical imaging and similar data sets. Such advanced software environments are in daily use by researchers and clinicians and in many nonmedical applications. 3D Slicer is unique through serving clinical users, multidisciplinary clinical research terms, and software architects within a single technology structure and user community. Functions such as interactive visualization, image registration, and model-based analysis are now being complemented by more advanced capabilities, most notably in neurological imaging and intervention. These functions, originally limited to offline use by technical factors, are integral to large scale, rapidly developing research studies, and they are being increasingly integrated into the management and delivery of care. This activity has been led by a community of basic, applied, and clinical scientists and engineers, from both academic and commercial perspectives. 3D Slicer, a free open-source software package, is based in this community; 3D Slicer provides a set of interactive tools and a stable platform that can quickly incorporate new analysis techniques and evolve to serve more sophisticated real-time applications while remaining compatible with the latest hardware and software generations of host computer systems.
Correlation is a statistical method used to assess a possible linear association between two continuous variables. It is simple both to calculate and to interpret. However, misuse of correlation is so common among researchers that some statisticians have wished that the method had never been devised at all. The aim of this article is to provide a guide to appropriate use of correlation in medical research and to highlight some misuse. Examples of the applications of the correlation coefficient have been provided using data from statistical simulations as well as real data. Rule of thumb for interpreting size of a correlation coefficient has been provided.