No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Glomerulosclerosis Identification in Whole Slide Images using Semantic Segmentation
Abstract
Background and objective:
Glomeruli identification, i.e., detection and characterization, is a key procedure in many nephropathology studies. In this paper, semantic segmentation based on convolutional neural networks (CNN) is proposed to detect glomeruli using Whole Slide Imaging (WSI) follows by a classification CNN to divide the glomeruli into normal and sclerosed.
Methods:
Comparison between U-Net and SegNet CNNs is performed for pixel-level segmentation considering both a two and three class problem, that is, a) non-glomerular and glomerular structures and b) non-glomerular normal glomerular and sclerotic structures. The two class semantic segmentation result is then used for a CNN classification where glomerular regions are divided into normal and global sclerosed glomeruli.
Results:
These methods were tested on a dataset composed of 47 WSIs belonging to human kidney sections stained with Periodic Acid Schiff (PAS). The best approach was the SegNet for two class segmentation follows by a fine-tuned AlexNet network to characterize the glomeruli. 98.16% of accuracy was obtained with this process of consecutive CNNs (SegNet-AlexNet) for segmentation and classification.
Conclusion:
The results obtained demonstrate that the sequential CNN segmentation-classification strategy achieves higher accuracy reducing misclassified cases and therefore being the methodology proposed for glomerulosclerosis detection.
... Automated glomeruli identification frameworks for kidney biopsies conducted by pathologists can be quite helpful because manual examination of kidney samples is time-consuming and error-prone [9], [8]. There are several segmentation methods to detect glomeruli in kidney images [10]. However, these methods require pixel-level annotation of the images. ...
... It is possible to group glomerular disorders according to their clinical symptoms, etymology, immunopathology, or morphological changes [9], [8]. A condition known as "glomerulosclerosis" is the result of the kidney lesion changing its morphology; this sclerosis can impact the kidney in many ways, depending on whether or not it is global or partial [10]. The number of glomeruli detected in each kidney biopsy should be counted in daily practice. ...
... The number of glomeruli detected in each kidney biopsy should be counted in daily practice. Per kidney biopsy, about 20 to 30 cuts are made [10]. Additionally, glomeruli that are completely sclerosed must be noted (the entire glomerulus). ...
Context: Analyzing digital pathology images is necessary to draw diagnostic conclusions by investigating tissue patterns and cellular morphology. However, manual evaluation can be time-consuming, expensive, and prone to inter- and intra-observer variability. Objective: To assist pathologists using computerized solutions, automated tissue structure detection and segmentation must be proposed. Furthermore, generating pixel-level object annotations for histopathology images is expensive and time-consuming. As a result, detection models with bounding box labels may be a feasible solution. Design: This paper studies. YOLO-v4 (You-Only-Look-Once), a real-time object detector for microscopic images. YOLO uses a single neural network to predict several bounding boxes and class probabilities for objects of interest. YOLO can enhance detection performance by training on whole slide images. YOLO-v4 has been used in this paper. for glomeruli detection in human kidney images. Multiple experiments have been designed and conducted based on different training data of two public datasets and a private dataset from the University of Michigan for fine-tuning the model. The model was tested on the private dataset from the University of Michigan, serving as an external validation of two different stains, namely hematoxylin and eosin (H&E) and periodic acid-Schiff (PAS). Results: Average specificity and sensitivity for all experiments, and comparison of existing segmentation methods on the same datasets are discussed. Conclusions: Automated glomeruli detection in human kidney images is possible using modern AI models. The design and validation for different stains still depends on variability of public multi-stain datasets.
... Assim uma etapa importante do diagnóstico de inúmeras patologias renaiś e avaliar lesões glomerulares em lâminas histológicas [1]. Dentre as variadas lesões capazes de afetar os glomérulos, há a hipercelularidade, com aumento na quantidade de células nos glomérulos [1]; a glomeruloesclerose, com alterações morfológicas dos glomérulos pelo endurecimento patológico dos tecidos [2], as lesões membranonas, com o espessamento da parede dos capilares glomerulares [3], e a crescente glomerular, caracterizada por lesões que apresentam uma anormal proliferação de células, envolvendo 10% ou mais da circunferência da cápsula de Bowman. A presença de crescentes em biópsias renais pode acontecer em diversos distúrbios glomerulares, denominados glomerulopatias com crescentes [4]. ...
... Vários trabalhos de aplicações de machine learning e deep learning na patologia digital por análise de imagens da nefrologia estão presentes na literatura [1], [2], [5]- [10]. As técnicas utilizadas pelos autores dos trabalhos são variadas, foram identificadas pesquisas abordando problemas de classificação, detecção, e segmentação. ...
... Bueno et. al. [2] apresentam a segmentação semântica baseada em CNNs para detecção de glomérulos em WSI, seguida por uma CNN para a classificação dos glomérulos em normais ou esclerosados. Uchino et. ...
Glomérulos são estruturas localizadas nos rins e responsáveis por filtrar o sangue e podem ser acometidos por diversas lesões, como a crescente glomerular, que é caracterizada por apresentar uma anormal proliferação de células. Neste trabalho, são avaliados diferentes modelos e condições de aplicação de deep learning na tarefa de classificação de imagens histopatológicas de crescente glomerular. Para isso, foram comparadas as redes pré-treinadas Xception, InceptionV3, MobileNet, VGG16 e ResNet50, aplicando-se para a classificação de imagens com glomérulos com crescente vs normais. Comparando acurácia, precisão, recall e f1-score dos modelos, a ResNet50 apresentou desempenho significativamente superior ao das demais redes, em todas as medidas. A aplicação de data augmentation não resultou em melhora significativa nos resultados neste caso. Em experimento de classificação de glomérulos crescentes vs não crescentes, adicionando imagens de três outras lesões à base de dados, a aplicação do Focal Loss, apresentou maior acurácia e precisão.
... In the past decade, the number of studies aiming to develop deep learning applications for nephropathology has increased rapidly. Computational image recognition focusing on the glomerulus is generally classified into the following three types: the detection of glomeruli [11][12][13][14], the classification of the glomeruli [10,15], and the segmentation of the glomeruli [16][17][18][19][20][21][22][23][24]. The glomeruli that are detected in the WSI are localized by drawing bounding boxes. ...
... The segmentation of glomeruli localizes and quantifies every glomerulus by identifying the regions of each glomerulus in the pixels. Several studies have attempted to distinguish between the entire glomerulus and the background [16,17] or to distinguish between the normal and the sclerotic glomeruli [20,21,24]. Other studies have focused on the tubules, the blood vessels, and the interstitium, in addition to the glomerulus [19,23], or on the components inside of the glomerulus [18,22]. ...
... Several studies [18][19][20][21] aiming for pixel-level semantic segmentation for WSI of renal tissue sections have set the task of distinguishing between nonsclerotic and sclerotic glomeruli. Bueno et al. [20] sequentially applied SegNet-VGG19 [34] in order to segment glomeruli and applied AlexNet to classify them as nonsclerotic or sclerotic glomeruli. ...
The histopathological findings of the glomeruli from whole slide images (WSIs) of a renal biopsy play an important role in diagnosing and grading kidney disease. This study aimed to develop an automated computational pipeline to detect glomeruli and to segment the histopathological regions inside of the glomerulus in a WSI. In order to assess the significance of this pipeline, we conducted a multivariate regression analysis to determine whether the quantified regions were associated with the prognosis of kidney function in 46 cases of immunoglobulin A nephropathy (IgAN). The developed pipelines showed a mean intersection over union (IoU) of 0.670 and 0.693 for five classes (i.e., background, Bowman’s space, glomerular tuft, crescentic, and sclerotic regions) against the WSI of its facility, and 0.678 and 0.609 against the WSI of the external facility. The multivariate analysis revealed that the predicted sclerotic regions, even those that were predicted by the external model, had a significant negative impact on the slope of the estimated glomerular filtration rate after biopsy. This is the first study to demonstrate that the quantified sclerotic regions that are predicted by an automated computational pipeline for the segmentation of the histopathological glomerular components on WSIs impact the prognosis of kidney function in patients with IgAN.
... 10 Also, CNN have been applied to WSI to classify sclerosed and nonsclerosed glomeruli. 11,12 Kolachalama et al. demonstrated that CNN deep learning models can outperform the pathologist-estimated fibrosis score across the classification tasks and can be applied to routine renal biopsy images. 13 To date, most machine learning and deep learning algorithms applied to histopathology images have been based on supervised (training) approaches. ...
... In histopathology image analysis, artificial intelligence and machine learning methods have been used in computer-aided studies to solve diagnostic decision-making problems, and most of the machine learning methods applied to histopathology slides have relied on fully supervised learning and pixel-level expert annotations to extract features or train a model. 11,13,29,30 However, supervised machine learning presents some major challenges. First, supervised learning requires significant labeling efforts, which is a very time-consuming task that is often impractical in histopathology images due to their large image size with high resolution. ...
Machine learning applied to digital pathology has been increasingly used to assess kidney function and diagnose the underlying cause of chronic kidney disease (CKD). We developed a novel computational framework, Clustering-based Spatial Analysis (CluSA), that leverages unsupervised learning to learn spatial relationships between local visual patterns in kidney tissue. This framework minimizes the need for time-consuming and impractical expert annotations. To incorporate spatial information over the clustered image patterns on the biopsy sample, we spatially encoded clustered patterns with colors and performed spatial analysis through graph neural network. A random forest classifier with various groups of features were used to predict CKD. For predicting high/low eGFR at the biopsy, we achieved a sensitivity of 0.97, specificity of 0.90, and accuracy of 0.95. AUC was 0.96. For predicting eGFR changes in one-year, we achieved a sensitivity of 0.83, specificity of 0.85, and accuracy of 0.84. AUC was 0.85. This study presents the first spatial analysis based on unsupervised machine learning algorithms. Without expert annotation, CluSA framework can not only accurately classify and predict the degree of kidney function at the biopsy and in one year, but also identify novel predictors of kidney function and renal prognosis.
... (2) used traditional image analysis and modern machine learning techniques to grade the severity of diabetic glomerular changes, and this correlated with disease severity. Other studies (3,4) have used semantic segmentation using convolutional neural networks (CNNs) to detect and classify glomeruli as normal and sclerosed from WSIs. Taken together, this demonstrates that machine learning tools are feasible for some aspects of renal pathology classification. ...
... The cropped images are resized to these dimensions using bicubic interpolation and an example is shown in Figure 3 below. Color normalization was also investigated using a modified version of Reinhard's method which decreases color variability in WSIs (3,4,7). ...
Introduction
When assessing kidney biopsies, pathologists use light microscopy, immunofluorescence, and electron microscopy to describe and diagnose glomerular lesions and diseases. These methods can be laborious, costly, fraught with inter-observer variability, and can have delays in turn-around time. Thus, computational approaches can be designed as screening and/or diagnostic tools, potentially relieving pathologist time, healthcare resources, while also having the ability to identify novel biomarkers, including subvisual features.
Methods
Here, we implement our recently published biomarker feature extraction (BFE) model along with 3 pre-trained deep learning models (VGG16, VGG19, and InceptionV3) to diagnose 3 glomerular diseases using PAS-stained digital pathology images alone. The BFE model extracts a panel of 233 explainable features related to underlying pathology, which are subsequently narrowed down to 10 morphological and microstructural texture features for classification with a linear discriminant analysis machine learning classifier. 45 patient renal biopsies (371 glomeruli) from minimal change disease (MCD), membranous nephropathy (MN), and thin-basement membrane nephropathy (TBMN) were split into training/validation and held out sets. For the 3 deep learningmodels, data augmentation and Grad-CAM were used for better performance and interpretability.
Results
The BFE model showed glomerular validation accuracy of 67.6% and testing accuracy of 76.8%. All deep learning approaches had higher validation accuracies (most for VGG16 at 78.5%) but lower testing accuracies. The highest testing accuracy at the glomerular level was VGG16 at 71.9%, while at the patient-level was InceptionV3 at 73.3%.
Discussion
The results highlight the potential of both traditional machine learning and deep learning-based approaches for kidney biopsy evaluation.
... As these works achieved superior results in the segmentation of renal structures, we also adopted U-Net as our baseline architecture. In fact, 10 out of 18 works in Table 1 use only U-Net (Jayapandian et al., 2021;Davis et al., 2021;Hermsen et al., 2019;Jha et al., 2021;Bueno et al., 2020), variations of U-Net (Gadermayr et al., 2019;Bouteldja et al., 2021) or combine it with other methods (Mei et al., 2020;Zeng et al., 2020;de Bel et al., 2018). The remaining works explore other DL-based segmentation approaches such as one DL network: Mask-RCNN (Jiang et al., 2021) and DeepLabV2 Ginley et al., 2020); two separate DL networks: MaskRCNN and FastRCNN (Altini et al., 2020a), and SegNet and DeepLabV3+ (Altini et al., 2020b); three separate DL networks: Mask-RCNN, U-Net, and DeepLabV3 (Jha et al., 2021); a combination of two DL networks: SegNet and AlexNet (Bueno et al., 2020); and finally pipelines that combine DL approaches with conventional image processing methods (Marsh et al., 2018;Kannan et al., 2019;Ginley et al., 2019). ...
... In fact, 10 out of 18 works in Table 1 use only U-Net (Jayapandian et al., 2021;Davis et al., 2021;Hermsen et al., 2019;Jha et al., 2021;Bueno et al., 2020), variations of U-Net (Gadermayr et al., 2019;Bouteldja et al., 2021) or combine it with other methods (Mei et al., 2020;Zeng et al., 2020;de Bel et al., 2018). The remaining works explore other DL-based segmentation approaches such as one DL network: Mask-RCNN (Jiang et al., 2021) and DeepLabV2 Ginley et al., 2020); two separate DL networks: MaskRCNN and FastRCNN (Altini et al., 2020a), and SegNet and DeepLabV3+ (Altini et al., 2020b); three separate DL networks: Mask-RCNN, U-Net, and DeepLabV3 (Jha et al., 2021); a combination of two DL networks: SegNet and AlexNet (Bueno et al., 2020); and finally pipelines that combine DL approaches with conventional image processing methods (Marsh et al., 2018;Kannan et al., 2019;Ginley et al., 2019). ...
The growing availability of scanned whole-slide images (WSIs) has allowed nephropathology to open new possibilities for medical decision-making over high-resolution images. Diagnosis of renal WSIs includes locating and identifying specific structures in the tissue. Considering the glomerulus as one of the first structures analyzed by pathologists, we propose here a novel convolutional neural network for glomerulus segmentation. Our end-to-end network, named DS-FNet, combines the strengths of semantic segmentation and semantic boundary detection networks via an attention-aware mechanism. Although we trained the proposed network on periodic acid-Schiff (PAS)-stained WSIs, we found that our network was capable to segment glomeruli on WSIs stained with different techniques, such as periodic acid-methenamine silver (PAMS), hematoxylin-eosin (HE), and Masson trichrome (TRI). To assess the performance of the proposed method, we used three public data sets: HuBMAP (available in a Kaggle competition), a subset of the NEPTUNE data set, and a novel challenging data set, called WSI_Fiocruz. We compared the DS-FNet with six other deep learning networks: original U-Net, our attention version of U-Net called AU-Net, U-Net++, U-Net3Plus, ResU-Net, and DeepLabV3+. Results showed that DS-FNet achieved equivalent or superior results on all data sets: On the HuBMAP data set, it reached a dice score (DSC) of 95.05%, very close to the first place (95.15%); on the NEPTUNE and WSI_Fiocruz data sets, DS-FNet obtained the highest average DSC, whether on PAS-stained images or images stained with other techniques. To the best we know, this is the first work to show consistently high performance in a one-to-many-stain glomerulus segmentation following a thorough protocol on data sets from different medical labs.
... No article reported the use of ML on kidney histological samples before that year, but it must be emphasized that in 1999 a simple single-layer perceptron network was designed and trained with features extracted from about 100 kidney transplant biopsies to predict the final diagnosis [50]. Initially, the main efforts were made to perform structure recognition through semantic segmentation, that is labeling each pixel in an image to outline object boundaries [51]. Although segmentation alone may be of limited use in routine practice, it lays the groundwork for performing more sophisticated tasks [46]. ...
Introduction
Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments.
Methods
Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included.
Results
Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification.
Conclusion
Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.
Graphical abstract
... Image semantic segmentation [1][2][3] is a typical problem in the field of machine vision, which is based on the principle of labeling each pixel point in an image with a predefined category label, dividing the image into multiple regions, using different colors for different categories to identify them, and preserving the location information of different categories of pixel points to achieve pixel-level segmentation of images. In recent years, semantic segmentation has been widely used in various fields, such as satellite remote sensing [4,5], medical image analysis [6][7][8], autonomous driving [9][10][11], and intelligent agriculture [12,13]. Traditional semantic segmentation methods mainly construct classifiers for semantic segmentation by texture primitive forest [14] (TF), random forest [15] (RF), grayscale segmentation [16], and conditional random field [17] methods, but each classifier is designed for a single category only. ...
Semantic segmentation is a typical problem in the field of machine vision. Convolutional neural networks(CNNs)-based methods all have excellent performance in image semantic segmentation. Existing semantic segmentation models tend to focus only on the improvement of image segmentation performance, with little attention to the problems of network lightweighting and multi-scale feature information utilization. To address this problem, we design a multi-scale feature fusion lightweight semantic segmentation network(MFFLNet), which consists of two parts: a deep feature extraction module(DFEM) and a multi-scale feature extraction module(MFEM). First, the deep feature extraction module(DFEM) utilizes the deconvolution layer to replace the convolution layer, which can avoid the problem of feature information loss caused by cropping the feature map when feature fusion is performed. Meanwhile, the dimensionality of the feature map is compressed by using 1 × 1 convolutional layers after each upsampling layer, which can effectively reduce the number of parameters of the model. Then, the multi-scale feature extraction module(MFEM) employs multiple null convolutions with different expansion rates for feature extraction of the image to extract feature information on multiple scales. Finally, the deep features and multi-scale features extracted by the two modules respectively are fused to achieve semantic segmentation of the image. It is shown experimentally that the proposed MFFLNet outperforms the mainstream methods in semantic segmentation on two datasets, PASCAL VOC 2012 and Cityscapes, with mIoU reaching 71. 23% and 79. 24%, respectively, and improving 5. 8% and 8. 8% compared with the state-of-the-art DeepLab V3 + model, respectively.
... Coudray et al. used convolution neural networks (CNN) on whole-slide images (WSI) to classify them into lung adenocarcinoma (LUAD), lung squamous cell carcinoma (LUSC) or normal lung tissue 10 . Also, CNN have been applied to WSI to classify sclerosed and non-sclerosed glomeruli 11,12 . Kolachalama et al. demonstrated that CNN deep learning models can outperform the pathologist-estimated fibrosis score across the classification tasks and can be applied to routine renal biopsy images 13 . ...
Machine learning applied to digital pathology has been increasingly used to assess kidney function and diagnose the underlying cause of chronic kidney disease (CKD). We developed a novel computational framework, clustering-based spatial analysis (CluSA), that leverages unsupervised learning to learn spatial relationships between local visual patterns in kidney tissue. This framework minimizes the need for time-consuming and impractical expert annotations. 107,471 histopathology images obtained from 172 biopsy cores were used in the clustering and in the deep learning model. To incorporate spatial information over the clustered image patterns on the biopsy sample, we spatially encoded clustered patterns with colors and performed spatial analysis through graph neural network. A random forest classifier with various groups of features were used to predict CKD. For predicting eGFR at the biopsy, we achieved a sensitivity of 0.97, specificity of 0.90, and accuracy of 0.95. AUC was 0.96. For predicting eGFR changes in one-year, we achieved a sensitivity of 0.83, specificity of 0.85, and accuracy of 0.84. AUC was 0.85. This study presents the first spatial analysis based on unsupervised machine learning algorithms. Without expert annotation, CluSA framework can not only accurately classify and predict the degree of kidney function at the biopsy and in one year, but also identify novel predictors of kidney function and renal prognosis.
... Qu et al. took ResNet34 as a network encoder backbone and chose a fixed pre-trained VGG16 model as a loss network for enhancing segmentation details, while taking advantage of migration learning to facilitate the training on a densely distributed lung cancer dataset [28]. Bueno et al. were the first to use a convolutional neural network based semantic segmentation network to detect sclerotic glomeruli in WSIs [29]. Rijthovena et al. used multiple branches of encoder-decoder CNN to obtain contextual and detailed information while using concentric patches with multiple resolutions and different receptive fields to receive accurate resolution semantic segmentation in a WSI [30]. ...
... Using Google's Inception v3 [34] pre-trained model with slight adjustments to tailor it to their goal, Kannan et al. [15] implemented the more traditional segmentation task and achieved an accuracy of 95.06% on their enormous 171 biopsy slide dataset. A combination of both segmentation and classification was proposed by Buenos et al. [2]. Two different methods were put forward to address their task of glomerular segmentation: a 3-class segmentation among Non-Glomerulus, Sclerosed Glomerulus and Normal Glomerulus, and a two-fold segmentation involving semantic segmentation of glomerulus using different configurations of U-Net [29], SegNet [1], followed by classification using AlexNet [18]. ...
Glomerulus segmentation in kidney tissue segments is a key process in nephropathology used for the effective diagnosis of renal diseases. This study proposes a novel and robust application of MLP (Multi-Layer Perceptron) based architectures for the segmentation of glomeruli in PAS (Periodic Acid-Schiff) stained whole renal images for effective diagnosis of renal diseases. For the segmentation challenge, the proposed unique solution uses MLP-UNet (Multi-Layer Perceptron U-Net), a novel design that evades using conventional convolution and self-attention mechanisms. Additionally, the report compares various approaches, including U-Net, and for the first time, trains the TransUNet model on the kidney WSI (Whole Slide Image) dataset. Dice Score and Dice Loss were used as the metric and loss function for training these models. Results showed that MLP-based architectures provide comparable results to pre-trained architectures like TransUNet with effectively lesser parameters and no pre-training, and also produce superior Dice scores across the 5-fold cross-validation training and learn more quickly than conventional U-Net architectures.
... We apply our technique to conduct compound figure separation for renal pathology (inhouse data) as well as on the ImageCLEF 2016 Compound Figure Bueno et al., 2020;Govind et al., 2018;Kannan et al., 2019;Ginley et al., 2019). Due to the lack of a publicly available dataset for renal pathology, it is appealing to extract large-scale glomerular images from public databases (e.g., NIH Open-i ® search engine) for downstream self-supervised or semi-supervised learning (Huo et al., 2021). ...
With the rapid development of self-supervised learning (e.g., contrastive learning), the importance of having large-scale images (even without annotations) for training a more generalizable AI model has been widely recognized in medical image analysis. However, collecting large-scale task-specific unannotated data at scale can be challenging for individual labs. Existing online resources, such as digital books, publications, and search engines, provide a new resource for obtaining large-scale images. However, published images in healthcare (e.g., radiology and pathology) consist of a considerable amount of compound figures with subplots. In order to extract and separate compound figures into usable individual images for downstream learning, we propose a simple compound figure separation (SimCFS) framework without using the traditionally required detection bounding box annotations, with a new loss function and a hard case simulation. Our technical contribution is four-fold: (1) we introduce a simulation-based training framework that minimizes the need for resource extensive bounding box annotations; (2) we propose a new side loss that is optimized for compound figure separation; (3) we propose an intra-class image augmentation method to simulate hard cases; and (4) to the best of our knowledge, this is the first study that evaluates the efficacy of leveraging self-supervised learning with compound image separation. From the results, the proposed SimCFS achieved state-of-the-art performance on the ImageCLEF 2016 Compound Figure Separation Database. The pretrained self-supervised learning model using large-scale mined figures improved the accuracy of downstream image classification tasks with a contrastive learning algorithm. The source code of SimCFS is made publicly available at https://github.com/hrlblab/ImageSeperation.
... Glomerulosclerosis (GS) is a health condition that causes morphological alterations (including sclerosis scarring or hardening) of tiny blood vessels (known as glomeruli) in the kidneys (Risdon and Turner 2012;Bueno et al. 2020). Within the glomeruli, waste and fluid are filtered from the blood and removed from the body through urine. ...
Glomerulosclerosis characterizes many conditions of primary kidney disease in advanced stages. Its accurate diagnosis relies on histological analysis of renal cortex biopsy, and it is paramount to guide the appropriate treatment and minimize the chances of the disease progressing to chronic stages. This article presents an ensemble approach composed of five convolutional neural networks (CNNs) - VGG-19, Inception-V3, ResNet-50, DenseNet-201, and EfficientNet-B2 - to detect glomerulosclerosis in glomerulus images. We fine-tuned the CNNs and evaluated several configurations for the fully connected layers. In total, we analyzed 25 different models. These CNNs, individually, demonstrated effectiveness in the task; however, we verified that the union of these five well-known CNNs improved the detection rate while decreasing the standard deviations of current techniques. The experiments were carried out in a data set comprised of 1,028 images, on which we applied data-augmentation techniques in the training set. The proposed CNNs ensemble achieved a near-perfect accuracy of 99.0% and kappa of 98.0%.
... Glomerulosclerosis (GS) is a health condition that causes morphological alterations, including sclerosis scarring or hardening, of tiny blood vessels in the kidneys called glomeruli [2]. Within the glomeruli, waste and fluid are filtered from the blood and removed from the body through urine. ...
Glomerulosclerosis is a common kidney disease characterized by the deposition of scar tissue, which replaces the renal parenchyma, and is quantified by renal pathologists to indicate the presence and extent of renal damage. It is paramount to guide the appropriate treatment and minimize the chances of the disease progressing to chronic stages. Thus, to identify glomerulus with sclerosis, this article proposes a convolutional neural network (CNN) inspired by convolutional blocks of DenseNet-201 but with smaller dense layers. We analyzed five CNNs - VGG-19, Inception-V3, ResNet-50, DenseNet-201, and EfficientNet-B2 - to define the best CNN model and evaluated several configurations for the fully connected layers. In total, 25 different models were analyzed. The experiments were carried out in three datasets, composed of 1,062 images, on which we applied data-augmentation techniques in the training set. These CNNs demonstrated effectiveness in the task and achieved an accuracy of 92.7% and kappa of 85.3%, considered excellent.KeywordsTransfer learningKidney diseaseComputer-aided diagnosisImage analysis
... Initially, these studies were mainly focused on the glomerulus. The first attempt was to determine the glomerular pathology [23][24][25][26][27][28][29][30][31][32][33], with models to distinguish between 8 sclerotic and nonsclerotic glomeruli reported later [34][35][36][37][38][39][40][41][42]. The application of automated methods to measure the percentage of sclerotic glomeruli can reduce the variability of assessment, increase throughput, and reduce the burden on pathologists. ...
Renal biopsy pathology is an essential gold standard for the diagnosis of most kidney diseases. With the increase in the incidence rate of kidney diseases, the lack of renal pathologists, and an imbalance in their distribution, there is an urgent need for a new renal pathological diagnosis model. Advances in artificial intelligence (AI) along with the growing digitization of pathology slides for diagnosis are promising approach to meet the demand for more accurate detection, classification, and prediction of the outcome of renal pathology. AI has contributed substantially to a variety of clinical applications, including renal pathology. Deep learning, a subfield of AI that is highly flexible and supports automatic feature extraction, is increasingly being used in multiple areas of pathology. In this narrative review, we first provide a general description of AI methods, and then discuss the current and prospective applications of AI in the field of renal pathology. Both diagnostic and predictive prognostic applications are covered, emphasizing AI in renal pathology images, predictive models, and 3D in renal pathology. Finally, we outline the challenges associated with the implementation of AI platforms in renal pathology and provide our perspective on how these platforms might change in this field.
... In renal pathology, the ability to precisely segment glomeruli is critical to investigating several kidney diseases [2,7]. Most prior arts [1,3,4,8] are pixel-based where they perform instance segmentation within a regional proposal on the pixel level. A popular example, Mask R-CNN [5], first detects objects and then segments instances within the proposed boxes using a mask predictor. ...
Circle representation has recently been introduced as a medical imaging optimized representation for more effective instance object detection on ball-shaped medical objects. With its superior performance on instance detection, it is appealing to extend the circle representation to instance medical object segmentation. In this work, we propose CircleSnake, a simple end-to-end circle contour deformation-based segmentation method for ball-shaped medical objects. Compared to the prevalent DeepSnake method, our contribution is three-fold: (1) We replace the complicated bounding box to octagon contour transformation with a computation-free and consistent bounding circle to circle contour adaption for segmenting ball-shaped medical objects; (2) Circle representation has fewer degrees of freedom (DoF=2) as compared with the octagon representation (DoF=8), thus yielding a more robust segmentation performance and better rotation consistency; (3) To the best of our knowledge, the proposed CircleSnake method is the first end-to-end circle representation deep segmentation pipeline method with consistent circle detection, circle contour proposal, and circular convolution. The key innovation is to integrate the circular graph convolution with circle detection into an end-to-end instance segmentation framework, enabled by the proposed simple and consistent circle contour representation. Glomeruli are used to evaluate the performance of the benchmarks. From the results, CircleSnake increases the average precision of glomerular detection from 0.559 to 0.614. The Dice score increased from 0.804 to 0.849. The code has been released: https://github.com/hrlblab/CircleSnake
... Chen et al. performed a large comprehensive study on a big annotated image dataset to evaluate the feasibility of U-Net for segmentation of 6 renal histologic primitives on 4 different stains [17]. Bueno et al. used CNN-based semantic segmentation and classification CNN to realize the detection of glomeruli and the classification of normal and sclerosed glomeruli for LM images, respectively [18]. ...
The chronic kidney disease (CKD) accompanied by permanent kidney damage, has become a heavy burden for worldwide public health. Clinically, glomerular immunofluorescence (IF) images are widely-used to reveal the occurrence probability and type of CKD. In histopathological assessment for glomerular IF image, multiple descriptive indicators are used to characterize deposits from different aspects, which suggest associated kidney lesions. In this paper, we design a hierarchical feature fusion attention network (HFANet) to classify two main descriptive indicators, namely fluorescence intensity and distribution pattern. Through the hierarchical feature fusion attention (HFA) module, HFANet supplements deep semantic features using shallow texture features to maximize its feature extraction capability and efficiency of information fusion. Different from directly adding or concatenating, HFANet weighted concatenates the feature maps from different hierarchies to highlight more discriminative regions. Further, by integrating HFANet with the proposed intensity equalization (IE) algorithm, U-Net++, and Grad-CAM, a computer-aided diagnostic system for glomerular IF images is constructed. With this system, the classification accuracy of the fluorescence intensity and distribution pattern reaches 90.48% and 90.87%, respectively. Extensive comparative experiments and ablation studies demonstrate that HFANet outperforms other universal backbones with the help of HFA module, and the classification performance of the devised system is comparable to senior pathologists. The heatmap given by the system, which is similar to the classification evidence used by the clinicians, can be used as diagnostic reference and training material for pathologists. The systematic demonstration video is available in the supplementary material.
... This is why the development of automated methods to segment and count healthy glomeruli could be very helpful in speeding up renal tissue analysis. In recent years, many deep learning approaches have been proposed for automatic segmentation of human glomeruli [11,12,13,14]. However, no publicly accessible datasets are available that can be used to train deep learning architectures. ...
... Glomerular phenotyping (Koziell et al., 2002) is a fundamental task for efficient diagnosis and quantitative evaluations in renal pathology. Recently, deep learning techniques have played increasingly important roles in renal pathology to reduce clinical working load of pathologists and enable large-scale population based research (Gadermayr Bueno et al., 2020;Govind et al., 2018;Kannan et al., 2019;Ginley et al., 2019). Due to the lack of a publicly available dataset for renal pathology, it is appealing to extract large-scale glomerular images from public databases (e.g., NIH Open-i search engine) for downstream self-supervised or semi-supervised learning (Huo et al., 2021). ...
With the rapid development of self-supervised learning (e.g., contrastive learning), the importance of having large-scale images (even without annotations) for training a more generalizable AI model has been widely recognized in medical image analysis. However, collecting large-scale task-specific unannotated data at scale can be challenging for individual labs. Existing online resources, such as digital books, publications, and search engines, provide a new resource for obtaining large-scale images. However, published images in healthcare (e.g., radiology and pathology) consist of a considerable amount of compound figures with subplots. In order to extract and separate compound figures into usable individual images for downstream learning, we propose a simple compound figure separation (SimCFS) framework without using the traditionally required detection bounding box annotations, with a new loss function and a hard case simulation. Our technical contribution is four-fold: (1) we introduce a simulation-based training framework that minimizes the need for resource extensive bounding box annotations; (2) we propose a new side loss that is optimized for compound figure separation; (3) we propose an intra-class image augmentation method to simulate hard cases; and (4) to the best of our knowledge, this is the first study that evaluates the efficacy of leveraging self-supervised learning with compound image separation. From the results, the proposed SimCFS achieved state-of-the-art performance on the ImageCLEF 2016 Compound Figure Separation Database. The pretrained self-supervised learning model using large-scale mined figures improved the accuracy of downstream image classification tasks with a contrastive learning algorithm. The source code of SimCFS is made publicly available at https://github.com/hrlblab/ImageSeperation.
... Specifically, convolutional neural networks (CNNs), which are the most popular deep learning-based techniques, are mainly used for the automated detection and morphometric analysis of histological components and in the prediction of renal disease prognosis. The applications of CNNs in renal pathology include glomerular counting [4][5][6][7][8], global glomerulosclerosis [9][10][11][12][13][14], podocyte morphometric analysis [14][15][16][17], the classification of diabetic glomerulosclerosis [18], IgA nephropathy [19,20], glomerular hypercellularity [21], several glomerular changes [22], kidney transplant pathology [23][24][25], interstitial fibrosis and tubular atrophy [10,11,14,[26][27][28], vascular detection [28], immunofluorescence staining patterns [29], and the classification of normal and abnormal structures in the renal cortex [4,[30][31][32] (Table 1). However, studies on the development of CNNs that can be successfully applied in the classification of normal and abnormal renal tubules [4,5,11,30], which remains a challenging domain even among renal pathologists, are scarce. ...
Renal pathology is essential for diagnosing and assessing the severity and prognosis of kidney diseases. Deep learning-based approaches have developed rapidly and have been applied in renal pathology. However, methods for the automated classification of normal and abnormal renal tubules remain scarce. Using a deep learning-based method, we aimed to classify normal and abnormal renal tubules, thereby assisting renal pathologists in the evaluation of renal biopsy specimens. Consequently, we developed a U-Net-based segmentation model using randomly selected regions obtained from 21 renal biopsy specimens. Further, we verified its performance in multiclass segmentation by calculating the Dice coefficients (DCs). We used 15 cases of tubulointerstitial nephritis to assess its applicability in aiding routine diagnoses conducted by renal pathologists and calculated the agreement ratio between diagnoses conducted by two renal pathologists and the time taken for evaluation. We also determined whether such diagnoses were improved when the output of segmentation was considered. The glomeruli and interstitium had the highest DCs, whereas the normal and abnormal renal tubules had intermediate DCs. Following the detailed evaluation of the tubulointerstitial compartments, the proximal, distal, atrophied, and degenerated tubules had intermediate DCs, whereas the arteries and inflamed tubules had low DCs. The annotation and output areas involving normal and abnormal tubules were strongly correlated in each class. The pathological concordance for the glomerular count, t, ct, and ci scores of the Banff classification of renal allograft pathology remained high with or without the segmented images. However, in terms of time consumption, the quantitative assessment of tubulitis, tubular atrophy, degenerated tubules, and the interstitium was improved significantly when renal pathologists considered the segmentation output. Deep learning algorithms can assist renal pathologists in the classification of normal and abnormal tubules in renal biopsy specimens, thereby facilitating the enhancement of renal pathology and ensuring appropriate clinical decisions.
... A Convolutional Neural Network (CNN) was employed and trained on a small in-house collected dataset, composed of only 26 kidney biopsies. In [18], instead, a two-step deep learning approach was realized, segmenting and then classifying the segmented structures, using an AlexNetlike architecture, tested on 47 Whole Slide Images (WSIs). Similarly, in [19], some different staining modalities were compared in the framework of a classification model to predict diseased and non-diseased kidney tissues. ...
Deep learning is widely applied in bioinformatics and biomedical imaging, due to its ability to perform various clinical tasks automatically and accurately. In particular, the application of deep learning techniques for the automatic identification of glomeruli in histopathological kidney images can play a fundamental role, offering a valid decision support system tool for the automatic evaluation of the Karpinski metric. This will help clinicians in detecting the presence of sclerotic glomeruli in order to decide whether the kidney is transplantable or not. In this work, we implemented a deep learning framework to identify and segment sclerotic and non-sclerotic glomeruli from scanned Whole Slide Images (WSIs) of human kidney biopsies. The experiments were conducted on a new dataset collected by both the Siena and Trieste hospitals. The images were segmented using the DeepLab V2 model, with a pre-trained ResNet101 encoder, applied to 512 × 512 patches extracted from the original WSIs. The results obtained are promising and show a good performance in the segmentation task and a good generalization capacity, despite the different coloring and typology of the histopathological images. Moreover, we present a novel use of the CD10 staining procedure, which gives promising results when applied to the segmentation of sclerotic glomeruli in kidney tissues.
... Animal model-derived samples were deployed in several studies, most of which dealt with glomerular detection [25][26][27][28][29][30][31][32][33], or unspecified human kidney biopsies with no reference to the pre-implantation setting or specific pathology [34][35][36][37][38]. Only three larger-sized studies applied segmentation CNN models to the detection and simultaneous classification of multiple renal structures, not only glomeruli but also different kinds of tubuli and vessels [39][40][41]. ...
Background
Transplant nephropathology is a highly specialized field of pathology comprising both the evaluation of organ donor biopsy for organ allocation and post-transplant graft biopsy for assessment of rejection or graft damage. The introduction of digital pathology with whole-slide imaging (WSI) in clinical research, trials and practice has catalyzed the application of artificial intelligence (AI) for histopathology, with development of novel machine-learning models for tissue interrogation and discovery. We aimed to review the literature for studies specifically applying AI algorithms to WSI-digitized pre-implantation kidney biopsy.
Methods
A systematic search was carried out in the electronic databases PubMed-MEDLINE and Embase until 25th September, 2021 with a combination of the key terms “kidney”, “biopsy”, “transplantation” and “artificial intelligence” and their aliases. Studies dealing with the application of AI algorithms coupled with WSI in pre-implantation kidney biopsies were included. The main theme addressed was detection and quantification of tissue components. Extracted data were: author, year and country of the study, type of biopsy features investigated, number of cases, type of algorithm deployed, main results of the study in terms of diagnostic outcome, and the main limitations of the study.
Results
Of 5761 retrieved articles, 7 met our inclusion criteria. All studies focused largely on AI-based detection and classification of glomerular structures and to a lesser extent on tubular and vascular structures. Performance of AI algorithms was excellent and promising.
Conclusion
All studies highlighted the importance of expert pathologist annotation to reliably train models and the need to acknowledge clinical nuances of the pre-implantation setting. Close cooperation between computer scientists and practicing as well as expert renal pathologists is needed, helping to refine the performance of AI-based models for routine pre-implantation kidney biopsy clinical practice.
Glomeruli is a collection of blood vessels present in kidneys of the human body. Since kidneys are one of the most important human body organs, diagnosis of any abnormality becomes very important. This work focuses on the binary classification of normal and scleroses glomeruli using convolutional neural networks from whole slide images (WSI). These images are periodic acid-Schiff (PAS) stained and the glomerulus can be seen as circular areas of dark stains on the slide. The main purpose of this work is to make the diagnoses of the kidney glomeruli fast and accurate since manual detection is quite time-consuming and has many human errors as well. In our work, we have performed the classification of microscopic images to detect the scleroses glomerulus from the kidney. This study deployed, four different types of CNN models and subsequently evaluated, they are AlexNet, Visual Geometry Group-19, GoogleNet, and a customized model. The comparative analysis has been made by considering several parameters such as the number of epochs, optimizers, batch size, and learning rate in which the customized model achieved the accuracy of 97.86%. The results of the proposed work are quite promising. The performances of the models used in this work are compared using various metrics such as accuracy, recall, precision, and F1-score, and the results from each of the models are noted.KeywordsConvolutional neural networksGoogleNetAlexNetVisual geometry group-19Whole slide imagesPeriodic acid-SchiffOptimizersPrecisionRecallF1-score
Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.
Computational Pathology (CoPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CoPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology facilitating transformational changes in the diagnosis and treatment of cancer diseases. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CoPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CoPath. In this article we provide a comprehensive review of more than 700 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CoPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CoPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CoPath.
Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studies have typically trained multiple segmentation networks in order to match the optimal pixel resolution of heterogeneous tissue types. This multi-network solution is resource-intensive and fails to model the spatial relationship between tissue types. In this paper, we propose the Omni-Seg network, a scale-aware dynamic neural network that achieves multi-object (six tissue types) and multi-scale (5× to 40× scale) pathological image segmentation via a single neural network. The contribution of this paper is three-fold: (1) a novel scale-aware controller is proposed to generalize the dynamic neural network from single-scale to multi-scale; (2) semi-supervised consistency regularization of pseudo-labels is introduced to model the inter-scale correlation of unannotated tissue types into a single end-to-end learning paradigm; and (3) superior scale-aware generalization is evidenced by directly applying a model trained on human kidney images to mouse kidney images, without retraining. By learning from 150,000 human pathological image patches from six tissue types at three different resolutions, our approach achieved superior segmentation performance according to human visual assessment and evaluation of image-omics (i.e., spatial transcriptomics). The official implementation is available at
https://github.com/ddrrnn123/Omni-Seg
.
For medical diagnostic tests, kidney segmentation from high-volume imagery is an important major. Since 3D medical images need a lot of GPU memory, slices and patches are used for training and inference in traditional neural network variant architectures, which necessarily slows down contextual learning. In this research, Mobile Net and Efficient Net CNN models were trained for segmenting human kidney images generated from The Human Biomolecular Atlas Program (HuBMAP). The purpose of this work is to evaluate the effectiveness of different strategies for Glomeruli identification in order to solve the issue. The high size images were decoded to be fitted and trained in the models first, then the CNN models were trained. The CNN models result show that the Efficient Net has the highest accuracy rate with 99.49 %, and Mobile Net with 99.33 %.
Dans le cadre de cette thèse, nous nous intéressons à des données histopathologiques rénales, et en particulier à la segmentation de glomérules. Ces structures sont complexes et comportent de multiples sous-structures rendant leur segmentation automatique particulièrement difficile. Notre objectif est d'améliorer la segmentation automatique de glomérules dans des coupes complètes en utilisant un CNN de type U-Net. L'entraînement d'un tel modèle nécessite une grande quantité d'images annotées. Or, dans notre contexte, le nombre d'images annotées disponibles est de l'ordre de quelques centaines seulement, ce qui pose la question des augmentations de données. Nous proposons d'étudier l'application et l'impact d'augmentations de deux types. Nous étudions premièrement les variations géométriques, introduites à l'aide de déformations spatiales aléatoires. Deuxièmement, nous étudions les variations de texture, introduites à l'aide de méthodes de synthèse de texture et de modèles génératifs.
Circle representation has recently been introduced as a “medical imaging optimized" representation for more effective instance object detection on ball-shaped medical objects. With its superior performance on instance detection, it is appealing to extend the circle representation to instance medical object segmentation. In this work, we propose CircleSnake, a simple end-to-end circle contour deformation-based segmentation method for ball-shaped medical objects. Compared to the prevalent DeepSnake method, our contribution is threefold: (1) We replace the complicated bounding box to octagon contour transformation with a computation-free and consistent bounding circle to circle contour adaption for segmenting ball-shaped medical objects; (2) Circle representation has fewer degrees of freedom (DoF = 2) as compared with the octagon representation (DoF = 8), thus yielding a more robust segmentation performance and better rotation consistency; (3) To the best of our knowledge, the proposed CircleSnake method is the first end-to-end circle representation deep segmentation pipeline method with consistent circle detection, circle contour proposal, and circular convolution. The key innovation is to integrate the circular graph convolution with circle detection into an end-to-end instance segmentation framework, enabled by the proposed simple and consistent circle contour representation. Glomeruli are used to evaluate the performance of the benchmarks. From the results, CircleSnake increases the average precision of glomerular detection from 0.559 to 0.614. The Dice score increased from 0.804 to 0.849. The code has been released:
.KeywordsInstance segmentationGraph convolutionPathologySnake
INTRODUCTION
Pathology diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable while digital and machine learning technologies open opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity.
MATERIALS AND METHODS
We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. Quality of classifier-produced heatmaps was evaluated by an intersection over union metrics (IoU) between predicted and ground truth localization heatmaps.
RESULTS
A proposed spatially guided modification of CNN classifier achieved the highest glomerular pattern classification accuracies with AUC values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, spatially guided classifier achieved significantly higher generalized mean IoU value, compared with single-multiclass and multiple-binary classifiers.
CONCLUSIONS
We propose a spatially guided CNN classifier which in our experiments reveals the potential to achieve high accuracy for intraglomerular pattern localization.
Background and Objective
Whole slide image (WSI) classification and lesion localization within giga-pixel slide are challenging tasks in computational pathology that requires context-aware representations of histological features to adequately infer nidus. The existing weakly supervised learning methods mainly treat different locations in the slide as independent regions and cannot learn potential nonlinear interactions between instances based on i.i.d assumption, resulting in the model unable to effectively utilize context-ware information to predict the labels of WSIs and locate the region of interest (ROI).
Methods
Here, we propose an interpretable classification model named bidirectional Attention-based Multiple Instance Learning Graph Convolutional Network (ABMIL-GCN), which hierarchically aggregates context-aware features of instances into a global representation in a topology fashion to predict the slide labels and localize the region of lymph node metastasis in WSIs.
Results
We verified the superiority of this method on the Camelyon16 dataset, and the results show that the average predicted ACC and AUC of the proposed model after flooding optimization can reach 90.89% and 0.9149, respectively. The average accuracy and ACC score are improved by more than 7% and 4% compared with the existing state-of-the-art algorithms.
Conclusions
The results demonstrate that context-aware GCN outperforms existing weakly supervised learning methods by introducing spatial correlations between the neighbor image patches, which also addresses the ‘accuracy-interpretability trade-off’ problem. The framework provides a novel paradigm for the clinical application of computer-aided diagnosis and intelligent systems.
With the increasing number of patients with chronic kidney disease (CDK), the workload of pathologists is heavy and the diagnostic efficiency is low. Therefore, the use of computer technology to assist the diagnosis of nephropathy becomes the trend of future development with a great application space. In recent years, UNet network has been widely used in medical image segmentation, and many improved algorithms have appeared. Most of existing methods focus on adding new modules or incorporating other design concepts on the basis of UNet, however, the structure of UNet network has not been fully analyzed. This paper points out two problems existing in UNet network: insufficient feature extraction in encoder interferes with the accuracy of image segmentation, and the fixed mode of skip connection in each layer leads to information redundancy in the network. To solve these problems, we improve the encoder and feature fusion method of UNet and named the new network as Relay-UNet. Experiment results show that under the condition of almost no change in consumption time, the Dice coefficient of Relay UNet on the glomerular dataset reaches 97.5%, and the training process is more stable than that of UNet.
Digital Pathology is an area prone to high variation due to multiple factors which can strongly affect diagnostic quality and visual appearance of the Whole-Slide-Images (WSIs). The state-of-the art methods to deal with such variation tend to address this through style-transfer inspired approaches. Usually, these solutions directly apply successful approaches from the literature, potentially with some task-related modifications. The majority of the obtained results are visually convincing, however, this paper shows that this is not a guarantee that such images can be directly used for either medical diagnosis or reducing domain shift.This article shows that slight modification in a stain transfer architecture, such as a choice of normalisation layer, while resulting in a variety of visually appealing results, surprisingly greatly effects the ability of a stain transfer model to reduce domain shift. By extensive qualitative and quantitative evaluations, we confirm that translations resulting from different stain transfer architectures are distinct from each other and from the real samples. Therefore conclusions made by visual inspection or pretrained model evaluation might be misleading.
The human body is complex but very interesting to discover. The human body has many unsolved riddles which yet haven’t been answered and sometimes even astonishes scientists and doctors when something new is discovered. The whole review consists of the functionality of the kidney; it causes of dysfunction, methods of detection, replacement techniques, and kidney-on-chip. Since every organ in the human body malfunctions due to various complications in its working which leads to various diseases. About, 13% of the world’s population is affected by kidney diseases. Since replacement techniques like dialysis and kidney transplantation are costly and sometimes, not available also due to lack of donors (in case of kidney transplantation). Researchers and scientists are trying to develop the kidney’s function artificially which is termed kidney-on-chip which is a revolutionary idea.KeywordsProximal convoluted tubule (PCT)Distal convoluted tubule (DCT)Acute kidney injury (AKI)Chronic kidney disease (CKD)Blood urea nitrogen (BUN)Glomerular filtration rate (GFR)
We have developed Histo-Cloud, a tool for segmentation of whole slide images (WSIs) that has an easy-to-use graphical user interface. This tool runs a state-of-the-art convolutional neural network (CNN) for segmentation of WSIs in the cloud and allows the extraction of features from segmented regions for further analysis. By segmenting glomeruli, interstitial fibrosis and tubular atrophy, and vascular structures from renal and non-renal WSIs, we demonstrate the scalability, best practices for transfer learning, and effects of dataset variability. Finally, we demonstrate an application for animal model research, analyzing glomerular features in murine models of aging, diabetic nephropathy, and HIV associated nephropathy. The ability to access this tool over the internet will facilitate widespread use by computational non-experts. Histo-Cloud is open source and adaptable for segmentation of any histological structure regardless of stain.
Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.
Background:
Image-based machine learning tools hold great promise for clinical applications in pathology research. However, the ideal end-users of these computational tools (e.g., pathologists and biological scientists) often lack the programming experience required for the setup and use of these tools which often rely on the use of command line interfaces.
Methods:
We have developed Histo-Cloud, a tool for segmentation of whole slide images (WSIs) that has an easy-to-use graphical user interface. This tool runs a state-of-the-art convolutional neural network (CNN) for segmentation of WSIs in the cloud and allows the extraction of features from segmented regions for further analysis.
Results:
By segmenting glomeruli, interstitial fibrosis and tubular atrophy, and vascular structures from renal and non-renal WSIs, we demonstrate the scalability, best practices for transfer learning, and effects of dataset variability. Finally, we demonstrate an application for animal model research, analyzing glomerular features in three murine models.
Conclusions:
Histo-Cloud is open source, accessible over the internet, and adaptable for segmentation of any histological structure regardless of stain.
In kidney transplant biopsies, both inflammation and chronic changes are important features that predict long-term graft survival. Quantitative scoring of these features is important for transplant diagnostics and kidney research. However, visual scoring is poorly reproducible and labor-intensive. The goal of this study was to investigate the potential of convolutional neural networks (CNNs) to quantify inflammation and chronic features in kidney transplant biopsies.
A structure segmentation CNN and a lymphocyte detection CNN were applied on 125 whole-slide image pairs of PAS-, and CD3-stained slides. The CNN results were used to quantify healthy and sclerotic glomeruli, interstitial fibrosis, tubular atrophy, and inflammation both within non-atrophic and atrophic tubuli, and in areas of interstitial fibrosis. The computed tissue features showed high correlations with Banff lesion scores of five pathologists. Analyses on a small subset showed a moderate correlation towards higher CD3⁺ cell density within scarred regions and higher CD3⁺ cell count inside atrophic tubuli correlated with long-term change of estimated glomerular filtration rate.
The presented CNNs are valid tools to yield objective quantitative information on glomeruli number, fibrotic tissue, and inflammation within scarred and non-scarred kidney parenchyma in a reproducible fashion. CNNs have the potential to improve kidney transplant diagnostics and will benefit the community as a novel method to generate surrogate endpoints for large-scale clinical studies.
Purpose: The quantitative detection, segmentation, and characterization of glomeruli from high-resolution whole slide imaging (WSI) play essential roles in the computer-assisted diagnosis and scientific research in digital renal pathology. Historically, such comprehensive quantification requires extensive programming skills to be able to handle heterogeneous and customized computational tools. To bridge the gap of performing glomerular quantification for non-technical users, we develop the Glo-In-One toolkit to achieve holistic glomerular detection, segmentation, and characterization via a single line of command. Additionally, we release a large-scale collection of 30,000 unlabeled glomerular images to further facilitate the algorithmic development of self-supervised deep learning. Approach: The inputs of the Glo-In-One toolkit are WSIs, while the outputs are (1) WSI-level multi-class circle glomerular detection results (which can be directly manipulated with ImageScope), (2) glomerular image patches with segmentation masks, and (3) different lesion types. In the current version, the fine-grained global glomerulosclerosis (GGS) characterization is provided, including assessed-solidified-GSS (associated with hypertension-related injury), disappearing-GSS (a further end result of the SGGS becoming contiguous with fibrotic interstitium), and obsolescent-GSS (nonspecific GGS increasing with aging) glomeruli. To leverage the performance of the Glo-In-One toolkit, we introduce self-supervised deep learning to glomerular quantification via large-scale web image mining. Results: The GGS fine-grained classification model achieved a decent performance compared with baseline supervised methods while only using 10% of the annotated data. The glomerular detection achieved an average precision of 0.627 with circle representations, while the glomerular segmentation achieved a 0.955 patch-wise Dice dimilarity coefficient. Conclusion: We develop and release an open-source Glo-In-One toolkit, a software with holistic glomerular detection, segmentation, and lesion characterization. This toolkit is user-friendly to non-technical users via a single line of command. The toolbox and the 30,000 web mined glomerular images have been made publicly available at https://github.com/hrlblab/Glo-In-One.
Purpose of review:
We seek to determine recent advances in kidney pathophysiology that have been enabled or enhanced by artificial intelligence. We describe some of the challenges in the field as well as future directions.
Recent findings:
We first provide an overview of artificial intelligence terminologies and methodologies. We then describe the use of artificial intelligence in kidney diseases to discover risk factors from clinical data for disease progression, annotate whole slide imaging and decipher multiomics data. We delineate key examples of risk stratification and prognostication in acute kidney injury (AKI) and chronic kidney disease (CKD). We contextualize these applications in kidney disease oncology, one of the subfields to benefit demonstrably from artificial intelligence using all if these approaches. We conclude by elucidating technical challenges and ethical considerations and briefly considering future directions.
Summary:
The integration of clinical data, patient derived data, histology and proteomics and genomics can enhance the work of clinicians in providing more accurate diagnoses and elevating understanding of disease progression. Implementation research needs to be performed to translate these algorithms to the clinical setting.
The quantitative detection, segmentation, and characterization of glomeruli from high-resolution whole slide imaging (WSI) play essential roles in the computer-assisted diagnosis and scientific research in digital renal pathology. Historically, such comprehensive quantification requires extensive programming skills in order to be able to handle heterogeneous and customized computational tools. To bridge the gap of performing glomerular quantification for non-technical users, we develop the Glo-In-One toolkit to achieve holistic glomerular detection, segmentation, and characterization via a single line of command. Additionally, we release a large-scale collection of 30,000 unlabeled glomerular images to further facilitate the algorithmic development of self-supervised deep learning. The inputs of the Glo-In-One toolkit are WSIs, while the outputs are (1) WSI-level multi-class circle glomerular detection results (which can be directly manipulated with ImageScope), (2) glomerular image patches with segmentation masks, and (3) different lesion types. To leverage the performance of the Glo-In-One toolkit, we introduce self-supervised deep learning to glomerular quantification via large-scale web image mining. The GGS fine-grained classification model achieved a decent performance compared with baseline supervised methods while only using 10% of the annotated data. The glomerular detection achieved an average precision of 0.627 with circle representations, while the glomerular segmentation achieved a 0.955 patch-wise Dice Similarity Coefficient (DSC).
Background and objective:
The effective application of deep learning to digital histopathology is hampered by the shortage of high-quality annotated images. In this paper we focus on the supervised segmentation of glomerular structures in patches of whole slide images of renal histopathological slides. Considering a U-Net model employed for segmentation, our goal is to evaluate the impact of augmenting training data with random spatial deformations.
Methods:
The effective application of deep learning to digital histopathology is hampered by the shortage of high-quality annotated images. In this paper we focus on the supervised segmentation of glomerular structures in patches of whole slide images of renal histopathological slides. Considering a U-Net model employed for segmentation, our goal is to evaluate the impact of augmenting training data with random spatial deformations.
Results:
We show that augmenting training data with spatially deformed images yields an improvement of up to 0.23 in average Dice score, with respect to training with no augmentation. We demonstrate that deformations with relatively strong distortions yield the best performance increase, while previous work only report the use of deformations with low distortions. The selected deformation models yield similar performance increase, provided that their parameters are properly adjusted. We provide bounds on the optimal parameter values, obtained through parameter sampling, which is achieved in a lower computational complexity with our single-parameter method. The paper is accompanied by a framework for evaluating the impact of random spatial deformations on the performance of any U-Net segmentation model.
Conclusion:
To our knowledge, this study is the first to evaluate the impact of random spatial deformations on the segmentation of histopathological images. Our study and framework provide tools to help practitioners and researchers to make a better usage of random spatial deformations when training deep models for segmentation.
The largest bottleneck to the development of convolutional neural network (CNN) models in the computational pathology domain is the collection and curation of diverse training datasets. Training CNNs requires large cohorts of image data, and model generalizability is dependent on training data heterogeneity. Including data from multiple centers enhances the generalizability of CNN based models, but this is hindered by the logistical challenges of sharing medical data. In this paper we explore the feasibility of training our recently developed cloud-based segmentation tool (Histo-Cloud) using federated learning. Using a dataset of renal tissue biopsies we show that federated training to segment interstitial fibrosis and tubular atrophy (IFTA) using datasets from three institutions is not found to be different from a training by pooling the data on one server when tested on a fourth (holdout) institution’s data. Further, training a model to segment glomeruli for a federated dataset (split by staining) demonstrates similar performance.
Rationale & Objective
The number of glomeruli is often used to determine the adequacy of a kidney biopsy (eg, at least 10 glomeruli). It is often assumed that biopsy specimens with limited amounts of cortex are too imprecise for detection of focal pathology.
Study Design
Clinical-pathologic correlation (cross-sectional).
Setting & Participants
Living kidney donors who underwent a needle core biopsy of their kidney at the time of donation.
Exposure
The amount of cortex biopsied as determined by either the number of glomeruli or area of cortex on histology.
Outcome
The percentage of globally sclerotic glomeruli, density of interstitial fibrosis foci, and severity of arteriosclerosis were determined.
Analytical approach
A beta-binomial model assessed how the mean percentage of globally sclerotic glomeruli and patient variability in percentage of globally sclerotic glomeruli differed with the number of glomeruli on the biopsy specimen. Additional models assessed the association of interstitial fibrosis and arteriosclerosis with number of glomeruli.
Results
There were 2,915 kidney donors studied. Fewer glomeruli on the biopsy specimen associated with higher mean percentage of globally sclerotic glomeruli and higher patient variability in percentage of globally sclerotic glomeruli. Smaller cortical volume on imaging correlated with both less cortex on biopsy and higher percentage of globally sclerotic glomeruli. Based on a statistical simulation, the probability of patient percentage of globally sclerotic glomeruli ≥ 10% if the biopsy percentage of globally sclerotic glomeruli is ≥10% (positive predictive value) was 45% with 1 to 9 glomeruli versus 31% with 10 or more glomeruli; the negative predictive value was 91% versus 98%. Fewer glomeruli also associated with more interstitial fibrosis and arteriosclerosis.
Limitations
The study was limited to living kidney donors. Patient variability in percentage of globally sclerotic glomeruli was based on a statistical model because multiple biopsy specimens per patient were not available.
Conclusions
The amount of cortex on a needle core biopsy is not completely random. Chronic changes from loss of cortex contribute to low amounts of cortex on a kidney biopsy specimen.
Advancements in computer vision and artificial intelligence (AI) carry the potential to make significant contributions to health care, particularly in diagnostic specialties such as radiology and pathology. The impact of these technologies on physician stakeholders is the subject of significant speculation. There is however a dearth of information regarding the opinions, enthusiasm, and concerns of the pathology community at large. Here, we report results from a survey of 487 pathologist-respondents practicing in 54 countries, conducted to examine perspectives on AI implementation in clinical practice. Despite limitations, including difficulty with quantifying response bias and verifying identity of respondents to this anonymous and voluntary survey, several interesting findings were uncovered. Overall, respondents carried generally positive attitudes towards AI, with nearly 75% reporting interest or excitement in AI as a diagnostic tool to facilitate improvements in workflow efficiency and quality assurance in pathology. Importantly, even within the more optimistic cohort, a significant number of respondents endorsed concerns about AI, including the potential for job displacement and replacement. Overall, around 80% of respondents predicted the introduction of AI technology in the pathology laboratory within the coming decade. Attempts to identify statistically significant demographic characteristics (e.g., age, sex, type/place of practice) predictive of attitudes towards AI using Kolmogorov–Smirnov (KS) testing revealed several associations. Important themes which were commented on by respondents included the need for increasing efforts towards physician training and resolving medical-legal implications prior to the generalized implementation of AI in pathology.
Introduction:
The number of glomeruli and glomerulosclerosis evaluated on kidney biopsy slides constitute standard components of a renal pathology report. Prevailing methods for glomerular assessment remain manual, labor intensive, and nonstandardized. We developed a deep learning framework to accurately identify and segment glomeruli from digitized images of human kidney biopsies.
Methods:
Trichrome-stained images (n = 275) from renal biopsies of 171 patients with chronic kidney disease treated at the Boston Medical Center from 2009 to 2012 were analyzed. A sliding window operation was defined to crop each original image to smaller images. Each cropped image was then evaluated by at least 3 experts into 3 categories: (i) no glomerulus, (ii) normal or partially sclerosed (NPS) glomerulus, and (iii) globally sclerosed (GS) glomerulus. This led to identification of 751 unique images representing nonglomerular regions, 611 images with NPS glomeruli, and 134 images with GS glomeruli. A convolutional neural network (CNN) was trained with cropped images as inputs and corresponding labels as output. Using this model, an image processing routine was developed to scan the test images to segment the GS glomeruli.
Results:
The CNN model was able to accurately discriminate nonglomerular images from NPS and GS images (performance on test data: Accuracy: 92.67% ± 2.02% and Kappa: 0.8681 ± 0.0392). The segmentation model that was based on the CNN multilabel classifier accurately marked the GS glomeruli on the test data (Matthews correlation coefficient = 0.628).
Conclusion:
This work demonstrates the power of deep learning for assessing complex histologic structures from digitized human kidney biopsies.
The pathologists follow a systematic and partially manual process to obtain histological tissue sections from the biological tissue extracted from patients. This process is far from being perfect and can introduce some errors in the quality of the tissue sections (distortions, deformations, folds and tissue breaks). In this paper, we propose a deep learning (DL) method for the detection and segmentation of these damaged regions in whole slide images (WSIs). The proposed technique is based on convolutional neural networks (CNNs) and uses the U-net model to achieve the pixel-wise segmentation of these unwanted regions. The results obtained show that this technique yields satisfactory results and can be applied as a pre-processing step for automatic WSI analysis in order to prevent the use of the damaged areas in the evaluation processes.
Gleason grading of histological images is important in risk assessment and treatment planning for prostate cancer patients. Much research has been done in classifying small homogeneous cancer regions within histological images. However, semi-supervised methods published to date depend on pre-selected regions and cannot be easily extended to an image of heterogeneous tissue composition. In this paper, we propose a multi-scale U-Net model to classify images at the pixel-level using 224 histological image tiles from radical prostatectomies of 20 patients. Our model was evaluated by a patient-based 10-fold cross validation, and achieved a mean Jaccard index of 65.8% across 4 classes (stroma, Gleason 3, Gleason 4 and benign glands), and 75.5% for 3 classes (stroma, benign glands, prostate cancer), outperforming other methods.
Glomerulus classification and detection in kidney tissue segments are key processes in nephropathology used for the correct diagnosis of the diseases. In this paper, we deal with the challenge of automating Glomerulus classification and detection from digitized kidney slide segments using a deep learning framework. The proposed method applies Convolutional Neural Networks (CNNs) between two classes: Glomerulus and Non-Glomerulus, to detect the image segments belonging to Glomerulus regions. We configure the CNN with the public pre-trained AlexNet model and adapt it to our system by learning from Glomerulus and Non-Glomerulus regions extracted from training slides. Once the model is trained, labeling is performed by applying the CNN classification to the image blocks under analysis. The results of the method indicate that this technique is suitable for correct Glomerulus detection in Whole Slide Images (WSI), showing robustness while reducing false positive and false negative detections.
Due to the increasing availability of whole slide scanners facilitating digitization of histopathological tissue, there is a strong demand for the development of computer based image analysis systems. In this work, the focus is on the segmentation of the glomeruli constituting a highly relevant structure in renal histopathology, which has not been investigated before in combination with CNNs. We propose two different CNN cascades for segmentation applications with sparse objects. These approaches are applied to the problem of glomerulus segmentation and compared with conventional fully-convolutional networks. Overall, with the best performing cascade approach, single CNNs are outperformed and a pixel-level Dice similarity coefficient of 0.90 is obtained. Combined with qualitative and further object-level analyses the obtained results are assessed as excellent also compared to recent approaches. In conclusion, we can state that especially one of the proposed cascade networks proved to be a highly powerful tool for segmenting the renal glomeruli providing best segmentation accuracies and also keeping the computing time at a low level.
Glomerulus classification in kidney tissue segments is a key process in nephropathology to obtain correct diseases diagnosis. In this paper, we deal with the challenge to automate the Glomerulus classification from digitized kidney slide segments using a deep learning framework. The proposed method applies Convolutional Neural Networks (CNNs) classification between two classes: Glomerulus and Non-Glomerulus, to detect the image segments belonging to Glomerulus regions. We configure the CNN with the public pre-trained AlexNet model, and adapt it to our system by learning from Glomerulus and Non-Glomerulus regions extracted from training slides. Once the model is trained, the labelling is performed applying the CNN classification to the image segments under analysis. The results obtained indicate that this technique is suitable for correct Glomerulus classification, showing robustness while reducing false positive and false negative detections.
PathoSpotter is a computational system designed to assist pathologists in teaching about and researching kidney diseases. PathoSpotter-K is the version that was developed to detect nephrological lesions in digital images of kidneys. Here, we present the results obtained using the first version of PathoSpotter-K, which uses classical image processing and pattern recognition methods to detect proliferative glomerular lesions with an accuracy of 88.3 ± 3.6%. Such performance is only achieved by similar systems if they use images of cell in contexts that are much less complex than the glomerular structure. The results indicate that the approach can be applied to the development of systems designed to train pathology students and to assist pathologists in determining large-scale clinicopathological correlations in morphological research.
This paper presents the development of several models of a deep convolutional auto-encoder in the Caffe deep learning framework and their experimental evaluation on the example of MNIST dataset. We have created five models of a convolutional auto-encoder which differ architecturally by the presence or absence of pooling and unpooling layers in the auto-encoder's encoder and decoder parts. Our results show that the developed models provide very good results in dimensionality reduction and unsupervised clustering tasks, and small classification errors when we used the learned internal code as an input of a supervised linear classifier and multi-layer perceptron. The best results were provided by a model where the encoder part contains convolutional and pooling layers, followed by an analogous decoder part with deconvolution and unpooling layers without the use of switch variables in the decoder part. The paper also discusses practical details of the creation of a deep convolutional auto-encoder in the very popular Caffe deep learning framework. We believe that our approach and results presented in this paper could help other researchers to build efficient deep neural network architectures in the future.
The clinical specialty of adult nephrology has enjoyed spectacular growth during the past three decades. Such a statement is no less true for pediatric nephrology. This book stands in quiet testimony to that fact. Practitioners of pediatric nephrology are now concerned with the diagnosis and treat ment of young patients with a widened spectrum of primary and sec ondary diseases of the kidneys and urinary tract, hypertension and disorders of water, electrolyte and acid-base metabolism. Their science, deriving from an exciting blend of physiology, morphology, pathology, immunology, biochemistry, microbiology, genetics and pharmacology, must also include an understanding of human developmental biology an insight that colleagues who practice adult nephrology require to a somewhat lesser extent. Dramatic, continuing advances in our understanding of the pathogen esis, pathophysiology, diagnosis and treatment of kidney diseases has led to a cascade of books and monographs on various aspects of the subject. Nevertheless, in view of the clear emergence of pediatric nephrology as a distinct medical specialty, it is most appropriate that a practical book, which focuses almost exclusively on approaches to the evaluation and diagnosis of young patients with kidney disease and related disorders, should appear. In that sense this book is unique. It provides an easily accessible, practical compendium or guide to the clinical investigation of all facets of kidney disease in children. Its special and unique emphasis is directed toward various approaches to the evaluation of such patients and the interpretation of associated laboratory or radiographical data.
This study develops a new computational method for high-throughput detection of glomeruli by introducing the Histogram of Oriented Gradients (HOG), an object detection algorithm widely used in computer vision. While methods operating through segmentation exist, they have assumed glomerulus presence in images and are not designed for huge microscopic images, making comprehensive detection of numerous glomeruli per image difficult. Experiments performed under various conditions indicate favorable outcomes from linear support vector machine combined with HOG features in terms of accuracy and speed. The method was applied to the whole microscopic kidney images of Spontaneously Diabetic Torii (SDT) rats, generally used as animal model of diabetic nephropathy, and their controls, Sprague-Dawley (SD) rats. It was revealed that the immunostaining level of an injury marker, desmin, in glomeruli was significantly higher in SDT rats compared to corresponding SD rats. Furthermore, desmin immunostaining levels in glomeruli were highly variable even within SDT rats of the same age, and highly immunostained glomeruli were likely to appear near the juxtamedullary region than in superficial regions. These suggest that the proposed method is a promising approach in quantifying injury level while avoiding possible local biases and for unraveling new insights into glomerulus injury in the SDT rats.
We present a novel and practical deep fully convolutional neural network
architecture for semantic pixel-wise segmentation termed SegNet. This core
trainable segmentation engine consists of an encoder network, a corresponding
decoder network followed by a pixel-wise classification layer. The architecture
of the encoder network is topologically identical to the 13 convolutional
layers in the VGG16 network . The role of the decoder network is to map the low
resolution encoder feature maps to full input resolution feature maps for
pixel-wise classification. The novelty of SegNet lies is in the manner in which
the decoder upsamples its lower resolution input feature map(s). Specifically,
the decoder uses pooling indices computed in the max-pooling step of the
corresponding encoder to perform non-linear upsampling. This eliminates the
need for learning to upsample. The upsampled maps are sparse and are then
convolved with trainable filters to produce dense feature maps. We compare our
proposed architecture with the fully convolutional network (FCN) architecture
and its variants. This comparison reveals the memory versus accuracy trade-off
involved in achieving good segmentation performance. The design of SegNet was
primarily motivated by road scene understanding applications. Hence, it is
efficient both in terms of memory and computational time during inference. It
is also significantly smaller in the number of trainable parameters than
competing architectures and can be trained end-to-end using stochastic gradient
descent. We also benchmark the performance of SegNet on Pascal VOC12 salient
object segmentation and the recent SUN RGB-D indoor scene understanding
challenge. We show that SegNet provides competitive performance although it is
significantly smaller than other architectures. We also provide a Caffe
implementation of SegNet and a webdemo at
http://mi.eng.cam.ac.uk/projects/segnet/
Background
The detection of the glomeruli is a key step in the histopathological evaluation of microscopic images of the kidneys. However, the task of automatic detection of the glomeruli poses challenges owing to the differences in their sizes and shapes in renal sections as well as the extensive variations in their intensities due to heterogeneity in immunohistochemistry staining.
Although the rectangular histogram of oriented gradients (Rectangular HOG) is a widely recognized powerful descriptor for general object detection, it shows many false positives owing to the aforementioned difficulties in the context of glomeruli detection.
Results
A new descriptor referred to as Segmental HOG was developed to perform a comprehensive detection of hundreds of glomeruli in images of whole kidney sections. The new descriptor possesses flexible blocks that can be adaptively fitted to input images in order to acquire robustness for the detection of the glomeruli. Moreover, the novel segmentation technique employed herewith generates high-quality segmentation outputs, and the algorithm is assured to converge to an optimal solution. Consequently, experiments using real-world image data revealed that Segmental HOG achieved significant improvements in detection performance compared to Rectangular HOG.
Conclusion
The proposed descriptor for glomeruli detection presents promising results, and it is expected to be useful in pathological evaluation.
There is large consent that successful training of deep networks requires
many thousand annotated training samples. In this paper, we present a network
and training strategy that relies on the strong use of data augmentation to use
the available annotated samples more efficiently. The architecture consists of
a contracting path to capture context and a symmetric expanding path that
enables precise localization. We show that such a network can be trained
end-to-end from very few images and outperforms the prior best method (a
sliding-window convolutional network) on the ISBI challenge for segmentation of
neuronal structures in electron microscopic stacks. Using the same network
trained on transmitted light microscopy images (phase contrast and DIC) we won
the ISBI cell tracking challenge 2015 in these categories by a large margin.
Moreover, the network is fast. Segmentation of a 512x512 image takes less than
a second on a recent GPU. The full implementation (based on Caffe) and the
trained networks are available at
http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .
The ImageNet Large Scale Visual Recognition Challenge is a benchmark in
object category classification and detection on hundreds of object categories
and millions of images. The challenge has been run annually from 2010 to
present, attracting participation from more than fifty institutions.
This paper describes the creation of this benchmark dataset and the advances
in object recognition that have been possible as a result. We discuss the
challenges of collecting large-scale ground truth annotation, highlight key
breakthroughs in categorical object recognition, provide detailed a analysis of
the current state of the field of large-scale image classification and object
detection, and compare the state-of-the-art computer vision accuracy with human
accuracy. We conclude with lessons learned in the five years of the challenge,
and propose future directions and improvements.
Histopathology diagnosis is based on visual examination of the morphology of histological sections under a microscope. With the increasing popularity of digital slide scanners, decision support systems based on the analysis of digital pathology images are in high demand. However, computerized decision support systems are fraught with problems that stem from color variations in tissue appearance due to variation in tissue preparation, variation in stain reactivity from different manufacturers/batches, user or protocol variation, and the use of scanners from different manufacturers. In this paper, we present a novel approach to stain normalization in histopathology images. The method is based on nonlinear mapping of a source image to a target image using a representation derived from color deconvolution. Color deconvolution is a method to obtain stain concentration values when the stain matrix, describing how the color is affected by the stain concentration, is given. Rather than relying on standard stain matrices, which may be inappropriate for a given image, we propose the use of a color-based classifier that incorporates a novel stain color descriptor to calculate image-specific stain matrix. In order to demonstrate the efficacy of the proposed stain matrix estimation and stain normalization methods, they are applied to the problem of tumor segmentation in breast histopathology images. The experimental results suggest that the paradigm of color normalization, as a preprocessing step, can significantly help histological image analysis algorithms to demonstrate stable performance which is insensitive to imaging conditions in general and scanner variations in particular.
Often this means removing a dominant and undesirable color cast, such as the yellow in photos taken under incandescent illumination. This article describes a method for a more general form of color correction that borrows one image's color characteristics from anoth-er. Figure 1 shows an example of this process, where we applied the colors of a sunset photograph to a daytime computer graphics rendering. We can imagine many methods for applying the colors of one image to another. Our goal is to do so with a simple algorithm, and our core strategy is to choose a suitable color space and then to apply simple oper-ations there. When a typical three channel image is represented in any of the most well-known color spaces, there will be correlations between the different channels'val-ues. For example, in RGB space, most pixels will have large values for the red and green channel if the blue channel is large. This implies that if we want to change the appearance of a pixel's color in a coherent way, we must modify all color channels in tandem. This complicates any color modification process. What we want is an orthogonal color space without correlations between the axes. Recently, Ruderman et al. developed a color space, called lab, which minimizes correlation between chan-nels for many natural scenes., This space is based on data-driven human perception research that assumes the human visual system is ideally suited for processing natural scenes. The authors discovered lab color space in the context of understanding the human visual sys-tem, and to our knowledge, lab space has never been applied otherwise or compared to other color spaces. There's little correlation between the axes in lab space,
Transplantable kidneys are in very limited supply. Accurate viability assessment prior to transplantation could minimize organ discard. Rapid and accurate evaluation of intra-operative donor kidney biopsies is essential for determining which kidneys are eligible for transplantation. The criteria for accepting or rejecting donor kidneys relies heavily on pathologist determination of the percent of glomeruli (determined from a frozen section) that are normal and sclerotic. This percentage is a critical measurement that correlates with transplant outcome. Inter- and intra-observer variability in donor biopsy evaluation is, however, significant. An automated method for determination of percent global glomerulosclerosis could prove useful in decreasing evaluation variability, increasing throughput, and easing the burden on pathologists. Here, we describe the development of a deep learning model that identifies and classifies non-sclerosed and sclerosed glomeruli in whole-slide images of donor kidney frozen section biopsies. This model extends a convolutional neural network (CNN) pre-trained on a large database of digital images. The extended model, when trained on just 48 whole slide images, exhibits slide-level evaluation performance on par with expert renal pathologists. The model substantially outperforms a model trained on image patches of isolated glomeruli. Encouragingly, the model’s performance is robust to slide preparation artifacts associated with frozen section preparation. As the first model reported that identifies and classifies normal and sclerotic glomeruli in frozen kidney sections, and thus the first model reported in the literature relevant to kidney transplantation, it may become an essential part of donor kidney biopsy evaluation in the clinical setting.
Transplantable kidneys are in very limited supply. Accurate viability assessment prior to transplantation could minimize organ discard. Rapid and accurate evaluation of intraoperative donor kidney biopsies is essential for determining which kidneys are eligible for transplantation. The criteria for accepting or rejecting donor kidneys relies heavily on pathologist determination of the percent of glomeruli (determined from a frozen section) that are normal and sclerotic. This percentage is a critical measurement that correlates with transplant outcome. Inter- and intra-observer variability in donor biopsy evaluation is, however, significant. An automated method for determination of percent global glomerulosclerosis could prove useful in decreasing evaluation variability, increasing throughput, and easing the burden on pathologists. Here, we describe the development of a deep learning model that identifies and classifies non-sclerosed and sclerosed glomeruli in whole-slide images of donor kidney frozen section biopsies. This model extends a convolutional neural network (CNN) pre-trained on a large database of digital images. The extended model, when trained on just 48 whole slide images, exhibits slide-level evaluation performance on par with expert renal pathologists. Encouragingly, the model’s performance is robust to slide preparation artifacts associated with frozen section preparation. The model substantially outperforms a model trained on image patches of isolated glomeruli, in terms of both accuracy and speed. The methodology overcomes the technical challenge of applying a pretrained CNN bottleneck model to whole-slide image classification. The traditional patch-based approach, while exhibiting deceptively good performance classifying isolated patches, does not translate successfully to whole-slide image segmentation in this setting. As the first model reported that identifies and classifies normal and sclerotic glomeruli in frozen kidney sections, and thus the first model reported in the literature relevant to kidney transplantation, it may become an essential part of donor kidney biopsy evaluation in the clinical setting.
Background Histologic examination of fixed renal tissue is widely used to assess morphology and the progression of disease. Commonly reported metrics include glomerular number and injury. However, characterization of renal histology is a time-consuming and user-dependent process. To accelerate and improve the process, we have developed a glomerular localization pipeline for trichrome-stained kidney sections using a machine learning image classification algorithm.
Methods We prepared 4- μ m slices of kidneys from rats of various genetic backgrounds that were subjected to different experimental protocols and mounted the slices on glass slides. All sections used in this analysis were trichrome stained and imaged in bright field at a minimum resolution of 0.92 μ m per pixel. The training and test datasets for the algorithm comprised 74 and 13 whole renal sections, respectively, totaling over 28,000 glomeruli manually localized. Additionally, because this localizer will be ultimately used for automated assessment of glomerular injury, we assessed bias of the localizer for preferentially identifying healthy or damaged glomeruli.
Results Localizer performance achieved an average precision and recall of 96.94% and 96.79%, respectively, on whole kidney sections without evidence of bias for or against glomerular injury or the need for manual preprocessing.
Conclusions This study presents a novel and robust application of convolutional neural nets for the localization of glomeruli in healthy and damaged trichrome-stained whole-renal section mounts and lays the groundwork for automated glomerular injury scoring.
Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we formulate the semantic segmentation problem and define the terminology of this field as well as interesting background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and goals. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. We also devote a part of the paper to review common loss functions and error metrics for this problem. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif- ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implemen- tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called dropout that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry
Immunohistochemical (IHC) biomarkers in breast tissue microarray (TMA) samples are used daily in pathology departments. In recent years, automatic methods to evaluate positive staining have been investigated since they may save time and reduce errors in the diagnosis. These errors are mostly due to subjective evaluation. The aim of this work is to develop a density tool able to automatically quantify the positive brown IHC stain in breast TMA for different biomarkers. To avoid the problem of colour variation and make a robust tool independent of the staining process, several colour standardization methods have been analysed. Four colour standardization methods have been compared against colour model segmentation. The standardization methods have been compared by means of NBS colour distance. The use of colour standardization helps to reduce noise due to stain and histological sample preparation. However, the most reliable and robust results have been obtained by combining the HSV and RGB colour models for segmentation with the HSB channels. The segmentation provides three outputs based on three saturation values for weak, medium and strong staining. Each output image can be combined according to the type of biomarker staining. The results with 12 biomarkers were evaluated and compared to the segmentation and density calculation done by expert pathologists. The Hausdorff distance, sensitivity and specificity have been used to quantitative validate the results. The tests carried out with 8000 TMA images provided an average of 95.94% accuracy applied to the total tissue cylinder area. Colour standardization was used only when the tissue core had blurring and fading stain and the expert could not evaluate them without a pre-processing.
To make reliable diagnosis, pathologists often need to identify certain special regions in medical images. In inflammatory bowel disease (IBD) diagnosis via histology tissue image examination, muscle regions are known to have no immune cell infiltration, and thus are ignored by pathologists. Also, messy regions (e.g., due to distortion and poor staining) are low in diagnostic yield. Hence, excluding muscle and messy regions to focus on vital regions is crucial for accurate diagnosis of IBD. In this paper, we propose a novel deep neural network based on fully convolutional networks (FCN) to identify muscle and messy regions, in an end-to-end fashion. First, we address the challenge of having limited medical training data, for training our deep neural network (a common problem for medical image processing, which may impede the application of the powerful deep learning method). Second, to deal with target regions of largely different sizes and arbitrary shapes, our deep neural network explores multi-scale information and structural information. Experimental results on clinical images show that our approach outperforms the state-of-the-art FCN for semantic segmentation of muscle and messy regions. Our approach may be readily extended to identify other types of regions in a variety of medical imaging applications.
Although transient vestigial excretory organs (the pro- and mesonephros) are recognizable in the human embryo, the definitive kidney is the meta- nephros. This is formed in two parts: the nephrons from the nephrogenic cord and the excretory ducts (collecting tubules, calyces, pelvis and ureter) from the ureteric bud which grows as a branch from the caudal portion of the mesonephric (Wolffian) duct. During early development the ureteric bud grows cranially and impinges on the caudal end of the nephrogenic cord (called the metanephric blastema) where it begins a process of rapid dichotomous branching. The first few generations of branches coalesce to form the renal pelvis and calyces. As further branching occurs, condensations of meta-nephric blastema, from which the nephrons develop, become related to the dilated tip, or ampulla, of each branch of the ureteric bud (Figure 2.1). As the nephrons form they become attached to the ampullae which in turn develop into collecting ducts. Attachment to the growing tip of the ureteric bud branches ensures that the nephrons are carried outwards as they develop.
Histogram Matching (HM) is a common technique for finding a monotonic map between two histograms. However, HM cannot deal with cases where a single mapping is sought between two sets of histograms. This paper presents a novel technique that finds such a mapping in an optimal manner under various histograms distance measures.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make train-ing faster, we used non-saturating neurons and a very efficient GPU implemen-tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
- T Kato
- R Relator
- H Ngouv
- Y Hirohashi
- O Takaki
- T Kakimoto
- K Okada
T. Kato, R. Relator, H. Ngouv, Y. Hirohashi, O. Takaki, T. Kakimoto, K. Okada,
Segmental hog: new descriptor for glomerulus detection in kidney microscopy
image, BMC Bioinform. 16 (316) (2015) 1-16.