No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Glomerulosclerosis Identification in Whole Slide Images using Semantic Segmentation
Abstract
Background and objective:
Glomeruli identification, i.e., detection and characterization, is a key procedure in many nephropathology studies. In this paper, semantic segmentation based on convolutional neural networks (CNN) is proposed to detect glomeruli using Whole Slide Imaging (WSI) follows by a classification CNN to divide the glomeruli into normal and sclerosed.
Methods:
Comparison between U-Net and SegNet CNNs is performed for pixel-level segmentation considering both a two and three class problem, that is, a) non-glomerular and glomerular structures and b) non-glomerular normal glomerular and sclerotic structures. The two class semantic segmentation result is then used for a CNN classification where glomerular regions are divided into normal and global sclerosed glomeruli.
Results:
These methods were tested on a dataset composed of 47 WSIs belonging to human kidney sections stained with Periodic Acid Schiff (PAS). The best approach was the SegNet for two class segmentation follows by a fine-tuned AlexNet network to characterize the glomeruli. 98.16% of accuracy was obtained with this process of consecutive CNNs (SegNet-AlexNet) for segmentation and classification.
Conclusion:
The results obtained demonstrate that the sequential CNN segmentation-classification strategy achieves higher accuracy reducing misclassified cases and therefore being the methodology proposed for glomerulosclerosis detection.
... Glomeruli are tiny tufts of blood vessels found in the kidneys which are responsible for filtering and expelling out waste from human blood [1]. However, various diseases such as diabetes, sickle cell disease, and obesity cause glomeruli to sclerose or scar. ...
... Previous studies have utilized CNNs to diagnose whole slide images. Bueno et al. has been able to obtain an accuracy of 98.16% when classifying WSIs of glomeruli using CNNs, increasing accuracy [1]. Furthermore, ensemble learning (EL), a method that aggregates the results of two or more base CNNs [7], may be used to increase accuracy. ...
... Several studies that tries to address the aforementioned problem exist in literature. For example, Bueno et al performed considered two semantic segmentation approaches to classify WSIs of glomeruli to detect glomerulosclerosis [1]. In their first approach, they considered a three-class classification problem, where images are classified into "no glomerulus", "has normal glomerulus", or "has sclerosed glomerulus." ...
... Automated glomeruli identification frameworks for kidney biopsies conducted by pathologists can be quite helpful because manual examination of kidney samples is time-consuming and error-prone [9], [8]. There are several segmentation methods to detect glomeruli in kidney images [10]. However, these methods require pixel-level annotation of the images. ...
... It is possible to group glomerular disorders according to their clinical symptoms, etymology, immunopathology, or morphological changes [9], [8]. A condition known as "glomerulosclerosis" is the result of the kidney lesion changing its morphology; this sclerosis can impact the kidney in many ways, depending on whether or not it is global or partial [10]. The number of glomeruli detected in each kidney biopsy should be counted in daily practice. ...
... The number of glomeruli detected in each kidney biopsy should be counted in daily practice. Per kidney biopsy, about 20 to 30 cuts are made [10]. Additionally, glomeruli that are completely sclerosed must be noted (the entire glomerulus). ...
Context: Analyzing digital pathology images is necessary to draw diagnostic conclusions by investigating tissue patterns and cellular morphology. However, manual evaluation can be time-consuming, expensive, and prone to inter- and intra-observer variability. Objective: To assist pathologists using computerized solutions, automated tissue structure detection and segmentation must be proposed. Furthermore, generating pixel-level object annotations for histopathology images is expensive and time-consuming. As a result, detection models with bounding box labels may be a feasible solution. Design: This paper studies. YOLO-v4 (You-Only-Look-Once), a real-time object detector for microscopic images. YOLO uses a single neural network to predict several bounding boxes and class probabilities for objects of interest. YOLO can enhance detection performance by training on whole slide images. YOLO-v4 has been used in this paper. for glomeruli detection in human kidney images. Multiple experiments have been designed and conducted based on different training data of two public datasets and a private dataset from the University of Michigan for fine-tuning the model. The model was tested on the private dataset from the University of Michigan, serving as an external validation of two different stains, namely hematoxylin and eosin (H&E) and periodic acid-Schiff (PAS). Results: Average specificity and sensitivity for all experiments, and comparison of existing segmentation methods on the same datasets are discussed. Conclusions: Automated glomeruli detection in human kidney images is possible using modern AI models. The design and validation for different stains still depends on variability of public multi-stain datasets.
... Fifteen H&E kidneys WSIs were used from the AIDPATH [40] kidney dataset. On each of these WSIs, we annotated dilated tubules but also "false dilated tubules" corresponding to white areas in the tissue that are not the lumens of dilated tubules (see Figure 10). ...
... Its advantage lies in its ability to consider both local and global information in images using deep learning features that facilitate the contour's evolution. We also provide annotations of dilated tubules from the AIDPATH dataset [40]. From the results obtained, our algorithm demonstrates that it is possible to exclusively extract raw features using a VGG16 trained on ImageNet in order to segment histology images. ...
This paper introduces a novel approach that combines unsupervised active contour models with deep learning for robust and adaptive image segmentation. Indeed, traditional active contours, provide a flexible framework for contour evolution and learning offers the capacity to learn intricate features and patterns directly from raw data. Our proposed methodology leverages the strengths of both paradigms, presenting a framework for both unsupervised and one-shot approaches for image segmentation. It is capable of capturing complex object boundaries without the need for extensive labeled training data. This is particularly required in histology, a field facing a significant shortage of annotations due to the challenging and time-consuming nature of the annotation process. We illustrate and compare our results to state of the art methods on a histology dataset and show significant improvements.
... To obtain great accuracy in diagnosis and quantitative analysis in renal pathology, the task of kidney glomeruli segmentation has received many considerations. Recently, deep learning techniques have become essential in this field, helping to enable studies on large-scale population-based [4,15,17,18,21,30,31,28], as well as alleviating the clinical workload for pathologists. Numerous traditional, featurebased image processing methods have also been developed for glomeruli segmentation. ...
Segmentation has long been essential in computer vision due to its numerous real-world applications. However, most traditional deep learning and machine learning models need help to capture geometric features such as size and convexity of the segmentation targets, resulting in suboptimal outcomes. To resolve this problem, we propose using a CovHuSeg algorithm to solve the problem of kidney glomeruli segmentation. This simple post-processing method is specified to adapt to the segmentation of ball-shaped anomalies, including the glomerulus. Unlike other post-processing methods, the CovHuSeg algorithm assures that the outcome mask does not have holes in it or comes in unusual shapes that are impossible to be the shape of a glomerulus. We illustrate the effectiveness of our method by experimenting with multiple deep-learning models in the context of segmentation on kidney pathology images. The results show that all models have increased accuracy when using the CovHuSeg algorithm.
... Following the tremendous success of deep learning in the field of computer vision, numerous computer-aided diagnostics systems for glomerulus detection on whole-slide images (WSI) have been proposed [2,3,4,5,6]. Despite the success of the above studies, their results are either reported on the well-known HuBMAP dataset [7] or on private datasets. ...
Whole-slide images (WSI) glomerulus segmentation is essential for accurately diagnosing kidney diseases. In this work, we propose a practical pipeline for glomerulus segmentation that effectively enhances both patch-level and WSI-level segmentation tasks. Our approach leverages stitching on overlapping patches, increasing the detection coverage, especially when glomeruli are located near patch image borders. In addition, we conduct comprehensive evaluations from different segmentation models across two large and diverse datasets with over 30K glomerulus annotations. Experimental results demonstrate that models using our pipeline outperform the previous state-of-the-art method, achieving superior results across both datasets and setting a new benchmark for glomerulus segmentation in WSIs. The code and pre-trained models are available at https://github.com/huuquan1994/wsi_glomerulus_seg.
... For example, Bel and Hermsen et al. developed and compared two fully convolutional networks, assessing the performance of UNet for classification tasks [12,14,15]. Meanwhile, Salvi and Bueno et al. employed semantic segmentation techniques to identify glomeruli and analyze their morphological changes [6,16]. Other research has utilized object detection methods to recognize glomeruli within WSI [17,18], and additional studies have classified pathological changes in glomeruli [19,20]. ...
Recent advancements in computer vision within the field of artificial intelligence (AI) have made significant inroads into the medical domain. However, the application of AI for classifying renal pathology remains challenging due to the subtle variations in multiple renal pathological classifications. Vision Transformers (ViT), an adaptation of the Transformer model for image recognition, have demonstrated superior capabilities in capturing global features and providing greater explainability. In our study, we developed a ViT model using a diverse set of stained renal histopathology images to evaluate its effectiveness in classifying renal pathology. A total of 1861 whole slide images (WSI) stained with HE, MASSON, PAS, and PASM were collected from 635 patients. Renal tissue images were then extracted, tiled, and categorized into 14 classes on the basis of renal pathology. We employed the classic ViT model from the Timm library, utilizing images sized 384 × 384 pixels with 16 × 16 pixel patches, to train the classification model. A comparative analysis was conducted to evaluate the performance of the ViT model against traditional convolutional neural network (CNN) models. The results indicated that the ViT model demonstrated superior recognition ability (accuracy: 0.96–0.99). Furthermore, we visualized the identification process of the ViT models to investigate potentially significant pathological ultrastructures. Our study demonstrated that ViT models outperformed CNN models in accurately classifying renal pathology. Additionally, ViT models are able to focus on specific, significant structures within renal histopathology, which could be crucial for identifying novel and meaningful pathological features in the diagnosis and treatment of renal disease.
... The current study utilized glomerular image patches from seven sources: Besusparis et al. [5] (Besusparis2023, n=3993), Bueno et al. [35,36] (Bueno2020, n=946), Gallego et al. [37] (two datasets: Gallego2021-HE/Gallego-PAS, n=611/527) [38], the Kidney Precision Medicine Project (KPMP, n=5978) [34], the Human BioMolecular Atlas Program (HuBMAP, n=4130) [39], the Norwegian Renal Registry (three datasets: NRR-PAS/NRR-HE/NRR-SIL, n=250/555/568), and Weis et al. [8] (Weis2022, n=5210). The sources displayed different properties, e.g. with respect to image formats, histological stains, availability of glomerular segmentations and/or class labels, which diagnoses and morphological lesions (beyond GS) are present in the dataset, and the proportion of GS and nonGS images, which are further described in the supplemental material and in supplementary tables 1 and 2. ...
Current deep learning models for classifying glomeruli in nephropathology are trained almost exclusively in a supervised manner, requiring expert-labeled images. Very little is known about the potential for unsupervised learning to overcome this bottleneck. To address this open question in a proof-of-concept, the project focused on the most fundamental classification task: globally sclerosed versus non-globally sclerosed glomeruli. The performance of clustering between the two classes was extensively studied across a variety of labeled datasets with diverse compositions and histological stains, and across the feature embeddings produced by 34 different pre-trained CNN models. As demonstrated by the study, clustering of globally and non-globally sclerosed glomeruli is generally highly feasible, yielding accuracies of over 95% in most datasets. Further work will be required to expand these experiments towards the clustering of additional glomerular lesion categories. We are convinced that these efforts (i) will open up opportunities for semi-automatic labeling approaches, thus alleviating the need for labor-intensive manual labeling, and (ii) illustrate that glomerular classification models can potentially be trained even in the absence of expert-derived class labels.
... Recent advances in deep learning with CNN and transformers [15,18] achieve significant improvement in the field of pathology segmentation. Several works were proposed to address the challenges of microscopic imaging data, including H&E stained pathology images, fluoresce data, or other cell imaging modalities [3,22]. Numerous datasets for cell and tissue segmentation, including MoNuSeg [25] and NEPTUNE [2], are available for identifying a variety of glomerular structures. ...
In digital pathology, the traditional method for deep learning-based image segmentation typically involves a two-stage process: initially segmenting high-resolution whole slide images (WSI) into smaller patches (e.g., 256x256, 512x512, 1024x1024) and subsequently reconstructing them to their original scale. This method often struggles to capture the complex details and vast scope of WSIs. In this paper, we propose the holistic histopathology (HoloHisto) segmentation method to achieve end-to-end segmentation on gigapixel WSIs, whose maximum resolution is above 80,00070,000 pixels. HoloHisto fundamentally shifts the paradigm of WSI segmentation to an end-to-end learning fashion with 1) a large (4K) resolution base patch for elevated visual information inclusion and efficient processing, and 2) a novel sequential tokenization mechanism to properly model the contextual relationships and efficiently model the rich information from the 4K input. To our best knowledge, HoloHisto presents the first holistic approach for gigapixel resolution WSI segmentation, supporting direct I/O of complete WSI and their corresponding gigapixel masks. Under the HoloHisto platform, we unveil a random 4K sampler that transcends ultra-high resolution, delivering 31 and 10 times more pixels than standard 2D and 3D patches, respectively, for advancing computational capabilities. To facilitate efficient 4K resolution dense prediction, we leverage sequential tokenization, utilizing a pre-trained image tokenizer to group image features into a discrete token grid. To assess the performance, our team curated a new kidney pathology image segmentation (KPIs) dataset with WSI-level glomeruli segmentation from whole mouse kidneys. From the results, HoloHisto-4K delivers remarkable performance gains over previous state-of-the-art models.
... 59 Lee et al. employed an unsupervised bag-ofwords model on histopathology images from kidney biopsies, revealing morphological features predicting CKD existence and outcomes with a 0.93 AUC for GFR and loss of function after 1 year. 60 Bueno et al. 61 demonstrated a sequential CNNs segmentationclassification strategy with 98% accuracy in detecting and classifying normal and sclerotic glomeruli. Predicting postoperative acute kidney injury risk in renal cell carcinoma patients, ML models, including SVM, RF, extreme gradient boosting, and light GBM (gightGBM), outperformed LR. 62 Kim et al. 63 delved into predicting dialysis adequacy in chronic hemodialysis patients using ML algorithms, specifically RF, and XGBoost, alongside deep learning models such as CNN and gated recurrent unit. ...
Chronic diseases such as heart disease, cancer, and diabetes are leading drivers of mortality worldwide, underscoring the need for improved efforts around early detection and prediction. The pathophysiology and management of chronic diseases have benefitted from emerging fields in molecular biology like genomics, transcriptomics, proteomics, glycomics, and lipidomics. The complex biomarker and mechanistic data from these “omics” studies present analytical and interpretive challenges, especially for traditional statistical methods. Machine learning (ML) techniques offer considerable promise in unlocking new pathways for data‐driven chronic disease risk assessment and prognosis. This review provides a comprehensive overview of state‐of‐the‐art applications of ML algorithms for chronic disease detection and prediction across datasets, including medical imaging, genomics, wearables, and electronic health records. Specifically, we review and synthesize key studies leveraging major ML approaches ranging from traditional techniques such as logistic regression and random forests to modern deep learning neural network architectures. We consolidate existing literature to date around ML for chronic disease prediction to synthesize major trends and trajectories that may inform both future research and clinical translation efforts in this growing field. While highlighting the critical innovations and successes emerging in this space, we identify the key challenges and limitations that remain to be addressed. Finally, we discuss pathways forward toward scalable, equitable, and clinically implementable ML solutions for transforming chronic disease screening and prevention.
... With the advent of deep learning, significant advancements have been made in the field of medical image segmentation, leveraging the powerful capabilities of neural networks [3]. In the specific context of renal pathology, deep-learning approaches have been increasingly applied to glomerular segmentation, demonstrating promising results with improved accuracy and efficiency [4]. These methods typically utilize convolutional neural networks (CNs) to discern the complex patterns and structures within kidney tissues, allowing for the identification and segmentation of glomeruli from a vast array of tissue samples [5]. ...
In the specialized field of renal histology, precise segmentation of glomeruli in microscopic images is crucial for accurate clinical diagnosis and pathological analysis. Facing the challenge of discerning complex visual features, such as shape, texture, and size within these images, we introduce a novel segmentation model that innovatively combines convolutional neural networks (CNNs) with the advanced TransXNet block, specifically tailored for glomerular segmentation. This innovative model is designed to capture the intricate details and broader contextual features within the images, ensuring a comprehensive and precise segmentation process. The model's architecture unfolds in two primary phases: the down-sampling phase, which utilizes CNNs structures within the TransXNet block for meticulous extraction of detailed features, and the up-sampling phase, which employs CNNs deconvolution techniques to restore spatial resolution and enhance macroscopic feature representation. A critical innovation in our model is the implementation of residual connections between these two phases, which facilitate the seamless integration of features and minimize loss of precision during image reconstruction. Experimental results demonstrate a significant improvement in our model’s performance compared to existing medical image segmentation methods. We report enhancements in mean Pixel Accuracy (mPA) and mean Intersection over Union (mIoU), with increases of approximately 3–5% and 3–8%, respectively. Additionally, the segmented outputs exhibit higher subjective visual quality with fewer noise artifacts. These findings suggest that our model offers promising applications in the segmentation of medical microscopic images, marking a significant contribution to the domain.
... As a result, specific pathological patterns and classifications of severity have been defined for each possible cause of the disease (e.g., diabetic nephropathy [4] and IgA nephropathy [5]). In the field of glomerular pathology analysis, various AI systems AUTOMATED SCORING OF GLOMERULAR INJURY 4 have been reported, including AI for diagnosing membranous nephropathy [6], AI for classifying the severity of diabetic nephropathy [7], AI for classifying the severity of IgA nephropathy (MEST-C) [8], AI for classifying the severity of lupus nephritis [9], AI for detecting specific glomerular lesions or classifying lesion patterns [10][11][12][13], and AI for diagnosing nephropathies [14]. These projects were primarily developed for use in clinical settings. ...
Several artificial intelligence (AI) systems have been developed for glomerular pathology analysis in clinical settings. However, the application of AI systems in nonclinical fields remains limited. In this study, we trained a convolutional neural network model, which is an AI algorithm, to classify the severity of Tensin 2 (TNS2)-deficient nephropathy into seven categories. A dataset consisting of 803 glomerular images was generated from kidney sections of TNS2-deficient and wild-type mice. Manual evaluations of the images were conducted to assess their glomerular injury scores. The trained AI achieved approximately 70% accuracy in predicting the glomerular injury score for TNS2-deficient nephropathy. However, the AI achieved approximately 100% accuracy when considering predictions within one score of the true label as correct. The AI’s predicted mean score closely matched the true mean score. In conclusion, while the AI model may not replace human judgment entirely, it can serve as a reliable second assessor in scoring glomerular injury, offering potential benefits in enhancing the accuracy and objectivity of such assessments.
... Over the last decade, numerous studies have focused on the development of deep-learning models for nephropathology. In several previous studies, neural networks have been trained and successfully applied to specific glomerular segmentation tasks, such as distinguishing between glomerular and non-glomerular regions and classifying healthy and injured glomeruli in WSIs of both human disease and animal models [25][26][27] . In 2020, Uchino et al. developed a comprehensive deep-learning model to classify multiple glomerular images and suggested its potential use in enhancing the diagnostic accuracy for clinicians 28 . ...
Tubular injury is the most common cause of acute kidney injury. Histopathological diagnosis may help distinguish between the different types of acute kidney injury and aid in treatment. To date, a limited number of study has used deep-learning models to assist in the histopathological diagnosis of acute kidney injury. This study aimed to perform histopathological segmentation to identify the four structures of acute renal tubular injury using deep-learning models. A segmentation model was used to classify tubule-specific injuries following cisplatin treatment. A total of 45 whole-slide images with 400 generated patches were used in the segmentation model, and 27,478 annotations were created for four classes: glomerulus, healthy tubules, necrotic tubules, and tubules with casts. A segmentation model was developed using the DeepLabV3 architecture with a MobileNetv3-Large backbone to accurately identify the four histopathological structures associated with acute renal tubular injury in PAS-stained mouse samples. In the segmentation model for four structures, the highest Intersection over Union and the Dice coefficient were obtained for the segmentation of the “glomerulus” class, followed by “necrotic tubules,” “healthy tubules,” and “tubules with cast” classes. The overall performance of the segmentation algorithm for all classes in the test set included an Intersection over Union of 0.7968 and a Dice coefficient of 0.8772. The Dice scores for the glomerulus, healthy tubules, necrotic tubules, and tubules with cast are 91.78 ± 11.09, 87.37 ± 4.02, 88.08 ± 6.83, and 83.64 ± 20.39%, respectively. The utilization of deep learning in a predictive model has demonstrated promising performance in accurately identifying the degree of injured renal tubules. These results may provide new opportunities for the application of the proposed methods to evaluate renal pathology more effectively.
... , [11], [12], [13], 80 [14], [15], [16]. For the segmentation of whole kidney tissue, 81 Hermsen et al. train a network using multi-tissue annotations 82 from PAS-stained kidney transplant biopsy WSIs [17]. ...
Accurately diagnosing chronic kidney disease requires pathologists to assess the structure of multiple tissues under different stains, a process that is timeconsuming and labor-intensive. Current AI-based methods for automatic structure assessment, like segmentation, often demand extensive manual annotation and focus on single stain domain. To address these challenges, we introduce MSMTSeg, a generative self-supervised meta-learning framework for multi-stained multi-tissue segmentation in renal biopsy whole slide images (WSIs). MSMTSeg incorporates multiple stain transform models for style translation of inter-stain domains, a self-supervision module for obtaining pre-trained models with the domain-specific feature representation, and a meta-learning strategy that leverages generated virtual data and pre-trained models to learn the domain-invariant feature representation across multiple stains, thereby enhancing segmentation performance. Experimental results demonstrate that MSMTSeg achieves superior and robust performance, with mDSC of 0.836 and mIoU of 0.718 for multiple tissues under different stains, using only one annotated training sample for each stain. Our ablation study confirms the effectiveness of each component, positioning MSMTSeg ahead of classic advanced segmentation networks, recent few-shot segmentation methods, and unsupervised domain adaptation methods. In conclusion, our proposed few-shot cross-domain technology offers a feasible and cost-effective solution for multi-stained renal histology segmentation, providing convenient assistance to pathologists in clinical practice. The source code and conditionally accessible data are available at
https://github.com/SnowRain510/MSMTSeg.</uri
... Content courtesy of Springer Nature, terms of use apply. Rights reserved glomeruli, tubules, peritubular capillaries, and other components21,22,32 . Similar to Madabhushi et al. ...
Managing patients with kidney allografts largely depends on biopsy diagnosis which is based on semiquantitative assessments of rejection features and extent of acute and chronic changes within the renal parenchyma. Current methods lack reproducibility while digital image data-driven computational models enable comprehensive and quantitative assays. In this study we aimed to develop a computational method for automated assessment of histopathology transformations within the tubulointerstitial compartment of the renal cortex. Whole slide images of modified Picrosirius red-stained biopsy slides were used for the training (n = 852) and both internal (n = 172) and external (n = 94) tests datasets. The pipeline utilizes deep learning segmentations of renal tubules, interstitium, and peritubular capillaries from which morphometry features were extracted. Seven indicators were selected for exploring the intrinsic spatial interactions within the tubulointerstitial compartment. A principal component analysis revealed two independent factors which can be interpreted as representing chronic and acute tubulointerstitial injury. A K-means clustering classified biopsies according to potential phenotypes of combined acute and chronic transformations of various degrees. We conclude that multivariate analyses of tubulointerstitial morphometry transformations enable extraction of and quantification of acute and chronic components of injury. The method is developed for renal allograft biopsies; however, the principle can be applied more broadly for kidney pathology assessment.
... Currently, digital pathology research is relatively scarce in the realm of renal pathology. Existing studies primarily focus on recognizing cell images with straightforward tissue structures, detecting and extracting glomeruli [26,27], and segmenting glomeruli [28,29]. To our knowledge, limited research has tackled the classification of specific glomeruli. ...
The primary pathological feature of Membranous Nephropathy (MN) is the presence of tiny spike-like projections on the glomerular basement membrane. Early detection and efficient treatment of spike-like projections are essential in halting disease progression. Renal pathology biopsy stands as the gold standard for diagnosing MN, and the accurate identification of glomerular spike-like projections plays a vital role in aiding diagnosis. Nevertheless, the tiny spike-like projection lesions and constraints in data quantity pose considerable challenges for supervised learning-based glomerular classification and quantification. We develop a Multi-Scale Annotation-based Multiple Instance Learning (MSA-MIL) model to address the issues. MSA-MIL utilizes image labels and box-level labels to jointly enhance the classification performance of the MIL model. Specifically, we first employ U-Net for glomerular image edge segmentation and subsequently train the MIL model on the dataset with image-level labels. Then, to overcome the limitations arising from the scarcity of positive instances and the relatively small size of spike-like projection features, we manually augment the number of instances with spike-like projections via using box-level annotation to further enhance the MIL model's classification performance. The designed MSA-MIL model enables the classification, visualization, and quantitative analysis of glomeruli with spike-like projections in renal pathology images. We validated and evaluated the designed MSA-MIL model. The model performed exceptionally well, achieving a high accuracy of 0.9847 and demonstrating a high recall rate, effectively preventing misdiagnosis. Additionally, we utilized heatmaps to visualize the locations of spike-like projections within glomeruli, enhancing the model's interpretability. Furthermore, through an analysis of the correlation between the stages of membranous nephropathy and the proportion of spike-like projections, we observed that as the disease advances, the proportion of spike-like projections increases. This finding serves to further validate the results obtained by the model. The MSA-MIL model is the first one specifically designed for classifying glomerular spike-like projections. It not only enhances classification performance but also proves to be more suitable for categorizing minute lesions compared to conventional Convolutional Neural Network (CNN) models. The visualization of glomerular lesions and the proportion of spike-like projections provide doctors with insights into the model's inference process, offering intuitive assistance for accurate diagnoses. This model brings significant hope for advancing research and diagnosis in the field of kidney diseases.
... Moreover, these models have proven instrumental in the domain of transplant kidney biopsies, facilitating the detection of organ quality, the prediction of rejection risks and the diagnosis of other related conditions [112,114]. Many other CNN algorithms are applied in kidney histological investigations, such as U-Net algorithms that were shown to be particularly relevant for segmentation processes [115][116][117][118]. U-Net training can be supported by other software, including QuPath. ...
In the era of digitalization, the biomedical sector has been affected by the spread of artificial intelligence. In recent years, the possibility of using deep and machine learning methods for clinical diagnostic and therapeutic interventions has been emerging as an essential resource for biomedical imaging. Digital pathology represents innovation in a clinical world that looks for faster and better-performing diagnostic methods, without losing the accuracy of current human-guided analyses. Indeed, artificial intelligence has played a key role in a wide variety of applications that require the analysis of a massive amount of data, including segmentation processes in medical imaging. In this context, artificial intelligence enables the improvement of image segmentation methods, moving towards the development of fully automated systems of analysis able to support pathologists in decision-making procedures. The aim of this review is to aid biologists and clinicians in discovering the most common segmentation open-source tools, including ImageJ (v. 1.54), CellProfiler (v. 4.2.5), Ilastik (v. 1.3.3) and QuPath (v. 0.4.3), along with their customized implementations. Additionally, the tools’ role in the histological imaging field is explored further, suggesting potential application workflows. In conclusion, this review encompasses an examination of the most commonly segmented tissues and their analysis through open-source deep and machine learning tools.
... The loss values of each part are added by a certain weight to obtain the final loss function, as shown in Eq. (12). ...
The percent global glomerulosclerosis is a key factor in determining the outcome of renal transfer surgery. At present, the rate is typically computed by pathologists, which is labour intensive and nonstandardized. With the development of Deep Learning (DL), DL-based segmentation models can be used to better identify and segment normal and sclerosed glomeruli. Based on this, we can better quantify percent global glomerulosclerosis to reduce the discard rate of donor kidneys. We used 51 whole slide images (WSIs) from different institutions that are publicly available on the internet. However, the number of sclerosed glomeruli is much smaller than that of normal glomeruli in different WSIs, which can reduce the effectiveness of Deep Learning. For better sclerosed glomerular identification and segmentation performance, we modified and trained a GAN (generative adversarial network)-based image inpainting model to obtain more synthetic sclerosed glomeruli. Our proposed inpainting method achieved an average SSIM (Structural Similarity) of 0.8086 and an average PSNR (Peak Signal-to-Noise Ratio) of 22.8943 dB in the area of generated sclerosed glomeruli. We obtained sclerosed glomerular segmentation performance improvement by adding synthetic sclerosed glomerular images and achieved the best Dice of glomerular segmentation in different test sets based on the modified Unet model.
... The main contributions of this work are, 1 To propose a novel deep learning model, such as a Conditional Variational Generative Adversarial Network for categorizing Chronic Kidney disease. 2 To employ a recently developed Squirrel Search Algorithm to select the most optimal features from the benchmark CKD dataset to improve the efficacy of the classification performed by the CVGAN deep learning model. 3 To assess the performance of the proposed model by comparing it against conventional deep learning models with and without feature selection techniques and further compare the proposed model with existing works from the literature to demonstrate its performance supremacy. ...
Globally, the prevalence of chronic kidney disease (CKD) is steadily increasing. Computer-aided automated diagnostic (CAD) methods play a significant part in predicting CKD. Due to their highly effective classification accuracy, CAD systems like deep learning algorithms are essential in diagnosing diseases. This research creates an innovative categorization model with a metaheuristic algorithm based on the best characteristic selection to diagnose chronic kidney disease. Data with the absence of values were first removed during the pre-processing phase. Then, the optimal assortment of attributes is chosen using the Squirrel Search algorithm, a metaheuristic method that aids in more precise disorder prediction or categorization. Conditional Variational Generative Adversarial Networks were suggested for classification to identify the presence of CKD. Performance measures such as accuracy, precision, recall, and F1 score were evaluated on the benchmark CKD dataset to determine the efficiency of the suggested feature selection-based classifier. According to the experimental findings, the proposed method outperformed existing classification models with accuracy, precision, recall, and F1 score values of 99.2%, 98.4%, 98.6%, and 98.9%, respectively.
... The majority of artificial intelligence algorithms are designed to cover a wide range of possible applications in surgical pathology, such as cancer grading, classification, molecular subtyping, outcome prediction, and segmentation. [1][2][3][4][5][6][7][8] AI applications for OSCC whole slide image (WSI) analysis are becoming popular. 9 Several AI-based feature approaches have been described in oral and oropharyngeal lesions. ...
Anatomical pathology is undergoing its third revolution, transitioning from analogical to digital pathology and incorporating new artificial intelligence technologies into clinical practice. Aside from classification, detection, and segmentation models, predictive models are gaining traction since they can impact diagnostic processes and laboratory activity, lowering consumable usage and turnaround time. Our research aimed to create a deep-learning model to generate synthetic Ki-67 immunohistochemistry from Haematoxylin and Eosin (H&E) stained images. We used 175 oral squamous cell carcinoma (OSCC) from the University Federico II’s Pathology Unit’s archives to train our model to generate 4 Tissue Micro Arrays (TMAs). We sectioned one slide from each TMA, first stained with H&E and then re-stained with anti-Ki-67 immunohistochemistry (IHC). In digitised slides, cores were disarrayed, and the matching cores of the 2 stained were aligned to construct a dataset to train a Pix2Pix algorithm to convert H&E images to IHC. Pathologists could recognise the synthetic images in only half of the cases in a specially designed likelihood test. Hence, our model produced realistic synthetic images. We next used QuPath to quantify IHC positivity, achieving remarkable levels of agreement between genuine and synthetic IHC.
Furthermore, a categorical analysis employing 3 Ki-67 positivity cut-offs (5%, 10%, and 15%) revealed high positive-predictive values. Our model is a promising tool for collecting Ki-67 positivity information directly on H&E slides, reducing laboratory demand and improving patient management. It is also a valuable option for smaller laboratories to easily and quickly screen bioptic samples and prioritise them in a digital pathology workflow.
... Recent progress in digital image analysis and machine-learning applications has opened new prospects for automated renal pathology assays for both segmentation and quantification tasks [16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35]. Several studies have shown deep-learning algorithms for automated recognition and segmentation of kidney histology compartments. ...
Unlabelled:
Introduction The diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable, while digital and machine-learning technologies present opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity.
Materials and methods:
We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. The quality of classifier-produced heatmaps was evaluated by intersection over union metrics (IoU) between predicted and ground truth localization heatmaps.
Results:
A proposed spatially guided modification of the CNN classifier achieved the highest glomerular pattern classification accuracies, with area under curve (AUC) values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, the spatially guided classifier achieved a significantly higher generalized mean IoU value compared to single-multiclass and multiple-binary classifiers.
Conclusions:
We propose a spatially guided CNN classifier that in our experiments reveals the potential to achieve high accuracy for the localization of intraglomerular patterns.
... The identification of glomerulosclerosis in whole slide images through semantic segmentation was proposed by Bueno et al. [11]. A method for detecting glomeruli using WSI is presented, which utilizes semantic segmentation based on CNNs. ...
... No article reported the use of ML on kidney histological samples before that year, but it must be emphasized that in 1999 a simple single-layer perceptron network was designed and trained with features extracted from about 100 kidney transplant biopsies to predict the final diagnosis [50]. Initially, the main efforts were made to perform structure recognition through semantic segmentation, that is labeling each pixel in an image to outline object boundaries [51]. Although segmentation alone may be of limited use in routine practice, it lays the groundwork for performing more sophisticated tasks [46]. ...
Introduction
Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments.
Methods
Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included.
Results
Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification.
Conclusion
Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.
Graphical abstract
... CNNs are a typical class of deep neural networks. They are the most famous and commonly employed in many fields, such as computer vision (Qin et al., 2020), semantic segmentation (Bueno et al., 2020) and time-series forecasting. The uses of the 1D-CNN model offers several advantages for cryptocurrency forecasting. ...
Purpose
This study aims to assess the predictive performance of various factors on Bitcoin returns, used for the development of a robust forecasting support decision model using machine learning techniques, before and during the COVID-19 pandemic. More specifically, the authors investigate the impact of the investor's sentiment on forecasting the Bitcoin returns.
Design/methodology/approach
This method uses feature selection techniques to assess the predictive performance of the different factors on the Bitcoin returns. Subsequently, the authors developed a forecasting model for the Bitcoin returns by evaluating the accuracy of three machine learning models, namely the one-dimensional convolutional neural network (1D-CNN), the bidirectional deep learning long short-term memory (BLSTM) neural networks and the support vector machine model.
Findings
The findings shed light on the importance of the investor's sentiment in enhancing the accuracy of the return forecasts. Furthermore, the investor's sentiment, the economic policy uncertainty (EPU), gold and the financial stress index (FSI) are the top best determinants before the COVID-19 outbreak. However, there was a significant decrease in the importance of financial uncertainty (FSI and EPU) during the COVID-19 pandemic, proving that investors attach much more importance to the sentimental side than to the traditional uncertainty factors. Regarding the forecasting model accuracy, the authors found that the 1D-CNN model showed the lowest prediction error before and during the COVID-19 and outperformed the other models. Therefore, it represents the best-performing algorithm among its tested counterparts, while the BLSTM is the least accurate model.
Practical implications
Moreover, this study contributes to a better understanding relevant for investors and policymakers to better forecast the returns based on a forecasting model, which can be used as a decision-making support tool. Therefore, the obtained results can drive the investors to uncover potential determinants, which forecast the Bitcoin returns. It actually gives more weight to the sentiment rather than financial uncertainties factors during the pandemic crisis.
Originality/value
To the authors’ knowledge, this is the first study to have attempted to construct a novel crypto sentiment measure and use it to develop a Bitcoin forecasting model. In fact, the development of a robust forecasting model, using machine learning techniques, offers a practical value as a decision-making support tool for investment strategies and policy formulation.
... Image semantic segmentation [1][2][3] is a typical problem in the field of machine vision, which is based on the principle of labeling each pixel point in an image with a predefined category label, dividing the image into multiple regions, using different colors for different categories to identify them, and preserving the location information of different categories of pixel points to achieve pixel-level segmentation of images. In recent years, semantic segmentation has been widely used in various fields, such as satellite remote sensing [4,5], medical image analysis [6][7][8], autonomous driving [9][10][11], and intelligent agriculture [12,13]. Traditional semantic segmentation methods mainly construct classifiers for semantic segmentation by texture primitive forest [14] (TF), random forest [15] (RF), grayscale segmentation [16], and conditional random field [17] methods, but each classifier is designed for a single category only. ...
Semantic segmentation is a typical problem in the field of machine vision. Convolutional neural networks(CNNs)-based methods all have excellent performance in image semantic segmentation. Existing semantic segmentation models tend to focus only on the improvement of image segmentation performance, with little attention to the problems of network lightweighting and multi-scale feature information utilization. To address this problem, we design a multi-scale feature fusion lightweight semantic segmentation network(MFFLNet), which consists of two parts: a deep feature extraction module(DFEM) and a multi-scale feature extraction module(MFEM). First, the deep feature extraction module(DFEM) utilizes the deconvolution layer to replace the convolution layer, which can avoid the problem of feature information loss caused by cropping the feature map when feature fusion is performed. Meanwhile, the dimensionality of the feature map is compressed by using 1 × 1 convolutional layers after each upsampling layer, which can effectively reduce the number of parameters of the model. Then, the multi-scale feature extraction module(MFEM) employs multiple null convolutions with different expansion rates for feature extraction of the image to extract feature information on multiple scales. Finally, the deep features and multi-scale features extracted by the two modules respectively are fused to achieve semantic segmentation of the image. It is shown experimentally that the proposed MFFLNet outperforms the mainstream methods in semantic segmentation on two datasets, PASCAL VOC 2012 and Cityscapes, with mIoU reaching 71. 23% and 79. 24%, respectively, and improving 5. 8% and 8. 8% compared with the state-of-the-art DeepLab V3 + model, respectively.
... Coudray et al. used convolution neural networks (CNN) on whole-slide images (WSI) to classify them into lung adenocarcinoma (LUAD), lung squamous cell carcinoma (LUSC) or normal lung tissue 10 . Also, CNN have been applied to WSI to classify sclerosed and non-sclerosed glomeruli 11,12 . Kolachalama et al. demonstrated that CNN deep learning models can outperform the pathologist-estimated fibrosis score across the classification tasks and can be applied to routine renal biopsy images 13 . ...
Machine learning applied to digital pathology has been increasingly used to assess kidney function and diagnose the underlying cause of chronic kidney disease (CKD). We developed a novel computational framework, clustering-based spatial analysis (CluSA), that leverages unsupervised learning to learn spatial relationships between local visual patterns in kidney tissue. This framework minimizes the need for time-consuming and impractical expert annotations. 107,471 histopathology images obtained from 172 biopsy cores were used in the clustering and in the deep learning model. To incorporate spatial information over the clustered image patterns on the biopsy sample, we spatially encoded clustered patterns with colors and performed spatial analysis through graph neural network. A random forest classifier with various groups of features were used to predict CKD. For predicting eGFR at the biopsy, we achieved a sensitivity of 0.97, specificity of 0.90, and accuracy of 0.95. AUC was 0.96. For predicting eGFR changes in one-year, we achieved a sensitivity of 0.83, specificity of 0.85, and accuracy of 0.84. AUC was 0.85. This study presents the first spatial analysis based on unsupervised machine learning algorithms. Without expert annotation, CluSA framework can not only accurately classify and predict the degree of kidney function at the biopsy and in one year, but also identify novel predictors of kidney function and renal prognosis.
... Qu et al. took ResNet34 as a network encoder backbone and chose a fixed pre-trained VGG16 model as a loss network for enhancing segmentation details, while taking advantage of migration learning to facilitate the training on a densely distributed lung cancer dataset [28]. Bueno et al. were the first to use a convolutional neural network based semantic segmentation network to detect sclerotic glomeruli in WSIs [29]. Rijthovena et al. used multiple branches of encoder-decoder CNN to obtain contextual and detailed information while using concentric patches with multiple resolutions and different receptive fields to receive accurate resolution semantic segmentation in a WSI [30]. ...
Hematoxylin and eosin stained whole slide images (WSIs) are the gold standard for pathologists and medical professionals for tumor diagnosis, surgery planning, and postoperative examinations. In recent years, due to the rapidly emerging field of deep learning, there have been many convolutional neural networks (CNNs) and Transformer based models applied to computational pathology for accurate segmentation. However,
the generalization ability and robustness of models often affect the diagnosis and prognosis of cancer, and we attempt to effectively combine the advantage of CNN which excels in sparse WSI segmentation while Transformer excels in dense cases. In this paper, We propose a novel feature fusion strategy, DHUnet, which utilizes Swin Transformer and ConvNeXt modules with a dual-branch hierarchical U-shaped architecture to fuse global and local features for WSI segmentation. Firstly, a WSI is divided into small patches, which are put into the global and local encoders to generate hierarchical features in parallel. Then, with the help of
global–local fusion modules and skip connections, the decoder can fully obtain the global coarse and local fine-grained information during the upsampling process. The proposed Cross-scale Expand Layer can make the patches with the same center but different scales recover the input resolution gradually at each stage. Finally, all the projected pixel-level patch masks are merged to restore the final WSI tumor segmentation. Extensive experiments demonstrate that DHUnet has excellent performance and generalization ability for WSI segmentation, achieving the best segmentation results on three datasets with different types of cancer, densities, and target sizes. The code and pre-processed datasets will be publicly available at https://github.com/pengsllab/DHUnet.
As global life expectancy increases, so does the burden of chronic diseases, yet individuals exhibit considerable variability in the rate at which they age. Identifying biomarkers that distinguish fast from slow ageing is crucial for understanding the biology of ageing, enabling early disease detection, and improving prevention strategies. Using contrastive deep learning, we show that skin biopsy images alone are sufficient to determine an individual's age. We then use visual features in histopathology slides of the skin biopsies to construct a novel biomarker of ageing. By linking with comprehensive health registers in Denmark, we demonstrate that visual features in histopathology slides of skin biopsies predict mortality and the prevalence of chronic age-related diseases. Our work highlights how routinely collected health data can provide additional value when used together with deep learning, by creating a new biomarker for ageing which can be actively used to determine mortality over time.
In pathology images, different stains highlight different glomerular structures, so a supervised deep learning-based glomerular instance segmentation model trained on individual stains performs poorly on other stains. However, it is difficult to obtain a training set with multiple stains because the labeling of pathology images is very time-consuming and tedious. Therefore, in this paper, we proposed an unsupervised stain augmentation-based method for segmentation of glomerular instances.
In this study, we successfully realized the conversion between different staining methods such as PAS, MT and PASM by contrastive unpaired translation (CUT), thus improving the staining diversity of the training set. Moreover, we replaced the backbone of mask R-CNN with swin transformer to further improve the efficiency of feature extraction and thus achieve better performance in instance segmentation task.
To validate the method presented in this paper, we constructed a dataset from 216 WSIs of the three stains in this study. After conducting in-depth experiments, we verified that the instance segmentation method based on stain augmentation outperforms existing methods across all metrics for PAS, PASM, and MT stains. Furthermore, ablation experiments are performed in this paper to further demonstrate the effectiveness of the proposed module.
This study successfully demonstrated the potential of unsupervised stain augmentation to improve glomerular segmentation in pathology analysis. Future research could extend this approach to other complex segmentation tasks in the pathology image domain to further explore the potential of applying stain augmentation techniques in different domains of pathology image analysis.
Ovarian cancer is a significant health concern because of its high mortality rates and potential to cause glomerular injury, which can obstruct the urinary tract. It is very crucial to diagnose and treat these diseases accurately as well as timely. In the era of artificial intelligence, deep learning models have emerged as powerful tools in analysing medical images as they showcase exceptional capabilities to detect diseases. In this study, an innovative approach has been proposed that uses deep transfer learning classifiers for the detection as well as classification of ovarian cancer, sclerosed glomeruli, and normal glomeruli in histopathological images. To gather relevant data, two different repositories have been explored which contain images of ovarian cancer, sclerosed glomeruli, and normal glomeruli. These images are thoroughly pre-processed by converting them into grayscale. Afterwards, advanced segmentation techniques are applied such as image equalization, thresholding, image inversion, and morphological opening which effectively highlight the affected areas using contour features, and various measurements such as area, mean intensity, height, width, and epsilon are calculated. Our study employed a range of deep learning techniques such as AlexNet2, InceptionV3, EfficientNetB0, EfficientNetB5, DenseNet121, Xception, MobileNetV2, and InceptionResNetV2 along with the two optimization techniques: Adam and RMSprop optimizer. Remarkably, during experimentation, AlexNet2 demonstrated exceptional accuracy by achieving 99.74%, with a low loss of 0.0018 and a root mean square error of 0.042426 when incorporating the Adam optimizer. Similarly, using the RMSprop optimizer, Xception delivered outstanding results with an accuracy of 99.74%, a minimal loss of 0.0027, and a root mean square error of 0.051962. This pioneering research significantly contributes to the field of medical diagnostics by harnessing deep learning technology to enhance the precision and efficiency of ovarian cancer and sclerosed glomeruli detection.
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field’s future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Background
Machine learning (ML) models can help assisting diagnosis by rapidly localising and classifying regions of interest (ROIs) within whole slide images (WSIs). Effective ML models for clinical decision support require a substantial dataset of ‘real’ data, and in reality, it should be robust, user-friendly and universally applicable.
Methods
WSIs of primary IgAN were collected and annotated. The H-AI-L algorithm which could facilitate direct WSI viewing and potential ROI detection for clinicians was built on the cloud server of matpool, a shared internet-based service platform. Model performance was evaluated using F1-score, precision, recall and Matthew’s correlation coefficient (MCC).
Results
The F1-score of glomerular localisation in WSIs was 0.85 and 0.89 for the initial and pretrained models, respectively, with corresponding recall values of 0.79 and 0.83, and precision scores of 0.92 and 0.97. Dichotomous differentiation between global sclerotic (GS) and other glomeruli revealed F1-scores of 0.70 and 0.91, and MCC values of 0.55 and 0.87, for the initial and pretrained models, respectively. The overall F1-score of multiclassification was 0.81 for the pretrained models. The total glomerular recall rate was 0.96, with F1-scores of 0.68, 0.56 and 0.26 for GS, segmental glomerulosclerosis and crescent (C), respectively. Interstitial fibrosis/tubular atrophy lesion similarity between the true label and model predictions was 0.75.
Conclusions
Our results underscore the efficacy of the ML integration algorithm in segmenting ROIs in IgAN WSIs, and the internet-based model deployment is in favour of widespread adoption and utilisation across multiple centres and increased volumes of WSIs.
Motivation
Pediatric kidney disease is a widespread, progressive condition that severely impacts growth and development of children. Chronic Kidney Disease (CKD) is often more insidious in children than in adults, usually requiring a renal biopsy for diagnosis. Biopsy evaluation requires copious examination by trained pathologists, which can be tedious and prone to human error. In this study, we propose an Artificial Intelligence (AI) method to assist pathologists in accurate segmentation and classification of pediatric kidney structures, named as AI-based Pediatric Kidney Diagnosis (APKD).
Results
We collected 2,935 pediatric patients diagnosed with kidney disease for the development of APKD. The dataset comprised 93,932 histological structures annotated manually by three skilled nephropathologists. APKD scored an average accuracy of 94% for each kidney structure category, including 99% in the glomerulus. We found strong correlation between the model and manual detection in detected glomeruli (Spearman correlation coefficient r = 0.98, p-value < 0.001; intraclass correlation coefficient ICC = 0.98, 95% CI = 0.96-0.98). Compared to manual detection, APKD was approximately 5.5 times faster in segmenting glomeruli. Finally, we show how the pathological features extracted by APKD can identify focal abnormalities of the glomerular capillary wall to aid in the early diagnosis of pediatric kidney disease.
Availability
https://github.com/ChunyueFeng/Kidney-DataSet.
Supplementary information
Supplementary data are available at Bioinformatics online.
Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by “engineered” methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.
Both health and disease are closely correlated with lung and kidney functions. The maintenance of blood pressure, the regulation of carbon dioxide and bicarbonate partial pressures, the preservation of fluid homeostasis, and the maintenance of acid–base balance all depend on renal and pulmonary functions. The challenging part of lung and kidney diseases is prediction and detection. Predicting earlier will help to find a better solution for lung and kidney diseases. Like prediction, detection is also a major part of lung and kidney diseases. By detecting, it will be easier to find a better treatment for the diseases. Both prediction and detection of lung and kidney disease are satisfied using the transfer learning approach. So, in this paper, the prediction and detection of lung and kidney diseases using transfer learning and transfer learning models used in both prediction and detection of lung and kidney diseases had been analyzed.
Glomeruli is a collection of blood vessels present in kidneys of the human body. Since kidneys are one of the most important human body organs, diagnosis of any abnormality becomes very important. This work focuses on the binary classification of normal and scleroses glomeruli using convolutional neural networks from whole slide images (WSI). These images are periodic acid-Schiff (PAS) stained and the glomerulus can be seen as circular areas of dark stains on the slide. The main purpose of this work is to make the diagnoses of the kidney glomeruli fast and accurate since manual detection is quite time-consuming and has many human errors as well. In our work, we have performed the classification of microscopic images to detect the scleroses glomerulus from the kidney. This study deployed, four different types of CNN models and subsequently evaluated, they are AlexNet, Visual Geometry Group-19, GoogleNet, and a customized model. The comparative analysis has been made by considering several parameters such as the number of epochs, optimizers, batch size, and learning rate in which the customized model achieved the accuracy of 97.86%. The results of the proposed work are quite promising. The performances of the models used in this work are compared using various metrics such as accuracy, recall, precision, and F1-score, and the results from each of the models are noted.KeywordsConvolutional neural networksGoogleNetAlexNetVisual geometry group-19Whole slide imagesPeriodic acid-SchiffOptimizersPrecisionRecallF1-score
Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.
The histopathological image analysis is one of the most crucial diagnostic procedures to identify Invasive ductal carcinoma (IDC) in breast cancers. However, this diagnosis process is currently time-consuming and heavily dependent on human expertise. Prior research has shown that different degrees of tumors present various microstructures in the histopathological images. However, very little has been done to utilize spatial recurrence features of microstructures for identifying IDC. This paper presents a novel recurrence analysis methodology for automatic image-guided IDC detection. We first utilize wavelet decomposition to delineate the subtle information in the images. Then, we model the patches with a weighted recurrence network approach to characterize the recurrence patterns of the histopathological images. Finally, we develop automated IDC detection models leveraging machine learning methods with spatial recurrence features extracted. The developed recurrence analysis models successfully characterize the complex microstructures of histopathological images and achieve the IDC detection performances of at least AUC = 0.96. This research developed a spatial recurrence analysis methodology to effectively identify IDC regions in histopathological images for BC. It shows a high potential to assist physicians in the decision-making process. The proposed methodology can further be applicable to image processing for other medical or biological applications.
Rationale & Objective
The number of glomeruli is often used to determine the adequacy of a kidney biopsy (eg, at least 10 glomeruli). It is often assumed that biopsy specimens with limited amounts of cortex are too imprecise for detection of focal pathology.
Study Design
Clinical-pathologic correlation (cross-sectional).
Setting & Participants
Living kidney donors who underwent a needle core biopsy of their kidney at the time of donation.
Exposure
The amount of cortex biopsied as determined by either the number of glomeruli or area of cortex on histology.
Outcome
The percentage of globally sclerotic glomeruli, density of interstitial fibrosis foci, and severity of arteriosclerosis were determined.
Analytical approach
A beta-binomial model assessed how the mean percentage of globally sclerotic glomeruli and patient variability in percentage of globally sclerotic glomeruli differed with the number of glomeruli on the biopsy specimen. Additional models assessed the association of interstitial fibrosis and arteriosclerosis with number of glomeruli.
Results
There were 2,915 kidney donors studied. Fewer glomeruli on the biopsy specimen associated with higher mean percentage of globally sclerotic glomeruli and higher patient variability in percentage of globally sclerotic glomeruli. Smaller cortical volume on imaging correlated with both less cortex on biopsy and higher percentage of globally sclerotic glomeruli. Based on a statistical simulation, the probability of patient percentage of globally sclerotic glomeruli ≥ 10% if the biopsy percentage of globally sclerotic glomeruli is ≥10% (positive predictive value) was 45% with 1 to 9 glomeruli versus 31% with 10 or more glomeruli; the negative predictive value was 91% versus 98%. Fewer glomeruli also associated with more interstitial fibrosis and arteriosclerosis.
Limitations
The study was limited to living kidney donors. Patient variability in percentage of globally sclerotic glomeruli was based on a statistical model because multiple biopsy specimens per patient were not available.
Conclusions
The amount of cortex on a needle core biopsy is not completely random. Chronic changes from loss of cortex contribute to low amounts of cortex on a kidney biopsy specimen.
Advancements in computer vision and artificial intelligence (AI) carry the potential to make significant contributions to health care, particularly in diagnostic specialties such as radiology and pathology. The impact of these technologies on physician stakeholders is the subject of significant speculation. There is however a dearth of information regarding the opinions, enthusiasm, and concerns of the pathology community at large. Here, we report results from a survey of 487 pathologist-respondents practicing in 54 countries, conducted to examine perspectives on AI implementation in clinical practice. Despite limitations, including difficulty with quantifying response bias and verifying identity of respondents to this anonymous and voluntary survey, several interesting findings were uncovered. Overall, respondents carried generally positive attitudes towards AI, with nearly 75% reporting interest or excitement in AI as a diagnostic tool to facilitate improvements in workflow efficiency and quality assurance in pathology. Importantly, even within the more optimistic cohort, a significant number of respondents endorsed concerns about AI, including the potential for job displacement and replacement. Overall, around 80% of respondents predicted the introduction of AI technology in the pathology laboratory within the coming decade. Attempts to identify statistically significant demographic characteristics (e.g., age, sex, type/place of practice) predictive of attitudes towards AI using Kolmogorov–Smirnov (KS) testing revealed several associations. Important themes which were commented on by respondents included the need for increasing efforts towards physician training and resolving medical-legal implications prior to the generalized implementation of AI in pathology.
Introduction:
The number of glomeruli and glomerulosclerosis evaluated on kidney biopsy slides constitute standard components of a renal pathology report. Prevailing methods for glomerular assessment remain manual, labor intensive, and nonstandardized. We developed a deep learning framework to accurately identify and segment glomeruli from digitized images of human kidney biopsies.
Methods:
Trichrome-stained images (n = 275) from renal biopsies of 171 patients with chronic kidney disease treated at the Boston Medical Center from 2009 to 2012 were analyzed. A sliding window operation was defined to crop each original image to smaller images. Each cropped image was then evaluated by at least 3 experts into 3 categories: (i) no glomerulus, (ii) normal or partially sclerosed (NPS) glomerulus, and (iii) globally sclerosed (GS) glomerulus. This led to identification of 751 unique images representing nonglomerular regions, 611 images with NPS glomeruli, and 134 images with GS glomeruli. A convolutional neural network (CNN) was trained with cropped images as inputs and corresponding labels as output. Using this model, an image processing routine was developed to scan the test images to segment the GS glomeruli.
Results:
The CNN model was able to accurately discriminate nonglomerular images from NPS and GS images (performance on test data: Accuracy: 92.67% ± 2.02% and Kappa: 0.8681 ± 0.0392). The segmentation model that was based on the CNN multilabel classifier accurately marked the GS glomeruli on the test data (Matthews correlation coefficient = 0.628).
Conclusion:
This work demonstrates the power of deep learning for assessing complex histologic structures from digitized human kidney biopsies.
The pathologists follow a systematic and partially manual process to obtain histological tissue sections from the biological tissue extracted from patients. This process is far from being perfect and can introduce some errors in the quality of the tissue sections (distortions, deformations, folds and tissue breaks). In this paper, we propose a deep learning (DL) method for the detection and segmentation of these damaged regions in whole slide images (WSIs). The proposed technique is based on convolutional neural networks (CNNs) and uses the U-net model to achieve the pixel-wise segmentation of these unwanted regions. The results obtained show that this technique yields satisfactory results and can be applied as a pre-processing step for automatic WSI analysis in order to prevent the use of the damaged areas in the evaluation processes.
Gleason grading of histological images is important in risk assessment and treatment planning for prostate cancer patients. Much research has been done in classifying small homogeneous cancer regions within histological images. However, semi-supervised methods published to date depend on pre-selected regions and cannot be easily extended to an image of heterogeneous tissue composition. In this paper, we propose a multi-scale U-Net model to classify images at the pixel-level using 224 histological image tiles from radical prostatectomies of 20 patients. Our model was evaluated by a patient-based 10-fold cross validation, and achieved a mean Jaccard index of 65.8% across 4 classes (stroma, Gleason 3, Gleason 4 and benign glands), and 75.5% for 3 classes (stroma, benign glands, prostate cancer), outperforming other methods.
Glomerulus classification and detection in kidney tissue segments are key processes in nephropathology used for the correct diagnosis of the diseases. In this paper, we deal with the challenge of automating Glomerulus classification and detection from digitized kidney slide segments using a deep learning framework. The proposed method applies Convolutional Neural Networks (CNNs) between two classes: Glomerulus and Non-Glomerulus, to detect the image segments belonging to Glomerulus regions. We configure the CNN with the public pre-trained AlexNet model and adapt it to our system by learning from Glomerulus and Non-Glomerulus regions extracted from training slides. Once the model is trained, labeling is performed by applying the CNN classification to the image blocks under analysis. The results of the method indicate that this technique is suitable for correct Glomerulus detection in Whole Slide Images (WSI), showing robustness while reducing false positive and false negative detections.
Due to the increasing availability of whole slide scanners facilitating digitization of histopathological tissue, there is a strong demand for the development of computer based image analysis systems. In this work, the focus is on the segmentation of the glomeruli constituting a highly relevant structure in renal histopathology, which has not been investigated before in combination with CNNs. We propose two different CNN cascades for segmentation applications with sparse objects. These approaches are applied to the problem of glomerulus segmentation and compared with conventional fully-convolutional networks. Overall, with the best performing cascade approach, single CNNs are outperformed and a pixel-level Dice similarity coefficient of 0.90 is obtained. Combined with qualitative and further object-level analyses the obtained results are assessed as excellent also compared to recent approaches. In conclusion, we can state that especially one of the proposed cascade networks proved to be a highly powerful tool for segmenting the renal glomeruli providing best segmentation accuracies and also keeping the computing time at a low level.
Glomerulus classification in kidney tissue segments is a key process in nephropathology to obtain correct diseases diagnosis. In this paper, we deal with the challenge to automate the Glomerulus classification from digitized kidney slide segments using a deep learning framework. The proposed method applies Convolutional Neural Networks (CNNs) classification between two classes: Glomerulus and Non-Glomerulus, to detect the image segments belonging to Glomerulus regions. We configure the CNN with the public pre-trained AlexNet model, and adapt it to our system by learning from Glomerulus and Non-Glomerulus regions extracted from training slides. Once the model is trained, the labelling is performed applying the CNN classification to the image segments under analysis. The results obtained indicate that this technique is suitable for correct Glomerulus classification, showing robustness while reducing false positive and false negative detections.
PathoSpotter is a computational system designed to assist pathologists in teaching about and researching kidney diseases. PathoSpotter-K is the version that was developed to detect nephrological lesions in digital images of kidneys. Here, we present the results obtained using the first version of PathoSpotter-K, which uses classical image processing and pattern recognition methods to detect proliferative glomerular lesions with an accuracy of 88.3 ± 3.6%. Such performance is only achieved by similar systems if they use images of cell in contexts that are much less complex than the glomerular structure. The results indicate that the approach can be applied to the development of systems designed to train pathology students and to assist pathologists in determining large-scale clinicopathological correlations in morphological research.
This paper presents the development of several models of a deep convolutional auto-encoder in the Caffe deep learning framework and their experimental evaluation on the example of MNIST dataset. We have created five models of a convolutional auto-encoder which differ architecturally by the presence or absence of pooling and unpooling layers in the auto-encoder's encoder and decoder parts. Our results show that the developed models provide very good results in dimensionality reduction and unsupervised clustering tasks, and small classification errors when we used the learned internal code as an input of a supervised linear classifier and multi-layer perceptron. The best results were provided by a model where the encoder part contains convolutional and pooling layers, followed by an analogous decoder part with deconvolution and unpooling layers without the use of switch variables in the decoder part. The paper also discusses practical details of the creation of a deep convolutional auto-encoder in the very popular Caffe deep learning framework. We believe that our approach and results presented in this paper could help other researchers to build efficient deep neural network architectures in the future.
The clinical specialty of adult nephrology has enjoyed spectacular growth during the past three decades. Such a statement is no less true for pediatric nephrology. This book stands in quiet testimony to that fact. Practitioners of pediatric nephrology are now concerned with the diagnosis and treat ment of young patients with a widened spectrum of primary and sec ondary diseases of the kidneys and urinary tract, hypertension and disorders of water, electrolyte and acid-base metabolism. Their science, deriving from an exciting blend of physiology, morphology, pathology, immunology, biochemistry, microbiology, genetics and pharmacology, must also include an understanding of human developmental biology an insight that colleagues who practice adult nephrology require to a somewhat lesser extent. Dramatic, continuing advances in our understanding of the pathogen esis, pathophysiology, diagnosis and treatment of kidney diseases has led to a cascade of books and monographs on various aspects of the subject. Nevertheless, in view of the clear emergence of pediatric nephrology as a distinct medical specialty, it is most appropriate that a practical book, which focuses almost exclusively on approaches to the evaluation and diagnosis of young patients with kidney disease and related disorders, should appear. In that sense this book is unique. It provides an easily accessible, practical compendium or guide to the clinical investigation of all facets of kidney disease in children. Its special and unique emphasis is directed toward various approaches to the evaluation of such patients and the interpretation of associated laboratory or radiographical data.
This study develops a new computational method for high-throughput detection of glomeruli by introducing the Histogram of Oriented Gradients (HOG), an object detection algorithm widely used in computer vision. While methods operating through segmentation exist, they have assumed glomerulus presence in images and are not designed for huge microscopic images, making comprehensive detection of numerous glomeruli per image difficult. Experiments performed under various conditions indicate favorable outcomes from linear support vector machine combined with HOG features in terms of accuracy and speed. The method was applied to the whole microscopic kidney images of Spontaneously Diabetic Torii (SDT) rats, generally used as animal model of diabetic nephropathy, and their controls, Sprague-Dawley (SD) rats. It was revealed that the immunostaining level of an injury marker, desmin, in glomeruli was significantly higher in SDT rats compared to corresponding SD rats. Furthermore, desmin immunostaining levels in glomeruli were highly variable even within SDT rats of the same age, and highly immunostained glomeruli were likely to appear near the juxtamedullary region than in superficial regions. These suggest that the proposed method is a promising approach in quantifying injury level while avoiding possible local biases and for unraveling new insights into glomerulus injury in the SDT rats.
We present a novel and practical deep fully convolutional neural network
architecture for semantic pixel-wise segmentation termed SegNet. This core
trainable segmentation engine consists of an encoder network, a corresponding
decoder network followed by a pixel-wise classification layer. The architecture
of the encoder network is topologically identical to the 13 convolutional
layers in the VGG16 network . The role of the decoder network is to map the low
resolution encoder feature maps to full input resolution feature maps for
pixel-wise classification. The novelty of SegNet lies is in the manner in which
the decoder upsamples its lower resolution input feature map(s). Specifically,
the decoder uses pooling indices computed in the max-pooling step of the
corresponding encoder to perform non-linear upsampling. This eliminates the
need for learning to upsample. The upsampled maps are sparse and are then
convolved with trainable filters to produce dense feature maps. We compare our
proposed architecture with the fully convolutional network (FCN) architecture
and its variants. This comparison reveals the memory versus accuracy trade-off
involved in achieving good segmentation performance. The design of SegNet was
primarily motivated by road scene understanding applications. Hence, it is
efficient both in terms of memory and computational time during inference. It
is also significantly smaller in the number of trainable parameters than
competing architectures and can be trained end-to-end using stochastic gradient
descent. We also benchmark the performance of SegNet on Pascal VOC12 salient
object segmentation and the recent SUN RGB-D indoor scene understanding
challenge. We show that SegNet provides competitive performance although it is
significantly smaller than other architectures. We also provide a Caffe
implementation of SegNet and a webdemo at
http://mi.eng.cam.ac.uk/projects/segnet/
Background
The detection of the glomeruli is a key step in the histopathological evaluation of microscopic images of the kidneys. However, the task of automatic detection of the glomeruli poses challenges owing to the differences in their sizes and shapes in renal sections as well as the extensive variations in their intensities due to heterogeneity in immunohistochemistry staining.
Although the rectangular histogram of oriented gradients (Rectangular HOG) is a widely recognized powerful descriptor for general object detection, it shows many false positives owing to the aforementioned difficulties in the context of glomeruli detection.
Results
A new descriptor referred to as Segmental HOG was developed to perform a comprehensive detection of hundreds of glomeruli in images of whole kidney sections. The new descriptor possesses flexible blocks that can be adaptively fitted to input images in order to acquire robustness for the detection of the glomeruli. Moreover, the novel segmentation technique employed herewith generates high-quality segmentation outputs, and the algorithm is assured to converge to an optimal solution. Consequently, experiments using real-world image data revealed that Segmental HOG achieved significant improvements in detection performance compared to Rectangular HOG.
Conclusion
The proposed descriptor for glomeruli detection presents promising results, and it is expected to be useful in pathological evaluation.
There is large consent that successful training of deep networks requires
many thousand annotated training samples. In this paper, we present a network
and training strategy that relies on the strong use of data augmentation to use
the available annotated samples more efficiently. The architecture consists of
a contracting path to capture context and a symmetric expanding path that
enables precise localization. We show that such a network can be trained
end-to-end from very few images and outperforms the prior best method (a
sliding-window convolutional network) on the ISBI challenge for segmentation of
neuronal structures in electron microscopic stacks. Using the same network
trained on transmitted light microscopy images (phase contrast and DIC) we won
the ISBI cell tracking challenge 2015 in these categories by a large margin.
Moreover, the network is fast. Segmentation of a 512x512 image takes less than
a second on a recent GPU. The full implementation (based on Caffe) and the
trained networks are available at
http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .
The ImageNet Large Scale Visual Recognition Challenge is a benchmark in
object category classification and detection on hundreds of object categories
and millions of images. The challenge has been run annually from 2010 to
present, attracting participation from more than fifty institutions.
This paper describes the creation of this benchmark dataset and the advances
in object recognition that have been possible as a result. We discuss the
challenges of collecting large-scale ground truth annotation, highlight key
breakthroughs in categorical object recognition, provide detailed a analysis of
the current state of the field of large-scale image classification and object
detection, and compare the state-of-the-art computer vision accuracy with human
accuracy. We conclude with lessons learned in the five years of the challenge,
and propose future directions and improvements.
Histopathology diagnosis is based on visual examination of the morphology of histological sections under a microscope. With the increasing popularity of digital slide scanners, decision support systems based on the analysis of digital pathology images are in high demand. However, computerized decision support systems are fraught with problems that stem from color variations in tissue appearance due to variation in tissue preparation, variation in stain reactivity from different manufacturers/batches, user or protocol variation, and the use of scanners from different manufacturers. In this paper, we present a novel approach to stain normalization in histopathology images. The method is based on nonlinear mapping of a source image to a target image using a representation derived from color deconvolution. Color deconvolution is a method to obtain stain concentration values when the stain matrix, describing how the color is affected by the stain concentration, is given. Rather than relying on standard stain matrices, which may be inappropriate for a given image, we propose the use of a color-based classifier that incorporates a novel stain color descriptor to calculate image-specific stain matrix. In order to demonstrate the efficacy of the proposed stain matrix estimation and stain normalization methods, they are applied to the problem of tumor segmentation in breast histopathology images. The experimental results suggest that the paradigm of color normalization, as a preprocessing step, can significantly help histological image analysis algorithms to demonstrate stable performance which is insensitive to imaging conditions in general and scanner variations in particular.
Often this means removing a dominant and undesirable color cast, such as the yellow in photos taken under incandescent illumination. This article describes a method for a more general form of color correction that borrows one image's color characteristics from anoth-er. Figure 1 shows an example of this process, where we applied the colors of a sunset photograph to a daytime computer graphics rendering. We can imagine many methods for applying the colors of one image to another. Our goal is to do so with a simple algorithm, and our core strategy is to choose a suitable color space and then to apply simple oper-ations there. When a typical three channel image is represented in any of the most well-known color spaces, there will be correlations between the different channels'val-ues. For example, in RGB space, most pixels will have large values for the red and green channel if the blue channel is large. This implies that if we want to change the appearance of a pixel's color in a coherent way, we must modify all color channels in tandem. This complicates any color modification process. What we want is an orthogonal color space without correlations between the axes. Recently, Ruderman et al. developed a color space, called lab, which minimizes correlation between chan-nels for many natural scenes., This space is based on data-driven human perception research that assumes the human visual system is ideally suited for processing natural scenes. The authors discovered lab color space in the context of understanding the human visual sys-tem, and to our knowledge, lab space has never been applied otherwise or compared to other color spaces. There's little correlation between the axes in lab space,
Transplantable kidneys are in very limited supply. Accurate viability assessment prior to transplantation could minimize organ discard. Rapid and accurate evaluation of intra-operative donor kidney biopsies is essential for determining which kidneys are eligible for transplantation. The criteria for accepting or rejecting donor kidneys relies heavily on pathologist determination of the percent of glomeruli (determined from a frozen section) that are normal and sclerotic. This percentage is a critical measurement that correlates with transplant outcome. Inter- and intra-observer variability in donor biopsy evaluation is, however, significant. An automated method for determination of percent global glomerulosclerosis could prove useful in decreasing evaluation variability, increasing throughput, and easing the burden on pathologists. Here, we describe the development of a deep learning model that identifies and classifies non-sclerosed and sclerosed glomeruli in whole-slide images of donor kidney frozen section biopsies. This model extends a convolutional neural network (CNN) pre-trained on a large database of digital images. The extended model, when trained on just 48 whole slide images, exhibits slide-level evaluation performance on par with expert renal pathologists. The model substantially outperforms a model trained on image patches of isolated glomeruli. Encouragingly, the model’s performance is robust to slide preparation artifacts associated with frozen section preparation. As the first model reported that identifies and classifies normal and sclerotic glomeruli in frozen kidney sections, and thus the first model reported in the literature relevant to kidney transplantation, it may become an essential part of donor kidney biopsy evaluation in the clinical setting.
Transplantable kidneys are in very limited supply. Accurate viability assessment prior to transplantation could minimize organ discard. Rapid and accurate evaluation of intraoperative donor kidney biopsies is essential for determining which kidneys are eligible for transplantation. The criteria for accepting or rejecting donor kidneys relies heavily on pathologist determination of the percent of glomeruli (determined from a frozen section) that are normal and sclerotic. This percentage is a critical measurement that correlates with transplant outcome. Inter- and intra-observer variability in donor biopsy evaluation is, however, significant. An automated method for determination of percent global glomerulosclerosis could prove useful in decreasing evaluation variability, increasing throughput, and easing the burden on pathologists. Here, we describe the development of a deep learning model that identifies and classifies non-sclerosed and sclerosed glomeruli in whole-slide images of donor kidney frozen section biopsies. This model extends a convolutional neural network (CNN) pre-trained on a large database of digital images. The extended model, when trained on just 48 whole slide images, exhibits slide-level evaluation performance on par with expert renal pathologists. Encouragingly, the model’s performance is robust to slide preparation artifacts associated with frozen section preparation. The model substantially outperforms a model trained on image patches of isolated glomeruli, in terms of both accuracy and speed. The methodology overcomes the technical challenge of applying a pretrained CNN bottleneck model to whole-slide image classification. The traditional patch-based approach, while exhibiting deceptively good performance classifying isolated patches, does not translate successfully to whole-slide image segmentation in this setting. As the first model reported that identifies and classifies normal and sclerotic glomeruli in frozen kidney sections, and thus the first model reported in the literature relevant to kidney transplantation, it may become an essential part of donor kidney biopsy evaluation in the clinical setting.
Background Histologic examination of fixed renal tissue is widely used to assess morphology and the progression of disease. Commonly reported metrics include glomerular number and injury. However, characterization of renal histology is a time-consuming and user-dependent process. To accelerate and improve the process, we have developed a glomerular localization pipeline for trichrome-stained kidney sections using a machine learning image classification algorithm.
Methods We prepared 4- μ m slices of kidneys from rats of various genetic backgrounds that were subjected to different experimental protocols and mounted the slices on glass slides. All sections used in this analysis were trichrome stained and imaged in bright field at a minimum resolution of 0.92 μ m per pixel. The training and test datasets for the algorithm comprised 74 and 13 whole renal sections, respectively, totaling over 28,000 glomeruli manually localized. Additionally, because this localizer will be ultimately used for automated assessment of glomerular injury, we assessed bias of the localizer for preferentially identifying healthy or damaged glomeruli.
Results Localizer performance achieved an average precision and recall of 96.94% and 96.79%, respectively, on whole kidney sections without evidence of bias for or against glomerular injury or the need for manual preprocessing.
Conclusions This study presents a novel and robust application of convolutional neural nets for the localization of glomeruli in healthy and damaged trichrome-stained whole-renal section mounts and lays the groundwork for automated glomerular injury scoring.
Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we formulate the semantic segmentation problem and define the terminology of this field as well as interesting background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and goals. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. We also devote a part of the paper to review common loss functions and error metrics for this problem. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif- ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implemen- tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called dropout that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry
Immunohistochemical (IHC) biomarkers in breast tissue microarray (TMA) samples are used daily in pathology departments. In recent years, automatic methods to evaluate positive staining have been investigated since they may save time and reduce errors in the diagnosis. These errors are mostly due to subjective evaluation. The aim of this work is to develop a density tool able to automatically quantify the positive brown IHC stain in breast TMA for different biomarkers. To avoid the problem of colour variation and make a robust tool independent of the staining process, several colour standardization methods have been analysed. Four colour standardization methods have been compared against colour model segmentation. The standardization methods have been compared by means of NBS colour distance. The use of colour standardization helps to reduce noise due to stain and histological sample preparation. However, the most reliable and robust results have been obtained by combining the HSV and RGB colour models for segmentation with the HSB channels. The segmentation provides three outputs based on three saturation values for weak, medium and strong staining. Each output image can be combined according to the type of biomarker staining. The results with 12 biomarkers were evaluated and compared to the segmentation and density calculation done by expert pathologists. The Hausdorff distance, sensitivity and specificity have been used to quantitative validate the results. The tests carried out with 8000 TMA images provided an average of 95.94% accuracy applied to the total tissue cylinder area. Colour standardization was used only when the tissue core had blurring and fading stain and the expert could not evaluate them without a pre-processing.
To make reliable diagnosis, pathologists often need to identify certain special regions in medical images. In inflammatory bowel disease (IBD) diagnosis via histology tissue image examination, muscle regions are known to have no immune cell infiltration, and thus are ignored by pathologists. Also, messy regions (e.g., due to distortion and poor staining) are low in diagnostic yield. Hence, excluding muscle and messy regions to focus on vital regions is crucial for accurate diagnosis of IBD. In this paper, we propose a novel deep neural network based on fully convolutional networks (FCN) to identify muscle and messy regions, in an end-to-end fashion. First, we address the challenge of having limited medical training data, for training our deep neural network (a common problem for medical image processing, which may impede the application of the powerful deep learning method). Second, to deal with target regions of largely different sizes and arbitrary shapes, our deep neural network explores multi-scale information and structural information. Experimental results on clinical images show that our approach outperforms the state-of-the-art FCN for semantic segmentation of muscle and messy regions. Our approach may be readily extended to identify other types of regions in a variety of medical imaging applications.
Although transient vestigial excretory organs (the pro- and mesonephros) are recognizable in the human embryo, the definitive kidney is the meta- nephros. This is formed in two parts: the nephrons from the nephrogenic cord and the excretory ducts (collecting tubules, calyces, pelvis and ureter) from the ureteric bud which grows as a branch from the caudal portion of the mesonephric (Wolffian) duct. During early development the ureteric bud grows cranially and impinges on the caudal end of the nephrogenic cord (called the metanephric blastema) where it begins a process of rapid dichotomous branching. The first few generations of branches coalesce to form the renal pelvis and calyces. As further branching occurs, condensations of meta-nephric blastema, from which the nephrons develop, become related to the dilated tip, or ampulla, of each branch of the ureteric bud (Figure 2.1). As the nephrons form they become attached to the ampullae which in turn develop into collecting ducts. Attachment to the growing tip of the ureteric bud branches ensures that the nephrons are carried outwards as they develop.
Histogram Matching (HM) is a common technique for finding a monotonic map between two histograms. However, HM cannot deal with cases where a single mapping is sought between two sets of histograms. This paper presents a novel technique that finds such a mapping in an optimal manner under various histograms distance measures.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make train-ing faster, we used non-saturating neurons and a very efficient GPU implemen-tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
- T Kato
- R Relator
- H Ngouv
- Y Hirohashi
- O Takaki
- T Kakimoto
- K Okada
T. Kato, R. Relator, H. Ngouv, Y. Hirohashi, O. Takaki, T. Kakimoto, K. Okada,
Segmental hog: new descriptor for glomerulus detection in kidney microscopy
image, BMC Bioinform. 16 (316) (2015) 1-16.