No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Dual-path Network with Synergistic Grouping Loss and Evidence Driven Risk Stratification for Whole Slide Cervical Image Analysis
Abstract
Cervical cancer has been one of the most lethal cancers threatening women’s health. Nevertheless, the incidence of cervical cancer can be effectively minimized with preventive clinical management strategies, including vaccines and regular screening examinations. Screening cervical smears under microscope by cytologist is a widely used routine in regular examination, which consumes cytologists’ large amount of time and labour. Computerized cytology analysis appropriately caters to such an imperative need, which alleviates cytologists’ workload and reduce potential misdiagnosis rate. However, automatic analysis of cervical smear via digitalized whole slide images (WSIs) remains a challenging problem, due to the extreme huge image resolution, existence of tiny lesions, noisy dataset and intricate clinical definition of classes with fuzzy boundaries. In this paper, we design an efficient deep convolutional neural network (CNN) with dual-path (DP) encoder for lesion retrieval, which ensures the inference efficiency and the sensitivity on both tiny and large lesions. Incorporated with synergistic grouping loss (SGL), the network can be effectively trained on noisy dataset with fuzzy inter-class boundaries. Inspired by the clinical diagnostic criteria from the cytologists, a novel smear-level classifier, i.e., rule-based risk stratification (RRS), is proposed for accurate smear-level classification and risk stratification, which aligns reasonably with intricate cytological definition of the classes. Extensive experiments on the largest dataset including 19,303 WSIs from multiple medical centers validate the robustness of our method. With high sensitivity of 0.907 and specificity of 0.80 being achieved, our method manifests the potential to reduce the workload for cytologists in the routine practice.
... First, cell-level identification is inherently challenging due to cytomorphology similarities with different cell types (squamous and glandular), locations (superficial, intermediate, and basal), and neoplasia (metaplastic, koilocytotic, dyskeratotic) [10]. Second, each specimen typically contains 20,000 to 50,000 cells with sparsely distributed lesion cells [11], and digitized slides can measure up to 100,000 × 100,000 pixels. Therefore, examining cytology specimens is tedious and time-consuming for cytologists. ...
... Unlike histology, where tissues are continuous and regional, cytology involves discrete cell objects and scattered diagnostic clues, necessitating tailored WSI analysis approaches [19]. Early efforts, such as the dual-path network for cell-level prediction and rule-based WSI classifiers [11], laid the foundation in this field. Following this, subsequent studies incorporated more specialized knowledge to provide enriched cytology characteristics and guidance, thereby improving screening performance. ...
... These methods generally follow a two-step CCS scheme: abnormal cell detection, where high-confidence candidates are identified, and slide-level aggregation for final prediction, illustrated in Extended Data Fig. 1(c). While effective on high-quality data [11,21], this CCS scheme faces significant challenges in real-world scenarios. Variations in patient demographics, sample preparation, and staining protocols across institutions often result in inconsistencies between training data and clinical evaluation, leading to a dramatic decline in screening performance [20,22]. ...
Cervical cancer is a leading malignancy in female reproductive system. While AI-assisted cytology offers a cost-effective and non-invasive screening solution, current systems struggle with generalizability in complex clinical scenarios. To address this issue, we introduced Smart-CCS, a generalizable Cervical Cancer Screening paradigm based on pretraining and adaptation to create robust and generalizable screening systems. To develop and validate Smart-CCS, we first curated a large-scale, multi-center dataset named CCS-127K, which comprises a total of 127,471 cervical cytology whole-slide images collected from 48 medical centers. By leveraging large-scale self-supervised pretraining, our CCS models are equipped with strong generalization capability, potentially generalizing across diverse scenarios. Then, we incorporated test-time adaptation to specifically optimize the trained CCS model for complex clinical settings, which adapts and refines predictions, improving real-world applicability. We conducted large-scale system evaluation among various cohorts. In retrospective cohorts, Smart-CCS achieved an overall area under the curve (AUC) value of 0.965 and sensitivity of 0.913 for cancer screening on 11 internal test datasets. In external testing, system performance maintained high at 0.950 AUC across 6 independent test datasets. In prospective cohorts, our Smart-CCS achieved AUCs of 0.947, 0.924, and 0.986 in three prospective centers, respectively. Moreover, the system demonstrated superior sensitivity in diagnosing cervical cancer, confirming the accuracy of our cancer screening results by using histology findings for validation. Interpretability analysis with cell and slide predictions further indicated that the system's decision-making aligns with clinical practice. Smart-CCS represents a significant advancement in cancer screening across diverse clinical contexts.
... Different labels assigned to similar samples can greatly affect the performance of the classification model. Lin et al. 37 used a synergistic grouping method to mitigate the impact of noisy samples on the network, but they did not specifically address the treatment of confusable samples. Unsupervised learning does not require samples to have fixed labels and can be used to remove the influence of confusable samples on the classification network. ...
... To address the issue of confusion between different categories of cervical cells, researchers have proposed synergistic grouping loss (SGL) 37 . SGL is specifically designed for multi-class tasks with potential confusion, especially when the distances between different categories of samples in cervical cell interpretation are anisotropic. ...
... To further validate the proposed learnable loss function in this paper, we compared WSGLoss with Cross-Entropy Loss (CELoss) and SGL (SGLoss) 37 . This experiment computed the classification accuracy for each class, which shown as Table 6. ...
Cervical cancer is one of the deadliest cancers that pose a significant threat to women’s health. Early detection and treatment are commonly used methods to prevent cervical cancer. The use of pathological image analysis techniques for the automatic interpretation of cervical cells in pathological slides is a prominent area of research in the field of digital medicine. According to The Bethesda System, cervical cytology necessitates further classification of precancerous lesions based on positive interpretations. However, clinical definitions among different categories of lesion are complex and often characterized by fuzzy boundaries. In addition, pathologists can deduce different criteria for judgment based on The Bethesda System, leading to potential confusion during data labeling. Noisy labels due to this reason are a great challenge for supervised learning. To address the problem caused by noisy labels, we propose a method based on label credibility correction for cervical cell images classification network. Firstly, a contrastive learning network is used to extract discriminative features from cell images to obtain more similar intra-class sample features. Subsequently, these features are fed into an unsupervised method for clustering, resulting in unsupervised class labels. Then unsupervised labels are corresponded to the true labels to separate confusable and typical samples. Through a similarity comparison between the cluster samples and the statistical feature centers of each class, the label credibility analysis is carried out to group labels. Finally, a cervical cell images multi-class network is trained using synergistic grouping method. In order to enhance the stability of the classification model, momentum is incorporated into the synergistic grouping loss. Experimental validation is conducted on a dataset comprising approximately 60,000 cells from multiple hospitals, showcasing the effectiveness of our proposed approach. The method achieves 2-class task accuracy of 0.9241 and 5-class task accuracy of 0.8598. Our proposed method achieves better performance than existing classification networks on cervical cancer.
... Recognizing this potential, Lin et al. [8] proposed DPNet, which utilizes instance-level annotations along with a hierarchical grouping loss in the instance detector and a rulebased classifier for slide-level predictions. Gao et al. [9] leverage information bottleneck theory to model pathologist-selected instances with hierarchical features within a multitask framework, which employs an auxiliary instance-level classifier to enrich the feature representation for slide-level classification. ...
... For example, Mercan et al. [7] employed a traditional max-pooling MIL approach constrained by a multi-label loss for breast cancer WSI classification, where the instances were regions of interest identified by pathologists. Lin et al. [8] explored cervical cancer screening on cytology WSIs using DPNet based on VGG-16 [46]. A hierarchical grouping loss is proposed for suspicious cell detection, and the detected instances were aggregated with fixed clinical rules at the bag level. ...
... Each hierarchy contains K c and K f classes. Under this setting, existing methods [8], [9] primarily leverage this mapping M at the instance level and requires instance annotation. In response, we advocate for using this mapping at both the instance and bag levels, while exploring the model's capability without instance annotations. ...
Fine-grained classification of whole slide images (WSIs) is essential in precision oncology, enabling precise cancer diagnosis and personalized treatment strategies. The core of this task involves distinguishing subtle morphological variations within the same broad category of gigapixel-resolution images, which presents a significant challenge. While the multi-instance learning (MIL) paradigm alleviates the computational burden of WSIs, existing MIL methods often overlook hierarchical label correlations, treating fine-grained classification as a flat multi-class classification task. To overcome these limitations, we introduce a novel hierarchical multi-instance learning (HMIL) framework. By facilitating on the hierarchical alignment of inherent relationships between different hierarchy of labels at instance and bag level, our approach provides a more structured and informative learning process. Specifically, HMIL incorporates a class-wise attention mechanism that aligns hierarchical information at both the instance and bag levels. Furthermore, we introduce supervised contrastive learning to enhance the discriminative capability for fine-grained classification and a curriculum-based dynamic weighting module to adaptively balance the hierarchical feature during training. Extensive experiments on our large-scale cytology cervical cancer (CCC) dataset and two public histology datasets, BRACS and PANDA, demonstrate the state-of-the-art class-wise and overall performance of our HMIL framework. Our source code is available at https://github.com/ChengJin-git/HMIL.
... To combat the challenge, our team collecting 139 cytopathology whole slide images (WSI) with our own designed endometrial sampling device Li Brush (20152660054, Xi'an Meijiajia Medical Technology Co., Ltd., China). Among them, 39 WSIs are papanicolaou stained, and 100 WSIs are hematoxylin and eosin (H&E) stained. These WSIs are annotated by two cytopathologists, thus building a dataset for cytological screening of endometrial cancer. ...
... Here we propose an innovative two-stage framework for endometrial cancer screening. In this study, we found that the staining styles of slides was performed differently in different medical centers [39]. Some endometrial samples were stained with H&E, while others were stained with papanicolaou. ...
... Some endometrial samples were stained with H&E, while others were stained with papanicolaou. In addition, the stained slides can also be highly variable due to the preservation environment, changes in the scanner, etc [39]. This can affect the final diagnosis results [40,41]. ...
Endometrial cancer screening is crucial for clinical treatment. Currently, cytopathologists analyze cytopathology images is considered a popular screening method, but manual diagnosis is time-consuming and laborious. Deep learning can provide objective guidance efficiency. But endometrial cytopathology images often come from different medical centers with different staining styles. It decreases the generalization ability of deep learning models in cytopathology images analysis, leading to poor performance. This study presents a robust automated screening framework for endometrial cancer that can be applied to cytopathology images with different staining styles, and provide an objective diagnostic reference for cytopathologists, thus contributing to clinical treatment. We collected and built the XJTU-EC dataset, the first cytopathology dataset that includes segmentation and classification labels. And we propose an efficient two-stage framework for adapting different staining style images, and screening endometrial cancer at the cellular level. Specifically, in the first stage, a novel CM-UNet is utilized to segment cell clumps, with a channel attention (CA) module and a multi-level semantic supervision (MSS) module. It can ignore staining variance and focus on extracting semantic information for segmentation. In the second stage, we propose a robust and effective classification algorithm based on contrastive learning, ECRNet. By momentum-based updating and adding labeled memory banks, it can reduce most of the false negative results. On the XJTU-EC dataset, CM-UNet achieves an excellent segmentation performance, and ECRNet obtains an accuracy of 98.50%, a precision of 99.32% and a sensitivity of 97.67% on the test set, which outperforms other competitive classical models. Our method robustly predicts endometrial cancer on cytopathologic images with different staining styles, which will further advance research in endometrial cancer screening and provide early diagnosis for patients. The code will be available on GitHub.
... Both cell-level features and patch-level features are crucial for the final slide-level prediction, as shown in Fig. 18. It was not until 2021 that automated cervical cytology screening entered the thorough WIS analysis stage with the presence of the first DL-based WSI analysis methods in cervical cytology screening [140]. In the past two years, several DL-based WSI analysis method for cervical cytology successively emerged. ...
... Lin et al. [140] presented the first work for the specific analysis of cervical whole slide images. Firstly, an efficient deep learning-based dual-path network (DP-Net) was designed for lesion detection. ...
... AIATBS system was validated at 11 medical centers, and the outstanding performance demonstrated its adoption applicability and robustness for routine assistive diagnostic screening which could reduce the workload of cytologists, and improve the accuracy of cervical cancer screening. Fig. 19 First WSI analysis framework in cervical cytology screening, DP-Net with synergistic grouping loss and rule-based risk stratification [140]. ...
Cervical cancer is one of the most common cancers in daily life. Early detection and diagnosis can effectively help facilitate subsequent clinical treatment and management. With the growing advancement of artificial intelligence (AI) and deep learning (DL) techniques, an increasing number of computer-aided diagnosis (CAD) methods based on deep learning have been applied in cervical cytology screening. In this paper, we survey more than 70 publications since 2016 to provide a systematic and comprehensive review of DL-based cervical cytology screening. First, we provide a concise summary of the medical and biological knowledge pertaining to cervical cytology, since we hold a firm belief that a comprehensive biomedical understanding can significantly contribute to the development of CAD systems. Then, we collect a wide range of public cervical cytology datasets. Besides, image analysis approaches and applications including cervical cell identification, abnormal cell or area detection, cell region segmentation and cervical whole slide image diagnosis are summarized. Finally, we discuss the present obstacles and promising directions for future researches in automated cervical cytology screening.
... With the overwhelming success in the analysis of natural images, deep learning (DL) has also been applied to cervical cell segmentation (Zhou et al., 2020;Song et al., 2015;Liang et al., 2022) and classification (Zhang et al., 2017). Very recently the DL-based object detection methods (Ren et al., 2017;Lin et al., 2017Lin et al., , 2018Redmon and Farhadi, 2018;Tian et al., 2019) are applied to directly detect the abnormal cells from cervical cytology images in an end-to-end manner and have taken this domain by storm (Zhu et al., 2021a;Lin et al., 2021;Liang et al., 2021a,b). ...
... As shown in Fig. 1, the cell in the red box (cell 1) can be identified as abnormal ('ASCUS') when referred to cell A, but as normal in comparison with cell B, namely the cervical cell classification can not just rely on the features extracted from the cell patch or the RoI. Existing methods (Zhu et al., 2021a;Lin et al., 2021;Liang et al., 2021a,b) often lack the feature interaction between cells, leading to the suboptimal classification performance. ...
... To employ the contextual information, Liang et al. (2021a) propose a global context-aware framework to reduce false positive predictions by introducing an extra image-level classification branch. Lin et al. (2021) present a DP-Net which concatenates both local cellular feature and global image feature for classification. Cao et al. (2021) add channel attention and spatial attention into Faster R-CNN to boost cell detection performance. ...
Cervical abnormal cell detection is a challenging task as the morphological differences between abnormal cells and normal cells are usually subtle. To determine whether a cervical cell is normal or abnormal, cytopathologists always take surrounding cells as references and make careful comparison to identify its abnormality. To mimic these clinical behaviors, we propose to explore contextual relationships to boost the performance of cervical abnormal cell detection. Specifically, both contextual relationships between cells and cell-to-global images are exploited to enhance features of each region of interest (RoI) proposals. Accordingly, two modules, termed as RoI-relationship attention module (RRAM) and global RoI attention module (GRAM) are developed and their combination strategies are also investigated. We setup strong baselines by using single-head or double-head Faster R-CNN with feature pyramid network (FPN) and integrate our RRAM and GRAM into them to validate the effectiveness of the proposed modules. Experiments conducted on a large cervical cell detection dataset consisting of 40,000 cytology images reveal that the introduction of RRAM and GRAM both achieves better average precision (AP) than the baseline methods. Moreover, when cascading RRAM and GRAM, our method outperforms the state-of-the-art (SOTA) methods. Furthermore, we also show the proposed feature enhancing scheme can facilitate the image-level and smear-level classification. The code and trained models are publicly available at https://github.com/CVIU-CSU/CR4CACD.
... Cytology screening, involving the identification of abnormal cells through the examination of thousands of cells under a microscope, is the primary approach for precancerous screening from Pap smears or liquid-based cytology specimens. However, a typical cytology test usually requires experienced cytologists to spend 5-10 minutes on analyzing cytology characteristics under a microscope to identify abnormal cells [10]. Computational cytology has made significant progress in accelerating this screening process [7]. ...
... Computational cytology has made significant progress in accelerating this screening process [7]. Cell detection is usually regarded as the prerequisite step for identifying suspicious cells throughout the entire process [5], [6], [10]. ...
Cytology screening from Papanicolaou (Pap) smears is a common and effective tool for the preventive clinical management of cervical cancer, where abnormal cell detection from whole slide images serves as the foundation for reporting cervical cytology. However, cervical cell detection remains challenging due to 1) hazily-defined cell types (e.g., ASC-US) with subtle morphological discrepancies caused by the dynamic cancerization process, i.e., cell class ambiguity, and 2) imbalanced class distributions of clinical data may cause missed detection, especially for minor categories, i.e., cell class imbalance. To this end, we propose a holistic and historical instance comparison approach for cervical cell detection. Specifically, we first develop a holistic instance comparison scheme enforcing both RoI-level and class-level cell discrimination. This coarse-to-fine cell comparison encourages the model to learn foreground-distinguishable and class-wise representations. To emphatically improve the distinguishability of minor classes, we then introduce a historical instance comparison scheme with a confident sample selection-based memory bank, which involves comparing current embeddings with historical embeddings for better cell instance discrimination. Extensive experiments and analysis on two large-scale cytology datasets including 42,592 and 114,513 cervical cells demonstrate the effectiveness of our method. The code is available at https://github.com/hjiangaz/HERO.
... Despite the advancements in cell-level and slide-level analysis, there remains a discernible gap between the two. To handle giga-pixel WSIs of cervical cytology containing tens of thousands of cells or instances (local cell images), researchers have adopted multi-stage or hybrid slide-level analysis methods based on large-number lesion cell annotations [25][26][27][28] . For instance, Cheng et al. 25 introduced a method that combines high-resolution (HR) and low-resolution (LR) images to recommend lesion cells from a WSI using CNNs, and classified a WSI by aggregating deep features of the recommended top 10 lesion instances through a recurrent neural network (using 79, 911 annotated lesion cells). ...
... For instance, Cheng et al. 25 introduced a method that combines high-resolution (HR) and low-resolution (LR) images to recommend lesion cells from a WSI using CNNs, and classified a WSI by aggregating deep features of the recommended top 10 lesion instances through a recurrent neural network (using 79, 911 annotated lesion cells). Lin et al. 26 designed a dual-path network employing a synergistic grouping loss function for lesion retrieval, and classified slides by aggregating the first stage predictions by rule-based risk stratification (using 202, 557 abnormal cell images). Cao et al. 27 proposed a patch-to-sample inference approach which trains patch-level classifier by a hard patch mining strategy and derives slide level diagnosis utilizing score embedding and token pooling techniques (using 25, 134 annotated lesion cells). ...
The insufficient coverage of cervical cytology screening in underdeveloped countries or remote areas is currently the bottleneck hurdle to its widespread implementation. Conventional centralized medical screening methods are heavily dependent on sizable, costly investments as well as sufficient qualified pathologists. In this paper, we have developed a cervical precancerous assisted-screening system for identifying high-risk squamous intraepithelial lesion (SIL) cases in regions with limited resources. This system utilizes a low-cost miniature microscope and a low-pathologist-reliance artificial intelligence algorithm. We design a low-cost compact microscope with pixel resolution about 0.87 mm/pixel for imaging cytology slides. To tackle the challenge of sparely-distributed lesion cells in cytology whole slide images (WSIs), we have developed a dual-stage slide classification model. In first stage, we train an instance-level classifier by self-supervised pretraining on large-number unlabeled cervical images and transfer learning on small-number labeled images, aiming to reduce negative cells within a slide. In the second stage, we employ our proposed Att-Transformer, which aggregates deep features extracted from the top 200 lesion probabilities instances, for slide-level classification. We train and validate our model on 3,510 low-resolution WSIs collected from four different centers, and evaluate our model on 364 slides from two external centers in remote areas, achieving AUC (area under receiver operating characteristic curve) of 0.87 and 0.89 respectively for screening high risk cases. We also evaluate it on new independent cohorts of 391 slides from the original four centers and achieve AUC of 0.89. Overall, all the results indicate that integration of our innovative algorithm together with the compact microscope represents a promising approach to cervical cytology precancerous screening for high-risk population in medical resource limited regions. This affordable and accessible screening is significant as it contributes towards the goal of eliminating cervical cancer worldwide.
... [100]. Therefore, several studies have addressed the subject of automatic cervical cancer diagnosis [64][65][66][67][68]74,75,80,. The investigations showed that AI-assisted methods were promising, and achieved a high sensitivity and specificity in clinical cervical cytological screening [66,126]. ...
... Pap smear testing is a fundamental procedure in protecting women from cervical cancer. However, the effort of a cytologist to detect morphologic changes in lesions with 20,000-50,000 cells on a single slide is tedious, arduous, and dependent on experience [67]. In a cross-sectional study by Wergeland Sørbye et al. [168], four pathologists at three hospitals in Norway evaluated one hundred Pap smears (20 cases normal, 20 cases LSIL, 20 cases HSIL, 20 cases atypical squamous cells of undetermined significance (ASC-US), and 20 high grade squamous intra-epithelial lesion (ASC-H)). ...
Objective:
The likelihood of timely treatment for cervical cancer increases with timely detection of abnormal cervical cells. Automated methods of detecting abnormal cervical cells were established because manual identification requires skilled pathologists and is time consuming and prone to error. The purpose of this systematic review is to evaluate the diagnostic performance of artificial intelligence (AI) technologies for the prediction, screening, and diagnosis of cervical cancer and pre-cancerous lesions.
Materials and methods:
Comprehensive searches were performed on three databases: Medline, Web of Science Core Collection (Indexes = SCI-EXPANDED, SSCI, A & HCI Timespan) and Scopus to find papers published until July 2022. Articles that applied any AI technique for the prediction, screening, and diagnosis of cervical cancer were included in the review. No time restriction was applied. Articles were searched, screened, incorporated, and analyzed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines.
Results:
The primary search yielded 2538 articles. After screening and evaluation of eligibility, 117 studies were incorporated in the review. AI techniques were found to play a significant role in screening systems for pre-cancerous and cancerous cervical lesions. The accuracy of the algorithms in predicting cervical cancer varied from 70% to 100%. AI techniques make a distinction between cancerous and normal Pap smears with 80-100% accuracy. AI is expected to serve as a practical tool for doctors in making accurate clinical diagnoses. The reported sensitivity and specificity of AI in colposcopy for the detection of CIN2+ were 71.9-98.22% and 51.8-96.2%, respectively.
Conclusion:
The present review highlights the acceptable performance of AI systems in the prediction, screening, or detection of cervical cancer and pre-cancerous lesions, especially when faced with a paucity of specialized centers or medical resources. In combination with human evaluation, AI could serve as a helpful tool in the interpretation of cervical smears or images.
... Clinically, cytopathologists typically compare the target cell with surrounding cells as a reference to determine whether it is normal or abnormal. Existing methods [11][12][13] lack feature interactions between cells, which can lead to suboptimal classification performance. ...
... With advancements in image processing methods and computational power, deep learning-based computer-aided diagnostic methods have gained widespread attention 7 . Presently, deep learning-based object detection algorithms 8-12 are applied to detect abnormal cells in cervical cytology images, achieving a series of remarkable results in this field [13][14][15] . ...
Cervical cancer poses a significant health risk to women. Deep learning methods can assist pathologists in quickly screening images of suspected lesion cells, greatly improving the efficiency of cervical cancer screening and diagnosis. However, existing deep learning methods rely solely on single-scale features and local spatial information, failing to effectively capture the subtle morphological differences between abnormal and normal cervical cells. To tackle this problem effectively, we propose a cervical cell detection method that utilizes multi-scale spatial information. This approach efficiently captures and integrates spatial information at different scales. Firstly, we design the Multi-Scale Spatial Information Augmentation Module (MSA), which captures global spatial information by introducing a multi-scale spatial information extraction branch during the feature extraction stage. Secondly, the Channel Attention Enhanced Module (CAE) is introduced to achieve channel-level weighted processing, dynamically optimizing each output feature using channel weights to focus on critical features. We use Sparse R-CNN as the baseline and integrate MSA and CAE into it. Experiments on the CDetector dataset achieved an Average Precision (AP) of 65.3%, outperforming the state-of-the-art (SOTA) methods.
... In contrast, ASC-H and HSIL, originating from basal squamous cells, are considered high-risk precancerous lesions. The characteristics of these two cell types show significant differences [31]. Based on this, we categorize precancerous cells into two subsets: epidermic cells and basal cells. ...
Cervical cancer is one of the most prevalent cancers among women, posing a significant threat to their health. Early screening can detect cervical precancerous lesions in a timely manner, thereby enabling the prevention or treatment of the disease. The use of pathological image analysis technology to automatically interpret cells in pathological slices is a hot topic in digital medicine research, as it can reduce the substantial effort required from pathologists to identify cells and can improve diagnostic efficiency and accuracy. Therefore, we propose a cervical cell detection network based on collecting prior knowledge and correcting confusing labels, called PGCC-Net. Specifically, we utilize clinical prior knowledge to break down the detection task into multiple sub-tasks for cell grouping detection, aiming to more effectively learn the specific structure of cells. Subsequently, we merge region proposals from grouping detection to achieve refined detection. In addition, according to the Bethesda system, clinical definitions among various categories of abnormal cervical cells are complex, and their boundaries are ambiguous. Differences in assessment criteria among pathologists result in ambiguously labeled cells, which poses a significant challenge for deep learning networks. To address this issue, we perform a labels correction module with feature similarity by constructing feature centers for typical cells in each category. Then, cells that are easily confused are mapped with these feature centers in order to update cells’ annotations. Accurate cell labeling greatly aids the classification head of the detection network. We conducted experimental validation on a public dataset of 7410 images and a private dataset of 13,526 images. The results indicate that our model outperforms the state-of-the-art cervical cell detection methods.
... Their model achieved an impressive accuracy of 99.63% on single-cell images and 96.37% on cell clusters within entire images. Lin et al. [21] introduced a deep convolutional neural network-based lesion detection system, optimized for high efficiency and sensitivity across various lesion sizes. CNN-based image segmentation algorithms show high efficiency in detecting tumors in pathological images. ...
In the intelligent medical diagnosis area, Artificial Intelligence (AI)’s trustworthiness, reliability, and interpretability are critical, especially in cancer diagnosis. Traditional neural networks, while excellent at processing natural images, often lack interpretability and adaptability when processing high-resolution digital pathological images. This limitation is particularly evident in pathological diagnosis, which is the gold standard of cancer diagnosis and relies on a pathologist’s careful examination and analysis of digital pathological slides to identify the features and progression of the disease. Therefore, the integration of interpretable AI into smart medical diagnosis is not only an inevitable technological trend but also a key to improving diagnostic accuracy and reliability. In this paper, we introduce an innovative Multi-Scale Multi-Branch Feature Encoder (MSBE) and present the design of the CrossLinkNet Framework. The MSBE enhances the network’s capability for feature extraction by allowing the adjustment of hyperparameters to configure the number of branches and modules. The CrossLinkNet Framework, serving as a versatile image segmentation network architecture, employs cross-layer encoder-decoder connections for multi-level feature fusion, thereby enhancing feature integration and segmentation accuracy. Comprehensive quantitative and qualitative experiments on two datasets demonstrate that CrossLinkNet, equipped with the MSBE encoder, not only achieves accurate segmentation results but is also adaptable to various tumor segmentation tasks and scenarios by replacing different feature encoders. Crucially, CrossLinkNet emphasizes the interpretability of the AI model, a crucial aspect for medical professionals, providing an in-depth understanding of the model’s decisions and thereby enhancing trust and reliability in AI-assisted diagnostics.
... The widespread application of cytology screening in recent decades has been proven essential for the early detection and timely treatment of cervical cancer [2]. However, screening cervical smears under a microscope by cytologists consumes a significant amount of time and labor [3], and the accuracy of cervical With the development of deep learning, convolutional neural networks (CNNs) are widely used to recognize cervical cytopathology images. Compared with traditional methods, CNNs can automatically extract features and learn mappings in an end-to-end way. ...
Cervical cancer is a major global health issue, particularly in developing countries where access to healthcare is limited. Early detection of pre-cancerous lesions is crucial for successful treatment and reducing mortality rates. However, traditional screening and diagnostic processes require cytopathology doctors to manually interpret a huge number of cells, which is time-consuming, costly, and prone to human experiences. In this paper, we propose a Multi-scale Window Transformer (MWT) for cervical cytopathology image recognition. We design multi-scale window multi-head self-attention (MW-MSA) to simultaneously integrate cell features of different scales. Small window self-attention is used to extract local cell detail features, and large window self-attention aims to integrate features from smaller-scale window attention to achieve window-to-window information interaction. Our design enables long-range feature integration but avoids whole image self-attention (SA) in ViT or twice local window SA in Swin Transformer. We find convolutional feed-forward networks (CFFN) are more efficient than original MLP-based FFN for representing cytopathology images. Our overall model adopts a pyramid architecture. We establish two multi-center cervical cell classification datasets of two-category 192,123 images and four-category 174,138 images. Extensive experiments demonstrate that our MWT outperforms state-of-the-art general classification networks and specialized classifiers for cytopathology images in the internal and external test sets. The results on large-scale datasets prove the effectiveness and generalization of our proposed model. Our work provides a reliable cytopathology image recognition method and helps establish computer-aided screening for cervical cancer. Our code is available at https://github.com/nmyz669/MWT, and our web service tool can be accessed at https://huggingface.co/spaces/nmyz/MWTdemo.
... In practical applications, the cervical cytology AI system needs to provide a WSI-level classi cation that considers all abnormal cells across the entire slide. Existing methods based on Convolutional Neural Networks (CNNs) have primarily focused on identifying local lesion cells 30 , thus the challenge of WSIlevel analysis has not been adequately addressed 31,32 . One of the key challenges in implementing AIbased CCa screening solutions is the generalization of the model in real-world scenario. ...
Cervical cancer is a major health concern for women worldwide, and cervical cytology screening is a widely used and effective technique for early detection. In this study, we built a large-scale database of digital WSIs from 49 hospitals in China, comprising of 76,614 WSIs with 3,435,463 cell-level annotations by 26 cytopathologists using manual and semi-automatic approaches. A novel AI diagnostic system called CCA-DIAG was developed for cervical cancer screening based on a hybrid machine learning framework, which is capable of efficient WSI-level classification for various sedimentations. Our results of multi-center validation show that the system can make classifications at the WSI-level with high sensitivity (ASCUS+:0.89, LSIL+:0.99) for diverse sedimentations and significantly improve the time efficiency of cytopathologists by approximately 4 times. These findings suggest that CCA-DIAG is a promising tool for cervical cancer screening and could potentially improve diagnosis accuracy and efficiency in clinical practice.
... Computational techniques enable efficient and accurate characterization of cells from cytology images [12,16]. Among all computational techniques, cell segmentation has been a fundamental and widely-studied task, since the acquisition of cell-level identification is a pre-requisition for further assessment and analysis [3,23]. * Figure 1. ...
Cell instance segmentation in cytology images has significant importance for biology analysis and cancer screening, while remains challenging due to 1) the extensive overlapping translucent cell clusters that cause the ambiguous boundaries, and 2) the confusion of mimics and debris as nuclei. In this work, we proposed a De-overlapping Network (DoNet) in a decompose-and-recombined strategy. A Dual-path Region Segmentation Module (DRM) explicitly decomposes the cell clusters into intersection and complement regions, followed by a Semantic Consistency-guided Recombination Module (CRM) for integration. To further introduce the containment relationship of the nucleus in the cytoplasm, we design a Mask-guided Region Proposal Strategy (MRP) that integrates the cell attention maps for inner-cell instance prediction. We validate the proposed approach on ISBI2014 and CPS datasets. Experiments show that our proposed DoNet significantly outperforms other state-of-the-art (SOTA) cell instance segmentation methods. The code is available at https://github.com/DeepDoNet/DoNet.
... Importantly, a routine scanning of LBC slides in a single layer of WSIs would be suitable for further high throughput analysis (e.g., automated image based cytological screening and medical image analysis) [20]. Indeed, deep learning approaches and its clinical application to classify cytopathological changes (e.g., neoplastic transformation) were reported in the recent years [32][33][34][35][36][37][38][39][40][41]. ...
Simple Summary
In this study, we aimed to investigate the use of deep learning for classifying whole-slide images of urine liquid-based cytology specimens into neoplastic and non-neoplastic (negative). To do so, we used a total of 786 whole-slide images to train models using four different approaches, and we evaluated them on 750 whole-slide images. The best model achieved good classification performance, demonstrating the promising potential of use of such models for aiding the screening process for urothelial carcinoma in routine clinical practices.
Abstract
Urinary cytology is a useful, essential diagnostic method in routine urological clinical practice. Liquid-based cytology (LBC) for urothelial carcinoma screening is commonly used in the routine clinical cytodiagnosis because of its high cellular yields. Since conventional screening processes by cytoscreeners and cytopathologists using microscopes is limited in terms of human resources, it is important to integrate new deep learning methods that can automatically and rapidly diagnose a large amount of specimens without delay. The goal of this study was to investigate the use of deep learning models for the classification of urine LBC whole-slide images (WSIs) into neoplastic and non-neoplastic (negative). We trained deep learning models using 786 WSIs by transfer learning, fully supervised, and weakly supervised learning approaches. We evaluated the trained models on two test sets, one of which was representative of the clinical distribution of neoplastic cases, with a combined total of 750 WSIs, achieving an area under the curve for diagnosis in the range of 0.984–0.990 by the best model, demonstrating the promising potential use of our model for aiding urine cytodiagnostic processes.
... However, current CNN-based computer-assisted diagnostic methods are dedicated to the recognition of local lesion cells and do not include whole slide image (WSI)-level diagnostic analysis. There exist a few approaches (Holmström et al., 2021;Lin et al., 2021) for analyzing whole-slide cervical cancer images on large-scale datasets, but they have yet to tackle the challenge of generalization in real-world applications along with clinical-level verification. Existing publicly available datasets, such as Herlev (Jantzen et al., 2005), ISBI14 (Lu et al., 2015), ISBI15 (Lu et al., 2016), and CERVIX93 (Phoulady & Mouton, 2018), are insufficient in terms of image volume and the number of annotations. ...
Cervical cancer affects around 0.5 million women per year, resulting in over 0.3 million fatalities. Therefore, repetitive screening for cervical cancer is of utmost importance. Computer-assisted diagnosis is key for scaling up cervical cancer screening. Current recognition algorithms, however, perform poorly on the whole-slide image (WSI) analysis, fail to generalize for different staining methods and on uneven distribution for subtype imaging, and provide sub-optimal clinical-level interpretations. Herein, we developed CervixFormer—an end-to-end, transformer-based adversarial ensemble learning framework to assess pre-cancerous and cancer-specific cervical malignant lesions on WSIs. The proposed framework consists of (1) a self-attention generative adversarial network (SAGAN) for generating synthetic images during patch-level training to address the class imbalanced problems; (2) a multi-scale transformer-based ensemble learning method for cell identification at various stages, including atypical squamous cells (ASC-H) and atypical squamous cells of undetermined significance (ASCUS), which have not been demonstrated in previous studies; and (3) a fusion model for concatenating ensemble-based results and producing final outcomes. In the evaluation, the proposed method is first evaluated on a private dataset of 717 annotated samples from six classes, obtaining a high recall and precision of 0.940 and 0.934, respectively, in roughly 1.2 minutes. To further examine the generalizability of CervixFormer, we evaluated it on four independent, publicly available datasets, namely, the CRIC cervix, Mendeley LBC, SIPaKMeD Pap Smear, and Cervix93 Extended Depth of Field (EDF) image datasets. CervixFormer obtained a fairly better performance on two-, three-, four-, and six-class classification of smear- and cell-level datasets. For clinical interpretation, we used GradCAM to visualize a coarse localization map, highlighting important regions in the WSI. Notably, the proposed framework extracts features mostly from the cell nucleus and partially from the cytoplasm. In comparison with the existing state-of-the-art benchmark methods, the proposed framework outperforms them in terms of recall, accuracy, and computing time.
... Importantly, a routine scanning of LBC slides in a single layer of WSIs would be suitable for further high throughput analysis (e.g., automated image based cytological screening and medical image analysis) [16]. Indeed, deep learning approaches and its clinical application to classify cytopathological changes (e.g., neoplastic transformation) were reported in the recent years [17][18][19][20][21][22][23][24][25][26]. In this study, we trained deep learning models based on convolutional neural networks (CNN) using a training dataset of 786 urine LBC (ThinPrep) WSIs. ...
Urinary cytology is a useful, essential diagnostic method in routine urological clinical practice. Liquid-based cytology (LBC) for urothelial carcinoma screening is commonly used in the routine clinical cytodiagnosis because of its high cell collection rate. Since conventional screening processes by cytoscreeners and cytopathologists using microscopes is limited in terms of human resources, it is important to integrate new deep learning methods that can automatically and rapidly diagnose a large amount of specimens without delay. The goal of this study was to investigate the use of deep learning models for the classification of urine LBC whole-slide images (WSIs) into neoplastic and non-neoplastic (negative). We trained deep learning models using 786 WSIs by transfer learning, fully supervised, and weakly supervised learning approaches. We evaluated the trained models on two test sets (equal and clinical balance) with a combined total of 750 WSIs, achieving ROC-AUCs for WSI diagnosis in the range of 0.984-0.990 by the best model, demonstrating the promising potential use of our model for aiding urine cytodiagnostic processes.
... However, far too little attention has been paid to the occurrence of cervical cancer that can be effectively reduced with preventive clinical management strategies, including vaccines and regular screening examinations [11]. It has previously been observed that early diagnosis and classification of cervical lesions greatly boost the chance of successful treatments of patients [12]. ...
Cervical cancer is a prevalent and deadly cancer that affects women all over the world. It affects about 0.5 million women anually and results in over 0.3 million fatalities. Diagnosis of this cancer was previously done manually, which could result in false positives or negatives. The researchers are still contemplating how to detect cervical cancer automatically and how to evaluate Pap smear images. Hence, this paper has reviewed several detection methods from the previous researches that has been done before. This paper reviews pre-processing, detection method framework for nucleus detection, and analysis performance of the method selected. There are four methods based on a reviewed technique from previous studies that have been running through the experimental procedure using Matlab, and the dataset used is established Herlev Dataset. The results show that the highest performance assessment metric values obtain from Method 1: Thresholding and Trace region boundaries in a binary image with the values of precision 1.0, sensitivity 98.77%, specificity 98.76%, accuracy 98.77% and PSNR 25.74% for a single type of cell. Meanwhile, the average values of precision were 0.99, sensitivity 90.71%, specificity 96.55%, accuracy 92.91% and PSNR 16.22%. The experimental results are then compared to the existing methods from previous studies. They show that the improvement method is able to detect the nucleus of the cell with higher performance assessment values. On the other hand, the majority of current approaches can be used with either a single or a large number of cervical cancer smear images. This study might persuade other researchers to recognize the value of some of the existing detection techniques and offer a strong approach for developing and implementing new solutions.
Background
Cervical cytology screening and colposcopy play crucial roles in cervical intraepithelial neoplasia (CIN) and cervical cancer prevention. Previous studies have provided evidence that artificial intelligence (AI) has remarkable diagnostic accuracy in these procedures. With this systematic review and meta-analysis, we aimed to examine the pooled accuracy, sensitivity, and specificity of AI-assisted cervical cytology screening and colposcopy for cervical intraepithelial neoplasia and cervical cancer screening.
Methods
In this systematic review and meta-analysis, we searched the PubMed, Embase, and Cochrane Library databases for studies published between January 1, 1986 and August 31, 2024. Studies investigating the sensitivity and specificity of AI-assisted cervical cytology screening and colposcopy for histologically verified cervical intraepithelial neoplasia and cervical cancer and a minimum of five cases were included. The performance of AI and experienced colposcopists was assessed via the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) through random effect models. Additionally, subgroup analyses of multiple diagnostic performance metrics in developed and developing countries were conducted. This study was registered with PROSPERO (CRD42024534049).
Findings
Seventy-seven studies met the eligibility criteria for inclusion in this study. The pooled diagnostic parameters of AI-assisted cervical cytology via Papanicolaou (Pap) smears were as follows: accuracy, 94% (95% CI 92–96); sensitivity, 95% (95% CI 91–98); specificity, 94% (95% CI 89–97); PPV, 88% (95% CI 78–96); and NPV, 95% (95% CI 89–99). The pooled accuracy, sensitivity, specificity, PPV, and NPV of AI-assisted cervical cytology via ThinPrep cytologic test (TCT) were 90% (95% CI 85–94), 97% (95% CI 95–99), 94% (95% CI 85–98), 84% (95% CI 64–98), and 96% (95% CI 94–98), respectively. Subgroup analysis revealed that, for AI-assisted cervical cytology diagnosis, certain performance indicators were superior in developed countries compared to developing countries. Compared with experienced colposcopists, AI demonstrated superior accuracy in colposcopic examinations (odds ratio (OR) 1.75; 95% CI 1.33–2.31; P < 0.0001; I² = 93%).
Interpretation
These results underscore the potential and practical value of AI in preventing and enabling early diagnosis of cervical cancer. Further research should support the development of AI for cervical cancer screening, including in low- and middle-income countries with limited resources.
Funding
This study was supported by the 10.13039/501100001809National Natural Science Foundation of China (No. 81901493) and the Shanghai Pujiang Program (No. 21PJD006).
High-quality cytopathology images are the guarantee of cervical cancer computer-aided screening. However, obtaining such images is dependent on expensive devices, which hinders the screening popularization in less developed areas. In this study, we propose a convolutional window-integration Transformer for cytopathology image super-resolution (SR) of portable microscope. We use self-attention within the window to integrate patches, and then design a convolutional windowintegration feed-forward network with two 5×5 size kernels to achieve cross-window patch integration. This design avoids long-range self-attention and facilitates SR local mapping learning. Besides, we design a multi-layer feature fusion in feature extraction to enhance high-frequency details, achieving better SR reconstruction. Finally, we register and establish a dataset of 239,100 paired portable microscope images and standard microscope images based on feature point matching. A series of experiments demonstrate that our model has the minimum parameter number and outperforms state-of-the-art CNN-based and recent Transformer-based SR models with PSNR improvement of 0.09-0.53dB. We release this dataset and codes publicly to promote the development of computational cytopathology imaging
In this sophisticated world living with CIN cervic cancer is much jeopardy. Cancer is a stochastic (random) process. So, in that CIN in initial stage is not jeopardy. It is totally because of hazardous malign cells. Using its modality in image is selected in existing system only automated classification depends on input image. But in proposed methodology the innovative alludes the jeopardy of CIN cancer is found using the size of /area of nucleus or cytoplasm. This proposed methodology was developed with an algorithm to find CIN area/size. This research work establishes a Cervic Cancer Classification Using Contour Based on Area of Nucleolus and Cytoplasm in Cells (CBANC) which classifies noise spread images into any one of five phases. A similarity measure produces 90% efficiency in proposed system as par with inefficient existing system which fetches us 50%. By pragmatic application it is proved that CBANC with fuzzy is better than Baye’s. This can be accomplished by removing well distinct consistency features and choosing preeminent classifier. Proposed work can extend with 3D input images for future use. It produces mightiest parameter shape and intensity which is very essential for 3D approach. The inference of proposed system can extend the latest classifier engines for more accuracy. It can easily predict more than 90% accuracy will be there. And also derive cancer growing and after therapy for cancer shrinking algorithm will be used for 2D or 3D CIN cancer classification. The outcomes of the proposed methodology CBANC shows that better when compared to the existing methodology like Bayes. It can be implemented in the real world environments of the medical field.
Cervical cancer is a severe threat to women’s health. The majority of cervical cancer cases occur in developing countries. The WHO has proposed screening 70% of women with high-performance tests between 35 and 45 years of age by 2030 to accelerate the elimination of cervical cancer. Due to an inadequate health infrastructure and organized screening strategy, most low- and middle-income countries are still far from achieving this goal. As part of the efforts to increase performance of cervical cancer screening, it is necessary to investigate the most accurate, efficient, and effective methods and strategies. Artificial intelligence (AI) is rapidly expanding its application in cancer screening and diagnosis and deep learning algorithms have offered human-like interpretation capabilities on various medical images. AI will soon have a more significant role in improving the implementation of cervical cancer screening, management, and follow-up. This review aims to report the state of AI with respect to cervical cancer screening. We discuss the primary AI applications and development of AI technology for image recognition applied to detection of abnormal cytology and cervical neoplastic diseases, as well as the challenges that we anticipate in the future.
Objetivo: Revisar a aplicação da Inteligência Artificial (IA) no diagnóstico de câncer através de exames de imagem, analisando tendências e avanços recentes na intersecção entre IA e medicina diagnóstica. Métodos: Foi realizada uma pesquisa bibliográfica em bases de dados eletrônicas, selecionando artigos científicos, revisões sistemáticas e meta-análises publicados entre 2017 e 2023. Os estudos incluíram algoritmos de IA, técnicas de aprendizado de máquina e redes neurais aplicadas a exames de imagem para detecção, classificação e diagnóstico de câncer. Resultados: A análise focou em modelos de reconhecimento de imagem para diagnóstico de câncer, priorizando métricas de sensibilidade e especificidade. Foram destacados estudos que compararam o desempenho de radiologistas com sistemas de IA mostrando que em alguns casos a IA superou os profissionais e, em outros, melhorou significativamente o desempenho dos radiologistas quando usada como assistência. Considerações finais: A IA mostrou-se uma ferramenta promissora no diagnóstico de câncer por imagem. A combinação de IA com dados clínicos pode melhorar as métricas de diagnóstico. Limitações incluem a qualidade e quantidade de imagens para treinamento, mas novas tecnologias como IA Generativa estão surgindo para superar esses desafios.
Artificial intelligence (AI) and machine learning (ML) have profoundly changed the landscape of healthcare. In pathology, AI/ML technologies are now able to enhance diagnostic accuracy, increase screening accuracy, and improve workflow efficiency. Notably, six such devices have been approved by the FDA. However, challenges persist, such as ensuring high-quality image datasets, robust validation processes, and managing regulatory hurdles. In this review, we cover AI/ML developments in pathology across several areas of study, including the cancers of the thyroid, cervix, bladder pancreas, and lung. Future advancements are expected to integrate cytopathology with radiology and clinical data, creating more robust and effective AI/ML tools for pathologists.
Cervical cancer is one of the most common cancers in daily life. Early detection and diagnosis can effectively help facilitate subsequent clinical treatment and management. With the growing advancement of artificial intelligence (AI) and deep learning (DL) techniques, an increasing number of computer-aided diagnosis (CAD) methods based on deep learning have been applied in cervical cytology screening. In this paper, we survey more than 80 publications since 2016 to provide a systematic and comprehensive review of DL-based cervical cytology screening. First, we provide a concise summary of the medical and biological knowledge pertaining to cervical cytology, since we hold a firm belief that a comprehensive biomedical understanding can significantly contribute to the development of CAD systems. Then, we collect a wide range of public cervical cytology datasets. Besides, image analysis approaches and applications including cervical cell identification, abnormal cell or area detection, cell region segmentation and cervical whole slide image diagnosis are summarized. Finally, we discuss the present obstacles and promising directions for future research in automated cervical cytology screening.
Integration of whole slide imaging (WSI) and deep learning technology has led to significant improvements in the screening and diagnosis of cervical cancer. WSI enables the examination of all cells on a slide simultaneously and deep learning algorithms can accurately label them as cancerous or non-cancerous. Although many studies have investigated the application of deep learning for diagnosing various diseases, there is a lack of research focusing on the evolution, limitations, and gaps of intelligent algorithms in conjunction with WSI for cervical cancer. This paper provides a comprehensive overview of the state-of-the-art deep learning algorithms used for the timely and precise analysis of cervical WSI images. A total of 115 relevant papers were reviewed, and 37 were selected after screening with specific inclusion and exclusion criteria. Methodological aspects including deep learning techniques, data sources, architectures, and classification techniques employed by the selected studies were analyzed. The review presents the most popular techniques and current trends in deep learning-based cervical classification systems, and categorizes the evolution of the domain based on deep learning techniques, citing an in-depth analysis of various models developed over time. The paper advocates for the implementation of transfer supervised learning when utilizing deep learning models such as ResNet, VGG19, and EfficientNet, and builds a solid foundation for applying relevant techniques in different fields. Although some progress has been made in developing novel models for the diagnosis of cervical cancer, substantial work remains to be done in creating standardized benchmark databases of WSI images for the research community. This paper serves as a comprehensive guide for understanding the fundamental concepts, benefits, and challenges related to various deep learning models on WSI, including their application for cervical system classification. Additionally, it provides valuable insights into future research directions in this area.
Background and objectives:
Cervical cancer affects around 0.5 million women per year, resulting in over 0.3 million fatalities. Therefore, repetitive screening for cervical cancer is of utmost importance. Computer-assisted diagnosis is key for scaling up cervical cancer screening. Current recognition algorithms, however, perform poorly on the whole-slide image (WSI) analysis, fail to generalize for different staining methods and on uneven distribution for subtype imaging, and provide sub-optimal clinical-level interpretations. Herein, we developed CervixFormer-an end-to-end, multi-scale swin transformer-based adversarial ensemble learning framework to assess pre-cancerous and cancer-specific cervical malignant lesions on WSIs.
Methods:
The proposed framework consists of (1) a self-attention generative adversarial network (SAGAN) for generating synthetic images during patch-level training to address the class imbalanced problems; (2) a multi-scale transformer-based ensemble learning method for cell identification at various stages, including atypical squamous cells (ASC) and atypical squamous cells of undetermined significance (ASCUS), which have not been demonstrated in previous studies; and (3) a fusion model for concatenating ensemble-based results and producing final outcomes.
Results:
In the evaluation, the proposed method is first evaluated on a private dataset of 717 annotated samples from six classes, obtaining a high recall and precision of 0.940 and 0.934, respectively, in roughly 1.2 minutes. To further examine the generalizability of CervixFormer, we evaluated it on four independent, publicly available datasets, namely, the CRIC cervix, Mendeley LBC, SIPaKMeD Pap Smear, and Cervix93 Extended Depth of Field image datasets. CervixFormer obtained a fairly better performance on two-, three-, four-, and six-class classification of smear- and cell-level datasets. For clinical interpretation, we used GradCAM to visualize a coarse localization map, highlighting important regions in the WSI. Notably, CervixFormer extracts feature mostly from the cell nucleus and partially from the cytoplasm.
Conclusions:
In comparison with the existing state-of-the-art benchmark methods, the CervixFormer outperforms them in terms of recall, accuracy, and computing time.
Cervical abnormal cell detection is a challenging task as the morphological discrepancies between abnormal and normal cells are usually subtle. To determine whether a cervical cell is normal or abnormal, cytopathologists always take surrounding cells as references to identify its abnormality. To mimic these behaviors, we propose to explore contextual relationships to boost the performance of cervical abnormal cell detection. Specifically, both contextual relationships between cells and cell-to-global images are exploited to enhance features of each region of interest (RoI) proposal. Accordingly, two modules, dubbed as RoI-relationship attention module (RRAM) and global RoI attention module (GRAM), are developed and their combination strategies are also investigated. We establish a strong baseline by using Double-Head Faster R-CNN with a feature pyramid network (FPN) and integrate our RRAM and GRAM into it to validate the effectiveness of the proposed modules. Experiments conducted on a large cervical cell detection dataset reveal that the introduction of RRAM and GRAM both achieves better average precision (AP) than the baseline methods. Moreover, when cascading RRAM and GRAM, our method outperforms the state-of-the-art (SOTA) methods. Furthermore, we show that the proposed feature-enhancing scheme can facilitate image- and smear-level classification. The code and trained models are publicly available at
https://github.com/CVIU-CSU/CR4CACD
.
Objective:
Artificial intelligence (AI)-based cytopathology studies conducted using deep learning have enabled cell detection and classification. Liquid-based cytology (LBC) has facilitated the standardisation of specimen preparation; however, cytomorphology varies according to the LBC processing technique used. In this study, we elucidated the relationship between two LBC techniques and cell detection and classification using a deep learning model.
Methods:
Cytological specimens were prepared using the ThinPrep and SurePath methods. The accuracy of cell detection and cell classification was examined using the one- and five-cell models, which were trained with one and five cell types, respectively.
Results:
When the same LBC processing techniques were used for the training and detection preparations, the cell detection and classification rates were high. The model trained on ThinPrep preparations was more accurate than that trained on SurePath. When the preparation types used for training and detection were different, the accuracy of cell detection and classification was significantly reduced (P < 0.01). The model trained on both ThinPrep and SurePath preparations exhibited slightly reduced cell detection and classification rates but was highly accurate.
Conclusions:
For the two LBC processing techniques, cytomorphology varied according to cell type; this difference affects the accuracy of cell detection and classification by deep learning. Therefore, for highly accurate cell detection and classification using AI, the same processing technique must be used for both training and detection. Our assessment also suggests that a deep learning model should be constructed using specimens prepared via a variety of processing techniques to construct a globally applicable AI model.
Cervical cancer is still a public health scourge in the developing countries due to the lack of organized screening programs. Though liquid-based cytology methods improved the performance of cervical cytology, the interpretation still suffers from subjectivity. Artificial intelligence (AI) algorithms have offered objectivity leading to better sensitivity and specificity of cervical cancer screening. Whole slide imaging (WSI) that converts a glass slide to a virtual slide provides a new perspective to the application of AI, especially for cervical cytology. In the recent years, there have been a few studies employing various AI algorithms on WSI images of conventional or LBC smears and demonstrating differing sensitivity/specificity or accuracy at detection of abnormalities in cervical smears. Considering the interest in AI-based screening modalities, this well-timed review intends to summarize the progress in this field while highlighting the research gaps and providing future research directions.
Computational cytology is a critical, rapid-developing, yet challenging topic in medical image computing concerned with analyzing digitized cytology images by computer-aided technologies for cancer screening. Recently, an increasing number of deep learning (DL) approaches have made significant achievements in medical image analysis, leading to boosting publications of cytological studies. In this article, we survey more than 120 publications of DL-based cytology image analysis to investigate the advanced methods and comprehensive applications. We first introduce various deep learning schemes, including fully supervised, weakly supervised, unsupervised, and transfer learning. Then, we systematically summarize public datasets, evaluation metrics, versatile cytology image analysis applications including cell classification, slide-level cancer screening, nuclei or cell detection and segmentation. Finally, we discuss current challenges and potential research directions of computational cytology.
Purpose
Artificial Intelligence (AI) is an emerging technology and turned into a field of knowledge that has been consistently displacing technologies for a change in human life. It is applied in all spheres of life as reflected in the review of the literature section here. As applicable in the field of libraries too, this study scientifically mapped the papers on AAIL and analyze its growth, collaboration network, trending topics, or research hot spots to highlight the challenges and opportunities in adopting AI-based advancements in library systems and processes.
Design/methodology/approach
The study was developed with a bibliometric approach, considering a decade, 2012 to 2021 for data extraction from a premier database, Scopus . The steps followed are (1) identification, selection of keywords, and forming the search strategy with the approval of a panel of computer scientists and librarians and (2) design and development of a perfect algorithm to verify these selected keywords in title-abstract-keywords of Scopus (3) Performing data processing in some state-of-the-art bibliometric visualization tools, Biblioshiny R and VOSviewer (4) discussing the findings for practical implications of the study and limitations.
Findings
As evident from several papers, not much research has been conducted on AI applications in libraries in comparison to topics like AI applications in cancer, health, medicine, education, and agriculture. As per the Price law, the growth pattern is exponential. The total number of papers relevant to the subject is 1462 (single and multi-authored) contributed by 5400 authors with 0.271 documents per author and around 4 authors per document. Papers occurred mostly in open-access journals. The productive journal is the Journal of Chemical Information and Modelling (NP = 63) while the highly consistent and impactful is the Journal of Machine Learning Research (z-index=63.58 and CPP = 56.17). In the case of authors, J Chen (z-index=28.86 and CPP = 43.75) is the most consistent and impactful author. At the country level, the USA has recorded the highest number of papers positioned at the center of the co-authorship network but at the institutional level, China takes the 1st position. The trending topics of research are machine learning, large dataset, deep learning, high-level languages, etc. The present information system has a high potential to improve if integrated with AI technologies.
Practical implications
The number of scientific papers has increased over time. The evolution of themes like machine learning implicates AI as a broad field of knowledge that converges with other disciplines. The themes like large datasets imply that AI may be applied to analyze and interpret these data and support decision-making in public sector enterprises. Theme named high-level language emerged as a research hotspot which indicated that extensive research has been going on in this area to improve computer systems for facilitating the processing of data with high momentum. These implications are of high strategic worth for policymakers, library stakeholders, researchers and the government as a whole for decision-making.
Originality/value
The analysis of collaboration, prolific authors/journals using consistency factor and CPP, testing the relationship between consistency (z-index) and impact (h-index), using state-of-the-art network visualization and cluster analysis techniques make this study novel and differentiates it from the traditional bibliometric analysis. To the best of the author's knowledge, this work is the first attempt to comprehend the research streams and provide a holistic view of research on the application of AI in libraries. The insights obtained from this analysis are instrumental for both academics and practitioners.
Eyelid malignant melanoma (MM) is a rare disease with high mortality. Accurate diagnosis of such disease is important but challenging. In clinical practice, the diagnosis of MM is currently performed manually by pathologists, which is subjective and biased. Since the heavy manual annotation workload, most pathological whole slide image (WSI) datasets are only partially labeled (without region annotations), which cannot be directly used in supervised deep learning. For these reasons, it is of great practical significance to design a laborsaving and high data utilization diagnosis method. In this paper, a self-supervised learning (SSL) based framework for automatically detecting eyelid MM is proposed. The framework consists of a self-supervised model for detecting MM areas at the patch-level and a second model for classifying lesion types at the slide level. A squeeze-excitation (SE) attention structure and a feature-projection (FP) structure are integrated to boost learning on details of pathological images and improve model performance. In addition, this framework also provides visual heatmaps with high quality and reliability to highlight the likely areas of the lesion to assist the evaluation and diagnosis of the eyelid MM. Extensive experimental results on different datasets show that our proposed method outperforms other state-of-the-art SSL and fully supervised methods at both patch and slide levels when only a subset of WSIs are annotated. It should be noted that our method is even comparable to supervised methods when all WSIs are fully annotated. To the best of our knowledge, our work is the first SSL method for automatic diagnosis of MM at the eyelid and has a great potential impact on reducing the workload of human annotations in clinical practice.
Introduction:
Deep learning is a subset of machine learning that has contributed to significant changes in feature extraction and image classification and is being actively researched and developed in the field of cytopathology. Liquid-based cytology (LBC) enables standardized cytological preparation and is also applied to artificial intelligence (AI) research, but cytological features differ depending on the LBC preservative solution types. In this study, the relationship between cell detection by AI and the type of preservative solution used was examined.
Methods:
The specimens were prepared from five preservative solutions of LBC and stained using the Papanicolaou method. The YOLOv5 deep convolutional neural network algorithm was used to create a deep learning model for each specimen, and a BRCPT model from five specimens was also created. Each model was compared to the specimen types used for detection.
Results:
Among the six models, a difference in the detection rate of approximately 25% was observed depending on the detected specimen, and within specimens, a difference in the detection rate of approximately 20% was observed depending on the model. The BRCPT model had little variation in the detection rate depending on the type of the detected specimen.
Conclusions:
The same cells were treated with different preservative solutions, the cytologic features were different, and AI clarified the difference in cytologic features depending on the type of solution. The type of preservative solution used for training and detection had an extreme influence on cell detection using AI. Although the accuracy of the deep learning model is important, it is necessary to understand that cell morphology differs depending on the type of preservative solution, which is a factor affecting the detection rate of AI.
Purpose:
To evaluate the ability of fine-grained annotations to overcome shortcut learning in deep learning (DL)-based diagnosis using chest radiographs.
Materials and methods:
Two DL models were developed using radiograph-level annotations (disease present: yes or no) and fine-grained lesion-level annotations (lesion bounding boxes), respectively named CheXNet and CheXDet. A total of 34 501 chest radiographs obtained from January 2005 to September 2019 were retrospectively collected and annotated regarding cardiomegaly, pleural effusion, mass, nodule, pneumonia, pneumothorax, tuberculosis, fracture, and aortic calcification. The internal classification performance and lesion localization performance of the models were compared on a testing set (n = 2922); external classification performance was compared on National Institutes of Health (NIH) Google (n = 4376) and PadChest (n = 24 536) datasets; and external lesion localization performance was compared on the NIH ChestX-ray14 dataset (n = 880). The models were also compared with radiologist performance on a subset of the internal testing set (n = 496). Performance was evaluated using receiver operating characteristic (ROC) curve analysis.
Results:
Given sufficient training data, both models performed similarly to radiologists. CheXDet achieved significant improvement for external classification, such as classifying fracture on NIH Google (CheXDet area under the ROC curve [AUC], 0.67; CheXNet AUC, 0.51; P < .001) and PadChest (CheXDet AUC, 0.78; CheXNet AUC, 0.55; P < .001). CheXDet achieved higher lesion detection performance than CheXNet for most abnormalities on all datasets, such as detecting pneumothorax on the internal set (CheXDet jackknife alternative free-response ROC [JAFROC] figure of merit [FOM], 0.87; CheXNet JAFROC FOM, 0.13; P < .001) and NIH ChestX-ray14 (CheXDet JAFROC FOM, 0.55; CheXNet JAFROC FOM, 0.04; P < .001).
Conclusion:
Fine-grained annotations overcame shortcut learning and enabled DL models to identify correct lesion patterns, improving the generalizability of the models.Keywords: Computer-aided Diagnosis, Conventional Radiography, Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms, Localization Supplemental material is available for this article © RSNA, 2022.
Traditional image-based survival prediction models rely on discriminative patch labeling which make those methods not scalable to extend to large datasets. Recent studies have shown Multiple Instance Learning (MIL) framework is useful for histopathological images when no annotations are available in classification task. Different to the current image-based survival models that limit to key patches or clusters derived from Whole Slide Images (WSIs), we propose Deep Attention Multiple Instance Survival Learning (DeepAttnMISL) by introducing both siamese MI-FCN and attention-based MIL pooling to efficiently learn imaging features from the WSI and then aggregate WSI-level information to patient-level. Attention-based aggregation is more flexible and adaptive than aggregation techniques in recent survival models. We evaluated our methods on two large cancer whole slide images datasets and our results suggest that the proposed approach is more effective and suitable for large datasets and has better interpretability in locating important patterns and features that contribute to accurate cancer survival predictions. The proposed framework can also be used to assess individual patient’s risk and thus assisting in delivering personalized medicine.
This preprint provides information about multi-organ nuclei segmentation and classification challenge, which is an official satellite event of ISBI 2020. This document summarizes the challenge participation rules and provides detailed information about its training and testing datasets.
Background:
The knowledge that persistent human papillomavirus (HPV) infection is the main cause of cervical cancer has resulted in the development of prophylactic vaccines to prevent HPV infection and HPV assays that detect nucleic acids of the virus. WHO has launched a Global Initiative to scale up preventive, screening, and treatment interventions to eliminate cervical cancer as a public health problem during the 21st century. Therefore, our study aimed to assess the existing burden of cervical cancer as a baseline from which to assess the effect of this initiative.
Methods:
For this worldwide analysis, we used data of cancer estimates from 185 countries from the Global Cancer Observatory 2018 database. We used a hierarchy of methods dependent on the availability and quality of the source information from population-based cancer registries to estimate incidence of cervical cancer. For estimation of cervical cancer mortality, we used the WHO mortality database. Countries were grouped in 21 subcontinents and were also categorised as high-resource or lower-resource countries, on the basis of their Human Development Index. We calculated the number of cervical cancer cases and deaths in a given country, directly age-standardised incidence and mortality rate of cervical cancer, indirectly standardised incidence ratio and mortality ratio, cumulative incidence and mortality rate, and average age at diagnosis.
Findings:
Approximately 570 000 cases of cervical cancer and 311 000 deaths from the disease occurred in 2018. Cervical cancer was the fourth most common cancer in women, ranking after breast cancer (2·1 million cases), colorectal cancer (0·8 million) and lung cancer (0·7 million). The estimated age-standardised incidence of cervical cancer was 13·1 per 100 000 women globally and varied widely among countries, with rates ranging from less than 2 to 75 per 100 000 women. Cervical cancer was the leading cause of cancer-related death in women in eastern, western, middle, and southern Africa. The highest incidence was estimated in Eswatini, with approximately 6·5% of women developing cervical cancer before age 75 years. China and India together contributed more than a third of the global cervical burden, with 106 000 cases in China and 97 000 cases in India, and 48 000 deaths in China and 60 000 deaths in India. Globally, the average age at diagnosis of cervical cancer was 53 years, ranging from 44 years (Vanuatu) to 68 years (Singapore). The global average age at death from cervical cancer was 59 years, ranging from 45 years (Vanuatu) to 76 years (Martinique). Cervical cancer ranked in the top three cancers affecting women younger than 45 years in 146 (79%) of 185 countries assessed.
Interpretation:
Cervical cancer continues to be a major public health problem affecting middle-aged women, particularly in less-resourced countries. The global scale-up of HPV vaccination and HPV-based screening-including self-sampling-has potential to make cervical cancer a rare disease in the decades to come. Our study could help shape and monitor the initiative to eliminate cervical cancer as a major public health problem.
Funding:
Belgian Foundation Against Cancer, DG Research and Innovation of the European Commission, and The Bill & Melinda Gates Foundation.
Although convolutional neural networks have achieved tremendous success on histopathology image classification, they usually require large-scale clean annotated data and are sensitive to noisy labels. Unfortunately, labeling large-scale images is laborious, expensive and lowly reliable for pathologists. To address these problems, in this paper, we propose a novel self-ensembling based deep architecture to leverage the semantic information of annotated images and explore the information hidden in unlabeled data, and meanwhile being robust to noisy labels. Specifically, the proposed architecture first creates ensemble targets for feature and label predictions of training samples, by using exponential moving average (EMA) to aggregate feature and label predictions within multiple previous training epochs. Then, the ensemble targets within the same class are mapped into a cluster so that they are further enhanced. Next, a consistency cost is utilized to form consensus predictions under different configurations. Finally, we validate the proposed method with extensive experiments on lung and breast cancer datasets that contain thousands of images. It can achieve 90.5% and 89.5% image classification accuracy using only 20% labeled patients on the two datasets, respectively. This performance is comparable to that of the baseline method with all labeled patients. Experiments also demonstrate its robustness to small percentage of noisy labels.
Automatic detection of cancer metastasis from whole slide images (WSIs) is a crucial step for following patient staging and prognosis. Recent convolutional neural network based approaches are struggling with the trade-off between accuracy and computational efficiency due to the difficulty in processing large-scale gigapixel WSIs. To meet this challenge, we propose a novel Pyramidal Feature Aggregation ScanNet (PFA-ScanNet) for robust and fast analysis of breast cancer metastasis. Our method mainly benefits from the aggregation of extracted local-to-global features with diverse receptive fields, as well as the proposed synergistic learning for training the main detector and extra decoder with semantic guidance. Furthermore, a high-efficiency inference mechanism is designed with dense pooling layers, which allows dense and fast scanning for gigapixel WSI analysis. As a result, the proposed PFA-ScanNet achieved the state-of-the-art FROC of 89.1% on the Camelyon16 dataset, as well as competitive kappa score of 0.905 on the Camelyon17 leaderboard without model ensemble. In addition, our method shows leading speed advantage over other methods, about 7.2 min per WSI with a single GPU, making automatic analysis of breast cancer metastasis more applicable in the clinical usage.
Computational image analysis is one means for evaluating digitized histopathology specimens that can increase the reproducibility and reliability with which cancer diagnoses are rendered while simultaneously providing insight as to the underlying mechanisms of disease onset and progression. A major challenge that is confronted when analyzing samples that have been prepared at disparate laboratories and institutions is that the algorithms used to assess the digitized specimens often exhibit heterogeneous staining characteristics because of slight differences in incubation times and the protocols used to prepare the samples. Unfortunately, such variations can render a prediction model learned from one batch of specimens ineffective for characterizing an ensemble originating from another site. In this work, we propose to adopt unsupervised domain adaptation to effectively transfer the discriminative knowledge obtained from any given source domain to the target domain without requiring any additional labeling or annotation of images at the target site. In this paper, our team investigates the use of two approaches for performing the adaptation: (1) color normalization and (2) adversarial training. The adversarial training strategy is implemented through the use of convolutional neural networks to find an invariant feature space and Siamese architecture within the target domain to add a regularization that is appropriate for the entire set of whole-slide images. The adversarial adaptation results in significant classification improvement compared with the baseline models under a wide range of experimental settings.
Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by: (1) noisy and expensive reference standards established by pathologists, (2) lack of generalization due to staining variation across laboratories, and (3) high computational requirements needed to process gigapixel whole-slide images (WSIs). In this paper, we present a method to train and evaluate CNNs to specifically solve these issues in the context of mitosis detection in breast cancer WSIs. First, by combining image analysis of mitotic activity in phosphohistone-H3 (PHH3) restained slides and registration, we built a reference standard for mitosis detection in entire H&E WSIs requiring minimal manual annotation effort. Second, we designed a data augmentation strategy that creates diverse and realistic H&E stain variations by modifying the hematoxylin and eosin color channels directly. Using it during training combined with network ensembling resulted in a stain invariant mitosis detector. Third, we applied knowledge distillation to reduce the computational requirements of the mitosis detection ensemble with a negligible loss of performance. The system was trained in a single-center cohort and evaluated in an independent multicenter cohort from The Cancer Genome Atlas on the three tasks of the Tumor Proliferation Assessment Challenge (TUPAC). We obtained a performance within the top-3 best methods for most of the tasks of the challenge.
Importance
Application of deep learning algorithms to whole-slide pathology images can potentially improve diagnostic accuracy and efficiency.
Objective
Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin–stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists’ diagnoses in a diagnostic setting.
Design, Setting, and Participants
Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC).
Exposures
Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation.
Main Outcomes and Measures
The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor.
Results
The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P < .001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC).
Conclusions and Relevance
In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting.
Recent work has shown that convolutional networks can be substantially deeper, more accurate and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper we embrace this observation and introduce the Dense Convolutional Network (DenseNet), where each layer is directly connected to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections, one between each layer and its subsequent layer (treating the input as layer 0), our network has L(L+1)/2 direct connections. For each layer, the feature maps of all preceding layers are treated as separate inputs whereas its own feature maps are passed on as inputs to all subsequent layers. Our proposed connectivity pattern has several compelling advantages: it alleviates the vanishing gradient problem and strengthens feature propagation; despite the increase in connections, it encourages feature reuse and leads to a substantial reduction of parameters; its models tend to generalize surprisingly well. We evaluate our proposed architecture on five highly competitive object recognition benchmark tasks. The DenseNet obtains significant improvements over the state-of-the-art on all five of them (e.g., yielding 3.74% test error on CIFAR-10, 19.25% on CIFAR-100 and 1.59% on SVHN).
Lymph node metastasis is one of the most significant diagnostic indicators in breast cancer, which is traditionally observed under the microscope by pathologists.
In recent years, computerized histology diagnosis has become one of the most rapidly expanding directions in the field of medical image computing, which aims to alleviate pathologists' workload and simultaneously reduce misdiagnosis rate.
However, automatic detection of lymph node metastases from whole slide images remains a challenging problem, due to the large-scale data with enormous resolutions and existence of hard mimics resulting in a large number of false positives.
In this paper, we propose a novel framework by leveraging fully convolutional networks for efficient inference to meet the speed requirement for clinical practice, while reconstructing dense predictions under different offsets for ensuring accurate detection on both micro- and macro-metastases.
Incorporating with the strategies of asynchronous sample prefetching and hard negative mining, the network can be effectively trained.
Extensive experiments on the benchmark dataset of 2016 Camelyon Grand Challenge corroborated the efficacy of our method.
Compared with the state-of-the-art methods, our method achieved superior performance with a faster speed on the tumor localization task and even surpassed human performance on the WSI classification task.
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections - one between each layer and its subsequent layer - our network has L(L+1)/2 direct connections. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). DenseNets obtain significant improvements over the state-of-the-art on most of them, whilst requiring less memory and computation to achieve high performance. Code and models are available at https://github.com/liuzhuang13/DenseNet.
The International Symposium on Biomedical Imaging (ISBI) held a grand challenge to evaluate computational systems for the automated detection of metastatic breast cancer in whole slide images of sentinel lymph node biopsies. Our team won both competitions in the grand challenge, obtaining an area under the receiver operating curve (AUC) of 0.925 for the task of whole slide image classification and a score of 0.7051 for the tumor localization task. A pathologist independently reviewed the same images, obtaining a whole slide image classification AUC of 0.966 and a tumor localization score of 0.733. Combining our deep learning system's predictions with the human pathologist's diagnoses increased the pathologist's AUC to 0.995, representing an approximately 85 percent reduction in human error rate. These results demonstrate the power of using deep learning to produce significant improvements in the accuracy of pathological diagnoses.
Cervical cancer is one of the most deadly and common forms of cancer among women if no action is taken to prevent it, yet it is preventable through a simple screening test, the so-called PAP-smear. This is the most effective cancer prevention measure developed so far. But the visual examination of the smears is time consuming and expensive and there have been numerous attempts at automating the analysis ever since the test was introduced more than 60 years ago. The first commercial systems for automated analysis of the cell samples appeared around the turn of the millennium but they have had limited impact on the screening costs. In this paper we examine the key issues that need to be addressed when an automated analysis system is developed and discuss how these challenges have been met over the years. The lessons learned may be useful in the efforts to create a cost-effective screening system that could make affordable screening for cervical cancer available for all women globally, thus preventing most of the quarter million annual unnecessary deaths still caused by this disease.
This paper presents a set of novel structural texture features for quantifying chromatin texture in cervical cells on a Pap smear. The paper also demonstrates the efficacy of these features for detecting malignancy associated changes (MACs) enabling the discrimination between normal and abnormal smears without performing an exhaustive review of all the cellular material for diagnostic cells. A feature selection experiment demonstrates that these features are among the top four features selected from a superset including more conventional features. Results show that a combination of new structural features and conventional features yields very good discrimination based on the MAC approach.
The number of mitoses per tissue area gives an important aggressiveness indication of the invasive breast carcinoma.However, automatic mitosis detection in histology images remains a challenging problem. Traditional methods either employ hand-crafted features to discriminate mitoses from other cells or construct a pixel-wise classifier to label every pixel in a sliding window way. While the former suffers from the large shape variation of mitoses and the existence of many mimics with similar appearance, the slow speed of the later prohibits its use in clinical practice.In order to overcome these shortcomings, we propose a fast and accurate method to detect mitosis by designing a novel deep cascaded convolutional neural network, which is composed of two components. First, by leveraging the fully convolutional neural network, we propose a coarse retrieval model to identify and locate the candidates of mitosis while preserving a high sensitivity.Based on these candidates, a fine discrimination model utilizing knowledge transferred from cross-domain is developed to further single out mitoses from hard mimics.Our approach outperformed other methods by a large margin in 2014 ICPR MITOS-ATYPIA challenge in terms of detection accuracy. When compared with the state-of-the-art methods on the 2012 ICPR MITOSIS data (a smaller and less challenging dataset), our method achieved comparable or better results with a roughly 60 times faster speed.
Detecting various types of cells in and around the tumor matrix holds a special significance in characterizing the tumor micro-environment for cancer prognostication and research. Automating the tasks of detecting, segmenting, and classifying nuclei can free up the pathologists’ time for higher value tasks and reduce errors due to fatigue and subjectivity. To encourage the computer vision research community to develop and test algorithms for these tasks, we prepared a large and diverse dataset of nucleus boundary annotations and class labels. The dataset has over 46,000 nuclei from 37 hospitals, 71 patients, four organs, and four nucleus types. We also organized a challenge around this dataset as a satellite event at the International Symposium on Biomedical Imaging (ISBI) in April 2020. The challenge saw a wide participation from across the world, and the top methods were able to match inter-human concordance for the challenge metric. In this paper, we summarize the dataset and the key findings of the challenge, including the commonalities and differences between the methods developed by various participants. We have released the MoNuSAC2020 dataset to the public.
Spermatogenesis in mammals is a cyclic process of spermatogenic cell development in the seminiferous epithelium that can be subdivided into 12 subsequent stages. Histological staging analysis of testis sections, specifically of seminiferous tubule cross-sections, is the only effective method to evaluate the quality of the spermatogenic process and to determine developmental defects leading to infertility. Such staging analysis, however, is tedious and time-consuming, and it may take a long time to become proficient. We now have developed a Computerized Staging system of Spermatogenesis (CSS) for mouse testis sections through learning of an expert with decades of experience in mouse testis staging. The development of the CSS system comprised three major parts: 1) Developing computational image analysis models for mouse testis sections; 2) Automated classification of each seminiferous tubule cross-section into three stage groups: Early Stages (ES: stages I-V), Middle Stages (MS: stages VI-VIII), and Late Stages (LS: stages IV-XII); 3) Automated classification of MS into distinct stages VI, VII-mVIII, and late VIII based on newly developed histomorphological features. A cohort of 40 H&E stained normal mouse testis sections was built according to three modules where 28 cross-sections were leveraged for developing tubule region segmentation, spermatogenic cells types and multi-concentric-layers segmentation models. The rest of 12 testis cross-sections, approximately 2314 tubules whose stages were manually annotated by two expert testis histologists, served as the basis for developing the CSS system. The CSS system’s accuracy of mean and standard deviation (MSD) in identifying ES, MS, and LS were 0.93 ± 0.03, 0.94 ± 0.11, and 0.89 ± 0.05 and 0.85 ± 0.12, 0.88 ± 0.07, and 0.96 ± 0.04 for one with 5 years of experience, respectively. The CSS system’s accuracy of MSD in identifying stages VI, VII-mVIII, and late VIII are 0.74 ± 0.03, 0.85 ± 0.04, and 0.78 ± 0.06 and 0.34 ± 0.18, 0.78 ± 0.16, and 0.44 ± 0.25 for one with 5 years of experience, respectively. In terms of time it takes to collect these data, it takes on average 3 hours for a histologist and 1.87 hours for the CSS system to finish evaluating an entire testis section (computed with a PC (I7-6800k 4.0 GHzwith 32GB of RAM & 256G SSD) and a Titan 1080Ti GPU). Therefore, the CSS system is more accurate and faster compared to a human histologist in staging, and further optimization and development will not only lead to a complete staging of all 12 stages of mouse spermatogenesis but also could aid in the future diagnosis of human infertility. Moreover, the top-ranking histomorphological features identified by the CSS classifier are consistent with the primary features used by histologists in discriminating stages VI, VII-mVIII, and late VIII.
Accurate and automated gland segmentation on histology tissue images is an essential but challenging task in the computer-aided diagnosis of adenocarcinoma. Despite their prevalence, deep learning models always require a myriad number of densely annotated training images, which are difficult to obtain due to extensive labor and associated expert costs related to histology image annotations. In this paper, we propose the pairwise relation-based semi-supervised (PRS) model for gland segmentation on histology images. This model consists of a segmentation network (S-Net) and a pairwise relation network (PR-Net). The S-Net is trained on labeled data for segmentation, and PR-Net is trained on both labeled and unlabeled data in an unsupervised way to enhance its image representation ability via exploiting the semantic consistency between each pair of images in the feature space. Since both networks share their encoders, the image representation ability learned by PR-Net can be transferred to S-Net to improve its segmentation performance. We also design the object-level Dice loss to address the issues caused by touching glands and combine it with other two loss functions for S-Net. We evaluated our model against five recent methods on the GlaS dataset and three recent methods on the CRAG dataset. Our results not only demonstrate the effectiveness of the proposed PR-Net and object-level Dice loss, but also indicate that our PRS model achieves the state-of-the-art gland segmentation performance on both benchmarks.
Deep learning methods show promising results for overlapping cervical cell instance segmentation. However, in order to train a model with good generalization ability, voluminous pixel-level annotations are demanded which is quite expensive and time-consuming for acquisition. In this paper, we propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining (MMT-PSM), which consists of a teacher and a student network during training. Two networks are encouraged to be consistent both in feature and semantic level under small perturbations. The teacher’s self-ensemble predictions from K-time augmented samples are used to construct the reliable pseudo-labels for optimizing the student. We design a novel strategy to estimate the sensitivity to perturbations for each proposal and select informative samples from massive cases to facilitate fast and effective semantic distillation. In addition, to eliminate the unavoidable noise from the background region, we propose to use the predicted segmentation mask as guidance to enforce the feature distillation in the foreground region. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only, and outperforms state-of-the-art semi-supervised methods. Code: https://github.com/SIAAAAAA/MMT-PSM.
Artificial intelligence has become a main driving force for a new round of industrial transformation around the world. Many countries including China are seizing the opportunity of the AI revolution to promote domestic economic and technological development. This Perspective briefly introduces the New Generation Artificial Intelligence Development Plan of China (2015–2030) from the point of view of the authors, a group of AI experts from academia and industry who have been involved in various stages of the plan. China’s AI development plan outlines a strategy for science and technology as well as education, tackling a number of challenges such as retaining talent, advancing fundamental research and exploring ethical issues. The New Generation Artificial Intelligence Development Plan is intended to be a blueprint for a complete AI ecosystem for the country. China’s New Generation Artificial Intelligence Development Plan was launched in 2017 and lays out an ambitious strategy, which intends to make China one of the world’s premier AI innovation centre by 2030. This Perspective presents the view from a group of Chinese AI experts from academia and industry about the origins of the plan, the motivations and main focus for attention from research and industry.
Every two minutes one woman dies of cervical cancer globally, due to lack of sufficient screening. Given a whole slide image (WSI) obtained by scanning a microscope glass slide for a Liquid Based Cytology (LBC) based Pap test, our goal is to assist the pathologist to determine the presence of pre-cancerous or cancerous cervical anomalies. Inter-annotator variation, large image sizes, data imbalance, stain variations, and lack of good annotation tools make this problem challenging. Existing related work has focused on sub-problems like cell segmentation and cervical cell classification but does not provide a practically feasible holistic solution. We propose a practical system architecture which is based on displaying regions of interest on WSIs containing potential anomaly for review by pathologists to increase productivity. We build multiple deep learning classifiers as part of the proposed architecture. Our experiments with a dataset of ∼19000 regions of interest provide an accuracy of ∼89% for a balanced dataset both for binary as well as 6-class classification settings. Our deployed system provides a top-5 accuracy of ∼94%.
Cervical intraepithelial neoplasia (CIN) grade of histopathology images is a crucial indicator in cervical biopsy results. Accurate CIN grading of epithelium regions helps pathologists with precancerous lesion diagnosis and treatment planning. Although an automated CIN grading system has been desired, supervised training of such a system would require a large amount of expert annotations, which are expensive and time-consuming to collect. In this paper, we investigate the CIN grade classification problem on segmented epithelium patches. We propose to use conditional Generative Adversarial Networks (cGANs) to expand the limited training dataset, by synthesizing realistic cervical histopathology images. While the synthetic images are visually appealing, they are not guaranteed to contain meaningful features for data augmentation. To tackle this issue, we propose a synthetic-image filtering mechanism based on the divergence in feature space between generated images and class centroids in order to control the feature quality of selected synthetic images for data augmentation. Our models are evaluated on a cervical histopathology image dataset with limited number of patch-level CIN grade annotations. Extensive experimental results show a significant improvement of classification accuracy from 66.3% to 71.7% using the same ResNet18 baseline classifier after leveraging our cGAN generated images with feature based filtering, which demonstrates the effectiveness of our models.
Cell instance segmentation in Pap smear image remains challenging due to the wide existence of occlusion among translucent cytoplasm in cell clumps. Conventional methods heavily rely on accurate nuclei detection results and are easily disturbed by miscellaneous objects. In this paper, we propose a novel Instance Relation Network (IRNet) for robust overlapping cell segmentation by exploring instance relation interaction. Specifically, we propose the Instance Relation Module to construct the cell association matrix for transferring information among individual cell-instance features. With the collaboration of different instances, the augmented features gain benefits from contextual information and improve semantic consistency. Meanwhile, we proposed a sparsity constrained Duplicate Removal Module to eliminate the misalignment between classification and localization accuracy for candidates selection. The largest cervical Pap smear (CPS) dataset with more than 8000 cell annotations in Pap smear image was constructed for comprehensive evaluation. Our method outperforms other methods by a large margin, demonstrating the effectiveness of exploring instance relation.
Accurate and automated gland instance segmentation on histology microscopy images can assist pathologists to diagnose the malignancy degree of colorectal adenocarcinoma. To address this problem, many deep convolutional neural network (DCNN) based methods have been proposed, most of which aim to generate better segmentation by improving the model structure and loss function. Few of them, however, focus on further emendating the inferred predictions, thus missing a chance to refine the obtained segmentation results. In this paper, we propose the deep segmentation-emendation (DSE) model for gland instance segmentation. This model consists of a segmentation network (Seg-Net) and an emendation network (Eme-Net). The Seg-Net is dedicated to generating segmentation results, and the Eme-Net learns to predict the inconsistency between the ground truth and the segmentation results generated by Seg-Net. The predictions made by Eme-Net can in turn be used to refine the segmentation result. We evaluated our DSE model against five recent deep learning models on the 2015 MICCAI Gland Segmentation challenge (GlaS) dataset and against two deep learning models on the colorectal adenocarcinoma (CRAG) dataset. Our results indicate that using Eme-Net results in significant improvement in segmentation accuracy, and the proposed DSE model is able to substantially outperform all the rest models in gland instance segmentation on both datasets.
Medical imaging analysis has witnessed impressive progress in recent years thanks to the development of large-scale labeled datasets. However, in many fields, including cervical cytology, a large well-annotated benchmark dataset remains missing. In this paper, we introduce by far the largest cervical cytology dataset, called Deep Cervical Cytological Lesions (referred to as DCCL). DCCL contains 14,432 image patches with around pixels cropped from 1,167 whole slide images collected from four medical centers and scanned by one of the three kinds of digital slide scanners. Besides patch level labels, cell level labels are provided, with 27,972 lesion cells labeled based on The 2014 Bethesda System and the bounding box by six board-certified pathologists with eight years of experience on the average. We also use deep learning models to generate the baseline performance for lesion cell detection and cell type classification on DCCL. We believe this dataset can serve as a valuable resource and platform for researchers to develop new algorithms and pipelines for advanced cervical cancer diagnosis and prevention.
Histopathology image analysis serves as the gold standard for cancer diagnosis. Efficient and precise diagnosis is quite critical for the subsequent therapeutic treatment of patients. So far, computer-aided diagnosis has not been widely applied in pathological field yet as currently well-addressed tasks are only the tip of the iceberg. Whole slide image (WSI) classification is a quite challenging problem. First, the scarcity of annotations heavily impedes the pace of developing effective approaches. Pixelwise delineated annotations on WSIs are time consuming and tedious, which poses difficulties in building a large-scale training dataset. In addition, a variety of heterogeneous patterns of tumor existing in high magnification field are actually the major obstacle. Furthermore, a gigapixel scale WSI cannot be directly analyzed due to the immeasurable computational cost. How to design the weakly supervised learning methods to maximize the use of available WSI-level labels that can be readily obtained in clinical practice is quite appealing. To overcome these challenges, we present a weakly supervised approach in this article for fast and effective classification on the whole slide lung cancer images. Our method first takes advantage of a patch-based fully convolutional network (FCN) to retrieve discriminative blocks and provides representative deep features with high efficiency. Then, different context-aware block selection and feature aggregation strategies are explored to generate globally holistic WSI descriptor which is ultimately fed into a random forest (RF) classifier for the image-level prediction. To the best of our knowledge, this is the first study to exploit the potential of image-level labels along with some coarse annotations for weakly supervised learning. A large-scale lung cancer WSI dataset is constructed in this article for evaluation, which validates the effectiveness and feasibility of the proposed method. Extensive experiments demonstrate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin with an accuracy of 97.3% . In addition, our method also achieves the best performance on the public lung cancer WSIs dataset from The Cancer Genome Atlas (TCGA). We highlight that a small number of coarse annotations can contribute to further accuracy improvement. We believe that weakly supervised learning methods have great potential to assist pathologists in histology image diagnosis in the near future.
We propose Neural Image Compression (NIC), a two-step method to build convolutional neural networks for gigapixel image analysis solely using weak image-level labels. First, gigapixel images are compressed using a neural network trained in an unsupervised fashion, retaining high-level information while suppressing pixel-level noise. Second, a convolutional neural network (CNN) is trained on these compressed image representations to predict image-level labels, avoiding the need for fine-grained manual annotations. We compared several encoding strategies, namely reconstruction error minimization, contrastive training and adversarial feature learning, and evaluated NIC on a synthetic task and two public histopathology datasets. We found that NIC can exploit visual cues associated with image-level labels successfully, integrating both global and local visual information. Furthermore, we visualized the regions of the input gigapixel images where the CNN attended to, and confirmed that they overlapped with annotations from human experts.
Accurate segmenting nuclei instances is a crucial step in computer-aided image analysis to extract rich features for cellular estimation and following diagnosis as well as treatment. While it still remains challenging because the wide existence of nuclei clusters, along with the large morphological variances among different organs make nuclei instance segmentation susceptible to over-/under-segmentation. Additionally, the inevitably subjective annotating and mislabeling prevent the network learning from reliable samples and eventually reduce the generalization capability for robustly segmenting unseen organ nuclei. To address these issues, we propose a novel deep neural network, namely Contour-aware Informative Aggregation Network (CIA-Net) with multi-level information aggregation module between two task-specific decoders. Rather than independent decoders, it leverages the merit of spatial and texture dependencies between nuclei and contour by bi-directionally aggregating task-specific features. Furthermore, we proposed a novel smooth truncated loss that modulates losses to reduce the perturbation from outliers. Consequently, the network can focus on learning from reliable and informative samples, which inherently improves the generalization capability. Experiments on the 2018 MICCAI challenge of Multi-Organ-Nuclei-Segmentation validated the effectiveness of our proposed method, surpassing all the other 35 competitive teams by a significant margin.
We present a novel approach to segmenting overlapping cytoplasm of cells in cervical smear images by leveraging adaptive shape priors extracted from cytoplasm’s contour fragments and shape statistics. The main challenge of this task is that many occluded boundaries in cytoplasm clumps are extremely difficult to be identified, sometimes even visually indistinguishable. Given a clump where multiple cytoplasms overlap, our method starts by cutting its contour into a set of contour fragments. We then locate the corresponding contour fragments of each cytoplasm by a grouping process. For each cytoplasm, according to the grouped fragments and a set of known shape references, we construct its shape and then connect the fragments to form a closed contour as the segmentation result, which is explicitly constrained by the constructed shape. We further integrate intensity and curvature information, which is complementary to the shape priors extracted from contour fragments, into our framework to improve the segmentation accuracy. We propose to iteratively conduct fragments grouping, shape constructing, and fragments connecting, for progressively refining the shape priors and improving the segmentation results. We extensively evaluate the effectiveness of our method on two typical cervical smear datasets. Experimental results demonstrate that our approach is highly effective and consistently outperforms the state-of-theart approaches. The proposed method is general enough to be applied to other similar microscopic image segmentation tasks where heavily overlapped objects exist.
Lymph node metastasis is one of the most important indicators in breast cancer diagnosis, that is traditionally observed under the microscope by pathologists. In recent years, with the dramatic advance of high-throughput scanning and deep learning technology, automatic analysis of histology from wholeslide images has received a wealth of interest in the field of medical image computing, which aims to alleviate pathologists’ workload and simultaneously reduce misdiagnosis rate. However, automatic detection of lymph node metastases from whole-slide images remains a key challenge because such images are typically very large, where they can often be multiple gigabytes in size. Also, the presence of hard mimics may result in a large number of false positives. In this paper, we propose a novel method with anchor layers for model conversion, which not only leverages the efficiency of fully convolutional architectures to meet the speed requirement in clinical practice, but also densely scans the wholeslide image to achieve accurate predictions on both micro- and macro-metastases. Incorporating the strategies of asynchronous sample prefetching and hard negative mining, the network can be effectively trained. The efficacy of our method are corroborated on the benchmark dataset of 2016 Camelyon Grand Challenge. Our method achieved significant improvements in comparison with the state-of-the-art methods on tumour localization accuracy with a much faster speed and even surpassed human performance on both challenge tasks.
This article provides a status report on the global burden of cancer worldwide using the GLOBOCAN 2018 estimates of cancer incidence and mortality produced by the International Agency for Research on Cancer, with a focus on geographic variability across 20 world regions. There will be an estimated 18.1 million new cancer cases (17.0 million excluding nonmelanoma skin cancer) and 9.6 million cancer deaths (9.5 million excluding nonmelanoma skin cancer) in 2018. In both sexes combined, lung cancer is the most commonly diagnosed cancer (11.6% of the total cases) and the leading cause of cancer death (18.4% of the total cancer deaths), closely followed by female breast cancer (11.6%), prostate cancer (7.1%), and colorectal cancer (6.1%) for incidence and colorectal cancer (9.2%), stomach cancer (8.2%), and liver cancer (8.2%) for mortality. Lung cancer is the most frequent cancer and the leading cause of cancer death among males, followed by prostate and colorectal cancer (for incidence) and liver and stomach cancer (for mortality). Among females, breast cancer is the most commonly diagnosed cancer and the leading cause of cancer death, followed by colorectal and lung cancer (for incidence), and vice versa (for mortality); cervical cancer ranks fourth for both incidence and mortality. The most frequently diagnosed cancer and the leading cause of cancer death, however, substantially vary across countries and within each country depending on the degree of economic development and associated social and life style factors. It is noteworthy that high‐quality cancer registry data, the basis for planning and implementing evidence‐based cancer control programs, are not available in most low‐ and middle‐income countries. The Global Initiative for Cancer Registry Development is an international partnership that supports better estimation, as well as the collection and use of local data, to prioritize and evaluate national cancer control efforts. CA: A Cancer Journal for Clinicians 2018;0:1‐31. © 2018 American Cancer Society
Automated detection of cancer metastases in lymph nodes has the potential to improve assessment of prognosis for patients. To enable fair comparison between the algorithms for this purpose, we set up the CAMELYON17 challenge in conjunction with the IEEE International Symposium on Biomedical Imaging 2017 conference in Melbourne. Over 300 participants registered on the challenge website, of which 23 teams submitted a total of 37 algorithms before the initial deadline. Participants were provided with 899 whole-slide images for developing their algorithms. The developed algorithms were evaluated based on the test set encompassing 100 patients and 500 whole-slide images. The evaluation metric used was a quadratic weighted Cohen’s kappa. We discuss the algorithmic details of the ten best pre-conference and two post-conference submissions. All these participants used convolutional neural networks in combination with pre-and postprocessing steps. Algorithms differed mostly in neural network architecture, training strategy and pre-and postprocessing methodology. Overall, the kappa metric ranged from 0.89 to -0.13 across all submissions. The best results were obtained with pre-trained architectures such as ResNet. Confusion matrix analysis revealed that all participants struggled with reliably identifying isolated tumor cells, the smallest type of metastasis, with detection rates below 40%. Qualitative inspection of the results of the top participants showed categories of false positives, such as nerves or contamination, which could be targets for further optimization. Last, we show that simple combinations of the top algorithms result in higher kappa metric values than any algorithm individually, with 0.93 for the best combination.
Hashing has become a popular tool on histopathology image analysis due to the significant gain in both computation and storage. However, most of current hashing techniques learn features and binary codes individually from whole images, or emphasize the inter-class difference but neglect the relevance order within the same classes. To alleviate these issues, in this paper, we propose a novel pairwise based deep ranking hashing framework. We first define a pairwise matrix to preserve intra-class relevance and inter-class difference. Then we propose an objective function that utilizes two identical continuous matrices generated by the hyperbolic tangent (tanh) function to approximate the pairwise matrix. Finally, we incorporate the objective function into a deep learning architecture to learn features and binary codes simultaneously. The proposed framework is validated on 5,356 skeletal muscle and 2,176 lung cancer images with four types of diseases, and it can achieve 97.49% classification accuracy, 97.49% mean average precision (MAP) with 100 returned images, and 0.51 NDCG score with 50 retrieved neighbors on 2,032 query images.
The visual attributes of cells, such as the nuclear morphology and chromatin openness, are critical for histopathology image analysis. By learning cell-level visual representation, we can obtain a rich mix of features that are highly reusable for various tasks, such as cell-level classification, nuclei segmentation, and cell counting. In this paper, we propose a unified generative adversarial networks architecture with a new formulation of loss to perform robust cell-level visual representation learning in an unsupervised setting. Our model is not only label-free and easily trained but also capable of cell-level unsupervised classification with interpretable visualization, which achieves promising results in the unsupervised classification of bone marrow cellular components. Based on the proposed cell-level visual representation learning, we further develop a pipeline that exploits the varieties of cellular elements to perform histopathology image classification, the advantages of which are demonstrated on bone marrow datasets.
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Each year, the treatment decisions for more than 230,000 breast cancer patients in the U.S. hinge on whether the cancer has metastasized away from the breast. Metastasis detection is currently performed by pathologists reviewing large expanses of biological tissues. This process is labor intensive and error-prone. We present a framework to automatically detect and localize tumors as small as 100 x 100 pixels in gigapixel microscopy images sized 100,000 x 100,000 pixels. Our method leverages a convolutional neural network (CNN) architecture and obtains state-of-the-art results on the Camelyon16 dataset in the challenging lesion-level tumor detection task. At 8 false positives per image, we detect 92.4% of the tumors, relative to 82.7% by the previous best automated approach. For comparison, a human pathologist attempting exhaustive search achieved 73.2% sensitivity. We achieve image-level AUC scores above 97% on both the Camelyon16 test set and an independent set of 110 slides. In addition, we discover that two slides in the Camelyon16 training set were erroneously labeled normal. Our approach could considerably reduce false negative rates in metastasis detection.
In histopathological image analysis, the morphology of histological structures, such as glands and nuclei, has been routinely adopted by pathologists to assess the malignancy degree of adenocarcinomas. Accurate detection and segmentation of these objects of interest from histology images is an essential prerequisite to obtain reliable morphological statistics for quantitative diagnosis. While manual annotation is error-prone, time-consuming and operator-dependant, automated detection and segmentation of objects of interest from histology images can be very challenging due to the large appearance variation, existence of strong mimics, and serious degeneration of histological structures. In order to meet these challenges, we propose a novel deep contour-aware network (DCAN) under a unified multi-task learning framework for more accurate detection and segmentation. In the proposed network, multi-level contextual features are explored based on an end-to-end fully convolutional network (FCN) to deal with the large appearance variation. We further propose to employ an auxiliary supervision mechanism to overcome the problem of vanishing gradients when training such a deep network. More importantly, our network can not only output accurate probability maps of histological objects, but also depict clear contours simultaneously for separating clustered object instances, which further boosts the segmentation performance. Our method ranked the first in two histological object segmentation challenges, including 2015 MICCAI Gland Segmentation Challenge and 2015 MICCAI Nuclei Segmentation Challenge. Extensive experiments on these two challenging datasets demonstrate the superior performance of our method, surpassing all the other methods by a significant margin.
Accurate segmentation of cervical cells in Pap smear images is an important step in automatic pre-cancer identification in the uterine cervix. One of the major segmentation challenges is overlapping of cytoplasm, which has not been well-addressed in previous studies. To tackle the overlapping issue, this paper proposes a learning-based method with robust shape priors to segment individual cell in Pap smear images to support automatic monitoring of changes in cells, which is a vital prerequisite of early detection of cervical cancer. We define this splitting problem as a discrete labeling task for multiple cells with a suitable cost function. The labeling results are then fed into our dynamic multi-template deformation model for further boundary refinement. Multi-scale deep convolutional networks are adopted to learn the diverse cell appearance features. We also incorporated high-level shape information to guide segmentation where cell boundary might be weak or lost due to cell overlapping. An evaluation carried out using two different datasets demonstrates the superiority of our proposed method over the state-of-the-art methods in terms of segmentation accuracy.
Deeper neural networks are more difficult to train. We present a residual
learning framework to ease the training of networks that are substantially
deeper than those used previously. We explicitly reformulate the layers as
learning residual functions with reference to the layer inputs, instead of
learning unreferenced functions. We provide comprehensive empirical evidence
showing that these residual networks are easier to optimize, and can gain
accuracy from considerably increased depth. On the ImageNet dataset we evaluate
residual nets with a depth of up to 152 layers---8x deeper than VGG nets but
still having lower complexity. An ensemble of these residual nets achieves
3.57% error on the ImageNet test set. This result won the 1st place on the
ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100
and 1000 layers.
The depth of representations is of central importance for many visual
recognition tasks. Solely due to our extremely deep representations, we obtain
a 28% relative improvement on the COCO object detection dataset. Deep residual
nets are foundations of our submissions to ILSVRC & COCO 2015 competitions,
where we also won the 1st places on the tasks of ImageNet detection, ImageNet
localization, COCO detection, and COCO segmentation.
This book offers clear, up-to-date guidance on how to report cytologic findings in cervical, vaginal and anal samples in accordance with the 2014 Bethesda System Update. The new edition has been expanded and revised to take into account the advances and experience of the past decade. A new chapter has been added, the terminology and text have been updated, and various terminological and morphologic questions have been clarified. In addition, new images are included that reflect the experience gained with liquid-based cytology since the publication of the last edition in 2004. Among more than 300 images, some represent classic examples of an entity while others illustrate interpretative dilemmas, borderline cytomorphologic features or mimics of epithelial abnormalities. The Bethesda System for Reporting Cervical Cytology, with its user-friendly format, is a “must have” for pathologists, cytopathologists, pathology residents, cytotechnologists, and clinicians.
Computer-aided image analysis of histopathology specimens could potentially provide support for early detection and improved characterization of diseases such as brain tumor, pancreatic neuroendocrine tumor (NET), and breast cancer. Automated nucleus segmentation is a prerequisite for various quantitative analyses including automatic morphological feature computation. However, it remains to be a challenging problem due to the complex nature of histopathology images. In this paper, we propose a learning-based framework for accurate and automatic nucleus segmentation with shape preservation. Given a nucleus image, it begins with a deep convolutional neural network (CNN) model to generate a probability map, to which an iterative region merging approach is performed for shape initializations. Next, a novel segmentation algorithm is exploited to separate individual nuclei combining a robust selection-based sparse shape model and a local repulsive deformable model. One of the significant benefits of the proposed framework is that it is applicable to different staining histopathology images. Due to the feature learning characteristic of the deep CNN and the high level shape prior modeling, the proposed method is general enough to perform well across multiple scenarios. We have tested the proposed algorithm on three large-scale pathology image datasets using a range of different tissue and stain preparations, and the comparative experiments with recent state of the arts demonstrate the superior performance of the proposed algorithm.
In this paper, a multi-scale convolutional network (MSCN) and graph partitioning based method is proposed for accurate segmentation of cervical cytoplasm and nuclei. Specifically, deep learning via MSCN is explored to extract scale invariant features and then segment regions centered at each pixel. The coarse segmentation is refined by an automated graph partitioning method based on the pre-trained feature. The texture, shape, and contextual information of the target objects are learned to localize the appearance of distinctive boundary, which is also explored to generate markers to split the touching nuclei. For further refinement of the segmentation, a coarse-to-fine nucleus segmentation framework is developed. The computational complexity of the segmentation is reduced by using superpixel instead of raw pixels. Extensive experimental results demonstrate that the proposed cervical nucleus cell segmentation delivers promising results and outperforms existing methods.
In this work we investigate the effect of the convolutional network depth on
its accuracy in the large-scale image recognition setting. Our main
contribution is a thorough evaluation of networks of increasing depth, which
shows that a significant improvement on the prior-art configurations can be
achieved by pushing the depth to 16-19 weight layers. These findings were the
basis of our ImageNet Challenge 2014 submission, where our team secured the
first and the second places in the localisation and classification tracks
respectively.
Screening plays an important role within the fight against cervical cancer. One of the most challenging parts in order to automate the screening process is the segmentation of nuclei in the cervical cell images, as the difficulty for performing this segmentation accurately varies widely within the nuclei. We present an algorithm to perform this task. After background determination in an overview image, and interactive identification of regions of interest (ROIs) at lower magnification levels, ROIs are extracted and processed at the full magnification level of 40x. Subsequent to initial background removal, the image regions are smoothed by mean-shift and median filtering. Then, segmentations are generated by an adaptive threshold. The connected components in the resulting segmentations are filtered with morphological operators by characteristics such as shape, size and roundness. The algorithm was tested on a set of 50 images and was found to outperform other methods.