No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Assessment of algorithms for mitosis detection in breast cancer histopathology images
Abstract
The proliferative activity of breast tumors, which is routinely estimated by
counting of mitotic figures in hematoxylin and eosin stained histology
sections, is considered to be one of the most important prognostic markers.
However, mitosis counting is laborious, subjective and may suffer from low
inter-observer agreement. With the wider acceptance of whole slide images in
pathology labs, automatic image analysis has been proposed as a potential
solution for these issues. In this paper, the results from the Assessment of
Mitosis Detection Algorithms 2013 (AMIDA13) challenge are described. The
challenge was based on a data set consisting of 12 training and 11 testing
subjects, with more than one thousand annotated mitotic figures by multiple
observers. Short descriptions and results from the evaluation of eleven methods
are presented. The top performing method has an error rate that is comparable
to the inter-observer agreement among pathologists.
... Potential use: The automated evaluation of immunohistochemistry (IHC) staining of receptors along with their percentage of staining, intensity and other parameters for scoring in breast carcinoma has attracted a lot of interest. Attempts have also been made to simultaneously predict biomarkers involved in carcinoma breast directly from H&E slides recently (18). These algorithms also need to consider the cost-savings aspect and the extent to which the objectivity and accuracy of AIassisted assessment adds value for clinical application. ...
... The diagnosis on H and E , the morphology of tumour is correlated with the IHC expression of positive and negative expression of receptors, their pattern of involvement and intensity and percentage before a final diagnosis is rendered. An AI algorithm based only on H&E is unlikely to diagnose such diagnostically challenging cases (18,20). Additionally, sufficient evidence of the effectiveness of the specific targeted therapy that the AI techniques are meant to address must be present before these tools may be implemented in clinical practice. ...
... Long-term storage is costly for most hospitals because WSIs demand hundreds of terabytes of space. Because of current guidelines requiring the storage time of glass slides for a longer time , expenses of storing diagnostic materials is increased (18,(23)(24). Furthermore, AI systems will need access to radiography, WSI, genomic, and in situ hybridization images in order to support the multimodal use case. This will complicate the integration of these disparate data sources (5,7). ...
Artificial intelligence is the future and its use in pathology can create a tremendous impact on health care in different aspects.Its use is being initiated in the field of pathology and is on the rise with a increasing acceptance.Pathology services will undergo a paradigm shift due to the implementation of computational pathology and the use of tools based on AI, which would increase the effectiveness and would be able to satisfy the demands of the precision medicine age. Moving AI models from research to clinical applications has been sluggish, notstanding their success. There may be too much distance and neglect between the clinical setting and self-contained research. The merge of AI technologies into pathology has significantly impacted diagnostic precision and speed. Digital pathology platforms equipped with machine learning algorithms enable pathologists to analyze large volumes of histological images with enhanced accuracy. These systems have demonstrated remarkable capabilities in identifying subtle morphological features indicative of various diseases such as cancerous lesions or infectious conditions. Moreover, AI-driven image analysis tools can assist pathologists in differentiating between benign and malignant tumors by quantifying cellular characteristics beyond human visual perception. Furthermore,AI-powered predictive models have the potential to refine prognostic assessments based on pathological findings. By leveraging vast datasets encompassing clinical outcomes and molecular profiles associated with specific diseases or tissue alterations, these algorithms can generate more tailored predictions regarding disease progression or treatment responsiveness. Through this approach,pathologists can offer more precise guidance on patient management while harnessing valuable insights from diverse sources for optimizing therapeutic intervention.The convergence of advanced image recognition techniques,virtual microscopy,and genomics data analysis could enable comprehensive profiling of individual disease phenotypes at an unprecedented level.In conclusion,AI technologies have already begun reshaping the landscapeof modern pathologypracticesthrough improved diagnostic capabilities,enriched prognostic insights,and envisaged pathways towards personalized healthcare delivery.The seamless integrationofAI-driven solutionsinto daily laboratory workflowswill undeniably propelpathologyintoa new era marked by heightened efficiencyand unparalleled precisionin diagnosticsand therapeuticsupport.
... Despite advances in molecular characterization of biological tumor behavior, morphological tumor classification using established histopathologic techniques remains an important factor in tumor prognostication (Makki, 2015;Soliman and Yussif, 2016). One criterion of particular interest within many tumor grading schemes is the density of cells undergoing division, which are visible as mitotic figures (MFs) in hematoxylin and eosin (H&E)-stained histopathological sections (Veta et al., 2015(Veta et al., , 2019. The number of MFs within a specific tumor area is enumerated by experienced pathologists, resulting in the mitotic count (MC). ...
... Despite the prognostic relevance of the MC, low inter-rater consistency on an object level has been reported in many studies (Veta et al., 2016;Meyer et al., 2005Meyer et al., , 2009Malon et al., 2012;Bertram et al., 2021). The recommendation for pathologists is to select the region of the suspected highest mitotic activity, which is considered to be the best predictor of tumor behavior (Azzola et al., 2003;Meuten et al., 2008;Veta et al., 2015). Selection of this regions of interest (ROI) within the tumor has a great impact on the MC , but is difficult for pathologists to reliably accomplish and is poorly reproducible Bertram et al., 2021). ...
... While this gave rise to algorithm development in the field, it was also an example of questionable dataset quality, as the training and test sets were selected from the same histology slides (Roux et al., 2013). More recent challenges (MITOS2014 (Roux et al., 2014), AMIDA13 (Veta et al., 2015), TUPAC16 (Veta et al., 2019)) also comprised breast cancer and incorporated a higher number of cases, yet, were still limited by having the same digitization device for the training and test set. ...
Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022). The challenge provided annotated histologic tumor images from six different domains and evaluated the algorith-mic approaches for mitotic figure detection provided by nine challenge participants on ten independent domains. Ground truth for mitotic figure detection was established in two ways: a three-expert consensus and an independent, immunohistochemistry-assisted set of labels. This work represents an overview of the challenge tasks, the algorithmic strategies employed by the participants, and potential factors contributing to their success. With an F 1 score of 0.764 for the top-performing team, we summarize that domain generalization across various tumor domains is possible with today's deep learning-based recognition pipelines. When assessed against the immunohistochemistry-assisted reference standard, all methods resulted in reduced recall scores, but with only minor changes in the order of participants in the ranking.
... Histopathological characterization is the only reliable existing method for diagnosing sessile serrated polyps, because other screening methods designed to detect pre-malignant lesions (such as fecal blood, fecal DNA, or virtual colonoscopy) are not well suited for differentiating sessile serrated polyps from other polyps [18]. However, differentiation between sessile serrated polyps and innocuous hyperplastic polyps is a challenging task for pathologists [45,2,16,44]. This is because sessile serrated polyps, like hyperplastic polyps, often lack the dysplastic nuclear changes that characterize conventional adenomatous polyps, and their histopathological diagnosis is entirely based on morphological features, such as serration, dilatation, and branching. ...
... These results are presented in Tables 3 and 4. As we can see in these tables, our wholeslide inferencing approach demonstrates a strong performance across different classes, with an over all accuracy of 93.0%, an over all precision of 89.7%, an over all recall of 88.3%, and an over all F1 score of 88.8%. As can be seen in the presented confusion matrix ( [45,16,44]. ...
Histopathological characterization of colorectal polyps is an important principle for determining the risk of colorectal cancer and future rates of surveillance for patients. This characterization is time-intensive, requires years of specialized training, and suffers from significant inter-observer and intra-observer variability. In this work, we built an automatic image-understanding method that can accurately classify different types of colorectal polyps in whole-slide histology images to help pathologists with histopathological characterization and diagnosis of colorectal polyps. The proposed image-understanding method is based on deep-learning techniques, which rely on numerous levels of abstraction for data representation and have shown state-of-the-art results for various image analysis tasks. Our image-understanding method covers all five polyp types (hyperplastic polyp, sessile serrated polyp, traditional serrated adenoma, tubular adenoma, and tubulovillous/villous adenoma) that are included in the US multi-society task force guidelines for colorectal cancer risk assessment and surveillance, and encompasses the most common occurrences of colorectal polyps. Our evaluation on 239 independent test samples shows our proposed method can identify the types of colorectal polyps in whole-slide images with a high efficacy (accuracy: 93.0%, precision: 89.7%, recall: 88.3%, F1 score: 88.8%). The presented method in this paper can reduce the cognitive burden on pathologists and improve their accuracy and efficiency in histopathological characterization of colorectal polyps, and in subsequent risk assessment and follow-up recommendations.
... Cruz-Roa et al. [16] surveyed and provided an overview of the use of DL and other ML techniques in analyzing breast cancer histopathology images. It covered various aspects, including feature extraction, classification, and tissue segmentation. ...
... Artificial Neural Network (ANN) techniques are frequently employed in the segmentation and classification tasks of breast histopathology images to increase the objectivity and accuracy of Breast histopathology Image Analysis (BHIA). The writers of the review [16] have provided a thorough synopsis of ANN-based BHIA approaches. For a more thorough analysis, they first divided the BHIA systems into deep and conventional neural networks. ...
... Cruz-Roa et al. [16] surveyed and provided an overview of the use of DL and other ML techniques in analyzing breast cancer histopathology images. It covered various aspects, including feature extraction, classification, and tissue segmentation. ...
... Artificial Neural Network (ANN) techniques are frequently employed in the segmentation and classification tasks of breast histopathology images to increase the objectivity and accuracy of Breast histopathology Image Analysis (BHIA). The writers of the review [16] have provided a thorough synopsis of ANN-based BHIA approaches. For a more thorough analysis, they first divided the BHIA systems into deep and conventional neural networks. ...
Breast cancer remains a significant global public health concern, emphasizing the critical role of accurate histopathological analysis in diagnosis and treatment planning. In recent years, the advent of deep learning techniques has showcased notable potential in elevating the precision and efficiency of histopathological data analysis. The proposed work introduces a novel approach that harnesses the power of Transfer Learning to capitalize on knowledge gleaned from pre-trained models, adapting it to the nuanced landscape of breast cancer histopathology. Our proposed model, a Transfer Learning-based concatenated model, exhibits substantial performance enhancements compared to traditional methodologies. Leveraging well-established pretrained models such as VGG-16, MobileNetV2, ResNet50, and DenseNet121—each Convolutional Neural Network architecture designed for classification tasks—this study meticulously tunes hyperparameters to optimize model performance. The implementation of a concatenated classification model is systematically benchmarked against individual classifiers on histopathological data. Remarkably, our concatenated model achieves an impressive training accuracy of 98%. The outcomes of our experiments underscore the efficacy of this four-level concatenated model in advancing the accuracy of breast cancer histopathological data analysis. By synergizing the strengths of deep learning and transfer learning, our approach holds the potential to augment the diagnostic capabilities of pathologists, thereby contributing to more informed and personalized treatment planning for individuals diagnosed with breast cancer. This research heralds a promising stride toward leveraging cutting-edge technology to refine the understanding and management of breast cancer, marking a significant advancement in the intersection of artificial intelligence and healthcare.
... Maintaining a standardized data preparation environment poses another significant challenge" [13]. Careful execution of biopsy, slide preparation, and scanning procedures is essential, as any issues during these processes can lead to subpar performance and data discrepancies [14]. However, these challenges include not only computer diagnosis, but also challenges for manual diagnosis, so it is necessary to develop powerful techniques in artificial intelligence to detect the mitotic number. ...
... The training data was provided only to the contestants, while the training data was maintained by the challenge organizers. The first task of the challenge was to predict the degrees of mitosis, while the second task of the challenge was to predict the gene expression based PAM50 proliferation scores from the WSI [14]. ...
Breast cancer consider as the second cause of death around the world after heart disease, and it is the primary cause of death for women. Timely detection of breast cancer plays a crucial role in lowering mortality rates, as it enhances the patient's prospects of survival through prompt diagnosis and appropriate treatment. The discovery of the mitotic number is one of the necessary procedures that must be performed for a person suffering from breast cancer because it is an important marker for determining the aggressiveness of the tumor. According to the Nottingham scale, it gives 3 degrees to determine the degree of the tumor, whether it is of the first degree, the second degree, or the third degree of seriousness. Deep learning algorithms have many contributions in the medical fields, including in the field of mitotic number discovery, as the mitotic number process is a difficult and tiring task that requires time and effort from pathologists (diagnostic doctors), because the work environment is under microscopes with high magnification degrees, for this reason deep learning techniques were used to reduce the burden on diagnostic doctors and save time for the patient to know the result of his examination, as the biopsy results in developed countries take from 10 days to two weeks for the results to appear. In this survey, we will evaluate the deep learning techniqus employed for mitotic number detection.
... This helps the evaluation process to be more comprehensive and tailored to every individual's demand. (1) One possible approach to apply artificial intelligence in QoL evaluation is by looking at patient-reported outcomes (PRRs), using NLP techniques. Understanding QoL requires knowledge of PROs as they are direct remarks from patients regarding their health conditions that have not been altered or understood by medical professionals. ...
Using artificial intelligence (AI) in the healthcare sector alters doctors' major decision-making process. Evaluating patients' quality of life (QoL) is one area where artificial intelligence seems rather promising. Understanding how various illnesses and therapies influence a person's overall health depends much on quality of life testing. Standard QoL exams, which rely on hand-written assessments and patient comments on their health, have issues like being subjective, biassed, and sluggish when it comes to analyse vast volumes of data. AI-powered testing tools can provide more accurate, quick, scalable methods to evaluate QoL if one is looking for a way around these challenges. This essay examines how artificial intelligence technology could alter the methodology of quality of life surveys. Diagnostics based on artificial intelligence are quite useful. For patient anecdotes, for instance, natural language processing (NLP) may be employed; machine learning techniques can then be used to project QoL values from medical data. AI systems can handle a lot of clinical data including medical records, imaging data, patient-reported results to generate objective, real-time, tailored QoL evaluations consistent and reusable once and again. Furthermore, these instruments can identify early warning indicators of deterioration that would not be evident using more conventional approaches. the usage of several sorts of records sources inclusive of clever tech and cellular fitness apps which increases the accuracy of stories in real time and allows non-stop tracking AI-driven checking out will also be led via This method not handiest courses medical doctors in making better selections however additionally affords people extra manipulate over their fitness, therefore improving their excellent of life over time. The studies additionally addresses moral questions arising from AI-primarily based QoL assessments consisting of data protection, patient permission, and what clinical professionals should do upon assessment of AI outcomes. through discussion of these issues, this take a look at emphasises the need of ensuring that synthetic intelligence generation be applied in a way that complements the interaction among the affected person and company in preference to replaces human know-how.
... During training, we randomly applied several techniques to augment the data and prevent overfitting, namely: (1), pruned from artifacts (2, 3, 4), a subset is manually annotated (5), and then used to train a CNN to distinguish between mitotic and non-mitotic patches (6), named CNN2. Bottom: inference stage, where candidates in a given PHH3 slide (7) are classified with CNN2 as mitotic or non-mitotic (8), then registered to their respective H&E slide pairs (9). rotations, vertical and horizontal mirroring, elastic deformation [32], Gaussian blurring, and translations. ...
Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by: (1) noisy and expensive reference standards established by pathologists, (2) lack of generalization due to staining variation across laboratories, and (3) high computational requirements needed to process gigapixel whole-slide images (WSIs). In this paper, we present a method to train and evaluate CNNs to specifically solve these issues in the context of mitosis detection in breast cancer WSIs. First, by combining image analysis of mitotic activity in phosphohistone-H3 (PHH3) restained slides and registration, we built a reference standard for mitosis detection in entire H&E WSIs requiring minimal manual annotation effort. Second, we designed a data augmentation strategy that creates diverse and realistic H&E stain variations by modifying the hematoxylin and eosin color channels directly. Using it during training combined with network ensembling resulted in a stain invariant mitosis detector. Third, we applied knowledge distillation to reduce the computational requirements of the mitosis detection ensemble with a negligible loss of performance. The system was trained in a single-center cohort and evaluated in an independent multicenter cohort from The Cancer Genome Atlas on the three tasks of the Tumor Proliferation Assessment Challenge (TUPAC). We obtained a performance within the top-3 best methods for most of the tasks of the challenge.
... Furthermore, IHC staining is often subjective and prone to inter-and intra-observer variability. 14,15 Therefore, developing computational methods to infer IHC staining from H&E staining can significantly benefit clinical diagnosis and research. One of the promising approaches to achieving this goal is to use deep learning models that can learn complex nonlinear mappings between H&E and IHC images. ...
In the current study, we introduced a unique method for identifying and segmenting oral squamous cell carcinoma (OSCC) nuclei, concentrating on those predicted to have significant CAF-1/p60 protein expression. Our suggested model uses the StarDist architecture, a deep-learning framework designed for biomedical image segmentation tasks. The training dataset comprises painstakingly annotated masks created from tissue sections previously stained with hematoxylin and eosin (H&E) and then restained with immunohistochemistry (IHC) for p60 protein. Our algorithm uses subtle morphological and colorimetric H&E cellular characteristics to predict CAF-1/p60 IHC expression in OSCC nuclei. The StarDist-based architecture performs exceptionally well in localizing and segmenting H&E nuclei, previously identified by IHC-based ground truth. In summary, our innovative approach harnesses deep learning and multimodal information to advance the automated analysis of OSCC nuclei exhibiting specific protein expression patterns. This methodology holds promise for expediting accurate pathological assessment and gaining deeper insights into the role of CAF-1/p60 protein within the context of oral cancer progression.
... ICPR12 [20], AMIDA13 [57], ICPR14 [58], TUPAC16 [5], MIDOG [59] datasets are widely used for mitosis detection. In this study, ICPR14 and TUPAC16 datasets were used in the training and testing stages to verify the validity of the Mi-DETR model. ...
In histopathological image analysis, the detection and count of mitotic cells are important biomarkers for determining the degree and aggressiveness of cancer prognosis. Manual detection of mitosis by pathologists is a lengthy and challenging process. With advancements in deep learning architectures, numerous automatic mitotic detection methods have been proposed. However, most mitotic detection methods have weak generalizability across image areas and lack reproducibility and validation in multicenter environments. To overcome these issues, a new automatic mitotic detection approach called the Mi-DETR, based on the DETR architecture, has been proposed. In the proposed Mi-DETR model, the backbone of the original DETR is replaced by CSPResNeXt. The aim of this is to strengthen the learning capacity in feature extraction and increase the variability of the learned features. In this way, information loss and unwanted gradient flow are avoided. In the decoder layer, unnecessary model parameters have been filtered out using a layer reduction strategy to improve model efficiency and reduce computational costs. Additionally, a more stable model has been obtained by using the CIoU loss function instead of the L1+GIoU loss function used in the DETR model. The publicly available ICPR14 and TUPAC16 breast histopathology datasets were used for training, validation, and testing in the experiments. The results provided more precise and compact bounding boxes close to clinically validated ground truth, demonstrating the accuracy and generalizability of the proposed model. As a result, the proposed Mi-DETR model achieved a 0.921 F1-Score on the ICPR14 dataset and a 0.950 F1-Score on the TUPAC16 dataset. The results obtained on both datasets demonstrate that the proposed model performs well enough to compete with state-of-the-art deep learning architectures.
... Early, precise endoscopic excision of precancerous lesions is considered the most effective method for preventing colorectal cancer [4]. While colonoscopy is widely regarded as the gold standard for conducting this excision of precancerous lesions, its manual implementation process by gastroenterologists, who are often overworked, is linked with an increased rate of missed cases [5]- [7]. Moreover, the utter reliance on human expertise could also introduce inconsistencies in polyp detection, especially in the early stages when the polyps are small and less detectable. ...
Each year, more than 1.9 million cases of colorectal cancer (CRC) are diagnosed worldwide. By 2040, the burden of colorectal cancer is estimated to reach 3.2 million new cases per year and 1.6 million deaths per year worldwide. As of 2024, it ranks as the third most prevalent form of cancer, contributing to over 10% of all new cancer cases annually, with a 5-year survival rate of only 65%. With effective early detection mechanisms in place, the survival rate could potentially increase to 90%. However, current detection mechanisms are manual and error-prone. This study presents a deep learning-based approach to automating the detection of polyps, the tumor that causes colorectal cancer, in the human colon. Various state-of-the-art deep learning models – including VGG, ResNet, DenseNet, and EfficientNet were trained and tested on a publicly available dataset. The findings of this study show that deep learning models can significantly automate the early diagnosis process of colorectal cancer with high accuracy, especially the DenseNet and EfficientNet models – attaining 99% and 99.4% respectively for both accuracy and F1 score metrics on the test dataset. This study validates the potential of deep learning to enhance the accuracy and reliability of colorectal cancer detection and prevention, ultimately contributing to better quality of diagnosis and patient outcomes.
... The Elston-Ellis revised Scarff-Bloom-Richardson grading system takes into account glandular formation, nuclear features, and mitotic activity [51]. The mitosis detection challenge presented by Veta et al. achieved a total F1 score of 0.61 using a model with a 10-layer deep convolutional neural network [52]. Tellez et al. used PHH3 stains in combination with CNN annotations, which, although not at the state-of-the-art level, demonstrated the potential of DL to improve the consistency of pathologists [53]. ...
Breast cancer is the most common malignant tumor among women worldwide and remains one of the leading causes of death among women. Its incidence and mortality rates are continuously rising. In recent years, with the rapid advancement of deep learning (DL) technology, DL has demonstrated significant potential in breast cancer diagnosis, prognosis evaluation, and treatment response prediction. This paper reviews relevant research progress and applies DL models to image enhancement, segmentation, and classification based on large-scale datasets from TCGA and multiple centers. We employed foundational models such as ResNet50, Transformer, and Hover-net to investigate the performance of DL models in breast cancer diagnosis, treatment, and prognosis prediction. The results indicate that DL techniques have significantly improved diagnostic accuracy and efficiency, particularly in predicting breast cancer metastasis and clinical prognosis. Furthermore, the study emphasizes the crucial role of robust databases in developing highly generalizable models. Future research will focus on addressing challenges related to data management, model interpretability, and regulatory compliance, ultimately aiming to provide more precise clinical treatment and prognostic evaluation programs for breast cancer patients.
... these alterations lead to functional changes in tumor cells, affecting their morphology and microenvironment [22]. hematoxylin and eosin (h&e)stained pathological slides reveal these complex morphological changes, including cell and nucleus shapes and sizes, mitotic activity [23], and alterations in the extracellular matrix [24]. While subtle changes may be difficult to detect with the naked eye, artificial intelligence (ai) technology, particularly deep learning (Dl) algorithms, enables the identification and quantification of these changes from digitized h&e slides [25]. ...
Background
The status of BRCA1/2 genes plays a crucial role in the treatment decision-making process for multiple cancer types. However, due to high costs and limited resources, a demand for BRCA1/2 genetic testing among patients is currently unmet. Notably, not all patients with BRCA1/2 mutations achieve favorable outcomes with poly (ADP-ribose) polymerase inhibitors (PARPi), indicating the necessity for risk stratification. In this study, we aimed to develop and validate a multimodal model for predicting BRCA1/2 gene status and prognosis with PARPi treatment.
Methods
We included 1695 slides from 1417 patients with ovarian, breast, prostate, and pancreatic cancers across three independent cohorts. Using a self-attention mechanism, we constructed a multi-instance attention model (MIAM) to detect BRCA1/2 gene status from hematoxylin and eosin (H&E) pathological images. We further combined tissue features from the MIAM model, cell features, and clinical factors (the MIAM-C model) to predict BRCA1/2 mutations and progression-free survival (PFS) with PARPi therapy. Model performance was evaluated using area under the curve (AUC) and Kaplan-Meier analysis. Morphological features contributing to MIAM-C were analyzed for interpretability.
Results
Across the four cancer types, MIAM-C outperformed the deep learning-based MIAM in identifying the BRCA1/2 genotype. Interpretability analysis revealed that high-attention regions included high-grade tumors and lymphocytic infiltration, which correlated with BRCA1/2 mutations. Notably, high lymphocyte ratios appeared characteristic of BRCA1/2 mutations. Furthermore, MIAM-C predicted PARPi therapy response (log-rank p < 0.05) and served as an independent prognostic factor for patients with BRCA1/2-mutant ovarian cancer (p < 0.05, hazard ratio:0.4, 95% confidence interval: 0.16–0.99).
Conclusions
The MIAM-C model accurately detected BRCA1/2 gene status and effectively stratified prognosis for patients with BRCA1/2 mutations.
... DL can detect both tubular formation (74) and nuclear pleomorphism (75) and count mitotic figures (76)(77)(78)(79)(80)(81). Hence, it can provide accurate histologic tumor grading. ...
... MFs that stained with PHH3 but showed only fine speckling indicated cells in prophase and were excluded from quantitation, as per criteria defined by previous literature on PHH3. [13][14][15][16] Only cells with crisp, coarse staining in the nuclei were included in the count. Once the hotspot was identified, PHH3-MI was determined by counting PHH3-positive cells and expressed as the number of PHH3-positive mitoses in 10 HPF (0.16 mm 2 ). ...
Background
The world health organization (WHO) classification of neuroendocrine neoplasms (NENs, i.e. neuroendocrine tumors (NETs) and neuroendocrine carcinomas (NECs)) of the gastrointestinal system involves grading of these tumors by mitotic count (i.e. H and E mitotic index or Haematoxylin and Eosin mitotic index [HE-MI] and Mindbomb E3 ubiquitin protein ligase 1 labelling index (MIB1-LI) into Grade 1 (G1), Grade 2 (G2), or Grade 3 (G3). However, the assessment of HE-MI and MIB1-LI is hindered by several factors that contribute to discordance between these two grading methods. Clinical data demonstrate the dependency of prognosis on grade.
Objectives
The objective of this study was to compare the grading of NENs of the hepatopancreatobiliary (HPB) system using Anti-phosphohistone H3 mitotic index (i.e. PHH3-MI), HE-MI and MIB1-LI.
Materials and Methods
In a cohort of 140 NENs selected from January 2011 to August 2019, the concordance and correlation between HE-MI, MIB1-LI and PHH3-MI grading methods were analysed using Cohen’s weighted kappa ( κ ) statistics and Spearman’s correlation (ρ), respectively. Receiver operating characteristic (ROC) curve and cut-off analyses were done to determine optimal PHH3-MI cut-off values to grade NENs.
Results
The rates of discordance between HE-MI vs. MIB1-LI, PHH3-MI vs. MIB1-LI and PHH3-MI vs. HE-MI were 52% ( κ =0.416), 29% ( κ =0.64) and 41% ( κ =0.508), respectively. There was a significant correlation between the grading methods. PHH3-MI had good overall sensitivity and specificity at cut-offs 2 and 17 in distinguishing between G1 vs. G2, and G2 vs. G3 tumors, respectively.
Conclusion
PHH3 immunolabeling allowed for quick and easy identification of mitotic figures (MF). It had the highest concordance with MIB1-LI. At cut-off values of 2 and 17, there was good overall sensitivity and specificity. The interobserver agreement was excellent.
... The earliest application of deep learning for image recognition, reported in a 2014 paper by Dong et al. (2015), first proposed the use of deep convolutional neural networks to learn the end-to-end mapping relationship between low-resolution images and high-resolution images in the task of image recognition. Recent advances in deep learning have shown that the diagnosis of various diseases based on the classification of radiographic images and tissue slices is almost beyond human capabilities (Teare et al. 2017;Veta et al. 2015). ...
In recent years, spatial transcriptomics (ST) research has become a popular field of study and has shown great potential in medicine. However, there are few bibliometric analyses in this field. Thus, in this study, we aimed to find and analyze the frontiers and trends of this medical research field based on the available literature. A computerized search was applied to the WoSCC (Web of Science Core Collection) Database for literature published from 2006 to 2023. Complete records of all literature and cited references were extracted and screened. The bibliometric analysis and visualization were performed using CiteSpace, VOSviewer, Bibliometrix R Package software, and Scimago Graphica. A total of 1467 papers and reviews were included. The analysis revealed that the ST publication and citation results have shown a rapid upward trend over the last 3 years. Nature Communications and Nature were the most productive and most co-cited journals, respectively. In the comprehensive global collaborative network, the United States is the country with the most organizations and publications, followed closely by China and the United Kingdom. The author Joakim Lundeberg published the most cited paper, while Patrik L. Ståhl ranked first among co-cited authors. The hot topics in ST are tissue recognition, cancer, heterogeneity, immunotherapy, differentiation, and models. ST technologies have greatly contributed to in-depth research in medical fields such as oncology and neuroscience, opening up new possibilities for the diagnosis and treatment of diseases. Moreover, artificial intelligence and big data drive additional development in ST fields.
... Zhang et al. [23] proposed a classification approach based on genetic markers analyzed from various microarray studies to predict clinical outcomes in cancer patients. However, they observed that multiple markers contributed to the study with a low density ratio when individual data points were widely dispersed, which is considered a significant finding. ...
Invasive breast cancer is a complex global health issue and the leading cause of women's mortality. Multiclassification in breast cancer, especially with high-resolution images, presents unique challenges. Clinical diagnosis relies on the cancer's pathological stage, requiring precise segmentation and adjustments. Complex structural changes during slide preparation and inconsistent image magnifications further complicate classification. To address these challenges, we propose a hybrid machine learning framework for accurate breast cancer detection and grading using large-scale pathological images. Our approach includes an improved Non-restricted Boltzmann Deep Belief Neural Network for nuclei segmentation, followed by feature extraction and novel feature selection using the Giraffe Kicking Optimization algorithm to mitigate overfitting. We implement an Optimal Kernel layer-based Support Vector Machine classifier to identify mitotic cells and nuclear atypia, using the Nottingham Grading System. Validation on the MITOSIS-ATYPIA-14 database demonstrates the framework's effectiveness, with performance metrics including accuracy, precision, recall, specificity, and F-measure. This approach addresses the complexities of breast cancer classification and grading in a streamlined manner, enhancing diagnostic accuracy and prognosis prediction.
... WSI eases the revision of old cases, data sharing and peerreview 9,10 . It has also created several research opportunities within the computer vision domain, especially due to the complexity of the problem and the high dimensions of WSIs [11][12][13][14] . Robust and high-performance systems can be valuable assets to the digital workflow of a laboratory, especially if they are transparent and interpretable 9,10 . ...
Considering the profound transformation affecting pathology practice, we aimed to develop a scalable artificial intelligence (AI) system to diagnose colorectal cancer from whole-slide images (WSI). For this, we propose a deep learning (DL) system that learns from weak labels, a sampling strategy that reduces the number of training samples by a factor of six without compromising performance, an approach to leverage a small subset of fully annotated samples, and a prototype with explainable predictions, active learning features and parallelisation. Noting some problems in the literature, this study is conducted with one of the largest WSI colorectal samples dataset with approximately 10,500 WSIs. Of these samples, 900 are testing samples. Furthermore, the robustness of the proposed method is assessed with two additional external datasets (TCGA and PAIP) and a dataset of samples collected directly from the proposed prototype. Our proposed method predicts, for the patch-based tiles, a class based on the severity of the dysplasia and uses that information to classify the whole slide. It is trained with an interpretable mixed-supervision scheme to leverage the domain knowledge introduced by pathologists through spatial annotations. The mixed-supervision scheme allowed for an intelligent sampling strategy effectively evaluated in several different scenarios without compromising the performance. On the internal dataset, the method shows an accuracy of 93.44% and a sensitivity between positive (low-grade and high-grade dysplasia) and non-neoplastic samples of 0.996. On the external test samples varied with TCGA being the most challenging dataset with an overall accuracy of 84.91% and a sensitivity of 0.996.
... With the continuous advancement of deep learning, convolutional neural networks (CNNs) have provided new solutions for mitotic cell detection (Lecun et al., 2015;Lin et al., 2016;Huang et al., 2017). Concurrently, the availability of publicly accessible datasets featuring expert-annotated images of mitotic cells, such as ICPR MITOS-2012 (Ludovic et al., 2013), AMIDA 2013 (Veta et al., 2015), ICPR MITOS-ATYPIA-2014 (MITOS- ATYPIA-14., 2014), and TUPAC 2016 (Veta et al., 2019), has facilitated the application of deep learning methods in mitotic cell detection. However, these datasets only contain annotated mitotic images corresponding to High Power Fields (HPF) in hotspots and lack annotations for most areas of whole slide images (WSI). ...
Background and object: Mitotic count (MC) is a critical histological parameter for accurately assessing the degree of invasiveness in breast cancer, holding significant clinical value for cancer treatment and prognosis. However, accurately identifying mitotic cells poses a challenge due to their morphological and size diversity.
Objective: We propose a novel end-to-end deep-learning method for identifying mitotic cells in breast cancer pathological images, with the aim of enhancing the performance of recognizing mitotic cells.
Methods: We introduced the Dilated Cascading Network (DilCasNet) composed of detection and classification stages. To enhance the model’s ability to capture distant feature dependencies in mitotic cells, we devised a novel Dilated Contextual Attention Module (DiCoA) that utilizes sparse global attention during the detection. For reclassifying mitotic cell areas localized in the detection stage, we integrate the EfficientNet-B7 and VGG16 pre-trained models (InPreMo) in the classification step.
Results: Based on the canine mammary carcinoma (CMC) mitosis dataset, DilCasNet demonstrates superior overall performance compared to the benchmark model. The specific metrics of the model’s performance are as follows: F1 score of 82.9%, Precision of 82.6%, and Recall of 83.2%. With the incorporation of the DiCoA attention module, the model exhibited an improvement of over 3.5% in the F1 during the detection stage.
Conclusion: The DilCasNet achieved a favorable detection performance of mitotic cells in breast cancer and provides a solution for detecting mitotic cells in pathological images of other cancers.
... Several challenges have been held in order to find novel and improved approaches for mitosis detection [17,22,23,28,29]. Some of these challenges and research on mitosis detection methods have also been conducted using tissue from the canine domain [30][31][32][33]. ...
Simple Summary
Performing a mitosis count (MC) is essential in grading canine Soft Tissue Sarcoma (cSTS) and canine Perivascular Wall Tumours (cPWTs), although it is subject to inter- and intra-observer variability. To enhance standardisation, an artificial intelligence mitosis detection approach was investigated. A two-step annotation process was utilised with a pre-trained Faster R-CNN model, refined through veterinary pathologists’ reviews of false positives, and subsequently optimised using an F1-score thresholding method to maximise accuracy measures. The study achieved a best F1-score of 0.75, demonstrating competitiveness in the field of canine mitosis detection.
Abstract
Performing a mitosis count (MC) is the diagnostic task of histologically grading canine Soft Tissue Sarcoma (cSTS). However, mitosis count is subject to inter- and intra-observer variability. Deep learning models can offer a standardisation in the process of MC used to histologically grade canine Soft Tissue Sarcomas. Subsequently, the focus of this study was mitosis detection in canine Perivascular Wall Tumours (cPWTs). Generating mitosis annotations is a long and arduous process open to inter-observer variability. Therefore, by keeping pathologists in the loop, a two-step annotation process was performed where a pre-trained Faster R-CNN model was trained on initial annotations provided by veterinary pathologists. The pathologists reviewed the output false positive mitosis candidates and determined whether these were overlooked candidates, thus updating the dataset. Faster R-CNN was then trained on this updated dataset. An optimal decision threshold was applied to maximise the F1-score predetermined using the validation set and produced our best F1-score of 0.75, which is competitive with the state of the art in the canine mitosis domain.
... Keywords: Mutation prediction, Machine learning, Representation learning, pan-cancer setting, dataset, self-supervised learning Cancer diagnosis heavily relies the examination of H&E stained tissue slides, which offer crucial insights into the disease and potential treatment options and which are routinely acquired in pathology labs. Digitizing these slides into Whole Slide Images (WSI) enables automated analysis, aiming at assisting clinicians in executing tedious tasks, such as counting mitoses [1], identification of metastases [2] and grading [3]. Furthermore, the availability of large data repositories, such as The Cancer Genome Atlas (TCGA) provides us with the challenging opportunity to identify morphological biomarkers related to survival [4] or treatment response [5], and to unravel the complex genotype-phenotype relationships by building predictive models for molecular features, such as single gene mutations [6], mutational signatures [7] and molecular subtypes [6]. ...
Automatic analysis of hematoxylin and eosin (H\&E) stained Whole Slide Images (WSI) bears great promise for computer assisted diagnosis and biomarker discovery. However, scarcity of annotated datasets leads to underperforming models. Furthermore, the size and complexity of the image data limit their integration into bioinformatic workflows and thus their adoption by the bioinformatics community.
Here, we present Giga-SSL, a self-supervised method for learning WSI representations without any annotation. We show that applying a simple linear classifier on the Giga-SSL representations improves classification performance over the fully supervised alternative on five benchmarked tasks and across different datasets. Moreover, we observe a substantial performance increase for small datasets (average gain of 7 AUC point) and a doubling of the number of mutations predictable from WSIs in a pan-cancer setting (from 45 to 93).
We make the WSI representations available, compressing the TCGA-FFPE images from 12TB to 23MB and enabling fast analysis on a laptop CPU. We hope this resource will facilitate multimodal data integration in order to analyze WSI in their genomic and transcriptomic context.
... For instance, the proliferative activity of breast tumors is routinely estimated by counting mitotic figures. In this context, deep learning approaches involving multi-layered CNNs for unsupervised feature learning have demonstrated an accuracy in counting mitotic figures comparable to the level of inter-observer variability [80]. Additionally, some have proposed a method for nuclei detection in colorectal adenocarcinoma by pinpointing their center positions [81]. ...
Introduction:
Artificial intelligence (AI) has the potential to transform oncologic care. There have been significant developments in AI applications in medical imaging and increasing interest in multimodal models. These are likely to enable improved oncologic care through more precise diagnosis, increasingly in a more personalized and less invasive manner. In this review, we provide an overview of the current state and challenges that clinicians, administrative personnel and policy makers need to be aware of and mitigate for the technology to reach its full potential.
Areas covered:
The article provides a brief targeted overview of AI, a high-level review of the current state and future potential AI applications in diagnostic radiology and to a lesser extent digital pathology, focusing on oncologic applications. This is followed by a discussion of emerging approaches, including multimodal models. The article concludes with a discussion of technical, regulatory challenges and infrastructure needs for AI to realize its full potential.
Expert opinion:
There is a large volume of promising research, and steadily increasing commercially available tools using AI. For the most advanced and promising precision diagnostic applications of AI to be used clinically, robust and comprehensive quality monitoring systems and informatics platforms will likely be required.
... Since deep learning exhibited outstanding performance in the ImageNet challenge [1], researchers have actively applied deep learning to pathological image analysis. Deep learning showed excellent performance in multiple challenges such as mitosis detection [2,3], breast cancer classification [4], and gland segmentation [5]. Currently, deep learning is widely used as a core algorithm in many pathological image analysis challenges [6]. ...
We propose a novel transfer learning framework for pathological image analysis, the Response-based Cross-task Knowledge Distillation (RCKD), which improves the performance of the model by pretraining it on a large unlabeled dataset guided by a high-performance teacher model. RCKD first pretrains a student model to predict the nuclei segmentation results of the teacher model for unlabeled pathological images, and then fine-tunes the pretrained model for the downstream tasks, such as organ cancer sub-type classification and cancer region segmentation, using relatively small target datasets. Unlike conventional knowledge distillation, RCKD does not require that the target tasks of the teacher and student models be the same. Moreover, unlike conventional transfer learning, RCKD can transfer knowledge between models with different architectures. In addition, we propose a lightweight architecture, the Convolutional neural network with Spatial Attention by Transformers (CSAT), for processing high-resolution pathological images with limited memory and computation. CSAT exhibited a top-1 accuracy of 78.6% on ImageNet with only 3M parameters and 1.08 G multiply-accumulate (MAC) operations. When pretrained by RCKD, CSAT exhibited average classification and segmentation accuracies of 94.2% and 0.673 mIoU on six pathological image datasets, which is 4% and 0.043 mIoU higher than EfficientNet-B0, and 7.4% and 0.006 mIoU higher than ConvNextV2-Atto pretrained on ImageNet, respectively.
Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the information that is important for the classification based on training data. The method requires a single anatomical MR image only. The segmentation method is applied to five different data sets: coronal T2-weighted images of preterm infants acquired at 30 weeks postmenstrual age (PMA) and 40 weeks PMA, axial T2- weighted images of preterm infants acquired at 40 weeks PMA, axial T1-weighted images of ageing adults acquired at an average age of 70 years, and T1-weighted images of young adults acquired at an average age of 23 years. The method obtained the following average Dice coefficients over all segmented tissue classes for each data set, respectively: 0.87, 0.82, 0.84, 0.86 and 0.91. The results demonstrate that the method obtains accurate segmentations in all five sets, and hence demonstrates its robustness to differences in age and acquisition protocol.
Mitosis detection, a crucial biomedical process, faces challenges like cell morphology variability, poor contrast, overcrowding, and limited annotated dataset availability. This research presents a novel method for mitosis detection in histopathological images highlighting two important contributions using a Bi-wolf optimization-based LP norm regularized deep Convolutional neural network (CNN) model. This hybrid optimization protocol is the key to the precise calibration of model parameters and effective training, which translates into optimal classifier performance. The results reveal that this model achieves high accuracy, sensitivity, and specificity values of 96.69%, 91.89%, and 97.74% respectively.
Abstract:
One of the most frequently diagnosed cancers in women is breast cancer. Mitotic cells in breast histopathological images are a very important biomarker to diagnose breast cancer. Mitotic scores help medical professionals to grade breast cancer appropriately. The procedure of identifying mitotic cells is quite time-consuming. To speed up and improve the process, automated deep learning methods can be used. The suggested study aims to conduct analysis on the detection of mitotic cells using U-Net and modified VGG16 technique. In this study, pre-processing of the input images is done using stain normalization and enhancement processes. A modified VGG16 classifier is used to classify the segmented results after the altered image has been segmented using U-Net technology. The suggested method's robustness is evaluated using data from the MITOSIS 2012 dataset. The proposed strategy performed better with a precision of 86%,recall of 75% and F1-Score of 80%.
Introduction. The identification of mitotic figures is essential for the diagnosis, grading, and classification of various different tumors. Despite its importance, there is a paucity of literature reporting the consistency in interpreting mitotic figures among pathologists. This study leverages publicly accessible datasets and social media to recruit an international group of pathologists to score an image database of more than 1000 mitotic figures collectively. Materials and Methods. Pathologists were instructed to randomly select a digital slide from The Cancer Genome Atlas (TCGA) datasets and annotate 10-20 mitotic figures within a 2 mm² area. The first 1010 submitted mitotic figures were used to create an image dataset, with each figure transformed into an individual tile at 40x magnification. The dataset was redistributed to all pathologists to review and determine whether each tile constituted a mitotic figure. Results. Overall pathologists had a median agreement rate of 80.2% (range 42.0%-95.7%). Individual mitotic figure tiles had a median agreement rate of 87.1% and a fair inter-rater agreement across all tiles (kappa = 0.284). Mitotic figures in prometaphase had lower percentage agreement rates compared to other phases of mitosis. Conclusion. This dataset stands as the largest international consensus study for mitotic figures to date and can be utilized as a training set for future studies. The agreement range reflects a spectrum of criteria that pathologists use to decide what constitutes a mitotic figure, which may have potential implications in tumor diagnostics and clinical management.
Breast cancer holds the second position for cancer deaths in women [12]. There are several Computer Aided Detection and Diagnosis (CAD) systems used today in order to aid radiologists in detecting malignant cancers at the early stage. Such systems along with suitable classifiers yield better prediction of cancerous masses. This paper presents a logistic regression model based mass detection and classification based on selected geometrical features from breast DICOM images with an accuracy of 93%. Previous work of Alima et al, resulted in an accuracy of 91% using ANN[4]. The performance of the feature extraction and classification system is developed using the database collected as a part of the dream challenge[2]. Performance results are given in terms of confusion matrices. 1. INTRODUCTION The morbidity and mortality rates of women affected with breast cancer are increasing day by day. The breast tissue comprises of glands used for milk production. These glands are otherwise called lobules. Breast cancer develops from these lobules and then spread to the ducts. The ducts are the connections between the lobules and the nipple. The rapid increase in mortality rate can be reduced as a result of an efficient detection of cancer. Namely, the breast regions classification is between mass and non-mass. There are several techniques for classifying benign and malignant neoplasy from digital images. Digital breast images are efficiently captured using mammography. X-Rays mammography is mainly used to detect asymmetry if any between the left and right breasts due to its Low amplitude [3]. This information is used by radiologist for detecting microcalcifications. In ultrasound technique, reflections from high frequency sound waves incident on breast tissues is used for imaging. Temperature variance in normal region and mass region is considered for breast thermography. It is an adjunct tool for accurate prediction of masses. Apart from these, for high risk women and to evaluate dense breasts, breast MRI of good resolution can be used [5]. Once the image acquisition phase is carried out, they have to be efficiently classified. For image classification, there are several conventional algorithms. Predictive analysis requires strong algorithms like SVM where a linear kernel is opted for a linear (separable) dataset and Gaussian kernel for a non-linear (non-separable) dataset classification. Binomial or multinomial logistic regression using sigmoid function is another method for classification. The two possible outcomes of binomial (binary) logistic regression give a straight forward interpretation of data. Input containing K closest training sets as neighbouring features is the K Nearest Neighbour (KNN) algorithm [3]. It is a non-parametric method for classification. Assumption of independence between each feature is the idea behind a Naïve Bayes classifier. It is named so because it works on the Bayes theorem for classification.
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field’s future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
The rapid development of precision medicine in recent years has started to challenge diagnostic pathology with respect to its ability to analyze histological images and increasingly large molecular profiling data in a quantitative, integrative, and standardized way. Artificial intelligence (AI) and, more precisely, deep learning technologies have recently demonstrated the potential to facilitate complex data analysis tasks, including clinical, histological, and molecular data for disease classification; tissue biomarker quantification; and clinical outcome prediction. This review provides a general introduction to AI and describes recent developments with a focus on applications in diagnostic pathology and beyond. We explain limitations including the black-box character of conventional AI and describe solutions to make machine learning decisions more transparent with so-called explainable AI. The purpose of the review is to foster a mutual understanding of both the biomedical and the AI side. To that end, in addition to providing an overview of the relevant foundations in pathology and machine learning, we present worked-through examples for a better practical understanding of what AI can achieve and how it should be done.
Expected final online publication date for the Annual Review of Pathology: Mechanisms of Disease, Volume 19 is January 2024. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by “engineered” methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.
Pathology clinical practice has evolved by adopting technological advancements initially regarded as potentially disruptive, such as electron microscopy, immunohistochemistry, and genomic sequencing. Breast pathology has a critical role as a medical domain, where the patient’s pathology diagnosis has significant implications for prognostication and treatment of diseases. The advent of digital and computational pathology has brought about significant advancements in the field, offering new possibilities for enhancing diagnostic accuracy and improving patient care. Digital slide scanning enables to conversion of glass slides into high-fidelity digital images, supporting the review of cases in a digital workflow. Digitization offers the capability to render specimen diagnoses, digital archival of patient specimens, collaboration, and telepathology. Integration of image analysis and machine learning–based systems layered atop the high-resolution digital images offers novel workflows to assist breast pathologists in their clinical, educational, and research endeavors. Decision support tools may improve the detection and classification of breast lesions and the quantification of immunohistochemical studies. Computational biomarkers may help to contribute to patient management or outcomes. Furthermore, using digital and computational pathology may increase standardization and quality assurance, especially in areas with high interobserver variability. This review explores the current landscape and possible future applications of digital and computational techniques in the field of breast pathology.
Breast cancer is one of the most common cancers and is one of the leading causes of cancer-related deaths in women worldwide. Early diagnosis and treatment are the key for a favorable prognosis. The application of artificial intelligence technology in the medical field is increasingly extensive, including image analysis, automated diagnosis, intelligent pharmaceutical system, personalized treatment and so on. AI-based breast cancer imaging, pathology and adjuvant therapy technology cannot only reduce the workload of clinicians, but also continuously improve the accuracy and sensitivity of breast cancer diagnosis and treatment. This paper reviews the application of AI in breast cancer, as well as looks ahead and poses challenges to the future development of AI for breast cancer detection and therapeutic, so as to provide ideas for future research.
Breast cancer (BCa) grading plays an important role in predicting disease aggressiveness and patient outcome. A key component of BCa grade is mitotic count, which involves quantifying the number of cells in the process of dividing (i.e. undergoing mitosis) at a specific point in time. Currently mitosis counting is done manually by a pathologist looking at multiple high power fields on a glass slide under a microscope, an extremely laborious and time consuming process. The development of computerized systems for automated detection of mitotic nuclei, while highly desirable, is confounded by the highly variable shape and appearance of mitoses. Existing methods use either handcrafted features that capture certain morphological, statistical or textural attributes of mitoses or features learned with convolutional neural networks (CNN). While handcrafted features are inspired by the domain and the particular application, the data-driven CNN models tend to be domain agnostic and attempt to learn additional feature bases that cannot be represented through any of the handcrafted features. On the other hand, CNN is computationally more complex and needs a large number of labeled training instances. Since handcrafted features attempt to model domain pertinent attributes and CNN approaches are largely unsupervised feature generation methods, there is an appeal to attempting to combine these two distinct classes of feature generation strategies to create an integrated set of attributes that can potentially outperform either class of feature extraction strategies individually. In this paper, we present a cascaded approach for mitosis detection that intelligently combines a CNN model and handcrafted features (morphology, color and texture features). By employing a light CNN model, the proposed approach is far less demanding computationally, and the cascaded strategy of combining handcrafted features and CNN-derived features enables the possibility of maximizing performance by leveraging the disconnected feature sets. Evaluation on the public ICPR12 mitosis dataset that has 226 mitoses annotated on 35 High Power Fields (HPF, x400 magnification) by several pathologists and 15 testing HPFs yielded an F-measure of 0.7345. Apart from this being the second best performance ever recorded for this MITOS dataset, our approach is faster and requires fewer computing resources compared to extant methods, making this feasible for clinical use.
This paper presents an overview of methods that have been proposed for the analysis of breast cancer histopathology images. This research area has become particularly relevant with the advent of whole slide imaging (WSI) scanners, which can perform cost-effective and high-throughput histopathology slide digitization, and which aim at replacing the optical microscope as the primary tool used by pathologist. Breast cancer is the most prevalent form of cancers among women, and image analysis methods that target this disease have a huge potential to reduce the workload in a typical pathology lab and to improve the quality of the interpretation. This paper is meant as an introduction for nonexperts. It starts with an overview of the tissue preparation, staining and slide digitization processes followed by a discussion of the different image processing techniques and applications, ranging from analysis of tissue staining to computer-aided diagnosis, and prognosis of breast cancer patients.
Histopathology diagnosis is based on visual examination of the morphology of histological sections under a microscope. With the increasing popularity of digital slide scanners, decision support systems based on the analysis of digital pathology images are in high demand. However, computerized decision support systems are fraught with problems that stem from color variations in tissue appearance due to variation in tissue preparation, variation in stain reactivity from different manufacturers/batches, user or protocol variation, and the use of scanners from different manufacturers. In this paper, we present a novel approach to stain normalization in histopathology images. The method is based on nonlinear mapping of a source image to a target image using a representation derived from color deconvolution. Color deconvolution is a method to obtain stain concentration values when the stain matrix, describing how the color is affected by the stain concentration, is given. Rather than relying on standard stain matrices, which may be inappropriate for a given image, we propose the use of a color-based classifier that incorporates a novel stain color descriptor to calculate image-specific stain matrix. In order to demonstrate the efficacy of the proposed stain matrix estimation and stain normalization methods, they are applied to the problem of tumor segmentation in breast histopathology images. The experimental results suggest that the paradigm of color normalization, as a preprocessing step, can significantly help histological image analysis algorithms to demonstrate stable performance which is insensitive to imaging conditions in general and scanner variations in particular.
Mitotic Activity Index (MAI) is an important independent prognostic factor and an integral part of the breast cancer grading system. Thus, correct estimation of this prognostically relevant feature is essential for guiding treatment decision and assessing patient prognosis. The aim of this study was to validate the use of high resolution Whole Slide Images (WSI) in estimating MAI in breast cancer specimens.
MAI was evaluated in 100 consecutive breast cancer specimens by three observers on two occasions, microscopically and on WSI with a wash out period of 4 months. MAI was also translated to mitotic scores as in grading. Inter- and intra-observer agreement between microscopic and digital MAI counts and scores was measured.
Almost perfect inter-observer agreements were obtained from counting MAI using a conventional microscope (intra-class correlation coefficient (ICCC) 0.879) as well as on WSI (ICCC 0.924). K coefficients reflected good inter-observer agreements among observers' microscopic mitotic scores (average kappa 0.642). Comparable results were also observed among digital mitotic scores (average kappa 0.635). There was strong to perfect intra-observer agreements between MAI counts and mitotic scores for the two diagnostic modalities (ICCC 0.716-0.863, kappa 0.506-0.617). There were no significant differences in mitotic scores using both diagnostic modalities.
Scoring mitoses using WSI in breast cancer seems to be just as reliable and reproducible as when using a microscope. Further development of software and image quality will definitely encourage the use of WSI in routine pathology practice.
During the last years, whole slide imaging has become more affordable and widely accepted in pathology labs. Digital slides are increasingly being used for digital archiving of routinely produced clinical slides, remote consultation and tumor boards, and quantitative image analysis for research purposes and in education. However, the implementation of a fully digital Pathology Department requires an in depth look into the suitability of digital slides for routine clinical use (the image quality of the produced digital slides and the factors that affect it) and the required infrastructure to support such use (the storage requirements and integration with lab management and hospital information systems). Optimization of digital pathology workflow requires communication between several systems, which can be facilitated by the use of open standards for digital slide storage and scanner management. Consideration of these aspects along with appropriate validation of the use of digital slides for routine pathology can pave the way for pathology departments to go "fully digital." In this paper, we summarize our experiences so far in the process of implementing a fully digital workflow at our Pathology Department and the steps that are needed to complete this process.
Mitosis count is one of the factors that pathologists use to assess the risk of metastasis and survival of the patients, which are affected by the breast cancer.
We investigate an application of a set of generic features and an ensemble of cascade adaboosts to the automated mitosis detection. Calculation of the features rely minimally on object-level descriptions and thus require minimal segmentation.
The proposed work was developed and tested on International Conference on Pattern Recognition (ICPR) 2012 mitosis detection contest data.
We plotted receiver operating characteristics curves of true positive versus false positive rates; calculated recall, precision, F-measure, and region overlap ratio measures.
WE TESTED OUR FEATURES WITH TWO DIFFERENT CLASSIFIER CONFIGURATIONS: 1) An ensemble of single adaboosts, 2) an ensemble of cascade adaboosts. On the ICPR 2012 mitosis detection contest evaluation, the cascade ensemble scored 54, 62.7, and 58, whereas the non-cascade version scored 68, 28.1, and 39.7 for the recall, precision, and F-measure measures, respectively. Mostly used features in the adaboost classifier rules were a shape-based feature, which counted granularity and a color-based feature, which relied on Red, Green, and Blue channel statistics.
The features, which express the granular structure and color variations, are found useful for mitosis detection. The ensemble of adaboosts performs better than the individual adaboost classifiers. Moreover, the ensemble of cascaded adaboosts was better than the ensemble of single adaboosts for mitosis detection.
In this paper, we propose a statistical approach for mitosis detection in breast cancer histological images. The proposed algorithm models the pixel intensities in mitotic and non-mitotic regions by a Gamma-Gaussian mixture model (GGMM) and employs a context aware post-processing (CAPP) in order to reduce false positives. Experimental results demonstrate the ability of this simple, yet effective method to detect mitotic cells (MCs) in standard H & E breast cancer histology images.
Counting of MCs in breast cancer histopathology images is one of three components (the other two being tubule formation, nuclear pleomorphism) required for developing computer assisted grading of breast cancer tissue slides. This is very challenging since the biological variability of the MCs makes their detection extremely difficult. In addition, if standard H & E is used (which stains chromatin rich structures, such as nucleus, apoptotic, and MCs dark blue) and it becomes extremely difficult to detect the latter given the fact that former two are densely localized in the tissue sections.
In this paper, a robust MCs detection technique is developed and tested on 35 breast histopathology images, belonging to five different tissue slides.
Our approach mimics a pathologists' approach to MCs detections. The idea is (1) to isolate tumor areas from non-tumor areas (lymphoid/inflammatory/apoptotic cells), (2) search for MCs in the reduced space by statistically modeling the pixel intensities from mitotic and non-mitotic regions, and finally (3) evaluate the context of each potential MC in terms of its texture.
Our experimental dataset consisted of 35 digitized images of breast cancer biopsy slides with paraffin embedded sections stained with H and E and scanned at × 40 using an Aperio scanscope slide scanner.
We propose GGMM for detecting MCs in breast histology images. Image intensities are modeled as random variables sampled from one of the two distributions; Gamma and Gaussian. Intensities from MCs are modeled by a gamma distribution and those from non-mitotic regions are modeled by a gaussian distribution. The choice of Gamma-Gaussian distribution is mainly due to the observation that the characteristics of the distribution match well with the data it models. The experimental results show that the proposed system achieves a high sensitivity of 0.82 with positive predictive value (PPV) of 0.29. Employing CAPP on these results produce 241% increase in PPV at the cost of less than 15% decrease in sensitivity.
In this paper, we presented a GGMM for detection of MCs in breast cancer histopathological images. In addition, we introduced CAPP as a tool to increase the PPV with a minimal loss in sensitivity. We evaluated the performance of the proposed detection algorithm in terms of sensitivity and PPV over a set of 35 breast histology images selected from five different tissue slides and showed that a reasonably high value of sensitivity can be retained while increasing the PPV. Our future work will aim at increasing the PPV further by modeling the spatial appearance of regions surrounding mitotic events.
According to Nottingham grading system, mitosis count plays a critical role in cancer diagnosis and grading. Manual counting of mitosis is tedious and subject to considerable inter- and intra-reader variations.
The aim is to improve the accuracy of mitosis detection by selecting the color channels that better capture the statistical and morphological features, which classify mitosis from other objects.
We propose a framework that includes comprehensive analysis of statistics and morphological features in selected channels of various color spaces that assist pathologists in mitosis detection. In candidate detection phase, we perform Laplacian of Gaussian, thresholding, morphology and active contour model on blue-ratio image to detect and segment candidates. In candidate classification phase, we extract a total of 143 features including morphological, first order and second order (texture) statistics features for each candidate in selected channels and finally classify using decision tree classifier.
The proposed method has been evaluated on Mitosis Detection in Breast Cancer Histological Images (MITOS) dataset provided for an International Conference on Pattern Recognition 2012 contest and achieved 74% and 71% detection rate, 70% and 56% precision and 72% and 63% F-Measure on Aperio and Hamamatsu images, respectively.
The proposed multi-channel features computation scheme uses fixed image scale and extracts nuclei features in selected channels of various color spaces. This simple but robust model has proven to be highly efficient in capturing multi-channels statistical features for mitosis detection, during the MITOS international benchmark. Indeed, the mitosis detection of critical importance in cancer diagnosis is a very challenging visual task. In future work, we plan to use color deconvolution as preprocessing and Hough transform or local extrema based candidate detection in order to reduce the number of candidates in mitosis and non-mitosis classes.
In the framework of the Cognitive Microscope (MICO) project, we have set up a contest about mitosis detection in images of H and E stained slides of breast cancer for the conference ICPR 2012. Mitotic count is an important parameter for the prognosis of breast cancer. However, mitosis detection in digital histopathology is a challenging problem that needs a deeper study. Indeed, mitosis detection is difficult because mitosis are small objects with a large variety of shapes, and they can thus be easily confused with some other objects or artefacts present in the image. We added a further dimension to the contest by using two different slide scanners having different resolutions and producing red-green-blue (RGB) images, and a multi-spectral microscope producing images in 10 different spectral bands and 17 layers Z-stack. 17 teams participated in the study and the best team achieved a recall rate of 0.7 and precision of 0.89.
Several studies on automatic tools to process digitized slides have been reported focusing mainly on nuclei or tubule detection. Mitosis detection is a challenging problem that has not yet been addressed well in the literature.
Mitotic count is an important parameter in breast cancer grading as it gives an evaluation of the aggressiveness of the tumor. However, consistency, reproducibility and agreement on mitotic count for the same slide can vary largely among pathologists. An automatic tool for this task may help for reaching a better consistency, and at the same time reducing the burden of this demanding task for the pathologists.
Professor Frιdιrique Capron team of the pathology department at Pitiι-Salpκtriθre Hospital in Paris, France, has selected a set of five slides of breast cancer. The slides are stained with H and E. They have been scanned by three different equipments: Aperio ScanScope XT slide scanner, Hamamatsu NanoZoomer 2.0-HT slide scanner and 10 bands multispectral microscope. The data set is made up of 50 high power fields (HPF) coming from 5 different slides scanned at ×40 magnification. There are 10 HPFs/slide. The pathologist has annotated all the mitotic cells manually. A HPF has a size of 512 μm × 512 μm (that is an area of 0.262 mm (2) , which is a surface equivalent to that of a microscope field diameter of 0.58 mm. These 50 HPFs contain a total of 326 mitotic cells on images of both scanners, and 322 mitotic cells on the multispectral microscope.
Up to 129 teams have registered to the contest. However, only 17 teams submitted their detection of mitotic cells. The performance of the best team is very promising, with F-measure as high as 0.78. However, the database we provided is by far too small for a good assessment of reliability and robustness of the proposed algorithms.
Mitotic count is an important criterion in the grading of many types of cancers, however, very little research has been made on automatic mitotic cell detection, mainly because of a lack of available data. A main objective of this contest was to propose a database of mitotic cells on digitized breast cancer histopathology slides to initiate works on automated mitotic cell detection. In the future, we would like to extend this database to have much more images from different patients and also for different types of cancers. In addition, mitotic cells should be annotated by several pathologists to reflect the partial agreement among them.
Deep Neural Networks now excel at image classification, detection and
segmentation. When used to scan images by means of a sliding window, however,
their high computational complexity can bring even the most powerful hardware
to its knees. We show how dynamic programming can speedup the process by orders
of magnitude, even when max-pooling layers are present.
This paper introduces a robust variant of AdaBoost, cw-AdaBoost, that uses weight perturbation to reduce variance error, and is particularly effective when dealing with data sets, such
as microarray data, which have large numbers of features and small number of instances. The algorithm is compared with AdaBoost,
Arcing and MultiBoost, using twelve gene expression datasets, using 10-fold cross validation. The new algorithm consistently
achieves higher classification accuracy over all these datasets. In contrast to other AdaBoost variants, the algorithm is
not susceptible to problems when a zero-error base classifier is encountered.
KeywordsBoosting-Bagging-Arcing-Multiboost-Ensemble machine learning-Random resampling weighted instances-Variance error-Bias error
Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe.
Class imbalance is a problem that is common to many application domains. When examples of one class in a training data set vastly outnumber examples of the other class(es), traditional data mining algorithms tend to create suboptimal classification models. Several techniques have been used to alleviate the problem of class imbalance, including data sampling and boosting. In this paper, we present a new hybrid sampling/boosting algorithm, called RUSBoost, for learning from skewed training data. This algorithm provides a simpler and faster alternative to SMOTEBoost, which is another algorithm that combines boosting and data sampling. This paper evaluates the performances of RUSBoost and SMOTEBoost, as well as their individual components (random undersampling, synthetic minority oversampling technique, and AdaBoost). We conduct experiments using 15 data sets from various application domains, four base learners, and four evaluation metrics. RUSBoost and SMOTEBoost both outperform the other procedures, and RUSBoost performs comparably to (and often better than) SMOTEBoost while being a simpler and faster technique. Given these experimental results, we highly recommend RUSBoost as an attractive alternative for improving the classification performance of learners built using imbalanced data.
Despite the prognostic importance of mitotic count as one of the components of the Bloom-Richardson grade, several studies have found that pathologists' agreement on the mitotic grade is fairly modest. Collecting a set of more than 4,200 candidate mitotic figures, we evaluate pathologists' agreement on individual figures, and train a computerized system for mitosis detection, comparing its performance to the classifications of three pathologists. The system's and the pathologists' classifications are based on evaluation of digital micrographs of hematoxylin and eosin stained breast tissue. On figures where the majority of pathologists agree on a classification, we compare the performance of the trained system to that of the individual pathologists. We find that the level of agreement of the pathologists ranges from slight to moderate, with strong biases, and that the system performs competitively in rating the ground truth set. This study is a step towards automatic mitosis count to accelerate a pathologist's work and improve reproducibility.
Digital slide scanners for scanning glass slides are becoming increasingly popular because current scanners are fast enough and produce good enough images for diagnostic purposes, education, and research. Also, the price for storing vast amounts of data has decreased over the last years, and this trend is expected to continue. Where most laboratories use their scanners mainly for education and research with limited financial and technical implications, we decided to face the huge challenges of prospectively setting up a fully digital pathology slide archive, primarily aiming to optimize the preparation and running of clinicopathological conferences. In this article, we describe the setup of our digital archiving solution and discuss the technical challenges we had to overcome. To give insight in the performance of our digital archive, we provide some statistics as well. We also present our thoughts on future developments in the area of digital slide scanning.
Live-cell imaging allows detailed dynamic cellular phenotyping for cell biology and, in combination with small molecule or drug libraries, for high-content screening. Fully automated analysis of live cell movies has been hampered by the lack of computational approaches that allow tracking and recognition of individual cell fates over time in a precise manner. Here, we present a fully automated approach to analyze time-lapse movies of dividing cells. Our method dynamically categorizes cells into seven phases of the cell cycle and five aberrant morphological phenotypes over time. It reliably tracks cells and their progeny and can thus measure the length of mitotic phases and detect cause and effect if mitosis goes awry. We applied our computational scheme to annotate mitotic phenotypes induced by RNAi gene knockdown of CKAP5 (also known as ch-TOG) or by treatment with the drug nocodazole. Our approach can be readily applied to comparable assays aiming at uncovering the dynamic cause of cell division phenotypes.
To develop a flexible method of separation and quantification of immunohistochemical staining by means of color image analysis.
An algorithm was developed to deconvolve the color information acquired with red-green-blue (RGB) cameras and to calculate the contribution of each of the applied stains based on stain-specific RGB absorption. The algorithm was tested using different combinations of diaminobenzidine, hematoxylin and eosin at different staining levels.
Quantification of the different stains was not significantly influenced by the combination of multiple stains in a single sample. The color deconvolution algorithm resulted in comparable quantification independent of the stain combinations as long as the histochemical procedures did not influence the amount of stain in the sample due to bleaching because of stain solubility and saturation of staining was prevented.
This image analysis algorithm provides a robust and flexible method for objective immunohistochemical analysis of samples stained with up to three different stains using a laboratory microscope, standard RGB camera setup and the public domain program NIH Image.
Counting of mitotic cells has been shown to be of prognostic value in breast cancer in different retrospective studies. Up to now the number of mitoses is assessed mainly manually according to a standardized but strict protocol. Although such a manual procedure is reasonably reproducible, automatic counting of mitotic cells offers the potential for greater objectivity and reproducibility. This paper describes the influence of resolution on automatic recognition by image processing of mitotic cells in Feulgen stained breast cancer sections. Using the image recording, correction and segmentation procedure described in a previous study, five specimens were analyzed: one was used to serve as a training set and four were put aside for later use as independent test set. For each slide, objects from a pre‐selected area were recorded at increasing resolution. For each object, contour features and optical density measurements were computed and stored in a data file for statistical analysis. The results showed that increased resolution using a 40× objective lowered the number of misclassified mitoses compared with a 20× objective (overall mean percentage of misclassified mitoses over training and all test specimens: 20x, 24.57; 40x, 7.96). The number of misclassifications of non‐mitoses was almost stable per specimen but varied between specimens (19–42%) due to differences among tissues.
Given the improvement in classifying mitoses and the possibility to evaluate interactively the measurement result, the described semi‐automated mitoses pre‐screener of histological sections may be suitable for further testing in a clinical setting. Cytometry 28:135–140, 1997. © 1997 Wiley‐Liss, Inc.
Morphological assessment of the degree of differentiation has been shown in numerous studies to provide useful prognostic information in breast cancer, but until recently histological grading has not been accepted as a routine procedure, mainly because of perceived problems with reproducibility and consistency. In the Nottingham/Tenovus Primary Breast Cancer Study the most commonly used method, described by Bloom & Richardson, has been modified in order to make the criteria more objective. The revised technique involves semiquantitative evaluation of three morphological features--the percentage of tubule formation, the degree of nuclear pleomorphism and an accurate mitotic count using a defined field area. A numerical scoring system is used and the overall grade is derived from a summation of individual scores for the three variables: three grades of differentiation are used. Since 1973, over 2200 patients with primary operable breast cancer have been entered into a study of multiple prognostic factors. Histological grade, assessed in 1831 patients, shows a very strong correlation with prognosis; patients with grade I tumours have a significantly better survival than those with grade II and III tumours (P less than 0.0001). These results demonstrate that this method for histological grading provides important prognostic information and, if the grading protocol is followed consistently, reproducible results can be obtained. Histological grade forms part of the multifactorial Nottingham prognostic index, together with tumour size and lymph node stage, which is used to stratify individual patients for appropriate therapy.
The scoring of mitotic figures is an integrated part of the Bloom and Richardson system for grading of invasive breast cancer. It is routinely done by pathologists by visual examination of hematoxylin and eosin (H&E) stained histology slides on a standard light microscope. As such, it is a tedious process prone to inter- and intra-observer variability. In the last decade, whole-slide imaging (WSI) has emerged as the “digital age” alternative to the classical microscope. The increasing acceptance of WSI in pathology labs has brought an interest in the application of automatic image analysis methods, with the goal of reducing or completely eliminating manual input to the analysis. In this paper, we present a method for automatic detection of mitotic figures in breast cancer histopathology images. The proposed method consists of two main components: candidate extraction and candidate classification. Candidate objects are extracted by image segmentation with the Chan-Vese level set method. The candidate classification component aims at classifying all extracted candidates as being a mitotic figure or a false object. A statistical classifier is trained with a number of features that describe the size, shape, color and texture of the candidate objects. The proposed detection procedure was developed using a set of 18 whole-slide images, with over 900 manually annotated mitotic figures, split into independent training and testing sets. The overall true positive rate on the testing set was 59.5% while achieving 4.2 false positives per one high power field (HPF).
We address a central problem of neuroanatomy, namely, the automatic segmen-tation of neuronal structures depicted in stacks of electron microscopy (EM) im-ages. This is necessary to efficiently map 3D brain structure and connectivity. To segment biological neuron membranes, we use a special type of deep artificial neural network as a pixel classifier. The label of each pixel (membrane or non-membrane) is predicted from raw pixel values in a square window centered on it. The input layer maps each window pixel to a neuron. It is followed by a succes-sion of convolutional and max-pooling layers which preserve 2D information and extract features with increasing levels of abstraction. The output layer produces a calibrated probability for each class. The classifier is trained by plain gradient descent on a 512 × 512 × 30 stack with known ground truth, and tested on a stack of the same size (ground truth unknown to the authors) by the organizers of the ISBI 2012 EM Segmentation Challenge. Even without problem-specific post-processing, our approach outperforms competing techniques by a large margin in all three considered metrics, i.e. rand error, warping error and pixel error. For pixel error, our approach is the only one outperforming a second human observer.
We present a new method for automatic classification of indirect immunoflourescence images of HEp-2 cells into different staining pattern classes. Our method is based on a new texture measure called shape index histograms that captures second-order image structure at multiple scales. Moreover we introduce a spatial decomposition scheme which is radially symmetric and suitable for cell images. The spatial decomposition is performed using donut-shaped pooling regions of varying sizes when gathering histogram contributions.We evaluate our method using both the ICIP 2013 and the ICPR 2012 competition datasets. Our results show that shape index histograms are superior to other popular texture descriptors for HEp-2 cell classification. Moreover, when comparing to other automated systems for HEp- 2 cell classification we show that shape index histograms are very competitive; especially considering the relatively low complexity of the method.
In this paper, we propose a statistical approach for mitosis detection in breast cancer histological images. The proposed algorithm models the pixel intensities in mitotic and non-mitotic regions by a Gamma-Gaussian mixture model and employs a context-aware post-processing in order to reduce false positives. Experimental results demonstrate the ability of this simple, yet effective method to detect mitotic cells in standard H&E stained breast cancer histology images.
We use deep max-pooling convolutional neural networks to detect mitosis in breast histology images. The networks are trained to classify each pixel in the images, using as context a patch centered on the pixel. Simple postprocessing is then applied to the network output. Our approach won the ICPR 2012 mitosis detection competition, outperforming other contestants by a significant margin.
The Multicenter Morphometric Mammary Carcinoma Project is a prospective study on the reproducibility and prognostic value of routine quantitative assessments, especially the mitotic activity index (MAI), the multivariate prognostic index (MPI; a combination of MAI, tumor size, and lymph node status), the mean nuclear area, and DNA ploidy assessments in patients with invasive breast cancer. Fourteen pathology laboratories providing routine services to 35 hospitals throughout The Netherlands are participating in this project. In this article, the reproducibility of MAI and MPI assessments is described. Assessment of the MAI was, according to a strict protocol, first performed in the participating laboratories; thereafter, slides were transferred to the coordination center in Amsterdam for quality control. Analysis of the reproducibility of the assessments in 2,469 patients showed correlation coefficients between 0.81 and 0.96 (mean, 0.91) for the MAI and between 0.91 and 0.97 (mean, 0.96) for the MPI. The reproducibility was fairly constant in time, although it showed a slight drop in the middle of the 2-year intake period. A prognostically relevant discrepancy in MPI (caused by differences in MAI) between the original and quality control assessments was found in only 7.2% of the cases. When analyzing the reasons for these discrepancies, a plausible explanation could be found in all cases: bad tissue processing and ignorance of or negligence in following the protocol guidelines for selection of the counting area or in the process of counting were the most important flaws. Since these errors are largely controllable, an even lower discrepancy rate is theoretically achievable. In conclusion, in a routine setting it can be learned, within a reasonable time, to perform mitosis counting in a highly reproducible way if a strict protocol is carefully followed. This opens the way for a wider application of the MAI and MPI in breast cancer patients. Motivation is, however, an important factor to obtain reproducible results, and ongoing quality control is essential to guarantee the reproducibility of the assessments.
Digital slide images have been used in many areas of pathology such as teaching, research, digital archiving, teleconsultation, and quality assurance testing. However, they have not much been used as yet for upfront diagnostics. The aim of this study was therefore to test the feasibility of digital slide image-based diagnosis of breast specimens. Sections of 100 breast specimens previously diagnosed conventionally were scanned and rediagnosed on digital slide images by the same pathologists who performed the initial light microscopy-based diagnosis. The digital slide image diagnoses were compared with the light microscopy diagnoses and classified as concordant, slightly discrepant (without clinical or prognostic consequences), or discrepant. The original light microscopy- and digital slide image-based diagnoses were concordant in 93% and slightly discrepant in 6% of cases. There was only 1 discrepant case with clinical or prognostic implication to the patient. However, for this case, no final agreement could be achieved. For 4 of the 6 slightly discrepant cases, digital slide image diagnosis was considered the better one, whereas the original diagnosis was preferred in only 1 case. In addition, for 1 case categorized as slightly discrepant, both the digital slide image and conventional diagnosis were imperfect according to 2 reviewing breast pathologists. This study demonstrates that upfront histopathologic diagnosis of breast biopsies and resections can reliably be done on digital slide image.
This paper presents a robust automated noninvasive video monitoring approach to recover the human pose in conditions with
persistent heavy obscuration. The proposed methods are compared with Ramanan’s stylized pose detection method and Wang’s sequential
pose model. The experimental results show that the proposed method performs significantly better than Ramanan’s approach,
is able to estimate the obscured body pose with various postures and obscuration levels in different environments, and is
not sensitive to illumination changes. The system is evaluated in two domains: sleeping human subjects obscured by a bed cover,
and pedestrians with a cluttered background scene, low feature contrast and baggy clothing. The body part detectors are trained
in the sleep monitoring domain but are still able to estimate the pose in the pedestrian domain, demonstrating the robustness
of the proposed technique.
KeywordsPose estimation–Covered body pose recognition–Sleep apnoea
Traditional methods of computer vision and machine learning cannot match
human performance on tasks such as the recognition of handwritten digits or
traffic signs. Our biologically plausible deep artificial neural network
architectures can. Small (often minimal) receptive fields of convolutional
winner-take-all neurons yield large network depth, resulting in roughly as many
sparsely connected neural layers as found in mammals between retina and visual
cortex. Only winner neurons are trained. Several deep neural columns become
experts on inputs preprocessed in different ways; their predictions are
averaged. Graphics cards allow for fast training. On the very competitive MNIST
handwriting benchmark, our method is the first to achieve near-human
performance. On a traffic sign recognition benchmark it outperforms humans by a
factor of two. We also improve the state-of-the-art on a plethora of common
image classification benchmarks.
Breiman's bagging and Freund and Schapire'sboosting are recent methods for improving the predictive power of classifier learning systems.Both form a set of classifiers that are combined by voting, bagging by generating replicated boots trap samples of the data, and boosting by adjustingthe weights of training instances. Thispaper reports results of applying both techniquesto a system that learns decision trees and testingon a representative collection of datasets. While both approaches...
We introduce a variational inference framework for training the Gaussian process latent variable model and thus performing Bayesian nonlinear dimensionality reduction. This method allows us to variationally integrate out the input vari- ables of the Gaussian process and compute a lower bound on the exact marginal likelihood of the nonlinear latent variable model. The maxi- mization of the variational lower bound provides a Bayesian training procedure that is robust to overfitting and can automatically select the di- mensionality of the nonlinear latent space. We demonstrate our method on real world datasets. The focus in this paper is on dimensionality re- duction problems, but the methodology is more general. For example, our algorithm is imme- diately applicable for training Gaussian process models in the presence of missing or uncertain inputs.
A growing number of screening applications require the automated monitoring of cell populations in a high-throughput, high-content environment. These applications depend on accurate cell tracking of individual cells that display various behaviors including mitosis, merging, rapid movement, and entering and leaving the field of view. Many approaches to cell tracking have been developed in the past, but most are quite complex, require extensive post-processing, and are parameter intensive. To overcome such issues, we present a general, consistent, and extensible tracking approach that explicitly models cell behaviors in a graph-theoretic framework. We introduce a way of extending the standard minimum-cost flow algorithm to account for mitosis and merging events through a coupling operation on particular edges. We then show how the resulting graph can be efficiently solved using algorithms such as linear programming to choose the edges of the graph that observe the constraints while leading to the lowest overall cost. This tracking algorithm relies on accurate denoising and segmentation steps for which we use a wavelet-based approach that is able to accurately segment cells even in images with very low contrast-to-noise. In addition, the framework is able to measure and correct for microscope defocusing and stage shift. We applied the algorithms on nearly 6000 images of 400,000 cells representing 32,000 tracks taken from five separate datasets, each composed of multiple wells. Our algorithm was able to segment and track cells and detect different cell behaviors with an accuracy of over 99%. This overall framework enables accurate quantitative analysis of cell events and provides a valuable tool for high-throughput biological studies.
High-throughput time-lapse microscopy is an excellent way of studying gene function by collecting time-resolved image data of the cellular responses to gene perturbations. With the increase in both data amount and complexity, computational methods capable of dealing with large image data sets are required. While image processing methods have been successfully applied to endpoint assays in the past, the analysis of complex time-resolved read-outs was so far still too immature to be applied on a large-scale. Here, we present a complete computational processing pipeline for such screens. By automatic image processing and machine learning, a quantitative description of phenotypic dynamics is obtained from the raw bitmaps. In order to visualize the resulting phenotypes in their temporal context, we introduce Event Order Maps allowing a concise representation of the major tendencies of causes and consequences of phenotypic classes. In order to cluster the phenotypic kinetics, we propose a novel technique based on trajectory representation of multidimensional time series. We demonstrate the use of these methods applying them on a genome wide RNAi screen by time-lapse microscopy.
This paper describes an image analysis technique for the counting of nuclei in mitosis in tissue sections. Five experienced pathologists scored mitoses in photographs of preselected areas of tissue sections of the breast. Objects consistently labelled as mitotic cells by all five pathologists were considered “mitoses” in the analysis. In total, there were 45 mitotic nuclei, 68 possible mitotic nuclei and 1,172 nonmitotic nuclei.
The image analysis procedure was designed to give priority to a low false negative rate, i.e., misclassification of mitoses. The procedure consists of three steps:
The objects remaining after the first two steps were visualized in a composite display for interactive evaluation: 10% of the mitotic nuclei were missed, and 85% of the nonmitotic nuclei were eliminated. The result of the fully automatic procedure described in this paper is rather disappointing and gave a loss of 37% of the mitoses while 5% of the nonmitotic nuclei remained.
This study describes an image processing method for the assessment of the mitotic count in Feulgen-stained breast cancer sections. The segmentation procedure was optimized to eliminate 95-98% of the nonmitoses, whereas 11% of the mitoses did not survive the segmentation procedure. Contour features and optical density measurements of the remaining objects were computed to allow for classification. Twelve specimens were analyzed, nine used to serve as a training set, and three put aside for later use as independent test set. The fully automatic image processing method correctly classified 81% of the mitoses at the specimen level while inserting 30% false positives. The automatic procedure strongly correlated with the interactive counting procedure (r = 0.98). Although the fully automatic method provided satisfactory results, it is not yet suited for clinical practice. The automated method with an interactive evaluation step gave an accurate reflection of the mitotic count showing an almost perfect correlation with the results of the interactive morphometry (r = 0.998). Therefore this semiautomated method may be useful as prescreening device.
Counting of mitotic cells has been shown to be of prognostic value in breast cancer in different retrospective studies. Up to now the number of mitoses is assessed mainly manually according to a standardized but strict protocol. Although such a manual procedure is reasonably reproducible, automatic counting of mitotic cells offers the potential for greater objectivity and reproducibility. This paper describes the influence of resolution on automatic recognition by image processing of mitotic cells in Feulgen stained breast cancer sections. Using the image recording, correction and segmentation procedure described in a previous study, five specimens were analyzed: one was used to serve as a training set and four were put aside for later use as independent test set. For each slide, objects from a pre-selected area were recorded at increasing resolution. For each object, contour features and optical density measurements were computed and stored in a data file for statistical analysis. The results showed that increased resolution using a 40x objective lowered the number of misclassified mitoses compared with a 20x objective (overall mean percentage of misclassified mitoses over training and all test specimens: 20x, 24.57; 40x, 7.96). The number of misclassifications of non-mitoses was almost stable per specimen but varied between specimens (19-42%) due to differences among tissues. Given the improvement in classifying mitoses and the possibility to evaluate interactively the measurement result, the described semi-automated mitoses pre-screener of histological sections may be suitable for further testing in a clinical setting.
Breast cancer is the leading cause of death among solid tumours in women, and its incidence is increasing in the West. Adjuvant chemotherapy and hormonal treatment improve survival but have potentially serious side effects, and are costly. Because adjuvant treatment should be given to high risk patients only, and traditional prognostic factors (lymph node status, tumour size) are insufficiently accurate, better predictors of high risk and treatment response are needed. Invasive breast cancer metastasises haematogenously very early on, so many breast cancer prognosticators are directly or indirectly related to proliferation. Although studies evaluating the role of individual proliferation regulating genes have greatly increased our knowledge of this complex process, the functional end result-cells dividing-has remained the most important prognostic factor. This article reviews the prognostic value of different proliferation assays in invasive breast cancer, and concludes that increased proliferation correlates strongly with poor prognosis, irrespective of the methodology used. Mitosis counting provides the most reproducible and independent prognostic value, and Ki67/MIB1 labelling and cyclin A index are promising alternatives that need methodological fine tuning.
This paper addresses the automatic detection of microaneurysms in color fundus images, which plays a key role in computer assisted diagnosis of diabetic retinopathy, a serious and frequent eye disease. The algorithm can be divided into four steps. The first step consists in image enhancement, shade correction and image normalization of the green channel. The second step aims at detecting candidates, i.e. all patterns possibly corresponding to MA, which is achieved by diameter closing and an automatic threshold scheme. Then, features are extracted, which are used in the last step to automatically classify candidates into real MA and other objects; the classification relies on kernel density estimation with variable bandwidth. A database of 21 annotated images has been used to train the algorithm. The algorithm was compared to manually obtained gradings of 94 images; sensitivity was 88.5% at an average number of 2.13 false positives per image.
Breiman's bagging and Freund and Schapire's boosting are recent methods for improving the predictive power of classifier learning systems. Both form a set of classifiers that are combined by voting, bagging by generating replicated bootstrap samples of the data, and boosting by adjusting the weights of training instances. This paper reports results of applying both techniques to a system that learns decision trees and testing on a representative collection of datasets. While both approaches substantially improve predictive accuracy, boosting shows the greater benefit. On the other hand, boosting also produces severe degradation on some datasets. A small change to the way that boosting combines the votes of learned classifiers reduces this downside and also leads to slightly better results on most of the datasets considered. Introduction Designers of empirical machine learning systems are concerned with such issues as the computational cost of the learning method and the accuracy and ...
Mitosis detection using generic features and an ensemble of cascade adaboosts doi:10.4103/2153-3539 Method for counting mitoses by image processing in feulgen stained breast cancer sections
- F B Tek
Tek, F.B., 2013. Mitosis detection using generic features and an ensemble of cascade adaboosts. Journal of Pathology Informatics 4, 12. doi:10.4103/2153-3539.112697 Ten Kate, T.K., Beliën, J.A.M., Smeulders, A.W.M., Baak, J.P.A., 1993. Method for counting mitoses by image processing in feulgen stained breast cancer sections. Cytometry 14, 241–250. doi:10.1002/cyto.990140302
Manifold Relevance Determination
- A Damianou
- C Ek
- M Titsias
- N Lawrence
Damianou, A., Ek, C., Titsias, M., Lawrence, N., 2012. Manifold Relevance Determination, in: International
Conference on Machine Learning (ICML). pp. 145-152.
Deep neural networks segment neuronal membranes in electron microscopy images
- D C Ciresan
- A Giusti
- L M Gambardella
- J Schmidhuber
Ciresan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J., 2012a. Deep neural
networks segment neuronal membranes in electron microscopy images. Neural
Inform Process Syst (NIPS), 2852-2860.