PreprintPDF Available

Multi-organ Nuclei Segmentation and Classification Challenge 2020

Authors:
  • Alberta Machine Intelligence Institute
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

This preprint provides information about multi-organ nuclei segmentation and classification challenge, which is an official satellite event of ISBI 2020. This document summarizes the challenge participation rules and provides detailed information about its training and testing datasets.
Content may be subject to copyright.
1
Multi-organ Nuclei Segmentation and Classification
Challenge 2020
Ruchika Verma, Neeraj Kumar, Abhijeet Patil, Nikhil Cherian Kurian, Swapnil Rane, and Amit Sethi
Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
Department of Electrical Engineering, Indian Institute of Technology Bombay, Mumbai, Maharashtra, India
Department of Pathology, Tata Memorial Cancer Centre, Mumbai, Maharashtra, India
Correspondence: monusac2020@gmail.com
I. INTRODUCTION
In order to assess the variations in the tumors and their
microenvironments across organs and patients, identification
of nuclei morphologies and classification of their types are
essential. In multi-organ nuclei segmentation and classification
(MoNuSAC) challenge, the organizers will provide carefully
annotated dataset of H&E stained whole slide images of
four organs (breast, kidney, lung and prostate) with hand-
annotated nuclei boundaries and cell-types. The participants
will use the training data of the challenge to build machine
learning models to segment and identify the type of cells
present in a given whole slide image. The participants are
also welcome to use machine-learning-free techniques for their
model development provided they solve both segmentation
and classification tasks. Subsequently, the participants will be
provided with a testing dataset of unseen patients to report the
results of their models to the organizers for evaluation. The
test annotations will be withheld from the participants and will
be used to rank the entries based on the performance of their
models. MoNuSAC is an official satellite event of the IEEE
International Symposium on Biomedical Imaging (ISBI) 2020.
This document provides information about the challenge
timeline, rules and regulations for participating in this chal-
lenge, registration procedure, details of training and test-
ing data, evaluation metric, and submission format. A post-
challenge journal paper summarizing the challenge outcomes
will be prepared after formally concluding the challenge at
ISBI 2020 workshop.
II. CH AL LE NG E TIMELINE
November 15, 2019: Challenge open for registration
(Please see Section III)
December 20, 2019: Training data release (Images +
Ground Truth)
February 01, 2020: Testing data release (Images only)
February 25, 2020: Submission of testing results along
with a manuscript describing the algorithm and the testing
code (Please check the submission instructions in Section
VI)
R. Verma, N. Kumar, A. Patil, N.C.Kurian, S.Rane and A.
Sethi are co-organizing the challenge; Address all correspondence to:
monusac2020@gmail.com
March 10, 2020: Preliminary leaderboard will be re-
leased online
April 3, 2020: Declaration of challenge winners at ISBI
2020 challenge workshop
III. REGISTRATION PROC ED UR E
Prospective participants can register for this challenge by
filling in their details in a Google form available here. Chal-
lenge participants should adhere to the following rules and
regulations, otherwise their participation may be cancelled
(without notice) by the organizers at any time during the
course of the challenge.
Anonymous registrations are strictly prohibited.
Each team is allowed to have at most 5 members but
the team registration should be done ONLY by ONE of
them. S/he will be the point of contact of the team with
the organizers.
For successful registration each team must provide com-
plete and correct information about the Name of the
contact person, Affiliation (including department, univer-
sity/institute/company, country) and valid E-mail address.
Participants should carefully select their team names
as they will not be changed during the course of the
challenge.
Redundant and incomplete registrations will be removed
without any notification.
Participating teams maintain copyright to the associated
intellectual property and software they develop in course
of participating in MoNuSAC 20201. The testing code
and model parameters submitted during the challenge will
be used by the organizers for the sole purpose of model
evaluation and will not be released publicly in any form
(even with the post-challenge journal paper).
IV. CHA LL EN GE DATA
H&E staining of human tissue sections is a routine and most
common protocol used by pathologists to enhance the contrast
of tissue sections for tumor assessment (grading, staging, etc.)
at multiple microscopic resolutions. Hence, we will provide
the annotated dataset of H&E stained digitized tissue images
of several patients acquired at multiple hospitals using one of
1https://monusac-2020.grand-challenge.org/
2
the most common 40x scanner magnification. The annotations
will be done with the help of expert pathologists.
A. Training Data
Training data spanning four organs (breast, kidney, lung and
prostate) with cell-boundary and cell-type annotations for ep-
ithelial cells, lymphocytes, macrophages and neutrophils was
prepared from the whole slide images of 45 patients (scanned
at 31 hospitals) downloaded from The Cancer Genome Atlas
(TCGA) data portal2. Plurality of patients, organs, hospitals
and disease states (benign and malignant tumors) will help
in learning the morphological variations of diverse cell-types
included in this challenge. The sources of such variations may
be the differences in the slide preparation protocols adopted at
multiple hospitals, patient-specific tumor biology or different
developmental stages of the annotated cell-types.
Training set annotations were done using Aperio
ImageScope R
and were saved in .xml files. Different
cell-types were annotated using a unique marker color:
epithelial cell were annotated in red, lymphocytes in yellow,
macrophages in green and neutrophils in blue. Figure 2
shows an example annotated image. In total, the training data
contains 31,411 hand-annotated nuclei instances including
14,539 epithelial cells, 15,654 lymphocytes, 587 macrophages
and 631 neutrophils.
The training data, comprising H&E stained images and .xml
annotation files, was released to the registered participants on
December 20, 2019. If you want to get access to the training
data, please register for the challenge (see Section III). The
code for reading .xml annotation files can be downloaded by
clicking here. Additionally, participants are free to use our
previous datasets released as part of MoNuSeg 20183to train
a generalized nuclei segmentation module [1], [2].
B. Testing Data
The testing data will be prepared using the similar protocol
as adopted for creating the challenge training data. However,
the testing set will be created from the patients not included in
the training set. The testing data will also contain annotations
for the ambiguous regions with white boundaries in addition
to the usual red (epithelial), yellow (lymphocytes), green
(macrophages) and blue (neutrophils) boundary annotations.
The ambiguous regions will be the ones which will not be
used for computing the evaluation metric for ranking the
participants because (1) these regions might have very faint
nuclei with unclear boundaries or (2) the annotators might be
unsure of their true class. Only H&E stained tissue images
and the ambiguous region annotations of the testing set will
be released to the participants on February 1, 2020. The
organizers will evaluate participant’s algorithms by using the
withheld testing cohort annotations (for the classes of interest).
2https://portal.gdc.cancer.gov/
3https://monuseg.grand-challenge.org/
V. EVAL UATION CRITERIA
The metric adopted for evaluating the participant’s per-
formance is weighted average of the class-specific panoptic
quality (PQ) [3]. We will match each predicted nucleus (p)
with the ground truth nucleus (g)if their intersection over
union (IoU) is strictly greater than 0.5, this matching will be
done separately for each cell-type. For a given class c, the
unique matching splits the predicted (pc)and ground truth
(gc)nuclei into three sets: true positives (T Pc), false positives
(F Pc), and false negatives (FNc), representing matched pairs
of segments, unmatched predicted segments, and unmatched
ground truth segments, respectively. Given these three sets,
class-specific P Qc(for cth class) will be computed as:
P Qc=P(pc,gc)∈{T Pc}IoU(pc, gc)
|T Pc|+1
2|F Pc|+1
2|F Nc|(1)
For each image in the test set, the weighted panoptic
quality will be computed as the weighted sum of the class-
specific panoptic quality, i.e. wP Q =P4
c=1 wcP Qc, where
the weights for the four cell-types are given as follows:
wc=(1for c = epithelial cells or lymphocytes
10 for c = macrophages or neutrophils (2)
Macrophages and neutrophils are given more weights in
metric computation because of their under-representation in
the training and testing sets. The average of the weighted PQs
across all images in the testing set will be used as the ranking
metric for this challenge.
VI. SU BM IS SI ON INSTRUCTIONS
Each participant should send an email to
monusac2020@gmail.com with three attachments (1)
zipped file containing prediction masks, (2) A manuscript
providing algorithm details and (3) Testing code for evaluating
the proposed algorithm, in the format described below.
Prediction mask submission format
1) Create a folder and name it with the patient name.
2) Within the folder created in step 1, create a sub-folder
for saving the results of each sub-image
3) Within the sub-folder created in step 2, create a sub-
sub-folder to save .mat files for segmented instances of
each cell-type. You can name the .mat file as per your
convenience. However, make sure that your folder name
represents patient name, sub-folder represents sub-image
name, and sub-sub-folder represents cell-type.
4) The folder hierarchy mentioned above is depicted in the
Figure 2.
The folders containing the results of all testing
images should be saved in a single zipped file as
“TeamName MoNuSAC test results.zip”. Click here to
see an example of submission file.
3
Fig. 1: (a) A sub-image cropped from whole slide image of a patient included in the training set, (b) boundary annotations of
different cell-types done using unique marker colors and (c) masks generated from the annotations - epithelial cells are shown
in red, lymphocytes in yellow, macrophages in green and neutrophils in blue.
Fig. 2: Folder hierarchy to save the testing results.
Composition of each .mat file
Each mat file should contain instances of only one class.
All pixels that belong to a segmented instance should be
assigned the same unique positive integer (>0). Hence, the
number of unique integers (excluding 0s) within a mat file
will correspond to the number of instances saved in that file
and 0 will represent the background.
Please click here to see the output file generated for a
couple of patients to familiarize yourself with the submission
format. The snapshot of the folder hierarchy is shown above.
Manuscript submission format
Each participating team must submit a manuscript describ-
ing their algorithm in detail.
The organizers will review the paper for sufficient details
required to understand and reproduce the algorithm and
hold the right to exclude participants in case their method
description is insufficient.
An example of a well written manuscript (from our
previous challenge - MoNuSeg) is provided on this link.
There is no page limit to allow you to give detailed
information about every step of your proposed algorithm.
Please include a flowchart illustrating your algorithm
along with the details of all parameters, hyper-parameters,
data augmentation methods, loss functions, etc. used for
training the models in your manuscript.
The manuscript should be submitted in a .pdf format with
name “TeamName MoNuSAC manuscript.pdf”.An example
of a well written manuscript (from our previous challenge -
MoNuSeg) is provided on this link.
Testing code submission format
Each team should submit a Python script and trained model
weights to facilitate organizers to evaluate their algorithms.
The submitted script should contain commented lines to load
the trained models, testing data and saving the predictions
(in the prediction mask submission format given above). The
testing code should also clearly mention the libraries required
to run the models.
The participants might have trained their models using
other programming languages but they should make sure that
their models can be readily tested in Python using popular
machine/deep learning libraries.
* The testing script should be named as “Team-
Name MoNuSAC testing code.py”.
* Participating teams maintain copyright of the associated
intellectual property and software they develop in course of
participating in MoNuSAC 2020. The testing code and model
parameters submitted during the challenge will be used by
the organizers for the sole purpose of model evaluation and
will not be released publicly in any form (even with the post-
challenge journal paper).
Please send one email to monusac2020@gmail.com with
subject ”TeamName MoNuSAC Submission” and 3 afore-
mentioned attachments to complete your participation in the
challenge.
4
Anyone who does not follow these submission instructions
will be disqualified.
VII. LEA DE RB OAR D ANNOUNCEMENT
The preliminary leaderboard will be announced publicly
during the challenge workshop at ISBI 2020. The repro-
ducibility of top 3 techniques will be ensured before the
ISBI workshop to declare the challenge winners. The final
leaderboard will be released on the challenge webpage after
verifying the results of all participants.
REFERENCES
[1] N. Kumar, R. Verma, D. Anand, Y. Zhou, O. F. Onder, E. Tsougenis,
H. Chen, P. A. Heng, J. Li, Z. Hu et al., “A multi-organ nucleus
segmentation challenge,” IEEE transactions on medical imaging, 2019.
[2] N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane, and A. Sethi,
“A dataset and a technique for generalized nuclear segmentation for com-
putational pathology,IEEE Transactions on Medical Imaging, vol. 36,
no. 7, pp. 1550–1560, July 2017.
[3] A. Kirillov, K. He, R. Girshick, C. Rother, and P. Doll´
ar, “Panoptic
segmentation,” in Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, 2019, pp. 9404–9413.
... A description of the challenge datasets, the organization of the competition, and the evaluation metrics was provided ahead of the challenge in [14] . A post-challenge report containing information on the competing algorithms and a discussion of the methods used and of the challenge results was published in [3] . ...
... The evaluation criterion differs slightly between the prechallenge [14] and post-challenge [3] publications. In both cases, the "panoptic quality" (PQ) is used [15] . ...
... The ground truth annotations are provided as.xml files with each annotation encoded as a polygon with the position of the vertices. For each sub-image, participants were asked to provide their predictions as "n-ary masks", with a separate file per class such that "all pixels that belong to a segmented instance should be assigned the same unique positive integer ( > 0 )" [14] . The "nary masks" were not directly released by the challenge organizers. ...
Article
Biomedical image analysis competitions often rank the participants based on a single metric that combines assessments of different aspects of the task at hand. While this is useful for declaring a single winner for a competition, it makes it difficult to assess the strengths and weaknesses of participating algorithms. By involving multiple capabilities (detection, segmentation and classification) and releasing the prediction masks provided by several teams, the MoNuSAC 2020 challenge provides an interesting opportunity to look at what information may be lost by using entangled metrics. We analyse the challenge results based on the ”Panoptic Quality” (PQ) used by the organizers, as well as on disentangled metrics that assess the detection, classification and segmentation abilities of the algorithms separately. We show that the PQ hides interesting aspects of the results, and that its sensitivity to small changes in the prediction masks makes it hard to interpret these results and to draw useful insights from them. Our results also demonstrate the necessity to have access, as much as possible, to the raw predictions provided by the participating teams so that challenge results can be more easily analysed and thus more useful to the research community.
... segmentation and classification) in a unified framework can improve the performance of both tasks. Besides, some public dataset are collected for nuclei classification, such as CoNSeP [5], PanNuke [15] and MoNuSAC [16]. Despite some works that pay attention to nuclei classification, most of them are coarse-grained and rarely focus on the fine-grained nuclei classification that grading the tumor nuclei by their appearance. ...
... It has been proven to be an accurate and interpretable metric for nuclei segmentation [5,15]. For classification, following the same idea of MoNuSAC [16], the PQ of each class (i.e., P Q 1 , P Q 2 , P Q 3 , P Q e ) and the average PQ (aPQ) are adopted. The aPQ is the average of each PQ per class. ...
Preprint
Full-text available
The grade of clear cell renal cell carcinoma (ccRCC) is a critical prognostic factor, making ccRCC nuclei grading a crucial task in RCC pathology analysis. Computer-aided nuclei grading aims to improve pathologists' work efficiency while reducing their misdiagnosis rate by automatically identifying the grades of tumor nuclei within histopathological images. Such a task requires precisely segment and accurately classify the nuclei. However, most of the existing nuclei segmentation and classification methods can not handle the inter-class similarity property of nuclei grading, thus can not be directly applied to the ccRCC grading task. In this paper, we propose a Composite High-Resolution Network for ccRCC nuclei grading. Specifically, we propose a segmentation network called W-Net that can separate the clustered nuclei. Then, we recast the fine-grained classification of nuclei to two cross-category classification tasks, based on two high-resolution feature extractors (HRFEs) which are proposed for learning these two tasks. The two HRFEs share the same backbone encoder with W-Net by a composite connection so that meaningful features for the segmentation task can be inherited for the classification task. Last, a head-fusion block is applied to generate the predicted label of each nucleus. Furthermore, we introduce a dataset for ccRCC nuclei grading, containing 1000 image patches with 70945 annotated nuclei. We demonstrate that our proposed method achieves state-of-the-art performance compared to existing methods on this large ccRCC grading dataset.
... If the rumour is tumour, the issue is tissue. In this paper, we tackle the problem of segmenting and classifying the type of cells present in a given H&E stained whole slide image (WSI) of multiple organs [5]. The cell types include epithelial cells, lymphocytes, macrophages and neutrophils. ...
... In total, the training data contain 31,411 hand-annotated nuclei instances, including 14,539 epithelial cells, 15,654 lymphocytes, 587 macrophages and 631 neutrophils. More details about the dataset can be found at [5]. ...
... It offered competitors a data set of~1 million images from 1000 object classes, made possible by the use of crowdsourced annotations from thousands of non-expert individuals. In contrast, computational biology competitions 4,12 typically offer only hundreds to thousands of labeled examples. The key bottlenecks are that annotators need to have certain levels of expertise and that annotation takes longer than conventional domains, making it difficult to obtain annotations at scale. ...
Article
Full-text available
Biology has become a prime area for the deployment of deep learning and artificial intelligence (AI), enabled largely by the massive data sets that the field can generate. Key to most AI tasks is the availability of a sufficiently large, labeled data set with which to train AI models. In the context of microscopy, it is easy to generate image data sets containing millions of cells and structures. However, it is challenging to obtain large-scale high-quality annotations for AI models. Here, we present HALS (Human-Augmenting Labeling System), a human-in-the-loop data labeling AI, which begins uninitialized and learns annotations from a human, in real-time. Using a multi-part AI composed of three deep learning models, HALS learns from just a few examples and immediately decreases the workload of the annotator, while increasing the quality of their annotations. Using a highly repetitive use-case—annotating cell types—and running experiments with seven pathologists—experts at the microscopic analysis of biological specimens—we demonstrate a manual work reduction of 90.60%, and an average data-quality boost of 4.34%, measured across four use-cases and two tissue stain types.
... Our approach draws on the Hover-Net [1] architecture to leverage multi-task transfer wherein representations from 3 different datasets (CPM17 [5] , Kumar [2] , CoNSeP [1] ) & corresponding tasks (segmentation/segmentation+classification) are learnt upstream before transferring to the MoNuSAC segmentation+classification task. We present our preliminary results for the same as part of MoNuSAC 2020's post-challenge. ...
... Five best models are chosen based on panoptic quality (PQ) [8] and final prediction masks are generated by major voting. ...
Article
Full-text available
A total of 209 training images from 46 patients were released for training. The training images were stained using H&E and scanned at 40× optical magnification. Each WSI was provided with four pixel-level ground truth labels (epithelial cells, lymphocytes, neutrophils and macrophages) annotated by pathologists. The original training images at 40× optical magnification are divided into smaller patches (256 × 256) using a sliding window with a stride of 64. When the training image's width or height is smaller than 256, the image is padded with white background. Out of 209 training images, 168 images are used for training and 41 images are set aside for validation. During training phase, data augmentations (RGBShift, ChannelShuffle, RandomBrightnessContrast, ShiftScaleRotate, ElasticTransform, Horizontal/Vertical flips and Transpose) are applied using albumentation library [1]. On top of four different nuclei types, we have additionally created a nuclei boundary mask and included in the training. The boundary information highlights the boundaries between adjacent nuclei which is import for separating different nuclei. 2. Proposed Model The model proposed in this task is inspired by U-Net architecture, which has a single encoder and a decoder to convert an image into a segmentation map [2]. As a backbone encoder, our proposed model uses 50 layers ResNet [3] with added Squeeze-and-Excitation (SE) blocks [4]. A noticeable gap between data distributions in train and test domains is observed, which results in severe performance loss at inferencing testing images. To address this challenge, an unsupervised domain adaptation (UDA) method is applied in our proposed model. The UDA method adapted to our model is adversarial entropy minimization [5]. This method utilizes the Shannon Entropy to constrain the model to produce high-confident prediction on testing images while minimizing loss value between prediction from source image and ground truth [5]. In this work, our proposed model utilized direct entropy minimization approach. Once the model is trained, testing
... This is a significant achievement in digital pathology and provide clear direction for other peers to develop an effective automated diagnosis system to further empower cytologists throughout their routine work. More and more public whole-slide datasets and challenges regarding different histopathology tasks present the growth of digital pathology, such as CAMELYON challenges for breast cancer metastasis detection ( Bejnordi et al., 2017;Bandi et al., 2018 ), PANDA-2020 challenge for prostate cancer grading ( Wouter et al., 2020 ), MoNuSAC-2020 challenge for multiple cell-types segmentation ( Verma et al., 2020 ), etc. We believe our work could enlighten the peers for future directions in exploring further advanced solutions of automated whole-slide analysis systems. ...
Article
Cervical cancer has been one of the most lethal cancers threatening women’s health. Nevertheless, the incidence of cervical cancer can be effectively minimized with preventive clinical management strategies, including vaccines and regular screening examinations. Screening cervical smears under microscope by cytologist is a widely used routine in regular examination, which consumes cytologists’ large amount of time and labour. Computerized cytology analysis appropriately caters to such an imperative need, which alleviates cytologists’ workload and reduce potential misdiagnosis rate. However, automatic analysis of cervical smear via digitalized whole slide images (WSIs) remains a challenging problem, due to the extreme huge image resolution, existence of tiny lesions, noisy dataset and intricate clinical definition of classes with fuzzy boundaries. In this paper, we design an efficient deep convolutional neural network (CNN) with dual-path (DP) encoder for lesion retrieval, which ensures the inference efficiency and the sensitivity on both tiny and large lesions. Incorporated with synergistic grouping loss (SGL), the network can be effectively trained on noisy dataset with fuzzy inter-class boundaries. Inspired by the clinical diagnostic criteria from the cytologists, a novel smear-level classifier, i.e., rule-based risk stratification (RRS), is proposed for accurate smear-level classification and risk stratification, which aligns reasonably with intricate cytological definition of the classes. Extensive experiments on the largest dataset including 19,303 WSIs from multiple medical centers validate the robustness of our method. With high sensitivity of 0.907 and specificity of 0.80 being achieved, our method manifests the potential to reduce the workload for cytologists in the routine practice.
Preprint
Full-text available
Biomedical image analysis competitions often rank the participants based on a single metric that combines assessments of different aspects of the task at hand. While this is useful for declaring a single winner for a competition, it makes it difficult to assess the strengths and weaknesses of participating algorithms. By involving multiple capabilities (detection, segmentation and classification) and releasing the prediction masks provided by several teams, the MoNuSAC 2020 challenge provides an interesting opportunity to look at what information may be lost by using entangled metrics. We analyse the challenge results based on the "Panoptic Quality" (PQ) used by the organizers, as well as on disentangled metrics that assess the detection, classification and segmentation abilities of the algorithms separately. We show that the PQ hides interesting aspects of the results, and that its sensitivity to small changes in the prediction masks makes it hard to interpret these results and to draw useful insights from them. Our results also demonstrate the necessity to have access, as much as possible, to the raw predictions provided by the participating teams so that challenge results can be more easily analysed and thus more useful to the research community.
Chapter
The grade of clear cell renal cell carcinoma (ccRCC) is a critical prognostic factor, making ccRCC nuclei grading a crucial task in RCC pathology analysis. Computer-aided nuclei grading aims to improve pathologists’ work efficiency while reducing their misdiagnosis rate by automatically identifying the grades of tumor nuclei within histopathological images. Such a task requires precisely segment and accurately classify the nuclei. However, most of the existing nuclei segmentation and classification methods can not handle the inter-class similarity property of nuclei grading, thus can not be directly applied to the ccRCC grading task. In this paper, we propose a Composite High-Resolution Network for ccRCC nuclei grading. Specifically, we propose a segmentation network called W-Net that can separate the clustered nuclei. Then, we recast the fine-grained nuclei classification into two cross-category classification tasks that are leaned by two newly designed high-resolution feature extractors (HRFEs). The two HRFEs share the same backbone encoder with W-Net by a composite connection so that meaningful features for the segmentation task can be inherited to the classification task. Last, a head-fusion block is applied to generate the predicted label of each nucleus. Furthermore, we introduce a dataset for ccRCC nuclei grading, containing 1000 image patches with 70945 annotated nuclei. We demonstrate that our proposed method achieves state-of-the-art performance compared to existing methods on this large ccRCC grading dataset.
Chapter
Histological subtype of papillary (p) renal cellular and cell-layer level patterns almost cannot be captured by existing CNN-based models in large-size histopathological images, which brings obstacles to directly applying these models to such a fine-grained classification task. This paper proposes a novel instance-based Vision Transformer (i-ViT) to learn robust representations of histopathological images for the pRCC subtyping task by extracting finer features from instance patches (by cropping around segmented nuclei and assigning predicted grades). The proposed i-ViT takes top-K instances as the inputs and aggregates them for capturing both the cellular and cell-layer level patterns by a position-embedding layer, a grade-embedding layer, and a multi-head multi-layer self-attention module. To evaluate the performance of the proposed framework, experienced pathologists select 1162 regions of interest from 171 whole slide images of type 1 and type 2 pRCC. Experimental results show that the proposed method achieves better performance than existing CNN-based models with a significant margin.
Article
Full-text available
Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. We summarize the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline 1. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net 2, FCN 3, and Mask-RCNN 4 were popularly used, typically based on ResNet 5 or VGG 6 base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.
Article
Full-text available
Nuclear segmentation in digital microscopic tissue images can enable extraction of high-quality features for nuclear morphometrics and other analysis in computational pathology. Conventional image processing techniques such as Otsu thresholding and watershed segmentation do not work effectively on challenging cases, such as chromatin-sparse and crowded nuclei. In contrast, machine learning-based segmentation can generalize across various nuclear appearances. However, training machine learning algorithms require datasets of images in which a vast number of nuclei have been annotated. Publicly accessible and annotated datasets, along with widely agreed upon metrics to compare techniques, have catalyzed tremendous innovation and progress on other image classification problems, particularly in object recognition. Inspired by their success, we introduce a large publicly accessible dataset of H&E stained tissue images with more than 21,000 painstakingly annotated nuclear boundaries, whose quality was validated by a medical doctor. Because our dataset is taken from multiple hospitals and includes a diversity of nuclear appearances from several patients, disease states, and organs, techniques trained on it are likely to generalize well and work right out-of-the-box on other H&E stained images. We also propose a new metric to evaluate nuclear segmentation results that penalizes object- and pixel-level errors in a unified manner, unlike previous metrics that penalize only one type of error. We also propose a segmentation technique based on deep learning that lays special emphasis on identifying the nuclear boundaries, including those between the touching or overlapping nuclei, and works well on a diverse set of test images.