Conference PaperPDF Available

Chromosome Classification with Convolutional Neural Network Based Deep Learning

Authors:

Figures

Content may be subject to copyright.
Chromosome Classification with Convolutional
Neural Network based Deep Learning
Wenbo ZhangSifan SongTianming BaiYanxin ZhaoFei Ma∗§ Jionglong Su∗‡§ Limin Yu†§
Mathematical Sciences Department, Xi’an Jiaotong-Liverpool University, Suzhou, China
Department of Electrical and Electronic Engineering, Xi’an Jiaotong-Liverpool University, Suzhou, China
Neusoft Corporation, Shenyang, China
§Research Center for Precision Medicine, HT-URC, Xi’an Jiaotong-Liverpool University
Suzhou, China, Tel: (86) 512 8816 1633, Email: limin.yu@xjtlu.edu.cn, jionglong.su@xjtlu.edu.cn
Abstract—Karyotyping plays a crucial role in genetic
disorder diagnosis. Currently Karyotyping requires consid-
erable manual efforts, domain expertise and experience,
and is very time consuming. Automating the karyotyping
process has been an important and popular task. This study
focuses on classification of chromosomes into 23 types, a
step towards fully automatic karyotyping. This study pro-
poses a convolutional neural network (CNN) based deep
learning network to automatically classify chromosomes.
The proposed method was trained and tested on a dataset
containing 10304 chromosome images, and was further tested
on a dataset containing 4830 chromosomes. The proposed
method achieved an accuracy of 92.5%, outperforming three
other methods appeared in the literature. To investigate
how applicable the proposed method is to the doctors, a
metric named proportion of well classified karyotype was
also designed. An result of 91.3% was achieved on this metric,
indicating that the proposed classification method could be
used to aid doctors in genetic disorder diagnosis.
I. INT RO DUCTI ON
The human cell normally contains 23 pairs of chromo-
somes, including 22 pairs of autosomes (the ones exist
in both males and females) as well as sex chromosomes
X and Y. Females have double X chromosomes as one
pair of sex chromosomes, while males have both X
and Y. Chromosome abnormality, namely aneuploidy
(having abnormal number of chromosomes in a cell)
and structural abnormalities (including deletions, du-
plication, translocation, inversion, insertions rings, and
isochromosome) may cause genetic disorder such as
Down’s syndrome. It is important to inspect the cells
of a patient and identify any irregular, extra or missing
parts for diagnostic purposes. Karyotyping, the process
of separating and classifying human chromosomes from
a cell image, plays a crucial role in this diagnosis process
[1].
However, accomplishing this work efficiently not only
requires considerable manual efforts, domain expertise
and experience, but also consumes a lot of time. Since
1980, with the motivation of lightening the load of cyto-
geneticists, automatic diagnosis systems for chromosome
karyotyping and analysis have become a popular and
important task.
J. M. Cho chose the two-layer artificial neural network
with the error backpropagation training as chromosome
classifier, which resulted an overall classification error
rate of 6.52% in the 460 chromosomes images [2]. To
overcome the higher classification error, J. Cho et. al. [3]
proposed a hierarchical multi-layer network as chromo-
some classifier and an error back-propagation training
algorithm. The overall result of classification error in this
method was 5.9% which was based on the 7 experiments.
S. Delshadpour [4] reduced the complexity of an
ANN in order to increase the performance of ANN and
combined an improved multi-layer perceptron neural
network for automated classification of chromosomes.
The overall accuracy of classification was increased to
88.3% on 304 chromosomes. However, their classification
results on 24 classes vary and on many classes the results
are not very accurate. To overcome the problems, B. C.
Oskouei and J. Shanbehzadeh [5] proposed a classifier
based on the wavelet neural network. They obtained an
accuracy rate of 93.35%. S. Gagulapalalic and M. Can
[6] proposed a novel method based on the Competi-
tive Neural Network Teams (CNNTs) to distinguish 22
types of the autosomes. Their method achieved better
classifying results, i.e. approximately 96.64%, on 150
chromosome images with each type of autosomes.
M. J. Roshtkhari and S. K. Setarehdan [7] presented a
wavelet transform based linear discriminant analysis to
classify normal and automatically straightened chromo-
somes, and a three layers feed-forward perceptron neural
network which was trained using the backpropagation
algorithm. The overall outcome of correct classification
was 99.3% after 303 highly curved chromosomes were
straightened. A subspace-based approach was proposed
by Q. Wu et. al., which synthesized the prototype chro-
mosome images and utilized transformation coefficients
as the feature measurements [8]. The result shows that
this method could synthesize highly visual prototype
chromosome images which were previously unseen in
2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI 2018)
978-1-5386-7604-2/18/$31.00 ©2018 IEEE
chromosome classification. Nevertheless, most of the
proposed systems are partially automated and still need
much manual assistance, since defining and extracting
features of images with overlapping or even touching
chromosomes are still difficult steps [6].
With the development of CNNs, they have been uti-
lized in medical image sector to deal with complicated
features. CNN based models have been explored to
analyze chromosome images. In 2016, Qiu et al. pro-
posed a 8-layer convolutional neural network and its
lowest testing error achieved 13.3%, which is the first
research proposing a CNN model to classify metaphase
chromosome images [9]. In 2017, Sharma et al. utilized a
4 deep learning blocks to construct a model and achieved
an accuracy of 68.5% (without straightening) and 86.7%
(with straightening) [10]. Swati et al. proposed a model
based on the Siamese networks which is composed by
a twin neural network and the best model yields 85.2%
classification accuracy [1].
Although above methods are based on convolutional
networks, their models do not utilize dropout layers
and norm regularization methods (such as L1 and L2)
to avoid overfitting, and the setting of parameters may
not be optimal. In this study, we consider to construct
a new CNN based model to address the above issues.
The rest of the paper is organized as follows. In Section
II, we outline the datasets used in the experiment and
pre-processing approaches. The structure of our CNNs
model is shown in Section III. In Section IV, we utilized
three metrics to evaluate our model and show the results.
Also, we compare our result with other methods. Finally,
conclusion and future work are presented in Section V.
II. DATA SET A ND SU MM ARY STATIST IC S
A. Data preprocessing
The raw data were collected from a local company.
The data set includes both cell images and their corre-
sponding keryotypings. An example of a cell image and
its karyotype is shown in Fig.1.
For image preprocessing, we first use an image bi-
narization operator on each karyotype to turn it into
binary images. We then applied an area filter to remove
noises such as the labels [11]. After that, the region-
prop function of Matlab can easily generate bounding
boxes to contain individual chromosomes [12]. On each
karyotype, 46 single chromosome can be identified in
sequence. An example is shown in Figure 1. Each single
chromosome were centrally placed in a 142x282 picture
with black background, as shown in Figure 2. In the
end, we can obtain 23 sets of chromosomes for each
karyotype.
B. training and test dataset
After data pre-processing, we obtained images with
each of which contains only one individual chromosome.
The images were adjusted to the size of 120 ×40. In
(a)
(b)
(c)
Fig. 1. (a) A raw cell image with its 46 chromosomes; (b) karyotype
of image (a); (c) individual chromosomes with bounding boxes.
Fig. 2. A standardized individual chromosome
TABLE I
THE DATASE TS U SED IN TH E EX PE RIMEN T
Dataset training test total
1 8243 2061 10304
2 (Test Set) 0 4830 4830
this study, we utilized two datasets from these processed
data and they are shown in Table I. The size of two
datasets were 10304 and 4830 respectively. For dataset
1, we randomly divided it into a training set (8243) and
a test set (2061). The labels of training set range from 1
to 24, corresponding to chromosome classes. Dataset 2 is
used only for test. It consists 105 chromosome images,
each with 46 chromosomes labeled with the same image
number. The dataset will be used with a new evaluation
metric which will be introduced in Section V.
III. CLASSIFICATION MODEL BASED ON CNNS
Convolutional neural networks (CNNs) belong to ar-
tificial neural networks which have been widely used in
computer vision, natural language processing and other
fields. VGG-16 [12], ResNet50 [13], Inception network
[14] have achieved excellent efficiencies in areas such
as image classification and understanding. Five main
operations in these methods are convolutional kernel,
nonlinear transformations (activation functions), pooling
method, fully connected layers and loss function (clas-
sification). The emerging of the convolutional kernel is
illumined by the unique structure of cerebral cortex neu-
rons of cats to learn and integrate deep features of image
[15]. The activation functions and pooling approaches
assist convolutional layers to subsequently extract and
filter more useful signals [16]. As classification engines,
final fully connected layers and loss function can screen
and supervise deeply learned features to their specific
labels [17].
In this study we construct a new deep network struc-
ture based on CNN methods. There are five types of
layers used in our model, including convolution layer,
pooling layer, dropout layer, flatten layer and dense
layers. The main structure of the network is shown in
Fig. 3. In our architecture, the number of convolution
layers is four and each convolution layer use filters
of size 3*3 except the first one (5*5). This is because
small size filters can decrease the amount of computation
and hence to train the model faster. Also, the activation
function of each convolution layer is rectified linear units
(ReLU) rather than sigmoid:
f(x) = (0x0
x x > 0(1)
The reason is that compared with sigmoid function,
ReLU could efficiently solve gradient vanishing problem
in the backpropagation process of updating parameters
and reduce computation [16].
At the beginning of our model, we pile two convolu-
tion layers together and the two layers have 256 and 128
filters respectively. The reason is that it could enhance
the ability of model in learning features. For example,
compared with one layer, this structure incorporates
two non-linear relu activation functions rather than one
single function, which makes the decision function more
discriminative [12].
After a couple of convolution layers, a pooling layer
is used, which could decrease the amount of parameters
and accelerate the computation. Then a dropout layer
with parameter 0.5 connects to the end of pooling layer.
Dropout layer is utilized to randomly set some dimen-
sions of input vector to be zero with certain probability
(according to the parameter we set). Therefore, it does
not have any trainable parameters, meaning that there is
no updating during the process of training. This kind of
layer can mitigate overfitting to a large extent [18]. After
that, there are the third convolution layer, consisting of
256 filters, and a pooling layer. The final convolution
layer with 128 filters take the input from the last layer.
In terms of flatten layers, since the shape of data flow
is image (the format is array) which does not match the
desired type (vector), flatten layer is utilized to transform
the data flow. The final fully connected dense layer with
120 neurons followed by one output layer produces a
distribution over the 24 output labels.
To cater to multi-class classification, the activation
function chosen in the output layer is softmax:
y(z) = ez
i
Pjez
j
(2)
Where zis the input of the softmax layer, irepresents
ith class.
The optimization algorithm used here is Adam [19]
with learning rate 0.0001. Since our task in this study is
multi-classification, the loss function used here is cross-
entropy objective function:
L({x, y}N
1) = X
nX
i
y(n)
ilog(ˆy(n)
i)(3)
where {x, y}N
1are the training data with corresponding
label, y(n)
irepresents whether nth sample belongs to
class i, and the value is 1 if it is true, and 0 otherwise.
ˆy(n)
iis the probability that the network assigns the nth
sample to ith class.
IV. EXP ERIMENT S AN D EVALU ATI ON M ET RICS
In the experiments, the pre-processed dataset 1 and
2 were used. Training set of dataset 1 was firstly fed
into our model and the batch size was set at 128. After
being trained through 100 epoches, this mode was used
to predict chromosome types on test set of dataset 1 and
dataset 2. In this study, the ground truth of karyotype
provided by doctors are used to evaluate the results
of the proposed model and then the performance is
captured by two metrics, accuracy and proportion of
well classified karyotype (PWCK).
A. Accuracy
Accuracy refers to the proportion of correctly classi-
fied chromosomes in all chromosomes. This metric is
widely used in classification tasks since it can evaluate
the performance efficiently and gives clear evaluations.
As mentioned before, we evaluated 2061 chromosome
images of testing set of dataset 1, by using this metric.
Fig. 3. CNN architecture used for chromosome classification
On this testing set, we obtained an accuracy of 92.5%.
Table II shows this result.
TABLE II
PERFORMANCE OF THE PROPOSED METHOD AND OTHER THREE
METHODS FROM THE LITERATURE
Author Method Accuracy
S. Delshadpour [4] Multi-layer Perceptron 88.3%
Swati [1] Siamese Network 85.6%
M. Sharma [10] deep CNN 86.7%
Proposed based on LeNet 92.5%
To further evaluate the performance of our proposed
method, We compared our result with one previous
chromosome classifying algorithms of M. Sharma [10] by
using the same evaluation metrics. Since the parameters
of Sharma’s model might not be suitable for our data
sets, we optimized the settings of data for Sharma’s
model. From the results of all methods shown in Tabel
II, our algorithm outperformed Sharma’s approaches in
terms of accuracy. S. Delshadpour ’s method [4] and
Swati’s [1] approach were also listed in table II. However,
their results come directly from their papers, which were
obtained in their studies with their own testing datasets
different to the ones used in this study.
B. Proportion of well classified karyotype (PWCK)
Doctors usually check a patient’s chromosomes by
taking a karyotype image as a unit. They often focus
on whether the chromosomes of one karyotype image
are well classified. If the accuracy achieves a certain
threshold, doctors would regard the karyotype image as
a qualified karyotype image, otherwise the image will
be discarded. Proportion of well classified karyotype
(PWCK) evaluates the proportion of acceptable kary-
otype classified by the proposed method. The definition
of PWCK is as follows:
P W CK =PiI(Accuracy(i)>80%)
N(4)
where I is the indicator function,
I(x) = (0xis false
1xis true (5)
N is the number of karyotype image, Accuracy(i) is
the classification accuracy of 46 chromosomes in ith
karyotyping image and its definition is as follows:
Accuracy(i) =correct classified chromosomes
46 ×100%
(6)
The threshold of 80% in (4) was identified by two
doctors in the research group who commented that the
classification results on a karyotype is acceptable if over
80% of chromosomes in the karyotype were correctly
classified. Here we use PWCK to evaluate dataset 2
with 105 karyotype images (4830 chromosomes). We
achieved a result of 91.3%, which shows that our method
is applicable in real life tasks.
V. DISCUSSION AND CONCLUSION
In this study, an automatic-classification method based
on CNNs was proposed. The model extracts chromo-
some images from karyotype and output their classes.
Compared with three other methods and deep learning
algorithms, our method achieved a better accuracy. Our
experiment also shows that our method is applicable
in real life tasks. The results of this study showed that
CNNs is useful in extracting features in terms of pre-
processed medical images. To investigate if our proposed
method is actually applicable and acceptable to doctors,
we proposed a new metric PWCK, which is closely
related to the medical application, since doctors focus
on the accuracy of individual karyotype images. The re-
sults suggest that this proposed automatic classification
method can be utilized in chromosome classification, and
can potentially help doctors to save a lot of time.
Furthermore, in the process of obtaining PWCK, we
found the average accuracy was not as good enough
as we expected, because there was a small discrepancy
between this value and the overall accuracy (92.5%). This
difference might be caused by the different construction
of datasets, since dataset 1 and dataset 2 we used in this
task contained different human chromosomes. Since the
model was trained by part of dataset 1, the model was
familiar with chromosome images from the people who
provided training data. Therefore, our model need to
consider the inner variance of chromosomes of different
people. A data set containing chromosomes from more
people would improve the ability of generalization of
our model.
In this study, we only focus on classifying vertical
chromosomes. In other situations, some chromosomes
have different orientations, which should be considered
in our future work.
ACKNOWLEDGMENT
This study is supported by the National Natural Sci-
ence Foundation of China (Grant No. 61501380), Key
Program Special Fund in XJTLU (KSF), Sando Medical
Laboratories, Inc and the Open Program of Neusoft
Corporation, Item number SKLSAOP1702.
REFER EN CES
[1] Swati, G. Gupta, M. Yadav, M. Sharma, and L. Vig, “Siamese
networks for chromosome classification,” in IEEE International
Conference on Computer Vision Workshop, 2018, pp. 72–81.
[2] J. M. Cho, “Chromosome classification using backpropagation
neural networks,” IEEE Engineering in Medicine and Biology Mag-
azine, vol. 19, no. 1, pp. 28–33, 2000.
[3] J. Cho, S. Y. Ryu, and S. H. Woo, “A study for the hierarchi-
cal artificial neural network model for giemsa-stained human
chromosome classification,” in International Conference of the IEEE
Engineering in Medicine & Biology Society, vol. 2, 2004, pp. 4588–
4591.
[4] S. Delshadpour, “Reduced size multi layer perceptron neural
network for human chromosome classification,” in International
Conference of the IEEE Engineering in Medicine & Biology Society,
vol. 3, 2003, pp. 2249–2252.
[5] B. C. Oskouei and J. Shanbehzadeh, “Chromosome classification
based on wavelet neural network,” in International Conference on
Digital Image Computing: Techniques and Applications, 2010, pp. 605–
610.
[6] S. Gagulapalalic and M. Can, “Human chromosome classification
using competitive neural network teams (cnnt) and nearest neigh-
bor,” in Ieee-Embs International Conference on Biomedical and Health
Informatics, 2014, pp. 626–629.
[7] M. J. Roshtkhari and S. K. Setarehdan, “Linear discriminant anal-
ysis of the wavelet domain features for automatic classification
of human chromosomes,” in 2008 9th International Conference on
Signal Processing, Oct 2008, pp. 849–852.
[8] Q. Wu, Z. Liu, T. Chen, Z. Xiong, and K. R. Castleman, “Subspace-
based prototyping and classification of chromosome images,”
IEEE Transactions on Image Processing, vol. 14, no. 9, pp. 1277–1287,
Sept 2005.
[9] Y. Qiu, X. Lu, S. Yan, M. Tan, S. Cheng, S. Li, H. Liu, and B. Zheng,
“Applying deep learning technology to automatically identify
metaphase chromosomes using scanning microscopic images: an
initial investigation,” Proceedings of SPIE, vol. 9709, 2016.
[10] M. Sharma, O. Saha, A. Sriraman, R. Hebbalaguppe, L. Vig, and
S. Karande, “Crowdsourcing for chromosome segmentation and
deep classification,” in Computer Vision and Pattern Recognition
Workshops, 2017, pp. 786–793.
[11] J. Blahuta, T. Soukup, and P. ermk, “Image processing of med-
ical diagnostic neurosonographical images in matlab,” in Wseas
International Conference on Computers, 2011, pp. 85–90.
[12] K. Simonyan and A. Zisserman, “Very deep convolutional net-
works for large-scale image recognition,” in International conference
on learning representations, 2015.
[13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
image recognition,” Computer vision and pattern recognition, pp.
770–778, 2016.
[14] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov,
D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with
convolutions,” Computer vision and pattern recognition, pp. 1–9,
2015.
[15] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recogni-
tion: a convolutional neural-network approach,” IEEE Transactions
on Neural Networks, vol. 8, no. 1, pp. 98–113, 1997.
[16] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT
Press, 2016.
[17] C. Bishop, Pattern Recognition and Machine Learning. Springer,
2006.
[18] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and
R. Salakhutdinov, “Dropout:a simple way to prevent neural net-
works from overfitting,” Journal of Machine Learning Research,
vol. 15, pp. 1929–1958, 2014.
[19] D. Kingma and J. Ba, “Adam: A method for stochastic optimiza-
tion,” Computer Science, 2014.
... In [25] convolutional blocks with DenseNet-like skip connections and "network-in-network" structures are used. Papers [26][27][28][29][30][31][32][33] have also proposed different deep CNN architectures for the classification. In [32], a chromosome classification approach based on a deep neural network with a triple attention mechanism (TAM), is proposed. ...
... Comparison of results of the proposed scheme with other methods with different datasets (percent)Sharma et al.[20] Images from a local hospital 86.7Zhang et al.[29] 224 karyotyping images from local company 92.5Qin et al.[28] Ning et al.[19] Wei et al.[33] 300 karyotype images from a local hospital Chen et al.[38] Ikaros dataset from Shanghai Xinhua Hospital ...
Article
Full-text available
Chromosome analysis and classification are essential in clinical applications to diagnose various structural and numerical abnormalities. Recently, karyotype analysis using intelligent image processing methods, especially deep learning, has attracted significant attention as a genetic abnormality test. This paper presents a novel chromosome classification algorithm that uses high-level features extracted from deep convolutional neural networks (DCNN) along with morphological features designed to identify and modify the classes of misclassified chromosomes. Initially, chromosomes are classified using a DCNN. Some structural features, such as centromere and banding profile, are then extracted to group chromosomes again. Based on the results of the two preceding methods, a decision strategy is utilized to identify misclassified chromosomes. Here, a final DCNN-based strategy is introduced to assign misclassified chromosomes to the associated classes. The proposed method can be used in parallel with other chromosome classification methods to modify misclassified chromosomes and promote the accuracy of the classification. Evaluation results show that the proposed algorithm outperforms relevant state-of-the-art algorithms regarding the classification precision and accuracy of 99.66 and 96.52%, respectively.
... While karyotyping is an essential procedure, it is also time-consuming and requires skilled cytologists to work manually on cells. As a result, researchers have developed computational techniques [4][5][6][7] to automate it, often adopting artificial neural networks to save time and effort. ...
... bounding_box ← find largest chromosome bounding box using x bw 6: ...
Article
Full-text available
Object classification is a crucial task in deep learning, which involves the identification and categorization of objects in images or videos. Although humans can easily recognize common objects, such as cars, animals, or plants, performing this task on a large scale can be time-consuming and error-prone. Therefore, automating this process using neural networks can save time and effort while achieving higher accuracy. Our study focuses on the classification step of human chromosome karyotyping, an important medical procedure that helps diagnose genetic disorders. Traditionally, this task is performed manually by expert cytologists, which is a time-consuming process that requires specialized medical skills. Therefore, automating it through deep learning can be immensely useful. To accomplish this, we implemented and adapted existing preprocessing and data augmentation techniques to prepare the chromosome images for classification. We used ResNet-50 convolutional neural network and Swin Transformer, coupled with an ensemble approach to classify the chromosomes, obtaining state-of-the-art performance in the tested dataset.
... 32 While karyotyping is an essential procedure, it is also time-consuming and requires 33 skilled cytologists to work manually on cells. As a result, researchers have developed 34 computational techniques [4][5][6][7] to automate it. 35 The two main challenges of automated karyotyping are chromosome segmentation 36 and chromosome classification. ...
... bounding_box ← find largest chromosome bounding box using x bw 6: ...
Preprint
Full-text available
Object classification is a crucial task in deep learning, which involves the identification and categorization of objects in images or videos. Although humans can easily recognize common objects, such as cars, animals, or plants, performing this task on a large scale can be time-consuming and error-prone. Therefore, automating this process using neural networks can save time and effort while achieving higher accuracy. Our study focuses on the classification step of human chromosome karyotyping, an important medical procedure that helps diagnose genetic disorders. Traditionally, this task is performed manually by expert cytologists, which is a time-consuming process that requires specialized medical skills. Therefore, automating it through deep learning can be immensely useful. To accomplish this, we implemented and adapted existing preprocessing and data augmentation techniques to prepare the chromosome images for classification. We used ResNet-50 convolutional neural networks and an ensemble approach to classify the chromosomes, obtaining state-of-the-art performance in the tested dataset.
... Deep learning (DL) technology [20][21][22][23][24] has recently shown outstanding performance in image analysis. In related research fields, the DL method has been actively applied to classify Giemsa-banded chromosomes for karyotyping [25][26][27][28][29][30][31]. In addition, Jang et al. [8] have successfully applied the DL method to identify DCs from ex vivo conventional Giemsa-stained chromosome images using the Faster Region-based Convolutional Network (Faster R-CNN) [32] with Feature Pyramid Networks (FPN) [33]. ...
... Studies have been conducted to automate DCA using machine learning [10,11] and deep learning [12][13][14] by using it to construct dose-response curves and calculate estimated doses [13] or as a classifier for individual chromosome patches extracted from images of individual metaphases [14]. In addition, several studies have been conducted on chromosome classification based on deep learning [15,16], and there have been studies on deep learning-based segmentation for biomedical images [17][18][19]. However, applying those findings to automate DCA has critical limitations because additional segmentation or localization methods are required before classification to obtain individual chromosome data within the image. ...
Article
Full-text available
Dicentric chromosome assay (DCA) is one of the cytogenetic dosimetry methods where the absorbed dose is estimated by counting the number of dicentric chromosomes, which is a major radiation-induced change in DNA. However, DCA is a time-consuming task and requires technical expertise. In this study, a neural network was applied for automating the DCA. We used YOLOv5, a one-stage detection algorithm, to mitigate these limitations by automating the estimation of the number of dicentric chromosomes in chromosome metaphase images. YOLOv5 was pretrained on common object datasets. For training, 887 augmented chromosome images were used. We evaluated the model using validation and test datasets with 380 and 300 images, respectively. With pretrained parameters, the trained model detected chromosomes in the images with a maximum F1 score of 0.94 and a mean average precision (mAP) of 0.961. Conversely, when the model was randomly initialized, the training performance decreased, with a maximum F1 score and mAP of 0.82 and 0.873%, respectively. These results confirm that the model could effectively detect dicentric chromosomes in an image. Consequently, automatic DCA is expected to be conducted based on deep learning for object detection, requiring a relatively small amount of chromosome data for training using the pretrained network.
... Due to their non-rigid nature, condensed chromosomes exhibit varying degrees of curvature under the microscope. Therefore, chromosome straightening is an important upstream task for chromosome classification [3] and the subsequent karyotype construction and cytogenetic map development [4]. ...
Article
Context.— Conventional karyotype analysis, which provides comprehensive cytogenetic information, plays a significant role in the diagnosis and risk stratification of hematologic neoplasms. The main limitations of this approach include long turnaround time and laboriousness. Therefore, we developed an integral R-banded karyotype analysis system for bone marrow metaphases, based on deep learning. Objective.— To evaluate the performance of the internal models and the entire karyotype analysis system for R-banded bone marrow metaphase. Design.— A total of 4442 sets of R-banded normal bone marrow metaphases and karyograms were collected. Accordingly, 4 deep learning–based models for different analytic stages of karyotyping, including denoising, segmentation, classification, and polarity recognition, were developed and integrated as an R-banded bone marrow karyotype analysis system. Five-fold cross validation was performed on each model. The whole system was implemented by 2 strategies of automatic and semiautomatic workflows. A test set of 885 metaphases was used to assess the entire system. Results.— The denoising model achieved an intersection-over-union (IoU) of 99.20% and a Dice similarity coefficient (DSC) of 99.58% for metaphase acquisition. The segmentation model achieved an IoU of 91.95% and a DSC of 95.79% for chromosome segmentation. The accuracies of the segmentation, classification, and polarity recognition models were 96.77%, 98.77%, and 99.93%, respectively. The whole system achieved an accuracy of 93.33% with the automatic strategy and an accuracy of 99.06% with the semiautomatic strategy. Conclusions.— The performance of both the internal models and the entire system is desirable. This deep learning–based karyotype analysis system has potential in clinical application.
Conference Paper
Full-text available
Karyotyping is the process of pairing and ordering 23 pairs of human chromosomes from cell images on the basis of size, centromere position, and banding pattern. Karyotyping during metaphase is often used by clinical cytogeneticists to analyze human chromosomes for diagnostic purposes. It requires experience, domain expertise and considerable manual effort to efficiently perform karyotyping and diagnosis of various disorders. Therefore, automation or even partial automation is highly desirable to assist technicians and reduce the cognitive load necessary for karyotyping. With these motivations, in this paper, we attempt to develop methods for chromosome classification by borrowing the latest ideas from deep learning. More specifically, we perform straightening on chromosomes and feed them into Siamese Networks to push the embeddings of samples coming from similar labels closer. Further, we propose to perform balanced sampling from the pairwise dataset while selecting dissimilar training pairs for Siamese Networks, and an MLP based prediction on top of the embeddings obtained from the trained Siamese Networks. We perform our experiments on a real world dataset of healthy patients collected from a hospital and exhaustively compare the effect of different straightening techniques, by applying them to chromosome images prior to classification. Results demonstrate that the proposed methods speed up both training and prediction by 83 and 3 folds, respectively; while surpassing the performance of a very competitive baseline created utilizing deep convolutional neural networks.
Technical Report
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Article
Automated high throughput scanning microscopy is a fast developing screening technology used in cytogenetic laboratories for the diagnosis of leukemia or other genetic diseases. However, one of the major challenges of using this new technology is how to efficiently detect the analyzable metaphase chromosomes during the scanning process. The purpose of this investigation is to develop a computer aided detection (CAD) scheme based on deep learning technology, which can identify the metaphase chromosomes with high accuracy. The CAD scheme includes an eight layer neural network. The first six layers compose of an automatic feature extraction module, which has an architecture of three convolution-max-pooling layer pairs. The 1st, 2nd and 3rd pair contains 30, 20, 20 feature maps, respectively. The seventh and eighth layers compose of a multiple layer perception (MLP) based classifier, which is used to identify the analyzable metaphase chromosomes. The performance of new CAD scheme was assessed by receiver operation characteristic (ROC) method. A number of 150 regions of interest (ROIs) were selected to test the performance of our new CAD scheme. Each ROI contains either interphase cell or metaphase chromosomes. The results indicate that new scheme is able to achieve an area under the ROC curve (AUC) of 0.886±0.043. This investigation demonstrates that applying a deep learning technique may enable to significantly improve the accuracy of the metaphase chromosome detection using a scanning microscopic imaging technology in the future. © (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Article
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets. © 2014 Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov.
Article
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
Conference Paper
This paper presents a novel approach to human chromosome classification. Human cell contains 22 pairs of autosomes and a pair of sex chromosomes. In this research, 22 types of autosomes represent 22 classes to be distinguished. New method of classification is based on the special organized committee of 462 simple perceptrons, called Competitive Neural Network Teams (CNNTs). Each perceptron is trained to differentiate two classes (i.e. two types of chromosome), hence there are 22 × 21 learning machines. Moreover, dummy perceptrons are set to zero for the chromosomes from the same class. The final outcome of the testing data is a 22×22 decision matrix, containing outcomes of each machine. With the special interpretation of these decisions, higher correct classification rate is achieved, reaching over 95%. The method can be further improved when testing is performed on a cell-by-cell basis by using CNNT complemented by Nearest Neighbor technique. The classification is applied to the Copenhagen chromosome data set and Sarajevo chromosome data set.
Article
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions. The method is straightforward to implement and is based an adaptive estimates of lower-order moments of the gradients. The method is computationally efficient, has little memory requirements and is well suited for problems that are large in terms of data and/or parameters. The method is also ap- propriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The method exhibits invariance to diagonal rescaling of the gradients by adapting to the geometry of the objective function. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. We demonstrate that Adam works well in practice when experimentally compared to other stochastic optimization methods.