Conference PaperPDF Available

Chromosome Classification with Convolutional Neural Network Based Deep Learning



Content may be subject to copyright.
Chromosome Classification with Convolutional
Neural Network based Deep Learning
Wenbo ZhangSifan SongTianming BaiYanxin ZhaoFei Ma∗§ Jionglong Su∗‡§ Limin Yu†§
Mathematical Sciences Department, Xi’an Jiaotong-Liverpool University, Suzhou, China
Department of Electrical and Electronic Engineering, Xi’an Jiaotong-Liverpool University, Suzhou, China
Neusoft Corporation, Shenyang, China
§Research Center for Precision Medicine, HT-URC, Xi’an Jiaotong-Liverpool University
Suzhou, China, Tel: (86) 512 8816 1633, Email:,
Abstract—Karyotyping plays a crucial role in genetic
disorder diagnosis. Currently Karyotyping requires consid-
erable manual efforts, domain expertise and experience,
and is very time consuming. Automating the karyotyping
process has been an important and popular task. This study
focuses on classification of chromosomes into 23 types, a
step towards fully automatic karyotyping. This study pro-
poses a convolutional neural network (CNN) based deep
learning network to automatically classify chromosomes.
The proposed method was trained and tested on a dataset
containing 10304 chromosome images, and was further tested
on a dataset containing 4830 chromosomes. The proposed
method achieved an accuracy of 92.5%, outperforming three
other methods appeared in the literature. To investigate
how applicable the proposed method is to the doctors, a
metric named proportion of well classified karyotype was
also designed. An result of 91.3% was achieved on this metric,
indicating that the proposed classification method could be
used to aid doctors in genetic disorder diagnosis.
The human cell normally contains 23 pairs of chromo-
somes, including 22 pairs of autosomes (the ones exist
in both males and females) as well as sex chromosomes
X and Y. Females have double X chromosomes as one
pair of sex chromosomes, while males have both X
and Y. Chromosome abnormality, namely aneuploidy
(having abnormal number of chromosomes in a cell)
and structural abnormalities (including deletions, du-
plication, translocation, inversion, insertions rings, and
isochromosome) may cause genetic disorder such as
Down’s syndrome. It is important to inspect the cells
of a patient and identify any irregular, extra or missing
parts for diagnostic purposes. Karyotyping, the process
of separating and classifying human chromosomes from
a cell image, plays a crucial role in this diagnosis process
However, accomplishing this work efficiently not only
requires considerable manual efforts, domain expertise
and experience, but also consumes a lot of time. Since
1980, with the motivation of lightening the load of cyto-
geneticists, automatic diagnosis systems for chromosome
karyotyping and analysis have become a popular and
important task.
J. M. Cho chose the two-layer artificial neural network
with the error backpropagation training as chromosome
classifier, which resulted an overall classification error
rate of 6.52% in the 460 chromosomes images [2]. To
overcome the higher classification error, J. Cho et. al. [3]
proposed a hierarchical multi-layer network as chromo-
some classifier and an error back-propagation training
algorithm. The overall result of classification error in this
method was 5.9% which was based on the 7 experiments.
S. Delshadpour [4] reduced the complexity of an
ANN in order to increase the performance of ANN and
combined an improved multi-layer perceptron neural
network for automated classification of chromosomes.
The overall accuracy of classification was increased to
88.3% on 304 chromosomes. However, their classification
results on 24 classes vary and on many classes the results
are not very accurate. To overcome the problems, B. C.
Oskouei and J. Shanbehzadeh [5] proposed a classifier
based on the wavelet neural network. They obtained an
accuracy rate of 93.35%. S. Gagulapalalic and M. Can
[6] proposed a novel method based on the Competi-
tive Neural Network Teams (CNNTs) to distinguish 22
types of the autosomes. Their method achieved better
classifying results, i.e. approximately 96.64%, on 150
chromosome images with each type of autosomes.
M. J. Roshtkhari and S. K. Setarehdan [7] presented a
wavelet transform based linear discriminant analysis to
classify normal and automatically straightened chromo-
somes, and a three layers feed-forward perceptron neural
network which was trained using the backpropagation
algorithm. The overall outcome of correct classification
was 99.3% after 303 highly curved chromosomes were
straightened. A subspace-based approach was proposed
by Q. Wu et. al., which synthesized the prototype chro-
mosome images and utilized transformation coefficients
as the feature measurements [8]. The result shows that
this method could synthesize highly visual prototype
chromosome images which were previously unseen in
2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI 2018)
978-1-5386-7604-2/18/$31.00 ©2018 IEEE
chromosome classification. Nevertheless, most of the
proposed systems are partially automated and still need
much manual assistance, since defining and extracting
features of images with overlapping or even touching
chromosomes are still difficult steps [6].
With the development of CNNs, they have been uti-
lized in medical image sector to deal with complicated
features. CNN based models have been explored to
analyze chromosome images. In 2016, Qiu et al. pro-
posed a 8-layer convolutional neural network and its
lowest testing error achieved 13.3%, which is the first
research proposing a CNN model to classify metaphase
chromosome images [9]. In 2017, Sharma et al. utilized a
4 deep learning blocks to construct a model and achieved
an accuracy of 68.5% (without straightening) and 86.7%
(with straightening) [10]. Swati et al. proposed a model
based on the Siamese networks which is composed by
a twin neural network and the best model yields 85.2%
classification accuracy [1].
Although above methods are based on convolutional
networks, their models do not utilize dropout layers
and norm regularization methods (such as L1 and L2)
to avoid overfitting, and the setting of parameters may
not be optimal. In this study, we consider to construct
a new CNN based model to address the above issues.
The rest of the paper is organized as follows. In Section
II, we outline the datasets used in the experiment and
pre-processing approaches. The structure of our CNNs
model is shown in Section III. In Section IV, we utilized
three metrics to evaluate our model and show the results.
Also, we compare our result with other methods. Finally,
conclusion and future work are presented in Section V.
A. Data preprocessing
The raw data were collected from a local company.
The data set includes both cell images and their corre-
sponding keryotypings. An example of a cell image and
its karyotype is shown in Fig.1.
For image preprocessing, we first use an image bi-
narization operator on each karyotype to turn it into
binary images. We then applied an area filter to remove
noises such as the labels [11]. After that, the region-
prop function of Matlab can easily generate bounding
boxes to contain individual chromosomes [12]. On each
karyotype, 46 single chromosome can be identified in
sequence. An example is shown in Figure 1. Each single
chromosome were centrally placed in a 142x282 picture
with black background, as shown in Figure 2. In the
end, we can obtain 23 sets of chromosomes for each
B. training and test dataset
After data pre-processing, we obtained images with
each of which contains only one individual chromosome.
The images were adjusted to the size of 120 ×40. In
Fig. 1. (a) A raw cell image with its 46 chromosomes; (b) karyotype
of image (a); (c) individual chromosomes with bounding boxes.
Fig. 2. A standardized individual chromosome
Dataset training test total
1 8243 2061 10304
2 (Test Set) 0 4830 4830
this study, we utilized two datasets from these processed
data and they are shown in Table I. The size of two
datasets were 10304 and 4830 respectively. For dataset
1, we randomly divided it into a training set (8243) and
a test set (2061). The labels of training set range from 1
to 24, corresponding to chromosome classes. Dataset 2 is
used only for test. It consists 105 chromosome images,
each with 46 chromosomes labeled with the same image
number. The dataset will be used with a new evaluation
metric which will be introduced in Section V.
Convolutional neural networks (CNNs) belong to ar-
tificial neural networks which have been widely used in
computer vision, natural language processing and other
fields. VGG-16 [12], ResNet50 [13], Inception network
[14] have achieved excellent efficiencies in areas such
as image classification and understanding. Five main
operations in these methods are convolutional kernel,
nonlinear transformations (activation functions), pooling
method, fully connected layers and loss function (clas-
sification). The emerging of the convolutional kernel is
illumined by the unique structure of cerebral cortex neu-
rons of cats to learn and integrate deep features of image
[15]. The activation functions and pooling approaches
assist convolutional layers to subsequently extract and
filter more useful signals [16]. As classification engines,
final fully connected layers and loss function can screen
and supervise deeply learned features to their specific
labels [17].
In this study we construct a new deep network struc-
ture based on CNN methods. There are five types of
layers used in our model, including convolution layer,
pooling layer, dropout layer, flatten layer and dense
layers. The main structure of the network is shown in
Fig. 3. In our architecture, the number of convolution
layers is four and each convolution layer use filters
of size 3*3 except the first one (5*5). This is because
small size filters can decrease the amount of computation
and hence to train the model faster. Also, the activation
function of each convolution layer is rectified linear units
(ReLU) rather than sigmoid:
f(x) = (0x0
x x > 0(1)
The reason is that compared with sigmoid function,
ReLU could efficiently solve gradient vanishing problem
in the backpropagation process of updating parameters
and reduce computation [16].
At the beginning of our model, we pile two convolu-
tion layers together and the two layers have 256 and 128
filters respectively. The reason is that it could enhance
the ability of model in learning features. For example,
compared with one layer, this structure incorporates
two non-linear relu activation functions rather than one
single function, which makes the decision function more
discriminative [12].
After a couple of convolution layers, a pooling layer
is used, which could decrease the amount of parameters
and accelerate the computation. Then a dropout layer
with parameter 0.5 connects to the end of pooling layer.
Dropout layer is utilized to randomly set some dimen-
sions of input vector to be zero with certain probability
(according to the parameter we set). Therefore, it does
not have any trainable parameters, meaning that there is
no updating during the process of training. This kind of
layer can mitigate overfitting to a large extent [18]. After
that, there are the third convolution layer, consisting of
256 filters, and a pooling layer. The final convolution
layer with 128 filters take the input from the last layer.
In terms of flatten layers, since the shape of data flow
is image (the format is array) which does not match the
desired type (vector), flatten layer is utilized to transform
the data flow. The final fully connected dense layer with
120 neurons followed by one output layer produces a
distribution over the 24 output labels.
To cater to multi-class classification, the activation
function chosen in the output layer is softmax:
y(z) = ez
Where zis the input of the softmax layer, irepresents
ith class.
The optimization algorithm used here is Adam [19]
with learning rate 0.0001. Since our task in this study is
multi-classification, the loss function used here is cross-
entropy objective function:
L({x, y}N
1) = X
where {x, y}N
1are the training data with corresponding
label, y(n)
irepresents whether nth sample belongs to
class i, and the value is 1 if it is true, and 0 otherwise.
iis the probability that the network assigns the nth
sample to ith class.
In the experiments, the pre-processed dataset 1 and
2 were used. Training set of dataset 1 was firstly fed
into our model and the batch size was set at 128. After
being trained through 100 epoches, this mode was used
to predict chromosome types on test set of dataset 1 and
dataset 2. In this study, the ground truth of karyotype
provided by doctors are used to evaluate the results
of the proposed model and then the performance is
captured by two metrics, accuracy and proportion of
well classified karyotype (PWCK).
A. Accuracy
Accuracy refers to the proportion of correctly classi-
fied chromosomes in all chromosomes. This metric is
widely used in classification tasks since it can evaluate
the performance efficiently and gives clear evaluations.
As mentioned before, we evaluated 2061 chromosome
images of testing set of dataset 1, by using this metric.
Fig. 3. CNN architecture used for chromosome classification
On this testing set, we obtained an accuracy of 92.5%.
Table II shows this result.
Author Method Accuracy
S. Delshadpour [4] Multi-layer Perceptron 88.3%
Swati [1] Siamese Network 85.6%
M. Sharma [10] deep CNN 86.7%
Proposed based on LeNet 92.5%
To further evaluate the performance of our proposed
method, We compared our result with one previous
chromosome classifying algorithms of M. Sharma [10] by
using the same evaluation metrics. Since the parameters
of Sharma’s model might not be suitable for our data
sets, we optimized the settings of data for Sharma’s
model. From the results of all methods shown in Tabel
II, our algorithm outperformed Sharma’s approaches in
terms of accuracy. S. Delshadpour ’s method [4] and
Swati’s [1] approach were also listed in table II. However,
their results come directly from their papers, which were
obtained in their studies with their own testing datasets
different to the ones used in this study.
B. Proportion of well classified karyotype (PWCK)
Doctors usually check a patient’s chromosomes by
taking a karyotype image as a unit. They often focus
on whether the chromosomes of one karyotype image
are well classified. If the accuracy achieves a certain
threshold, doctors would regard the karyotype image as
a qualified karyotype image, otherwise the image will
be discarded. Proportion of well classified karyotype
(PWCK) evaluates the proportion of acceptable kary-
otype classified by the proposed method. The definition
of PWCK is as follows:
P W CK =PiI(Accuracy(i)>80%)
where I is the indicator function,
I(x) = (0xis false
1xis true (5)
N is the number of karyotype image, Accuracy(i) is
the classification accuracy of 46 chromosomes in ith
karyotyping image and its definition is as follows:
Accuracy(i) =correct classified chromosomes
46 ×100%
The threshold of 80% in (4) was identified by two
doctors in the research group who commented that the
classification results on a karyotype is acceptable if over
80% of chromosomes in the karyotype were correctly
classified. Here we use PWCK to evaluate dataset 2
with 105 karyotype images (4830 chromosomes). We
achieved a result of 91.3%, which shows that our method
is applicable in real life tasks.
In this study, an automatic-classification method based
on CNNs was proposed. The model extracts chromo-
some images from karyotype and output their classes.
Compared with three other methods and deep learning
algorithms, our method achieved a better accuracy. Our
experiment also shows that our method is applicable
in real life tasks. The results of this study showed that
CNNs is useful in extracting features in terms of pre-
processed medical images. To investigate if our proposed
method is actually applicable and acceptable to doctors,
we proposed a new metric PWCK, which is closely
related to the medical application, since doctors focus
on the accuracy of individual karyotype images. The re-
sults suggest that this proposed automatic classification
method can be utilized in chromosome classification, and
can potentially help doctors to save a lot of time.
Furthermore, in the process of obtaining PWCK, we
found the average accuracy was not as good enough
as we expected, because there was a small discrepancy
between this value and the overall accuracy (92.5%). This
difference might be caused by the different construction
of datasets, since dataset 1 and dataset 2 we used in this
task contained different human chromosomes. Since the
model was trained by part of dataset 1, the model was
familiar with chromosome images from the people who
provided training data. Therefore, our model need to
consider the inner variance of chromosomes of different
people. A data set containing chromosomes from more
people would improve the ability of generalization of
our model.
In this study, we only focus on classifying vertical
chromosomes. In other situations, some chromosomes
have different orientations, which should be considered
in our future work.
This study is supported by the National Natural Sci-
ence Foundation of China (Grant No. 61501380), Key
Program Special Fund in XJTLU (KSF), Sando Medical
Laboratories, Inc and the Open Program of Neusoft
Corporation, Item number SKLSAOP1702.
[1] Swati, G. Gupta, M. Yadav, M. Sharma, and L. Vig, “Siamese
networks for chromosome classification,” in IEEE International
Conference on Computer Vision Workshop, 2018, pp. 72–81.
[2] J. M. Cho, “Chromosome classification using backpropagation
neural networks,” IEEE Engineering in Medicine and Biology Mag-
azine, vol. 19, no. 1, pp. 28–33, 2000.
[3] J. Cho, S. Y. Ryu, and S. H. Woo, “A study for the hierarchi-
cal artificial neural network model for giemsa-stained human
chromosome classification,” in International Conference of the IEEE
Engineering in Medicine & Biology Society, vol. 2, 2004, pp. 4588–
[4] S. Delshadpour, “Reduced size multi layer perceptron neural
network for human chromosome classification,” in International
Conference of the IEEE Engineering in Medicine & Biology Society,
vol. 3, 2003, pp. 2249–2252.
[5] B. C. Oskouei and J. Shanbehzadeh, “Chromosome classification
based on wavelet neural network,” in International Conference on
Digital Image Computing: Techniques and Applications, 2010, pp. 605–
[6] S. Gagulapalalic and M. Can, “Human chromosome classification
using competitive neural network teams (cnnt) and nearest neigh-
bor,” in Ieee-Embs International Conference on Biomedical and Health
Informatics, 2014, pp. 626–629.
[7] M. J. Roshtkhari and S. K. Setarehdan, “Linear discriminant anal-
ysis of the wavelet domain features for automatic classification
of human chromosomes,” in 2008 9th International Conference on
Signal Processing, Oct 2008, pp. 849–852.
[8] Q. Wu, Z. Liu, T. Chen, Z. Xiong, and K. R. Castleman, “Subspace-
based prototyping and classification of chromosome images,”
IEEE Transactions on Image Processing, vol. 14, no. 9, pp. 1277–1287,
Sept 2005.
[9] Y. Qiu, X. Lu, S. Yan, M. Tan, S. Cheng, S. Li, H. Liu, and B. Zheng,
“Applying deep learning technology to automatically identify
metaphase chromosomes using scanning microscopic images: an
initial investigation,” Proceedings of SPIE, vol. 9709, 2016.
[10] M. Sharma, O. Saha, A. Sriraman, R. Hebbalaguppe, L. Vig, and
S. Karande, “Crowdsourcing for chromosome segmentation and
deep classification,” in Computer Vision and Pattern Recognition
Workshops, 2017, pp. 786–793.
[11] J. Blahuta, T. Soukup, and P. ermk, “Image processing of med-
ical diagnostic neurosonographical images in matlab,” in Wseas
International Conference on Computers, 2011, pp. 85–90.
[12] K. Simonyan and A. Zisserman, “Very deep convolutional net-
works for large-scale image recognition,” in International conference
on learning representations, 2015.
[13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
image recognition,” Computer vision and pattern recognition, pp.
770–778, 2016.
[14] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov,
D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with
convolutions,” Computer vision and pattern recognition, pp. 1–9,
[15] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recogni-
tion: a convolutional neural-network approach,” IEEE Transactions
on Neural Networks, vol. 8, no. 1, pp. 98–113, 1997.
[16] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT
Press, 2016.
[17] C. Bishop, Pattern Recognition and Machine Learning. Springer,
[18] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and
R. Salakhutdinov, “Dropout:a simple way to prevent neural net-
works from overfitting,” Journal of Machine Learning Research,
vol. 15, pp. 1929–1958, 2014.
[19] D. Kingma and J. Ba, “Adam: A method for stochastic optimiza-
tion,” Computer Science, 2014.
... From the graphical illustrations of Figure 7, it is evident that the training accuracy increases as the epochs increases and reaches its maximum value at 200 epochs. The other important categorization metrics utilized to assess the performance of the proposed classifier are recall, F1 score, and precision (Zhang et al., 2018). ...
Full-text available
This article offers a framework of a hybrid convolution neural network (CNN) – support vector machine (SVM) model for the classification of human metaphase chromosome images. In this model, the features extracted by CNN were fed to the SVM classifier to label the chromosomes into 24 classes. Classification accuracy is generally affected by low‐resolution chromosome images. Hence, in the proposed work, a Laplacian pyramidal super‐resolution network (LaPSRN) was deployed to improve input chromosome images resolution before feeding them to the classifier network. The experimentation showed that due to LaPSRN, resolution‐enhanced images are obtained, which has improved the classification accuracy by 3% compared to the architecture without a super‐resolution block. In addition, instead of using the usual ReLU function in CNN architecture, a Swish activation function was utilized, which enhanced the validation accuracy by 0.8%. After experimenting with various hyper‐parameters for the CNN SVM architecture, the ADAM optimizer with step decay outperformed the other standard optimizers. The novel part of this work is the combination of Laplacian pyramidal super‐resolution with the hybrid CNN‐SVM model, which has not been employed yet for chromosome classification tasks as per our knowledge. The proposed hybrid CNN SVM architecture with the Swish activation function and super‐resolution techniques was tested on the metaphase chromosome images from the BioImLab dataset, and an improved classification accuracy of 94.6% was obtained. Moreover, the standard metrics for classification like F1 score, precision, support, and the confusion matrix values of the proposed CNN model with SVM classifier are superior to the prevailing CNN architectures for chromosome images.
Karyotyping is an important procedure to assess the possible existence of chromosomal abnormalities. However, because of the non-rigid nature, chromosomes are usually heavily curved in microscopic images and such deformed shapes hinder the chromosome analysis for cytogeneticists. In this paper, we present a self-attention guided framework to erase the curvature of chromosomes. The proposed framework extracts spatial information and local textures to preserve banding patterns in a regression module. With complementary information from the bent chromosome, a refinement module is designed to further improve fine details. In addition, we propose two dedicated geometric constraints to maintain the length and restore the distortion of chromosomes. To train our framework, we create a synthetic dataset where curved chromosomes are generated from the real-world straight chromosomes by grid-deformation. Quantitative and qualitative experiments are conducted on synthetic and real-world data. Experimental results show that our proposed method can effectively straighten bent chromosomes while keeping banding details and length.KeywordsKaryotypingChromosome straighteningSelf-attentionMicroscopy
Chromosome karyotyping analysis is a vital cytogenetics technique for diagnosing genetic and congenital malformations, analyzing gestational and implantation failures, etc. Since the chromosome classification as an essential stage in chromosome karyotype analysis is a highly time-consuming, tedious, and error-prone task, which requires a large amount of manual work of experienced cytogenetics experts. Many deep learning-based methods have been proposed to address the chromosome classification issues. However, two challenges still remain in current chromosome classification methods. First, most existing methods were developed by different private datasets, making these methods difficult to compare with each other on the same base. Second, due to the absence of reproducing details of most existing methods, these methods are difficult to be applied in clinical chromosome classification applications widely. To address the above challenges in the chromosome classification issue, this work builds and publishes a massive clinical dataset. This dataset enables the benchmarking and building chromosome classification baselines suitable for different scenarios. The massive clinical dataset consists of 126,453 privacy preserving G-band chromosome instances from 2,763 karyotypes of 408 individuals. To our best knowledge, it is the first work to collect, annotate, and release a publicly available clinical chromosome classification dataset whose data size scale is also over 120,000. Meanwhile, the experimental results show that the proposed dataset can boost performance of existing chromosome classification models at a varied range of degrees, with the highest accuracy improvement by 5.39 percentage points. Moreover, the best baseline with 99.33% accuracy reports state-of-the-art classification performance. The clinical dataset and state-of-the-art baselines can be found at
Full-text available
Chromosome karyotyping is pivotal in the diagnosis of many genetic disorders and birth defects. Manual karyotyping is a tedious procedure and various techniques were proposed in the literature to automate this process. As Artificial Intelligence algorithms are unfolding their vast potential in a plethora of applications, automatic karyotyping of chromosomes remains one of the critical challenges yet to be fully addressed. Deep learning techniques have gained rapid momentum in the computer vision domain specifically for chromosome image segmentation and classification. This paper proposes Chromenet—a simplified CNN architecture for classification of metaphase Q banded chromosome images and achieves good classification accuracy than the state of art methods proposed in recent times. This research discusses the comparison of newly suggested optimizers with the standard deep learning optimizers. Different optimizers like SGD, ADAM, Adagrad, Nadam, Adabound and RMSprop were used and a comparison of classification accuracy and loss was obtained. In this work, it was proposed to use step-based decay for ADAM optimizer in which the learning rate is modified every 15–20 epochs. After excessive experimentation, it was found that the ADAM optimizer with step-based decay achieved classification accuracy around 92.6% which is better in comparison to the other adaptive optimizers for the proposed CNN architecture. Moreover, the performance metrics like F1 score, precision, support, sensitivity, Negative predictive value, Jaccard score and confusion matrix used for the evaluation of the proposed model prove that the proffered architecture is more suitable for classification of human chromosome images when compared to the other methods proposed in the recent literature.
A fully automated chromosome analysis system can substitute cytogenetic experts for the task of chromosome karyotype analysis, which in turn can substantially increase the efficiency of disease diagnosis. However, the construction of such a system is most crucially restricted by the accuracy of chromosome classification, during karyotype analysis. To facilitate the construction of an automatic chromosome analysis system, an input-aware and probabilistic prediction convolutional neural network (IAPP-CNN) is presented in this paper for high accuracy of chromosome classification. The approach follows three stages and consists of one input-aware module, one feature extractor module and one probabilistic prediction module. In the first stage, the input-aware module develops raw images automatically into the global-scale image, the object-scale image and the part-scale image, by introducing an attention mechanism. In the second stage, the three scale images are input into the feature extraction module through three branches, then the respective feature operators are obtained via their independent CNN feature extractors. In the third stage, the probabilistic prediction module uses three dynamic probabilistic parameters to estimate the prediction of each CNN branch separately, and then combined the three CNN votes for the final decision. The feature expression ability of the key feature was improved and the network was enabled to focus on the recognizable regions in the image. Evaluation results from a large dataset of healthy patients showed that the proposed IAPP-CNN achieved the highest accuracy of 99.2% for the chromosome classification task, surpassing the performance of a competitive baseline created by state-of-the-art methods.
Karyotyping is a vital cytogenetics technique widely applied in prenatal diagnosis and genetic screening. Heavily dependent on the experience of the cytogeneticist and easily affected by the attention, karyotype analysis is a time-consuming and error-prone task, and incorrect karyotypes may result in misdiagnosis conclusions. This paper proposes an effective identification framework for incorrect karyotypes based on deep learning technology. Firstly, a chromosome classifier is trained and utilized to classify chromosome instances in karyotypes performed manually by cytogeneticists. Afterward, when the categories of chromosome instances classified by the classifier are not identical to those categories classified by cytogeneticists, the proposed framework identifies these corresponding karyotypes as unreliable. Finally, the expert team review these unreliable karyotypes and confirmed their correctness. Extensive experiments show that the proposed framework achieves 100% recall and 88.89% F1 score on incorrect karyotypes, which demonstrates the advancement and promising effectiveness of the proposed framework to address the issue of incorrect karyotypes.
Full-text available
In the field of cytogenetics, chromosome image analysis or karyotyping from metaphase images plays an imperative role in the diagnosis, prognosis and treatment assessment of different genetic disorders and cancers. This paper is a comprehensive review on different traditional and deep-based techniques, which are utilized in the design of automated karyotyping systems (AKSs). By this review, a detailed methodology is suggested for the design of end-to-end automated karyotyping system (EEAKS) which portrays a sequential multi stage approach. Methods related to all the stages in EEAKS are systematically surveyed by exploring the state of the art literature. Datasets and performance measures incorporated in the past studies are explored. Even though numerous methods were proposed throughout the past three decades, a completely automated framework has not yet been acknowledged. Inferences from this study show that, while various traditional image processing strategies are utilized for pre-processing and segmentation, machine learning techniques are used only for the classification purpose. In conventional classifiers, artificial neural networks are generally utilized even when the peak performance is given by support vector machines. However, owing to the recent prodigious breakthrough in computer vision, deep neural networks are progressively utilized for developing automated systems. It is seen that deep neural networks are not yet explored in the realm of pre-processing stage of EEAKS. However, limited number of methods based on convolutional neural networks (CNN) are utilized in all other stages. This review recommends a hybrid CNN for the design of EEAKS, in which all the stages can be automated by sub CNNs. Methodology for generating sufficient datasets is also discussed here which is, indeed, required for further research in this area. This paper concludes with future research directions for the development of a fully automated end-to-end karyotyping system.
Conference Paper
Full-text available
Karyotyping is the process of pairing and ordering 23 pairs of human chromosomes from cell images on the basis of size, centromere position, and banding pattern. Karyotyping during metaphase is often used by clinical cytogeneticists to analyze human chromosomes for diagnostic purposes. It requires experience, domain expertise and considerable manual effort to efficiently perform karyotyping and diagnosis of various disorders. Therefore, automation or even partial automation is highly desirable to assist technicians and reduce the cognitive load necessary for karyotyping. With these motivations, in this paper, we attempt to develop methods for chromosome classification by borrowing the latest ideas from deep learning. More specifically, we perform straightening on chromosomes and feed them into Siamese Networks to push the embeddings of samples coming from similar labels closer. Further, we propose to perform balanced sampling from the pairwise dataset while selecting dissimilar training pairs for Siamese Networks, and an MLP based prediction on top of the embeddings obtained from the trained Siamese Networks. We perform our experiments on a real world dataset of healthy patients collected from a hospital and exhaustively compare the effect of different straightening techniques, by applying them to chromosome images prior to classification. Results demonstrate that the proposed methods speed up both training and prediction by 83 and 3 folds, respectively; while surpassing the performance of a very competitive baseline created utilizing deep convolutional neural networks.
Technical Report
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Automated high throughput scanning microscopy is a fast developing screening technology used in cytogenetic laboratories for the diagnosis of leukemia or other genetic diseases. However, one of the major challenges of using this new technology is how to efficiently detect the analyzable metaphase chromosomes during the scanning process. The purpose of this investigation is to develop a computer aided detection (CAD) scheme based on deep learning technology, which can identify the metaphase chromosomes with high accuracy. The CAD scheme includes an eight layer neural network. The first six layers compose of an automatic feature extraction module, which has an architecture of three convolution-max-pooling layer pairs. The 1st, 2nd and 3rd pair contains 30, 20, 20 feature maps, respectively. The seventh and eighth layers compose of a multiple layer perception (MLP) based classifier, which is used to identify the analyzable metaphase chromosomes. The performance of new CAD scheme was assessed by receiver operation characteristic (ROC) method. A number of 150 regions of interest (ROIs) were selected to test the performance of our new CAD scheme. Each ROI contains either interphase cell or metaphase chromosomes. The results indicate that new scheme is able to achieve an area under the ROC curve (AUC) of 0.886±0.043. This investigation demonstrates that applying a deep learning technique may enable to significantly improve the accuracy of the metaphase chromosome detection using a scanning microscopic imaging technology in the future. © (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets. © 2014 Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov.
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
Conference Paper
This paper presents a novel approach to human chromosome classification. Human cell contains 22 pairs of autosomes and a pair of sex chromosomes. In this research, 22 types of autosomes represent 22 classes to be distinguished. New method of classification is based on the special organized committee of 462 simple perceptrons, called Competitive Neural Network Teams (CNNTs). Each perceptron is trained to differentiate two classes (i.e. two types of chromosome), hence there are 22 × 21 learning machines. Moreover, dummy perceptrons are set to zero for the chromosomes from the same class. The final outcome of the testing data is a 22×22 decision matrix, containing outcomes of each machine. With the special interpretation of these decisions, higher correct classification rate is achieved, reaching over 95%. The method can be further improved when testing is performed on a cell-by-cell basis by using CNNT complemented by Nearest Neighbor technique. The classification is applied to the Copenhagen chromosome data set and Sarajevo chromosome data set.
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions. The method is straightforward to implement and is based an adaptive estimates of lower-order moments of the gradients. The method is computationally efficient, has little memory requirements and is well suited for problems that are large in terms of data and/or parameters. The method is also ap- propriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The method exhibits invariance to diagonal rescaling of the gradients by adapting to the geometry of the objective function. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. We demonstrate that Adam works well in practice when experimentally compared to other stochastic optimization methods.