Determination of Mitotic Delays in 3D Fluorescence Microscopy Images of Human Cells Using an Error-Correcting Finite State Machine

Conference Paper (PDF Available)inProceedings / IEEE International Symposium on Biomedical Imaging: from nano to macro. IEEE International Symposium on Biomedical Imaging · January 2007with313 Reads
DOI: 10.1007/978-3-540-71091-2_49 · Source: DBLP
Conference: Bildverarbeitung für die Medizin 2007, Algorithmen, Systeme, Anwendungen, Proceedings des Workshops vom 25.-27. März 2007 in München
Abstract
In high-throughput cell phenotype screens large amounts of image data are acquired. The evaluation of these microscopy images re- quires automated image analysis methods. Here we introduce a compu- tational scheme to process 3D multi-cell image sequences as they are produced in large-scale RNAi experiments. We describe an approach to automatically segment, track, and classify cell nuclei into seven difierent mitotic phases. In particular, we present an algorithm based on a flnite state machine to check the consistency of the resulting sequence of mi- totic phases and to correct classiflcation errors. Our approach enables automated determination of the duration of the single phases and thus the identiflcation of cell cultures with delayed mitotic progression.

Figures

Determination of Mitotic Delays in 3D
Fluorescence Microscopy Images of Human Cells
using an Error-Correcting Finite State Machine
Nathalie Harder
1
, Felipe Mora-Berm´udez
2
, William J. Godinez
1
,
Jan Ellenberg
2
, Roland Eils
1
and Karl Rohr
1
1
University of Heidelberg, IPMB, and DKFZ Heidelberg, Dept. Bioinformatics and
Functional Genomics, Im Neuenheimer Feld 364, D-69120 Heidelberg,
2
Europ ean Molecular Biology Laboratory (EMBL), Gene Expression and Cell
Biology/Biophysics Programmes, Meyerhofstrasse 1, D-69117 Heidelberg
Email: n.harder@dkfz-heidelberg.de
Abstract. In high-throughput cell phenotype screens large amounts of
image data are acquired. The evaluation of these microscopy images re-
quires automated image analysis methods. Here we introduce a compu-
tational scheme to process 3D multi-cell image sequences as they are
pro duced in large-scale RNAi experiments. We describe an approach to
automatically segment, track, and classify cell nuclei into seven different
mitotic phases. In particular, we present an algorithm based on a finite
state machine to check the consistency of the resulting sequence of mi-
totic phases and to correct classification errors. Our approach enables
automated determination of the duration of the single phases and thus
the identification of cell cultures with delayed mitotic progression.
1 Introduction
RNA interference (RNAi) is an effective tool for identifying the biological func-
tion of genes. With this method genes are systematically silenced and the re-
sulting morphological changes are analyzed. However, such large-scale screens
provide large amounts of data which require tools for automated image analysis.
Our work is carried out within the project MitoCheck, which has the goal
to explore the processes of cell division (mitosis) in human cells at a molecular
level. In this project RNAi secondary screens are performed and fluorescence
microscopy image sequences of the treated cell cultures are acquired to study
the effects of the silenced genes on mitosis. This contribution is concerned with
the automated evaluation of an assay for studying delays in mitotic phases, which
are caused by gene silencing. Thus, the duration of the different phases of cell
division has to be measured for the treated cells and compared with the normal
cells from control experiments. Therefore, cells have to be observed throughout
their life cycle and for each time point the respective phase has to b e determined.
Automated analysis of high-throughput cell phenotype screens plays an in-
creasingly important role. Approaches to analyse single-frame multi-cell 2D im-
ages from large-scale experiments have been described, e.g., in [1]. Recently,
243
Fig. 1. Image analysis workflow: (1) Maximum intensity projection of multi-cell 3D
images, (2) Segmentation and tracking in 2D, (3) Extraction of 3D ROIs that include
single cells,(4) Selection of most informative slices, (5) Feature extraction, (6) Classifi-
cation, (7) Consistency check, error correction, and determination of phase durations
work has been done on automatically processing multi-cell 2D image sequences
for cell cycle analysis. With these approaches, cells are segmented, tracked, and
classified given phase-contrast [2] or fluorescence [3] microscopy images. In [2, 3]
cells or cell nuclei are classified into a maximum of four phases. In [4] 3D image
sequences are processed and cell nuclei are classified into seven cell cycle phases
but no consistency check of the resulting phase sequences has been performed.
Our approach allows to analyze 3D multi-cell image sequences from a confocal
fluorescence microscope. We classify cell nuclei into seven mitotic phases. To
this end, we have developed a workflow that comprises segmentation, tracking
of splitting nuclei, extraction of static and dynamic features, and classification.
In particular, we present a scheme to check the consistency of the resulting
sequences of mitotic phases of cells based on biological constraints. This scheme
is based on a finite state machine which enables to automatically resolve errors
using the confusion probabilities of the classifier. As a result, we can determine
the lengths of the seven mitotic phases of cell nuclei automatically.
2 Methods
Image analysis workflow To analyze the mitotic phases high-resolution con-
focal fluorescence microscopy 3D images of the DNA are acquired which consist
of three confocal planes (slices). Because of technical reasons in the image ac-
quisition process the number of slices is restricted to three. During mitosis the
cell changes its shape (gets more rounded) and therefore in different phases the
DNA is visible in different slices. Using multi-cell images that contain cells in
different mitotic phases, it is impossible to define one slice per time step that
well represents the DNA of all cells. Therefore, we have developed the workflow
shown in Fig. 1. First, we apply a maximum intensity projection (MIP) for each
time step, resulting in 2D images (Fig. 2 (left)). Based on these MIP images we
244
Fig. 2. (left) Original 3D image (Maximum intensity projection), (right) 1-4: Example
for the tracking of a mitotic nucleus in four consecutive time-steps, 4a: Result without
mitosis detection, 4b: Result after mitosis detection and track merging
Fig. 3. Example images for the seven different mitotic phases (from left to right): inter-,
pro-, prometa-, meta-, ana- 1, ana- 2, and telophase
perform segmentation and tracking to determine the correspondences in subse-
quent frames. We now go back to the 3D images and define 3D ROIs for each cell
based on the segmentation and tracking result. For each 3D ROI we choose the
most informative slice and extract static and dynamic features. Then we apply
a classifier which results in a sequence of mitotic phases for each cell trajectory.
Finally, the resulting phase sequences are parsed with a finite state machine to
check their consistency, resolve inconsistencies, and determine the phase lengths.
Segmentation and tracking of mitotic cell nuclei For segmentation we
apply region-adaptive thresholding which proved to be fast and robust in our
application. This scheme computes local intensity thresholds in overlapping im-
age regions using histogram-based threshold selection. To analyze the mitotic
behavior of single cells, a tracking scheme is required that determines the tem-
poral connections for splitting objects. We have developed the following two-step
scheme: First, initial, non-splitting trajectories are established, and second, mi-
totic events are detected and the related trajectories are merged resulting in
tree-structured trajectories (Fig. 2 (right)). For more details see [4].
Feature extraction and classification To compute image features we se-
lect for each cell nucleus its individual most informative slice. Our experiments
showed that the maximum total intensity performs very well as selection crite-
rion. Within the most informative slice we compute static and dynamic image
features. The static features comprise object- and edge-related features, texture
features, grey scale invariants, and Zernike moments. As dynamic features we
compute the difference of object size, intensity mean and standard deviation,
and circularity for each nucleus to its predecessor and successor. We apply a
support vector machine (SVM) classifier with a radial basis function (RBF) ker-
245
Fig. 4. Finite state machine to check the consistency of the computed phase sequences
nel to classify the nuclei into the classes interphase, prophase, prometaphase,
metaphase, anaphase 1, anaphase 2, and Telophase (Fig. 3).
Consistency check and error correction In order to determine the phase
lengths automatically it is necessary that the computed phase sequences are
consistent (which is not always the case due to classification errors). Therefore,
we have developed a finite state machine (FSM) that accepts only biologically
possible phase sequences. Each phase is represented by one state of the FSM. The
possible phase transitions are represented by the state relations. Fig. 4 shows a
sketch of the FSM; for clarity only the most important relations are displayed.
The FSM contains error states to resolve inconsistencies. An error state is quit
if the following phase is a valid phase. Then the most likely phase for the error
state is determined based on the confusion probabilities of the classifier that have
been established using cross-validation on the training set. In one run of the FSM
inconsistencies of length 1 are completely corrected. Errors of length larger than
1 can be successively resolved in multiple runs of the FSM. In addition, the FSM
determines the phase durations.
3 Results
Our experiments are based on four multi-cell 3D image sequences each consisting
of 124 time steps. 29 cell nuclei have been segmented and tracked over 124 time
steps resulting in 4225 single cell 3D image stacks (note that the cells proliferate).
Some few trajectories have been manually corrected since our tracking scheme
currently detects 80% of the o ccurring mitoses. We have performed a kind of two-
fold cross-validation on the 29 available trajectories where in each loop 18 tracks
have been used for training and 11 tracks for testing. In each cross-validation
loop first the confusion probabilities were determined using a five-fold cross-
validation on the training set. Then, the classifier was trained with the whole
training set and tested with the test set. We obtain an overall classification
246
Table 1. Sample numbers and classification accuracies for each class (22 tracks tested)
Inter Pro Prometa Meta Ana1 Ana2 Telo
No. of samples 2986 51 36 76 14 50 152
Class. accuracy 99.0% 84.3% 94.4% 79.0% 57.1% 86.0% 69.7%
accuracy of 96.6% (the sample numbers and accuracies per class for both cross-
validation loops are given in Tab. 1). The resulting phase sequences for the
22 classified trajectories were processed with the finite state machine using the
confusion matrices determined on the training sets to resolve consistency errors.
Finally, the corrected phase sequences have been compared with the manually
assigned correct phases (ground truth). It turned out that all inconsistencies of
length 1 have been resolved in the first run of the FSM. Applying the FSM a
second time on the corrected sequences resolved also all inconsistencies of length
2. Longer inconsistencies have partly been resolved. Note that this correction
step is essential for determining the phase lengths automatically.
4 Discussion
We have presented an approach for automated analysis of the duration of mitotic
phases in 3D confocal microscopy image sequences. Our approach segments and
tracks splitting nuclei and thus determines cell pedigrees. By using static and
dynamic features our scheme classifies the cells into seven mitotic phases. The
consistency of the computed phase sequences is checked and inconsistencies are
resolved using a priori knowledge. Finally, the phase durations are determined.
In future work, we plan to improve the performance of the tracking and clas-
sification schemes by including additional image features. We will also apply our
approach to a larger numb er of image sequences and evaluate its performance.
Acknowledgment
This work has been supported by the EU project MitoCheck.
References
1. Perlman ZE, Slack MD, Feng Y, et al. Multidimensional drug profiling by automated
microscopy. Science 2004;306:1194–1198.
2. Yang F, Mackey MA, Ianzini F, et al. Cell segmentation, tracking, and mitosis
detection using temporal context. LNCS 2005;3749:302–309.
3. Padfield DR, Rittscher J, Sebastian T, et al. Spatio-temporal cell cycle analysis
using 3D level set segmentation of unstained nuclei in line scan confocal fluorescence
images. Procs ISBI 2006; 1036–1039.
4. Harder N, Berm´udez F, Godinez WJ, et al. Automated analysis of the mitotic
phases of human cells in 3D fluorescence microscopy image sequences. LNCS
2006;4190:840–848.
    • "Several supervised methods have been proposed in this context. Cell nuclei were classified to mitotic phases using a support vector machine [11,12] and afterwards a finite state machine [13] or an HMM is used to correct for improbable transitions between the respective phases [14]. "
    [Show abstract] [Hide abstract] ABSTRACT: Gene perturbation experiments in combination with fluorescence time-lapse cell imaging are a powerful tool in reverse genetics. High content applications require tools for the automated processing of the large amounts of data. These tools include in general several image processing steps, the extraction of morphological descriptors, and the grouping of cells into phenotype classes according to their descriptors. This phenotyping can be applied in a supervised or an unsupervised manner. Unsupervised methods are suitable for the discovery of formerly unknown phenotypes, which are expected to occur in high-throughput RNAi time-lapse screens. We developed an unsupervised phenotyping approach based on Hidden Markov Models (HMMs) with multivariate Gaussian emissions for the detection of knockdown-specific phenotypes in RNAi time-lapse movies. The automated detection of abnormal cell morphologies allows us to assign a phenotypic fingerprint to each gene knockdown. By applying our method to the Mitocheck database, we show that a phenotypic fingerprint is indicative of a gene's function. Our fully unsupervised HMM-based phenotyping is able to automatically identify cell morphologies that are specific for a certain knockdown. Beyond the identification of genes whose knockdown affects cell morphology, phenotypic fingerprints can be used to find modules of functionally related genes.
    Full-text · Article · Oct 2013
  • [Show abstract] [Hide abstract] ABSTRACT: In high-throughput RNAi knockdown screens large amounts of image data are acquired. The evaluation of these microscopy images constitutes a bottleneck and motivates the devel- opment of automated image analysis methods. This contribution is concerned with the au- tomated evaluation of RNAi knockdown experiments for studying delays in mitotic phases. To this end, 3D multi-cell image sequences of living cell nuclei are acquired. Based on these images, the duration of the mitotic phases has to be measured for the treated cells and compared with the normal cells from control experiments. To automatically determine the lengths of the cell cycle phases, we have developed a workflow that comprises segmentation, tracking of splitting nuclei, extraction of static and dynamic features, classification, and phase length determination. For fast and accurate segmentation we use a region adaptive thresholding technique on the maximum intensity projected images (Fig. 1a,b). We perform tracking of the splitting cell nuclei using a two step approach. First, correspondences are determined by exploit- ing the smoothness of potential trajectories. Second, mitosis events are detected based on morphological properties and the corresponding trajectories are merged (Fig. 1c). Based on the tracking result we automatically select the most informative slice for each nucleus from the 3D image, which is then used for feature extraction. Besides static image features, we additionally include dynamic image features which represent temporal changes of the cell morphology between ancestrally related cells. A support vector machine classifier is used to classify the nuclei into the following seven cell cycle phases: Interphase, Prophase, Prometaphase, Metaphase, Anaphase1, Anaphase2, and Telophase (Fig. 2). Finally, we have
    Article · · BMC Bioinformatics
  • [Show abstract] [Hide abstract] ABSTRACT: Tracking and registration approaches have been developed for automatic analysis of multidimensional biomedical images. The tracking approach allows computing the trajectories of cells in fluorescence microscopy image sequences. The registration approach enables to geometrically align cell microscopy images by using elastic transformations.
    Chapter · Jan 2010 · BMC Bioinformatics
    K. RohrK. RohrW. J. GodinezW. J. GodinezN. HarderN. Harder+1more author...[...]
Show more