Abstract—The process required to track cellular structures is
a key task in the study of cell migration. This allows the
accurate estimation of motility indicators that help in the
understanding of mechanisms behind various biological
processes. This paper reports a particle-based fully automatic
tracking framework that is able to quantify the motility of
living cells in time-lapse images. Contrary to the standard
tracking methods based on predefined motion models, in this
paper we reformulate the tracking mechanism as a data driven
optimization process to remove its reliance on a priory motion
models. The proposed method has been evaluated on 2D and 3D
florescence in-vivo image sequences that describe the
development of the quail embryo.
HE study of cell migration is of a great significance in
various fields of research including embryonic
development, drug discovery and wound healing -. In
the quantification of the cell migration, the accurate tracking
of cellular structures is a key task that determines the
location and the association rules between the cells
throughout the image sequence. With the advent of modern
microscopy imaging modalities the volume of image data has
vastly increased and as a consequence the manual tracking of
individual cell has become tedious and even impossible
when dealing with large dense cellular data sets. Also, the
variety of cellular imaging techniques, different cell types,
changes in cell morphology over time and random motion of
the cells raise significant challenges that have to be
overcome by the automatic cellular tracking algorithms. As a
result, the development of a unique solution that works well
on all type of cells and imaging condition is impractical.
Thus, the study of cell migration using computer vision
based techniques entails a highly collaborative and multi-
Manuscript received April 15, 2011. This work was supported through
the National Biophotonics and Imaging Platform, Ireland, and funded by
the Irish Government's Programme for Research in Third Level Institutions,
Cycle 4, Ireland’s EU Structural Funds Programmes 2007-2013.
Microscopy data was obtained through support of the NIH (R01HL087136,
R01HL068855 and R01 HL068855).
M. J. Hossain is with the Centre for Image Processing and Analysis,
Dublin City University, Glasnevin, Dublin 9, Ireland (phone: +353-1-700-
7637; fax: +353-1-700-5508; e-mail: firstname.lastname@example.org).
P. F. Whelan is with the Department of Electronic Engineering, Dublin
City University, Glasnevin, Dublin 9, Ireland (e-mail: email@example.com)
A. Czirok is with the Department of Anatomy and Cell Biology, Kansas
University Medical Centre, KS 66160, USA (e-mail: firstname.lastname@example.org).
O. Ghita is with the Centre for Image Processing and Analysis, Dublin
City University, Glasnevin, Dublin 9, Ireland (e-mail: email@example.com).
disciplinary research . To this end engineering efforts
concentrate on the development of tracking solutions, while
molecular scientists focus on the investigation of the
biological implications associated with motility patterns.
The major challenges associated with automated cell
tracking are related to several factors such as the noise
inserted in the imaging process, random motion patterns,
changes in the appearance of cells over time, deformation,
cellular interaction, photo-bleaching, etc. Various methods
have been proposed to address the above challenges and they
can be broadly classified into two categories ,,. In
the first category cells are detected in each frame and the cell
to cell correspondence is carried out for each two adjacent
frames in the sequence ,. Methods included in the
second category detect cells in the first frame, identify model
parameters and perform tracking by evolving the model in
subsequent frames. The effectiveness of methods in the first
category is largely dependent on the success of segmentation
which is problematic when the number of cells is large and
they are not well separated. In this regard, to avoid the
segmentation of cells in each frame in the process of tracking
in-vivo cellular data, most of the research efforts have been
devoted to the development of model based tracking
techniques where the robust identification of the motion
patterns represents the major challenge.
Model based techniques based on Kalman and particle
based filters estimate the system model parameters to guide
the tracking process for each cell in the image sequence
,. However, due to the self propelled motility of cells,
their motion characteristics may not be known in advance to
obtain a suitable system model that incorporates all modes of
motion. To compensate for this problem, the estimation of
the motion patterns has been carried out using multiple
statistical models . Although these algorithms include
multiple models corresponding to different cellular
movement types, they entail a considerable level of
supervision that is required to identify the motility patterns.
Some methods incorporate the current observation data
along with the motion model to obtain better tracking
performance ,. In this regard, several hybrid
methods have been reported where a transition model was
combined with an independent mean shift algorithm
,. These methods lead to improvement in tracking
accuracy, but they cannot accommodate cell displacements
larger than the kernel size.
An Active Particle-based Tracking Framework for 2D and 3D Time-
lapse Microscopy Images
M. Julius Hossain, Member, IEEE, Paul F. Whelan, Senior Member, IEEE, Andras Czirok and Ovidiu
The analysis of 3D motion is becoming increasingly
important in the studies that address the molecular and
cellular domain, but so far only a limited number of papers
have been reported on 3D cell migration -. Most of
the existing methods that address the 3D cell motion rely on
the use of projected 2D data. Thus, the motility information
obtained through this 3D to 2D projection process cannot
precisely describe the 3D cell migration. For instance in 3D,
when one cell passes above or beneath another, the motion is
perceived as penetration or merging in the projected 2D data.
As a result, in this situation the cell tracking using projected
data will return an inaccurate estimation of the real 3D cell
In this paper we proposed a novel tracking framework that
is able to extract motility information on both 2D and 3D in-
vivo cellular data. The proposed method does not require
prior knowledge about the motion model or assumptions in
regard to the image noise. Here the target is represented by a
set of active particles and tracking is performed by evolving
these particles into subsequent frames. The particle tracking
process in the proposed method is data-driven where the
level of similarity between the target candidate and the target
model is evaluated in an adaptive framework to minimize the
occurrence of incorrect tracking decisions. To track cells in
3D within a series of image stacks, in addition to in-plane
searches, the algorithm evaluates the data in adjacent optical
planes to infer the cell motion along the z-axis.
II. OVERVIEW OF THE PROPOSED FRAMEWORK
The first step of the proposed tracking framework involves
cell detection which is a data dependent process. In the
proposed framework the cellular structures are automatically
detected (full details about this process is provided in
Section IV) where a set of particles is employed to sample
the area around the centroid of each of the detected cells.
Then, particles are propagated in the next frame based on the
displacement that is estimated for each cell using the Newton
Raphson iterative minimization with a pyramidal image
decomposition. The minimization procedure is carried out
starting from the image with the lowest resolution towards
the image with the highest resolution in the pyramidal image
decomposition. Once the displacement of a particle is
determined, the likelihood is estimated using the intensity
distribution calculated within the window W in two
successive frames. A penalty measure is calculated for each
particle in agreement to the distance transformed image to
prevent incorrect cellular associations when multiple cells
are spatially close. A cell is localized in next frame based on
the likelihood and penalty score associated with all the
particles that have been employed to represent the cell.
When the algorithm is applied to 3D data, in order to
address the cell motion along the z-axis two adjacent optical
planes immediately above and below the plane under
evaluation are also analyzed. The plane that has the highest
likelihood is selected as the target location for next frame. It
is useful to note here that in 3D stack images the spatial
resolution along the z-axis is much lower than that along xy-
plane. The datasets used in our experiments have only 7 to 9
planes in the z-axis, where an individual nucleus is fully
visible at most in two adjacent planes.
III. THE PROPOSED METHOD
To implement the cellular tracking, the proposed method
propagates the particles associated with the cell in the frame
under investigation to the next frame based on the image
information without any knowledge about a priory motion
model. Since it is not possible to determine the location of a
particle in the subsequent frame based only on the
information of a single pixel, a small window centered on the
pixels of interest is used. In our approach, the objective is to
determine the displacement d of a feature calculated within
the window W from frame
F to the next frame
minimizing the error,
h s I
e s d
where, h is a weighting function,
intensity value of
F at s and T defines the time index. The
estimation of d can be formulated as a standard optimization
problem. Several techniques have been proposed in the
literature to minimize e in (1) and the most common include
the steepest descent, conjugate gradient and Newton-
Raphson minimization. When d is small, the Newton-
Raphson minimization has been proved to be the most
efficient approach , and it has been also employed in
our implementation. In relation to (1) we define h as a
Gaussian kernel to emphasize the central area of the window.
where r is the distance from the center of the window. In our
algorithm, the size of the window is odd i.e.
(21) (2 1)
where w is the half window width.
The displacement d is solved using the following equation,
( ) ( )T
g s g s ds
is a 2 2
which is computed by estimating gradients and their second
order moments, and e is estimated using (1). Here, d is
assumed to be constant within the window and the optimum
solution is obtained by applying the Newton-Raphson
minimization. In our implementation to accommodate large
motion, a multi-resolution image pyramid is created where
the number of pyramid levels n is dynamically determined
based on w and the maximum instantaneous cell migration.
Let s be the location associated with a particle. The
Ls in a pyramid level L is computed
tracked from the coarsest to the finest resolution, i.e. the
result at each resolution provides the initial solution to be
analyzed at the subsequent resolution. Thus, the proposed
framework is able to deal with large displacements at the top
of the pyramid, while maintaining sub-pixel accuracy at the
bottom. Using this approach, the particle tracking process is
guided based on image properties rather than using a-priori
defined motion models.
Once a particle is tracked, i.e. the value of d is determined,
the similarity between two corresponding windows is
estimated by using a intensity histogram due to its
independence to scaling and rotation . In this
process, the weighting function h is used to emphasize more
the image data close to the central region of the window. We
( ) 1,2,..
as the bin index of the intensity
signal at s where m denotes the number of bins. The intensity
distribution at s is determined as follows,
. Within this framework, features are
[ ( )
where, [.] is the Kronecker delta function, K is a
normalization constant ensuring
, W is the
size of the feature window. The distribution calculated with
equation (5) assigns a probability for each of the m bins. The
likelihood between distribution
p is computed as follows,
sp and its corresponding
D p p
L p p
is the Hellinger distance calculated using
equation (7). As, the likelihood of a distribution
always calculated with respect to its corresponding
p, it is simply denoted by
D p p
s s d
D p ppp
Due to the unconstrained motion, cells may come close to
each other or interact. This situation leads to incorrect
tracking decisions when multiple cells in a frame tend to
match with a particular cell in the next frame. To deal with
this problem, we calculate a penalty force
E x associated
with each particle
other targets being tracked. Let
ix based on its distance with respect to
is be the location of a
ix that is associated with the
ro . Now, the
penalty force associate with
ix is defined as,
tMdis s s
ts represents the centroid of a target
dis s s represents the
is the total number of objects being tracked. Thus, to
determine the penalty measure associated with a particle, (8)
evaluates its distance with respect to the other objects being
tracked. It assigns a higher penalty force to the particle
having objects in closer neighborhood. To reduce the
computational time required to determine the pair-wise
distances in (8), a two-pass algorithms based on the Chamfer
5/7 distance has been used . It would be useful to note
that the penalty measure in (8) is dependent on the local
neighborhood around each particle, hence it models the
cellular interactions more accurately than the object-wise
repulsive force assignment based on Markov random field
. To achieve a better localization, the state of an object
ro is determined based on the likelihood and penalty force
of the particles associated with it.
2 L distance between
ts , M
number of particles that have been used to represent
ix is associated with target
N is the
r is a normalization factor that is computed as follows,
Thus, the likelihood function is computed based on the
similarity between two corresponding regions where the
correspondence is represented by the vector d which is
determined using a data-driven optimization process. It
would be useful to note here that equation (9) is used to track
cell migration in 2D.
To track cell nuclei in 3D within a series of image stacks,
the proposed method additionally searches two adjacent
optical planes immediately above and below the plane of
interest and determines the likelihood. Let z be the index of
current optical plane. So, to track a target
proposed method analyzes three frames
be the matching confidence of object
which is computed as follows:
zF represents the frame in plane z at time T. Let
, r k
ro in plane k,
E are the likelihood and the penalty measure
that are calculated when
is analyzed. The plane which
provides the highest matching confidence is selected as the
new z-coordinate of
IV. EXPERIMENTS AND RESULTS
The proposed tracking framework has been evaluated on
both 2D and 3D time-lapse cellular image sequences. The
experimental tests have been performed on a large number of
deconvolved (Autoquant X, Media Cybernetics, Bethesda,
MD, USA) time-lapse florescent microscopy image
sequences that record the in-vivo development of transgenic
quail embryos. In these embryos the nuclei of the endothelial
(vascular) cells are labeled with a GFP (green fluorescent
protein) variant . Images are captured in multiple
adjacent fields and optical planes using a wide-field
epifluorescence microscope  with an xy resolution of 1.3
μm/pixel and the time interval between two frames is ranging
between 4 to 8 minutes. The number of optical (z) planes
varies from 7 to 9 and the distance between two consecutive
planes is 42 μm. Fluorescence intensities are projected onto
a single plane where color-coding indicates their origin in the
various optical sections. The resulting color coded image
sequence is used for the cell tracking procedure. Fig. 1(a)
shows the DIC image of a quail embryo where a region of
interest (ROI) for cellular tracking is marked with a red
rectangle. Fig. 1(b) illustrates the corresponding florescent
image of the segment marked with the rectangle in Fig. 1(a).
Fig. 1(c) shows an image stack of 5 optical planes where an
individual nucleus is clearly visible in two consecutive
planes and it is not prominent in other planes. Thus, to track
a cell migration along z-axis, the proposed method probes
the current plane and the two immediately adjacent planes.
Cells are automatically detected using a multi-stage
segmentation approach. In the initial stage, an image
sharpening operation is performed on the intensity data to
highlight the inner region of the nuclei. Next, a given number
of cells (user defined) are detected progressively based on
the strength of the peaks in the intensity sharpened data and a
minimum distance among the cells is enforced during this
process to achieve a better distribution of the detected cells
within the image data domain. To enforce a minimum
distance between cells, once a cell is detected the
surrounding area within the given distance is marked to
disregard the potential peak on that region. In proposed
method, the cell detection process evaluates the relative
intensity difference between the foreground (cellular
structures) and background information, thus it can be used
in different imaging condition without resorting to arbitrary
thresholds. To detect cell nuclei in a 3D image stack, all the z
planes are taken into consideration while selecting the given
number cells. Thus, the number of detected cells in a
particular plane might be quite different from that detected in
Fig. 1. Sample microscopy images (a) DIC image of a quail embryo. (b) The
fluorescence image of the segment marked by the rectangle in Fig. 1(a). (c)
Five optical planes. (This diagram is best viewed in color)
To extract the motility information in 2D, the proposed
method is applied on several quail sequences where the
number of frames in a sequence varies between 87 and 160.
We used a 9×9 feature window where the maximum
expected cell displacement between two consecutive frames
is 15 pixels. Fig. 2 shows the visual tracking results obtained
when the algorithm is applied to in-vivo quail embryo data.
Fig. 2(a) and 2(b) show the result where 100 and 200 cells,
respectively, have been detected in the first image where Fig.
2(b) includes 100 new cells in addition to the cells that are
present in Fig. 2(a). Lineages of the tracked cells in Frames
80 and 87 are shown in Figs. 2(c) and 2(d), respectively. Fig.
2(e) shows the color coded trajectories of the tracked cells in
Frame 87 where red and blue denote initial position and final
position, respectively. These trajectories indicate the
formation of a well-defined crescent-shaped field from which
future endocardial cells are recruited. Their dynamics can be
validated using the corresponding DIC images shown in Fig.
2(f) where the arrow indicates the direction of the cell
migration during the heart development process.
Fig. 2. Visual tracking results for 2D image sequences. (a) and (b)
Detection results for 100 and 200 cells, respectively. (c) and (d)
Trajectories of the tracked nuclei at frames 80 and 87, respectively. (e)
Color coded trajectories during the early stage of heart development. (f)
Corresponding DIC image. (This diagram is best viewed in color)
Fig. 3 illustrates the tracking results in 3D for a series of
image stacks. Fig. 3(a) shows the detection results for 200
cells where a different color is used to mark the cells in each
plane. For clarity purposes, the 3D trajectories are displayed
in two forms. In Fig. 3(b) the color coded trajectories are
overlaid on the maximum intensity projected image. In this
diagram, the movement along the z axis is indicated by the
change in the color of the trajectory with respect to the color
associated with the plane it moves into. In Figs. 3(c) and 3(d)
the direction of the nuclei movements within the 3D volume
is illustrated from two different directions.
The tracking process records the locations of the cells in
each frame that are used to calculate different motility
statistics including instantaneous velocity, distance-traveled,
cellular-cycle, motion directionality, etc. These statistics are
useful in the analysis of the key processes in different stages
of the embryonic development. Fig. 4 depicts the normalized
instantaneous velocities of the cells in different stages of the
heart development. This diagram shows a reduction in the
growth rate of the heart development at the later stages of the
Fig. 3. Tracking results in 3D. (a) Detection results on five optical planes.
(b) Trajectories of tracked cells in maximum intensity projected image. (c)
and (d) Direction of cell motion in a 3D volume viewed from two different
directions. (This diagram is best viewed in color)
As discussed earlier, the velocity of the cells extracted by
the tracking algorithm is one of the key parameters employed
to measure the growth rate associated with different organ
formation processes. Thus, we evaluated the performance of
the developed tracking method in regard to how well it
computes the instantaneous cell velocity by comparing its
performance with respect to the manually annotated data.
The average deviation between the cell velocities that are
determined using the manual annotated data and the
proposed method is less than 8%. Fig. 5 shows the velocity Download full-text
normalized cumulative frequencies plotted against the
normalized instantaneous (frame to frame) velocities. This
graph illustrates a very good agreement between the manual
and automated tracking results which demonstrates the
versatility and robustness of our cellular tracking algorithm
when applied to challenging in-vivo data. The level of
accuracy attained by our method allows a detailed analysis of
biological implications associated with cellular migration.
Fig. 4. Average normalized instantaneous cell velocities during different
stages of the heart development.
Fig. 5. Normalized instantaneous cell velocities versus normalized
cumulative frequencies showing the amount of deviation between the cell
velocities determined using the manually annotated data and automated in-
The proposed framework introduces a data-driven cellular
tracking algorithm that does not require prior knowledge
about the motion patterns or assumptions in regard to the
image noise. The particles are independent and are able to
adapt to local conditions, and as a result, the proposed
framework is suitable to track the random motion of the cell
nuclei. The performance attained by the proposed method
when it has been applied to both 2D and 3D cellular data
demonstrates robust tracking results that allow a detailed
quantitative analysis of cell migration that helps in
understanding the mechanism behind key biological
We would like to express our gratitude to Dr. Rusty
Lansford (California Institute of Technology) for the
transgenic quails, and Computational Imaging Group of Drs.
Brenda Rongish and Charlie Little (University of Kansas
Medical Center) for the transgenic image sequences.
 O. Al-Kofahi, R. J. Radke, S. K. Goderie, Q. Shen, S. Temple and B.
Roysam “Automated cell lineage construction: a rapid method to
analyze clonal development established with Murine neural progenitor
cells,” Cell Cycle, vol. 5, no. 3, pp. 327-335, Feb. 2006.
 A. Mosig, S. Jager, C. Wang, S. Nath, I. Ersoy, K. Palaniappan and S.
Chen, “Tracking cells in life cell imaging videos using topological
alignments,” Algorithms for Molecular Biology., vol. 4:10, Jul. 2009.
 F. Li, X. Zhou, J. Ma, and S. T. C. Wong “Multiple nuclei tracking
using integer programming for quantitative cancer cell cycle
analysis,” IEEE Trans. Medical Imaging, vol. 29, pp. 96-105, Jan.
 A. R. Horwitz, N. Watson, and J.T. Parsons, “Breaking barriers
through collaboration: the example of the cell migration consortium,”
Genome Biology, vol. 3, no.11, pp. 2011.1-2011.4, 2002.
 K. Thirusittampalam, M.J. Hossain, O. Ghita, P.F. Whelan, “A novel
framework for tracking in-vitro cells in time-lapse phase contrast
data, in Proc. British Machine Vision Conference, UK, 2010, pp.
 M. Dewan, M. Ahmad, and M Swamy, “Tracking biological cells in
time-lapse microscopy: an adaptive technique combining motion and
topological features”, IEEE Trans. Biomedical Engineering, vo 58,
no. 6, pp. 1637-1647, Jun. 2011.
 M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial
on particle filters for online nonlinear/non-Gaussian Bayesian
tracking,” IEEE Trans. Signal Processing, vol. 50, pp. 174–188, Feb.
 Y Rui and Y. Chen, “Better proposal distributions: object tracking
using unscented particle filter,” in Proc. 2001 IEEE Int. Conf.
Computer Vision and Pattern Recognition, vol. 2, pp. 786-793
 K. Li, E.D. Miller, M. Chen, T. Kanade, L.E. Weiss and P.G.
Campbell, “Cell population tracking and lineage construction with
spatiotemporal context,” Medical Image Analysis, vol. 12, pp. 546-
 K Nummiaroa, E. Koller-Meierb, L. Goola, “An adaptive color-based
particle filter”, Image and Vision Computing, vol. 21, pp. 99–110,
 E. Maggio and A. Cavallaro, ”Hybrid particle filter and mean shift
tracker with adaptive transition model”, in Proc. IEEE Int. Conf.
Acoustics, Speech, and Signal Processing, 2005, vol.2 , pp. 221-224.
 I. Smal, K. Draegestein, N. Galjart, W. Niessen, and E. Meijering,
“Particle filtering for multiple object tracking in dynamic fluorescence
microscopy images: application to microtubule growth analysis,”
IEEE Trans. Medical Imaging, vol. 27, no. 6, pp. 789-804, Jun. 2008
 G. Rabut and J. Ellenberg, “Automatic real-time three-dimensional
cell tracking by fluorescence microscopy,” J. Microscopy, vol. 216,
no. 2, pp. 131–137, Nov. 2004
 J. Shi and C. Tomasi, “Good features to track”, in Proc. IEEE Int.
Conf. Computer Vision and Pattern Recognition, 1999, pp. 593–600.
 B.D. Lucas and T. Kanade. “An iterative image registration technique
with an application to stereo vision,” in Proc. Int. Joint Conf.
Artificial Intelligence, 1981, pp. 121-130
 G. Hager, M. Dewan and C. Stewart, “Multiple kernel tracking with
SSD,” in Proc. IEEE Conference on Computer Vision and Pattern
Recognition, 2004, vol. 1, pp. 790–797
 M. J. Hossain, M. Dewan, A. Kiok and O. Chae, “A Linear Time
Algorithm of Computing Hausdorff Distance for Content Based
Image Analysis,” Circuits, Systems, and Signal Processing, pp. 1-11.
doi:10.1007/s00034-011-9284-y, March 2011.
 Z. Khan, T. Balch, and F. Dellaert, “MCMC-based particle filtering
for tracking a variable number of interacting targets,” IEEE Trans.
Pattern Analysis and Machine Intelligence, vol. 27, pp. 1805-1819,
 Y. Sato, G. Poynter, D. Huss, M. B. Filla, A. Czirok, B. J. Rongish, C.
D. Little, S. E. Fraser, and R. Lansford, “Dynamic analysis of vascular
morphogenesis using transgenic quail embryos,” PLOS ONE, vol. 5,
no. 9, Sep. 2010.
 A Czirok, P.A. Rupp, B.J. Rongish and C.D. Little, “Multi-field 3D
scanning light microscopy of early embryogenesis.” J. Microscopy.
vol. 206, pp. 209-17, Jun. 2002.