Page 1

Evaluation of Approaches for Tracking Virus

Particles in Fluorescence Microscopy Images

W. J. Godinez1, M. Lampe2, S. W¨ orz1, B. M¨ uller2, R. Eils1, K. Rohr1

1University of Heidelberg, BIOQUANT, IPMB, and DKFZ Heidelberg,

Dept. Bioinformatics and Functional Genomics, Biomedical Computer Vision Group,

Im Neuenheimer Feld 267, 69120 Heidelberg, Germany

2University of Heidelberg, Dept. of Virology,

Im Neuenheimer Feld 324, 69120 Heidelberg, Germany

wgodinez@ieee.org

Abstract. Tracking virus particles in fluorescence microscopy image se-

quences enables the characterization of the dynamical behavior of these

objects. Several approaches have been developed for the task of virus

tracking. However, few studies have quantitatively evaluated the per-

formance of the different approaches. Such a comparison is essential to

predict the performance of the approaches under realistic conditions. In

this paper, we present a quantitative evaluation of eight approaches for

tracking virus particles. We have investigated deterministic and prob-

abilistic approaches. The evaluation is based on nine real microscopy

image sequences of virus particles, for which ground truth was obtained

by manual tracking.

1 Introduction

Tracking single virus particles in fluorescence time-lapse microscopy images

yields quantitative information that describes their dynamical behavior. Such

information can be employed to characterize the influence of antiviral drugs.

To obtain statistically sound conclusions, a large number of particles must be

tracked. Therefore, automatic tracking approaches are required to efficiently

handle the large amount of image data.

Several approaches for virus tracking have been described. Typically, deter-

ministic approaches have been employed (e.g., [1, 2]). More recently, probabilis-

tic approaches (e.g., [3, 4]) have been introduced. However, few studies have

quantitatively compared the performance of virus tracking approaches. Such

a comparison is needed to predict the performance of the approaches for real

images. The most detailed comparison of tracking approaches for fluorescent

particles has been presented in [5]. There, the authors evaluate the perfor-

mance of four deterministic tracking approaches using synthetic images with

a focus on object localization. One main finding is that the performance of

the tracking approaches declines as the signal-to-noise ratio (SNR) decreases.

While the study is relatively detailed, it has three shortcomings. First, the per-

formance measure is based on the localization error, and the influence of errors

Page 2

208 Godinez et al.

in correspondence finding is ignored. Second, no real images have been used.

Third, only deterministic approaches are considered.

In this work, we present a quantitative performance evaluation of approaches

for tracking multiple virus particles in microscopy image sequences. In total, we

have evaluated eight tracking approaches. We have analyzed two deterministic

approaches as well as six probabilistic approaches. The deterministic approaches

are based on either the spot-enhancing filter [6] or 2D Gaussian fitting for particle

localization, and a global nearest neighbor approach for motion correspondence.

The probabilistic approaches are based on Kalman filters, mixture of particle

filters (MPF), and independent particle filters (IPF). The approaches have been

applied to synthetic image sequences displaying virus-like objects, as well as to 9

different real microscopy image sequences (each comprising between 150 and 400

frames) displaying HIV-1 particles. In comparison to [5], the employed perfor-

mance measure reflects more comprehensively the performance of the evaluated

tracking approaches.

2 Materials and methods

2.1 Deterministic tracking approaches

The deterministic approaches follow a two-step paradigm consisting of object

localization and motion correspondence. For object localization, we employed

two approaches: an approach based on the spot-enhancing filter (SEF), and an

approach using 2D Gaussian fitting (GaussFit). For motion correspondence, we

used a global nearest neighbor (GNN) approach [2]. By combining the localiza-

tion schemes with the motion correspondence scheme we obtain two deterministic

approaches: 1) spot-enhancing filter with global nearest neighbor (SEF&GNN),

and 2) 2D Gaussian fitting with global nearest neighbor (GaussFit&GNN).

2.2 Probabilistic tracking approaches

The probabilistic approaches follow a Bayesian paradigm, where the aim is to

estimate the state xtof a virus particle at time step t given a sequence of mea-

surements y1:t. A solution to this problem involves computing the posterior

distribution p(xt|y1:t) using stochastic propagation and Bayes’ theorem:

p(xt|y1:t) ∝ p(yt|xt)p(xt|y1:t−1).

Given certain assumptions on the form of the distributions, the recursion can

be solved analytically using a Kalman filter. More generally, the recursive re-

lation can be solved via approximation using a particle filter. The idea behind

this filter is to approximate the posterior distribution using a set of weighted

random samples. When tracking multiple objects, multiple modes arise in the

posterior distribution. The multimodality can be modeled via a non-parametric

M-component mixture model, which can be computed using a mixture of parti-

cle filters [7]. Alternatively, one may track multiple objects by instantiating one

(1)

Page 3

Evaluation of Approaches for Tracking Virus Particles 209

independent particle filter per object (e.g., [8]). To prevent the problem of filter

coalescence, the IPF includes a penalization scheme [4]. Similarly, one may use

one Kalman filter per object to track multiple objects.

For all probabilistic approaches, we employed the two localization schemes de-

scribed above to detect virus particles. Note that the approaches using a mixture

of particle filters can only track a fixed number of objects. For the approaches

using independent particle filters and Kalman filters, the motion correspondence

problem, i.e., the problem of assigning the position measurements computed

by the localization schemes to each spatial-temporal filter, is addressed using

a global nearest neighbor approach. The combination of the two localization

algorithms with the different filters yields the following tracking approaches: 3)

spot-enhancing filter and Kalman filters (SEF&Kalman), 4) spot-enhancing fil-

ter and a mixture of particle filters (SEF&MPF), 5) spot-enhancing filter and

independent particle filters (SEF&IPF), 6) 2D Gaussian fitting and Kalman fil-

ters (GaussFit&Kalman), 7) 2D Gaussian fitting and a mixture of particle filters

(GaussFit&MPF), and 8) 2D Gaussian fitting and independent particle filters

(GaussFit&IPF).

2.3

To quantitatively assess the performance of each approach in each image se-

quence, we have employed the tracking accuracy Ptrack=

flects the ratio between the number of correctly computed trajectories ntrack,correct

and the number of true trajectories ntrack,total. The value ntrack,correctis com-

puted as the weighted sum of the percentage of tracked time steps rtracked,i

for each i-th true trajectory: ntrack,correct =?ntrack,total

the number of correctly computed trajectories ntrack,icorresponding to each i-th

true trajectory: wi= G(ntrack,i;µ = 1,σ = 1). The weighting scheme is intro-

duced to penalize computed trajectories that are broken. A computed trajectory

is assumed to be correct if the Euclidean distance between the measured object

position and the true object position is below 2 pixels.

Performance assessment

ntrack,correct

ntrack,total, which re-

i=1

wirtracked,i, where the

weight wi is given by a Gaussian function G(·), which takes as its argument

3 Results

We have applied all eight approaches to synthetic as well as real microscopy

images. Below, we present the results for nine real microscopy image sequences.

In these sequences, fluorescently labeled HIV-1 particles were imaged using a

fluorescence wide-field microscope; movies were recorded with a frequency of

10Hz. Ground truth for the virus positions was obtained by manual tracking

using the commercial software MetaMorph. For all sequences, we have employed

fixed parameter values for all approaches. Similarly, the noise parameters for the

dynamical model of the Kalman filter were set analogously as the ones employed

for the particle filter. Details for each sequence are given in Table 1. The quan-

titative experimental results for the nine sequences are presented in Table 2. As

an example, results for the real image sequence “Seq. 7” are shown in Fig. 1.

Page 4

210Godinez et al.

Table 1. Description of real microscopy image sequences.

Dimensions [pixels]

256×256

256×256

256×256

256×256

512×512

512×512

512×512

512×512

512×512

No. of time steps No. of objects

Seq. 1

Seq. 2

Seq. 3

Seq. 4

Seq. 5

Seq. 6

Seq. 7

Seq. 8

Seq. 9

250

250

250

150

200

400

400

400

400

23

10

5

21

15

29

31

43

24

Table 2. Tracking accuracy Ptrack for real microscopy image sequences.

SEF& SEF& SEF& SEF& GaussFit& GaussFit& GaussFit& GaussFit&

GNN Kalman MPF IPFGNNKalman MPFIPF

Seq. 1

Seq. 2

Seq. 3

Seq. 4

Seq. 5

Seq. 6

Seq. 7

Seq. 8

Seq. 9

75.24

60.78

45.07

61.71

93.54

57.79

64.65

74.61

72.55

81.12

68.70

32.29

68.89

93.54

74.59

80.60

74.09

82.55

84.82 86.73

71.93 83.52

76.67 42.55

28.52 71.10

84.64 93.54

39.64 63.98

48.26 82.48

50.62 74.64

48.51 73.55

71.29

30.79

40.69

70.53

81.09

55.18

53.21

75.10

67.78

81.30

44.64

60.18

74.59

93.54

67.97

77.47

74.11

80.55

81.95

63.49

60.00

23.81

85.64

41.73

31.23

49.04

52.61

82.61

84.67

80.00

83.39

93.54

62.10

77.67

67.76

76.89

Mean

Std. Dev. 13.68

67.33 72.93

17.08

59.29 74.68

20.62 15.00

60.63

16.82

72.71

13.99

54.39

20.96

78.74

9.32

4 Discussion

Our experimental results in Table 2 indicate that the performance of the deter-

ministic approaches is not very good (e.g., for the best deterministic approach,

namely SEF&GNN, we obtain a mean tracking accuracy of¯Ptrack= 67.33%).

The reason for this is that localization errors (e.g., detection failures) as well

as errors in correspondence finding (e.g, incorrect assignments) reduce the num-

ber of correctly computed trajectories. The approaches based on the Kalman

filter, in comparison to the deterministic approaches, yield an improved per-

formance (e.g.,¯Ptrack = 72.93% for the best Kalman-based approach, namely

SEF&Kalman). This suggests that the inclusion of a spatial-temporal filter-

ing step enhances the performance. For the approaches using particle filters, it

turned out that those using independent particle filters (IPF) outperform those

using a mixture of particle filters (MPF). The reason for this result is twofold:

Page 5

Evaluation of Approaches for Tracking Virus Particles 211

Fig.1. Tracking results for four approaches for the real image sequence“Seq. 7”(time

step t = 110).

(a) SEF&GNN(b) SEF&Kalman(c) SEF&MPF (d) SEF&IPF

first, MPF cannot track a variable number of objects (whereas in the real images,

the number of objects varies over time); second, the MPF uses a variable number

of samples to estimate each mixture component. As such, the estimation accu-

racy decreases for those components for which few samples have been assigned.

Among all approaches, the best tracking accuracy is obtained by GaussFit&IPF

(¯Ptrack= 78.74%). The superior performance is mainly due to the comprehen-

sive tracking machinery of the particle filter, which includes the steps of particle

localization, motion correspondence, and position estimation. In summary, the

quantitative results suggest that the probabilistic approaches are more accurate

than the deterministic schemes.

Acknowledgement. Support of the BMBF (FORSYS) project VIROQUANT

is gratefully acknowledged.

References

1. Schutz GJ, Schindler H, Schmidt T. Single-molecule microscopy on model mem-

branes reveals anomalous diffusion. Biophys J. 1997;73(2):1073–1080.

2. Sbalzarini IF, Koumoutsakos P. Feature point tracking and trajectory analysis for

video imaging in cell biology. J Struct Biol. 2005;151(2):182–195.

3. Arhel N, Genovesio A, Kim K, et al. Quantitative four-dimensional tracking of

cytoplasmic and nuclear HIV-1 complexes. Nat Methods. 2006;3(10):817–824.

4. Godinez WJ, Lampe M, W¨ orz S, et al. Probabilistic tracking of virus particles in

fluorescence microscopy images. Proc ISBI. 2008; p. 272–275.

5. Cheezum MK, Walker WF, Guilford WH. Quantitative comparison of algorithms

for tracking single fluorescent particles. Biophys J. 2001;81:2378–2388.

6. Sage D, Neumann FR, Hediger F, et al. Automatic tracking of individual fluores-

cence particles: Application to the study of chromosome dynamics. IEEE Trans

Image Process. 2005;14(9):1372–1382.

7. Vermaak J, Doucet A, P´ erez P; IEEE. Maintaining multi-modality through mixture

tracking. Proc ICCV. 2003; p. 1110–1116.

8. Qu W, Schonfeld D, Mohamed M; IEEE.

multi-object tracking using a magnetic-inertia potential model. Proc IEEE ICCV.

2005;1:535–540.

Real-time interactively distributed