Content uploaded by Antoine Basset

Author content

All content in this area was uploaded by Antoine Basset on Jul 17, 2015

Content may be subject to copyright.

SLT-LoG: A vesicle segmentation method with automatic scale selection

and local thresholding applied to TIRF microscopy

Antoine Basset∗, J´

erˆ

ome Boulanger†, Patrick Bouthemy∗, Charles Kervrann∗, and Jean Salamero†

∗Inria, Centre Rennes – Bretagne Atlantique, Campus Universitaire de Beaulieu, 35042 Rennes Cedex, France

†CNRS, Institut Curie, UMR 144, 12 rue Lhomond, 75005 Paris, France

Abstract— Accurately detecting cellular structures in ﬂuores-

cence microscopy is of primary interest for further quantitative

analysis such as counting, tracking or classiﬁcation. We aim at

segmenting vesicles in TIRF images. The optimal segmentation

scale is automatically selected, relying on a multiscale feature

detection stage, and the segmentation consists in thresholding

the Laplacian of Gaussian of the intensity image. In contrast

to other methods, the threshold is locally adapted, resulting

in better detection rates for complex images. Our method

is mostly on par with machine learning-based techniques,

while offering lower computation time and requiring no prior

training. It is very competitive with existing unsupervised

detection algorithms.

I. INT RODUCT IO N AN D RE LATED WORK

Since the early time of protein tagging with GFP, mi-

croscopy investigations at the single cell level were faced

with the problem of automatically characterizing particle be-

havior in space and time. Recovering particle dynamics is of

utmost importance for understanding biological mechanisms

such as cell-to-cell interaction and exchange, signaling and

cellular response, migration and division, among others. In

the case of membrane trafﬁcking that guarantees homeostasis

of cellular compartments, many studies deal with the problem

of tracking vesicles [8], classifying their trajectories [11], or

recognizing various dynamical events [2]. These high-level

analyses primarily require a reliable, accurate and efﬁcient

detection of particles and vesicles in ﬂuorescence microscopy

images. One of the characteristics of total internal reﬂection

ﬂuorescence (TIRF) microscopy is its very short depth of

ﬁeld (DOF) [1]. Moreover, the vesicles we are interested in

share a similar size, so they appear as spots of similar scale in

the sequence. Estimating the proper image scale to segment

the vesicles is then of key interest.

Many vesicle detection methods have already been pro-

posed, like wavelet multiscale product (WMP) [9], multiscale

variance-stabilizing transform detector (MSVST) [16], top-

hat ﬁlter (TH) [4], grayscale opening top-hat ﬁlter (MTH)

[13], H-dome based detector (HD) [15], spot-enhancing ﬁlter

(SEF) [11], image feature-based detector (IDF1and IDF2)

[14], or maximum possible height-dome (MPHD) [10]. An

extensive comparison was proposed in [12].

In this paper, we propose an original and efﬁcient method

for vesicle segmentation with fewer parameters than the

aforementioned methods. It exploits the Laplacian of Gaus-

sian (LoG) of the images at several scales. Since the vesicles

size is almost constant in space and time, a prominent

mode is expected in the empirical distribution of the scales

at which the minima of LoG values are detected. It will

precisely correspond to the optimal sought scale. The vesicle

segmentation map is then derived by thresholding the LoG

values obtained at this optimal scale. To set the threshold, we

assume that the values of the LoG locally follow a normal

distribution. For each point, we estimate the local mean and

variance, and the threshold is deduced from a user-selected

probability of false alarm (PFA).

We have evaluated our method on classical synthetic

sequences for which the performances of the above methods

are available [10], [12]. Comparative results on this dataset

demonstrate that our method outperforms well-known unsu-

pervised methods. We have also obtained very satisfactory

results on real TIRF sequences.

The remaining of the paper is organized as follows. In

Section II, we describe our vesicle segmentation method.

Comparative experimental results are reported in Section III

and we give concluding remarks in Section IV.

II. SEGMENTATION OF VESICLES

Our overall segmentation method called SLT-LoG pro-

ceeds in three steps: (1) off-line scale selection, (2) compu-

tation of the LoG ﬁeld at the selected scale and estimation

of the Gaussian parameters, (3) local thresholding.

A. Scale selection

We adopt the Lindeberg’s scale-space framework [7]. The

automatic scale selection consists in counting the number of

so-called blobs (corresponding to minima values) detected in

the LoG maps at different scales. Formally, the scale-space

representation {Lt}t∈R?

+of an image Iis deﬁned by:

∀t∈R?

+, Lt=gt⊗I, (1)

where gtis a 2D isotropic Gaussian convolution kernel of

variance (or scale)t. To highlight the vesicles, which appear

as bright spots in the image I, we apply the scale-normalized

Laplacian operator to Ltdeﬁned as:

t∇2Lt=t∂2Lt

∂x2+∂2Lt

∂y2,(2)

where ∇2is the Laplacian operator. Thanks to the associative

property of convolution, the computation time can be reduced

by combining the Gaussian and Laplacian ﬁlters using a

single normalized LoG kernel ht, such that t∇2Lt=ht⊗I.

(a) Input image I(b) LoG map H1(t= 1) (c) LoG map H3(t= 3) (d) LoG map H9(t= 9) (e) LoG map H27 (t= 27)

Fig. 1. Scale-space LoG-transform of a real TIRF image depicting a M10 cell (Rab11-mCherry).

(a) Input image (b) Blobs ground-truth (c) Detected blobs

Fig. 2. Blob detection in a synthetic sequence. (a) Gaussian spots with a

variance σ2of 9 are added to a cluttered background; a Poisson-Gaussian

noise is further added. (b) Ground truth of the added spots, the disks radius

is related to the Gaussian variance: r=√2σ2. (c) The detected blobs are

plotted in yellow: 20 blobs are detected at scale t= 3, 41 blobs at t= 9,

5 blobs at t= 27, and 1 blob at t= 81. We can deduce the optimal scale

t?= 9, which is indeed the true spots variance.

We thus obtain the multi-scale LoG ﬁeld:

∀t∈R?

+, Ht: Ω →R

(x, y)7→ (ht⊗I)(x, y),(3)

where Ω⊂R2is the image domain.

In [6] it was proven that under some assumptions, the

scale-space theory applies to discrete signals. Therefore, we

can use sampled LoG kernels of exponentially increasing

scales. In practice, two consecutive scales must be distant

from an odd multiplicative factor, so we use the smallest

possible step, that is 3. For illustration, Fig. 1 depicts the

scale-space LoG-transform of a TIRF microscopy image.

Ablob bis deﬁned by the triplet (tb, xb, yb)of a local min-

imum in the LoG ﬁeld, where tb- and (xb, yb)-coordinates

respectively correspond to the scale and spatial position of

the blob b[7]. Hence, bis a blob iff:

∀(t, x, y)∈N(b), Ht(x, y)> Htb(xb, yb),(4)

with N(b)a3×3×3neighborhood of b. The blob detection is

illustrated on Fig. 2. We do not exploit the detected positions

per themselves, since this method behaves poorly in terms of

vesicles detection. Indeed, as illustrated in Fig. 2c, only 52

spots over 60 are recovered, while 15 others are wrongly

detected. More interesting is the scale distribution of the

blobs, and particularly its main mode. For a disk of radius

r, the LoG response is minimum for t=r2/2, resulting

in a blob at this scale. As a consequence, since the TIRF

acquisition modality has a narrow DOF, and most exocytotic

or endocytotic vesicles are similar in size, most detected

blobs share the same scale, which is precisely the optimal

scale t?to be selected. As the optimal scale does not vary

in time, we only apply the scale selection to the ﬁrst frame

of the sequence in order to save computation time. Then, we

TABLE I

SEL ECT ED V ER SUS E XP EC TED S CA LE

Gaussian variance 1 4 9 16 25 36 49 64 81 100

Expected scale 1 3 9 9 27 27 27 81 81 81

Selected scale 3 3 9 9 27 27 27 27 81 81

The scales are growing by a factor 3, so we expect to ﬁnd the multiple of

3 closest to the Gaussian spots variance.

apply the t?-LoG on every image of the sequence.

To demonstrate the scale selection efﬁciency, we have gen-

erated different images containing isotropic Gaussian spots

for different variances. They are corrupted by a Poisson-

Gaussian noise. Table I summarizes the selected scales, and

Fig. 2 displays an example of blobs scale-space distribution

for a Gaussian spot variance of 9 pixels.

B. Estimation of the local distribution of Ht?

The segmentation of the vesicles consists in thresholding

the t?-LoG-ﬁltered images. As depicted in Fig. 3 and 5,

experiments demonstrate that a global threshold cannot prop-

erly cope with complex situations. In this example involving

a space-varying background overlaid with isotropic Gaussian

spots, the global approach both underdetects on the left part

of the image and overdetects on the right part. Thus, the

detection cannot be simultaneously improved for both sides.

To overcome this difﬁculty, we estimate a threshold at each

point according to the local distribution of Ht?computed in a

neighborhood of that point. This local distribution is assumed

to be Gaussian. Indeed, if we consider the pixels of the input

image independently and identically normally distributed, the

distribution of Ht?is theoretically normal since the LoG

operator is a ﬁnite convolution. Moreover, the parameters

of a local normal distribution can be estimated in constant

time with respect to the number of pixels in the window,

which is crucial for a point-wise estimation. However, for

TIRF microscopy, it is generally assumed that the noise

follows a Poisson-Gaussian distribution [1]. Thus, we ﬁrst

stabilize the variance by applying a generalized Anscombe

transform, whose parameters are estimated as in [3]. The

Anscombe transform is performed before the blob detection

step, because it modiﬁes the image intensity range.

To compute the local mean µ(p)and variance σ2(p)

of the Gaussian distribution, we have tested two types of

neighborhood: a square window W(p), in which the moment

evaluation reduces to a four-term addition using integral

images [5]; a Gaussian window G(p), whose weights are

(a) Input image (b) Global thresholding (c) Local thresholding

Fig. 3. Detection by applying global or local thresholding on Ht?. (b)

False negatives are framed in yellow and false positives in red. (c) Detection

is perfect with the local threshold.

recursively evaluated. In both cases, the computation time

does not depend on the size of the window.

C. Vesicle segmentation with local threshold

Given a p-value P, the threshold τ(p)is locally calculated

as: ∀p∈Ω, τ(p) = Φ−1(P)×σ2(p) + µ(p),(5)

where Φdenotes the cumulative distribution function of the

normal distribution. The function Φ−1is evaluated only once,

since it does not depend on point p. Hence, the complexity

of the overall estimation and thresholding process is linear

with the image size. Thresholding the LoG-ﬁltered image

Ht?results in a set V0of connected components, where false

detections are mostly due to noise and thus have a small

area. Hence, we discard the smallest connected components.

As said in II-A, the LoG favors rounded objects of radius

√2t?. We suppose that objects of half that size are irrelevant

as well as objects of only one or two-pixel size. Therefore,

the minimum acceptable area Amin is set to:

Amin = max(2,bπt?c).(6)

The ﬁnal set of vesicles Vis then:

V={v∈V0| |v|> Amin},(7)

where |v|denotes the area of the connected component v.

III. EXP ER IM EN TAL R ES ULT S

A. Comparative evaluation on synthetic sequences

We have compared our method with eight other unsuper-

vised detection methods evaluated in [10] and [12], namely,

WMP [9], MSVST [16], TH [4], MTH [13], HD [15], SEF

[11], IDF1and IDF2[14], and MPHD [10].

The benchmark comprises six different synthetic se-

quences introduced in [12], which involve two vesicle form

factors (round and elongated) and three types of background:

constant background (type A), background with a horizon-

tal intensity gradient (type B), and background with large

structures (type C). A Poisson noise is added, with a signal-

to-noise ratio (SNR) ranging from 1 to 5. Figure 4 depicts a

sample of each background type. Each sequence is 16-frame

long of 512 ×512 size and contains 256 vesicles per frame.

Round objects are generated as Gaussian spots of standard

deviation 2 pixels. For elongated spots, the standard deviation

is 5 pixels along the principal axis and 2 pixels along the

secondary axis. More details can be found in [12].

We report comparative results for SNR = 2, as done

in [10], [12]. Table II summarizes the true positive rates

(TPR) and modiﬁed false positive rates (FPR*) with this

benchmark conﬁguration. FPR* is deﬁned in [12] as

FPR* = NF P /(NT P +NF N ). The parameters involved in

(a) Type A, SNR = 3,

elongated objects

(b) Type B, SNR = 2,

elongated objects

(c) Type C, SNR = 1,

round objects

Fig. 4. Synthetic image samples.

TABLE II

COMPARISON WITH STATE-OF-TH E-A RT ME TH ODS AT SN R = 2

Object Background TPR of SLT-LoG Best TPR

shape type using W(p)using G(p)from [10], [12]

Round

Type A 0.990 0.996 0.99

Type B 0.974 0.987 0.99 (MSVST)

Type C 0.966 0.982 0.95 (SEF)

Elongated

Type A 1+(2.4×10−4)1+(0) 0.99

Type B 1+(9.8×10−4)1+(0) 0.99

Type C 0.981 0.999 0.97 (HD)

The FPR* is 0.010, except for 1+, where it is put in brackets.

each method (in our case P) were set to obtain FPR* = 0.01.

In [10], [12], an object is considered as detected if the

distance between ground truth and its estimate coordinates

is less than a threshold, arbitrarily set to 4 pixels. Since

our method supplies the entire spatial support of the vesicle,

we can evaluate it with a parameter-free criterion: a vesicle

is considered as correctly detected if the ground-truth is

included in the segmented connected component. As a matter

of fact, this criterion is tighter since here the diameter of the

extracted connected components is always less than 8 pixels.

Due to page limitation, we only report in Table II the results

of the best performer for each sequence drawn from [10],

[12]. In all cases but one, our method – denoted as SLT-LoG

(Scale-selected Local Thresholding of LoG) – outperforms

the other methods in terms of detection and false alarm rates.

For two sequences, we even obtain TPR = 1, and the FPR*

can be decreased to very low values without losing any true

positive. For most of the sequences, SLT-LoG also performs

better than the two learning methods presented in [12]

exploiting AdaBoost and Fisher discriminant analysis. For

these sequences, the performance of SLT-LoG is better using

G(p). Moreover, the sensitivity of the variance parameter is

very low: a standard deviation of 15 pixels has been chosen

for all our experiments, while we had to adapt sequence by

sequence the size of W(p)to get the best performance.

To be more exhaustive, other experiments were carried

out with different SNRs. For SNR = 1, the method per-

formance decreases, but even manually it becomes hard to

label these sequences, as shown in Fig. 4c. FROC curves

are given in Fig. 5: type A background with round objects

and type C background with elongated objects. This plot

also demonstrates the potent advantage of using a local

threshold for images presenting a complex background. For

the classical LoG method (with global threshold), we take

0!

0.2!

0.4!

0.6!

0.8!

1!

0!0.2!0.4!0.6!0.8!1!

TPR!

FPR*!

SLT-LoG with

Gaussian window!

SLT-LoG with

rectangular window!

LoG!

SLT-LoG with

Gaussian window!

SLT-LoG with

rectangular window!

LoG!

SNR 1, type A,!

round objects!

SNR 1, type C,!

elongated objects!

Fig. 5. Comparison with the classical LoG method at SNR = 1.

(a) Rab11-mCherry (b) Segmented vesicles in image (a)

(c) TfR-pHluorin, Rab11-mCherry (d) Segmented vesicles in image (c)

Fig. 6. Results on two real TIRF sequences of M10 cells.

the scale provided by the SLT-LoG method. In some cases,

W(p)behaves better than G(p). For SNR = 3 and above,

our method perfectly performs for all the sequences of the

benchmark, except for one of them (type C with round

objects at SNR = 3, using W(p)), however we get only two

false positives among 4094 true positives for that case.

For all the synthetic sequences, the execution time on a

laptop (4-core 2.3GHz CPU, 8GB 1.6GHz DDR3) is only

0.15s using W(p), or 0.25s using G(p), per 512×512 frame,

plus 3.5s for the off-line scale selection step.

B. Results on real sequences

We have applied our segmentation method to several real

TIRF microscopy sequences of M10 cell lines transfected

with different ﬂuorescently labeled cargo protein, namely

Langerine and Transferrin receptor (TfR), as well as the

Rab11 GTPase. These proteins involved in the recycling

pathway are associated to transport intermediates (such as

vesicles) and exhibit various appearance. On Rab11 frames,

several very elongated objects are visible (as in Fig. 6a),

which cannot be accurately modeled by anisotropic Gaussian

spots. The proposed SLT-LoG method provides the entire

spatial support of the vesicles, while other methods would

only output their center coordinates or ﬁt ellipses. Using

some beam-splitting techniques, two ﬂuorescent markers can

be captured side-by-side on the detector, resulting in two-part

images as shown in Fig. 6c. Despite the two very different

parts of the image, the segmentation obtained with our SLT-

LoG method is very satisfactory.

IV. DISCUSSION AND CONCLUSION

We have proposed a novel and efﬁcient vesicle segmenta-

tion method called SLT-LoG which involves an automatic

scale selection and a local threshold setting. After deter-

mining the optimal scale, a LoG operator is applied on

the images. The segmentation threshold is automatically and

locally set according to a given PFA value. Overall, SLT-LoG

outperforms state-of-the-art unsupervised methods. Except

the PFA which is in fact a detection sensitivity setting, the

only parameter of the method to be ﬁxed is the size of

the local estimation windows W(p)or G(p). Its sensitivity

rapidly decreases when SNR increases. Our future work will

mainly focus on the automatic adaptation of the W(p)size.

This project is partially supported by R´

egion Bretagne (Brittany Council)

through a contribution to A. Basset’s Ph.D. student grant.

REFERENCES

[1] D. Axelrod. Total internal reﬂection ﬂuorescence microscopy. Methods

in Cell Biology, 89:169–221, 2008.

[2] J. Boulanger, A. Gidon, C. Kervrann, and J. Salamero. A patch-based

method for repetitive and transcient event detection in ﬂuorescence

imaging. PLoS One, 5(10):e13190, Oct. 2010.

[3] J. Boulanger, C. Kervrann, P. Bouthemy, P. Elbau, J.-B. Sibarita,

and J. Salamero. Patch-based non-local functional for denoising

ﬂuorescence microscopy image sequences. IEEE Trans. Medical

Imaging, 29(2):442–453, Feb. 2010.

[4] D. S. Bright and E. B. Steel. Two-dimensional top hat ﬁlter for

extracting spots and spheres from digital images. J. Microscopy,

146(2):191–200, May 1987.

[5] F. C. Crow. Summed-area tables for texture mapping. ACM SIG-

GRAPH Comp. Graphics, 18(3):207–212, Jul. 1984.

[6] T. Lindeberg. Scale-space for discrete signals. IEEE Trans. Pattern

Analysis and Machine Intelligence, 12(3):234–254, Mar. 1990.

[7] T. Lindeberg. Feature detection with automatic scale selection. Int. J.

Comp. Vision, 30(2):79–116, Nov. 1998.

[8] E. Meijering, O. Dzyubachyk, and I. Smal. Methods for cell and

particle tracking. Elsevier, 2012.

[9] J.-C. Olivo-Marin. Extraction of spots in biological images using

multiscale products. Pattern Recog., 35(9):1989–1996, Sept. 2002.

[10] S. H. Rezatoﬁghi, R. Hartley, and W. E. Hughes. A new approach for

spot detection in total internal reﬂection ﬂuorescence microscopy. In

IEEE Int. Symp. Biomedical Imaging, ISBI’12, Barcelona, May 2012.

[11] D. Sage, F.R. Neumann, F. Hediger, S.M. Gasser, and M. Unser.

Automatic tracking of individual ﬂuorescence particles: Application

to the study of chromosome dynamics. IEEE Trans. Image Process.,

14(9):1372–1383, Sep. 2005.

[12] I. Smal, M. Loog, W. J. Niessen, and E. H. W. Meijering. Quantitative

comparison of spot detection methods in ﬂuorescence microscopy.

IEEE Trans. Medical Imaging, 29(2):282–301, Feb. 2010.

[13] P. Soille. Morphological image analysis: Principles and applications.

Springer, 2003.

[14] B. M. ter Haar Romeny. Front-end vision and multi-scale image

analysis. Computational Imaging and Vision. Springer, 2003.

[15] L. Vincent. Morphological grayscale reconstruction in image analysis:

Applications and efﬁcient algorithms. IEEE Trans. Image Process.,

2(2):176–201, Apr. 1993.

[16] B. Zhang, M. J. Fadili, J.-L. Starck, and J.-C. Olivo-Marin. Multiscale

variance-stabilizing transform for mixed-Poisson-Gaussian processes

and its applications in bioimaging. In IEEE Int. Conf. Image Process.,

ICIP’07, San Antonio, Oct. 2007.