Conference PaperPDF Available

Abstract and Figures

Accurately detecting cellular structures in fluorescence microscopy is of primary interest for further quantitative analysis such as counting, tracking or classification. We aim at segmenting vesicles in TIRF images. The optimal segmentation scale is automatically selected, relying on a multiscale feature detection stage, and the segmentation consists in thresholding the Laplacian of Gaussian of the intensity image. In contrast to other methods, the threshold is locally adapted, resulting in better detection rates for complex images. Our method is mostly on par with machine learning-based techniques, while offering lower computation time and requiring no prior training. It is very competitive with existing unsupervised detection algorithms.
Content may be subject to copyright.
SLT-LoG: A vesicle segmentation method with automatic scale selection
and local thresholding applied to TIRF microscopy
Antoine Basset, J´
erˆ
ome Boulanger, Patrick Bouthemy, Charles Kervrann, and Jean Salamero
Inria, Centre Rennes – Bretagne Atlantique, Campus Universitaire de Beaulieu, 35042 Rennes Cedex, France
CNRS, Institut Curie, UMR 144, 12 rue Lhomond, 75005 Paris, France
Abstract Accurately detecting cellular structures in fluores-
cence microscopy is of primary interest for further quantitative
analysis such as counting, tracking or classification. We aim at
segmenting vesicles in TIRF images. The optimal segmentation
scale is automatically selected, relying on a multiscale feature
detection stage, and the segmentation consists in thresholding
the Laplacian of Gaussian of the intensity image. In contrast
to other methods, the threshold is locally adapted, resulting
in better detection rates for complex images. Our method
is mostly on par with machine learning-based techniques,
while offering lower computation time and requiring no prior
training. It is very competitive with existing unsupervised
detection algorithms.
I. INT RODUCT IO N AN D RE LATED WORK
Since the early time of protein tagging with GFP, mi-
croscopy investigations at the single cell level were faced
with the problem of automatically characterizing particle be-
havior in space and time. Recovering particle dynamics is of
utmost importance for understanding biological mechanisms
such as cell-to-cell interaction and exchange, signaling and
cellular response, migration and division, among others. In
the case of membrane trafficking that guarantees homeostasis
of cellular compartments, many studies deal with the problem
of tracking vesicles [8], classifying their trajectories [11], or
recognizing various dynamical events [2]. These high-level
analyses primarily require a reliable, accurate and efficient
detection of particles and vesicles in fluorescence microscopy
images. One of the characteristics of total internal reflection
fluorescence (TIRF) microscopy is its very short depth of
field (DOF) [1]. Moreover, the vesicles we are interested in
share a similar size, so they appear as spots of similar scale in
the sequence. Estimating the proper image scale to segment
the vesicles is then of key interest.
Many vesicle detection methods have already been pro-
posed, like wavelet multiscale product (WMP) [9], multiscale
variance-stabilizing transform detector (MSVST) [16], top-
hat filter (TH) [4], grayscale opening top-hat filter (MTH)
[13], H-dome based detector (HD) [15], spot-enhancing filter
(SEF) [11], image feature-based detector (IDF1and IDF2)
[14], or maximum possible height-dome (MPHD) [10]. An
extensive comparison was proposed in [12].
In this paper, we propose an original and efficient method
for vesicle segmentation with fewer parameters than the
aforementioned methods. It exploits the Laplacian of Gaus-
sian (LoG) of the images at several scales. Since the vesicles
size is almost constant in space and time, a prominent
mode is expected in the empirical distribution of the scales
at which the minima of LoG values are detected. It will
precisely correspond to the optimal sought scale. The vesicle
segmentation map is then derived by thresholding the LoG
values obtained at this optimal scale. To set the threshold, we
assume that the values of the LoG locally follow a normal
distribution. For each point, we estimate the local mean and
variance, and the threshold is deduced from a user-selected
probability of false alarm (PFA).
We have evaluated our method on classical synthetic
sequences for which the performances of the above methods
are available [10], [12]. Comparative results on this dataset
demonstrate that our method outperforms well-known unsu-
pervised methods. We have also obtained very satisfactory
results on real TIRF sequences.
The remaining of the paper is organized as follows. In
Section II, we describe our vesicle segmentation method.
Comparative experimental results are reported in Section III
and we give concluding remarks in Section IV.
II. SEGMENTATION OF VESICLES
Our overall segmentation method called SLT-LoG pro-
ceeds in three steps: (1) off-line scale selection, (2) compu-
tation of the LoG field at the selected scale and estimation
of the Gaussian parameters, (3) local thresholding.
A. Scale selection
We adopt the Lindeberg’s scale-space framework [7]. The
automatic scale selection consists in counting the number of
so-called blobs (corresponding to minima values) detected in
the LoG maps at different scales. Formally, the scale-space
representation {Lt}tR?
+of an image Iis defined by:
tR?
+, Lt=gtI, (1)
where gtis a 2D isotropic Gaussian convolution kernel of
variance (or scale)t. To highlight the vesicles, which appear
as bright spots in the image I, we apply the scale-normalized
Laplacian operator to Ltdefined as:
t2Lt=t2Lt
∂x2+2Lt
∂y2,(2)
where 2is the Laplacian operator. Thanks to the associative
property of convolution, the computation time can be reduced
by combining the Gaussian and Laplacian filters using a
single normalized LoG kernel ht, such that t2Lt=htI.
(a) Input image I(b) LoG map H1(t= 1) (c) LoG map H3(t= 3) (d) LoG map H9(t= 9) (e) LoG map H27 (t= 27)
Fig. 1. Scale-space LoG-transform of a real TIRF image depicting a M10 cell (Rab11-mCherry).
(a) Input image (b) Blobs ground-truth (c) Detected blobs
Fig. 2. Blob detection in a synthetic sequence. (a) Gaussian spots with a
variance σ2of 9 are added to a cluttered background; a Poisson-Gaussian
noise is further added. (b) Ground truth of the added spots, the disks radius
is related to the Gaussian variance: r=2σ2. (c) The detected blobs are
plotted in yellow: 20 blobs are detected at scale t= 3, 41 blobs at t= 9,
5 blobs at t= 27, and 1 blob at t= 81. We can deduce the optimal scale
t?= 9, which is indeed the true spots variance.
We thus obtain the multi-scale LoG field:
tR?
+, Ht: Ω R
(x, y)7→ (htI)(x, y),(3)
where R2is the image domain.
In [6] it was proven that under some assumptions, the
scale-space theory applies to discrete signals. Therefore, we
can use sampled LoG kernels of exponentially increasing
scales. In practice, two consecutive scales must be distant
from an odd multiplicative factor, so we use the smallest
possible step, that is 3. For illustration, Fig. 1 depicts the
scale-space LoG-transform of a TIRF microscopy image.
Ablob bis defined by the triplet (tb, xb, yb)of a local min-
imum in the LoG field, where tb- and (xb, yb)-coordinates
respectively correspond to the scale and spatial position of
the blob b[7]. Hence, bis a blob iff:
(t, x, y)N(b), Ht(x, y)> Htb(xb, yb),(4)
with N(b)a3×3×3neighborhood of b. The blob detection is
illustrated on Fig. 2. We do not exploit the detected positions
per themselves, since this method behaves poorly in terms of
vesicles detection. Indeed, as illustrated in Fig. 2c, only 52
spots over 60 are recovered, while 15 others are wrongly
detected. More interesting is the scale distribution of the
blobs, and particularly its main mode. For a disk of radius
r, the LoG response is minimum for t=r2/2, resulting
in a blob at this scale. As a consequence, since the TIRF
acquisition modality has a narrow DOF, and most exocytotic
or endocytotic vesicles are similar in size, most detected
blobs share the same scale, which is precisely the optimal
scale t?to be selected. As the optimal scale does not vary
in time, we only apply the scale selection to the first frame
of the sequence in order to save computation time. Then, we
TABLE I
SEL ECT ED V ER SUS E XP EC TED S CA LE
Gaussian variance 1 4 9 16 25 36 49 64 81 100
Expected scale 1 3 9 9 27 27 27 81 81 81
Selected scale 3 3 9 9 27 27 27 27 81 81
The scales are growing by a factor 3, so we expect to find the multiple of
3 closest to the Gaussian spots variance.
apply the t?-LoG on every image of the sequence.
To demonstrate the scale selection efficiency, we have gen-
erated different images containing isotropic Gaussian spots
for different variances. They are corrupted by a Poisson-
Gaussian noise. Table I summarizes the selected scales, and
Fig. 2 displays an example of blobs scale-space distribution
for a Gaussian spot variance of 9 pixels.
B. Estimation of the local distribution of Ht?
The segmentation of the vesicles consists in thresholding
the t?-LoG-filtered images. As depicted in Fig. 3 and 5,
experiments demonstrate that a global threshold cannot prop-
erly cope with complex situations. In this example involving
a space-varying background overlaid with isotropic Gaussian
spots, the global approach both underdetects on the left part
of the image and overdetects on the right part. Thus, the
detection cannot be simultaneously improved for both sides.
To overcome this difficulty, we estimate a threshold at each
point according to the local distribution of Ht?computed in a
neighborhood of that point. This local distribution is assumed
to be Gaussian. Indeed, if we consider the pixels of the input
image independently and identically normally distributed, the
distribution of Ht?is theoretically normal since the LoG
operator is a finite convolution. Moreover, the parameters
of a local normal distribution can be estimated in constant
time with respect to the number of pixels in the window,
which is crucial for a point-wise estimation. However, for
TIRF microscopy, it is generally assumed that the noise
follows a Poisson-Gaussian distribution [1]. Thus, we first
stabilize the variance by applying a generalized Anscombe
transform, whose parameters are estimated as in [3]. The
Anscombe transform is performed before the blob detection
step, because it modifies the image intensity range.
To compute the local mean µ(p)and variance σ2(p)
of the Gaussian distribution, we have tested two types of
neighborhood: a square window W(p), in which the moment
evaluation reduces to a four-term addition using integral
images [5]; a Gaussian window G(p), whose weights are
(a) Input image (b) Global thresholding (c) Local thresholding
Fig. 3. Detection by applying global or local thresholding on Ht?. (b)
False negatives are framed in yellow and false positives in red. (c) Detection
is perfect with the local threshold.
recursively evaluated. In both cases, the computation time
does not depend on the size of the window.
C. Vesicle segmentation with local threshold
Given a p-value P, the threshold τ(p)is locally calculated
as: p, τ(p) = Φ1(P)×σ2(p) + µ(p),(5)
where Φdenotes the cumulative distribution function of the
normal distribution. The function Φ1is evaluated only once,
since it does not depend on point p. Hence, the complexity
of the overall estimation and thresholding process is linear
with the image size. Thresholding the LoG-filtered image
Ht?results in a set V0of connected components, where false
detections are mostly due to noise and thus have a small
area. Hence, we discard the smallest connected components.
As said in II-A, the LoG favors rounded objects of radius
2t?. We suppose that objects of half that size are irrelevant
as well as objects of only one or two-pixel size. Therefore,
the minimum acceptable area Amin is set to:
Amin = max(2,bπt?c).(6)
The final set of vesicles Vis then:
V={vV0| |v|> Amin},(7)
where |v|denotes the area of the connected component v.
III. EXP ER IM EN TAL R ES ULT S
A. Comparative evaluation on synthetic sequences
We have compared our method with eight other unsuper-
vised detection methods evaluated in [10] and [12], namely,
WMP [9], MSVST [16], TH [4], MTH [13], HD [15], SEF
[11], IDF1and IDF2[14], and MPHD [10].
The benchmark comprises six different synthetic se-
quences introduced in [12], which involve two vesicle form
factors (round and elongated) and three types of background:
constant background (type A), background with a horizon-
tal intensity gradient (type B), and background with large
structures (type C). A Poisson noise is added, with a signal-
to-noise ratio (SNR) ranging from 1 to 5. Figure 4 depicts a
sample of each background type. Each sequence is 16-frame
long of 512 ×512 size and contains 256 vesicles per frame.
Round objects are generated as Gaussian spots of standard
deviation 2 pixels. For elongated spots, the standard deviation
is 5 pixels along the principal axis and 2 pixels along the
secondary axis. More details can be found in [12].
We report comparative results for SNR = 2, as done
in [10], [12]. Table II summarizes the true positive rates
(TPR) and modified false positive rates (FPR*) with this
benchmark configuration. FPR* is defined in [12] as
FPR* = NF P /(NT P +NF N ). The parameters involved in
(a) Type A, SNR = 3,
elongated objects
(b) Type B, SNR = 2,
elongated objects
(c) Type C, SNR = 1,
round objects
Fig. 4. Synthetic image samples.
TABLE II
COMPARISON WITH STATE-OF-TH E-A RT ME TH ODS AT SN R = 2
Object Background TPR of SLT-LoG Best TPR
shape type using W(p)using G(p)from [10], [12]
Round
Type A 0.990 0.996 0.99
Type B 0.974 0.987 0.99 (MSVST)
Type C 0.966 0.982 0.95 (SEF)
Elongated
Type A 1+(2.4×104)1+(0) 0.99
Type B 1+(9.8×104)1+(0) 0.99
Type C 0.981 0.999 0.97 (HD)
The FPR* is 0.010, except for 1+, where it is put in brackets.
each method (in our case P) were set to obtain FPR* = 0.01.
In [10], [12], an object is considered as detected if the
distance between ground truth and its estimate coordinates
is less than a threshold, arbitrarily set to 4 pixels. Since
our method supplies the entire spatial support of the vesicle,
we can evaluate it with a parameter-free criterion: a vesicle
is considered as correctly detected if the ground-truth is
included in the segmented connected component. As a matter
of fact, this criterion is tighter since here the diameter of the
extracted connected components is always less than 8 pixels.
Due to page limitation, we only report in Table II the results
of the best performer for each sequence drawn from [10],
[12]. In all cases but one, our method – denoted as SLT-LoG
(Scale-selected Local Thresholding of LoG) – outperforms
the other methods in terms of detection and false alarm rates.
For two sequences, we even obtain TPR = 1, and the FPR*
can be decreased to very low values without losing any true
positive. For most of the sequences, SLT-LoG also performs
better than the two learning methods presented in [12]
exploiting AdaBoost and Fisher discriminant analysis. For
these sequences, the performance of SLT-LoG is better using
G(p). Moreover, the sensitivity of the variance parameter is
very low: a standard deviation of 15 pixels has been chosen
for all our experiments, while we had to adapt sequence by
sequence the size of W(p)to get the best performance.
To be more exhaustive, other experiments were carried
out with different SNRs. For SNR = 1, the method per-
formance decreases, but even manually it becomes hard to
label these sequences, as shown in Fig. 4c. FROC curves
are given in Fig. 5: type A background with round objects
and type C background with elongated objects. This plot
also demonstrates the potent advantage of using a local
threshold for images presenting a complex background. For
the classical LoG method (with global threshold), we take
0!
0.2!
0.4!
0.6!
0.8!
1!
0!0.2!0.4!0.6!0.8!1!
TPR!
FPR*!
SLT-LoG with
Gaussian window!
SLT-LoG with
rectangular window!
LoG!
SLT-LoG with
Gaussian window!
SLT-LoG with
rectangular window!
LoG!
SNR 1, type A,!
round objects!
SNR 1, type C,!
elongated objects!
Fig. 5. Comparison with the classical LoG method at SNR = 1.
(a) Rab11-mCherry (b) Segmented vesicles in image (a)
(c) TfR-pHluorin, Rab11-mCherry (d) Segmented vesicles in image (c)
Fig. 6. Results on two real TIRF sequences of M10 cells.
the scale provided by the SLT-LoG method. In some cases,
W(p)behaves better than G(p). For SNR = 3 and above,
our method perfectly performs for all the sequences of the
benchmark, except for one of them (type C with round
objects at SNR = 3, using W(p)), however we get only two
false positives among 4094 true positives for that case.
For all the synthetic sequences, the execution time on a
laptop (4-core 2.3GHz CPU, 8GB 1.6GHz DDR3) is only
0.15s using W(p), or 0.25s using G(p), per 512×512 frame,
plus 3.5s for the off-line scale selection step.
B. Results on real sequences
We have applied our segmentation method to several real
TIRF microscopy sequences of M10 cell lines transfected
with different fluorescently labeled cargo protein, namely
Langerine and Transferrin receptor (TfR), as well as the
Rab11 GTPase. These proteins involved in the recycling
pathway are associated to transport intermediates (such as
vesicles) and exhibit various appearance. On Rab11 frames,
several very elongated objects are visible (as in Fig. 6a),
which cannot be accurately modeled by anisotropic Gaussian
spots. The proposed SLT-LoG method provides the entire
spatial support of the vesicles, while other methods would
only output their center coordinates or fit ellipses. Using
some beam-splitting techniques, two fluorescent markers can
be captured side-by-side on the detector, resulting in two-part
images as shown in Fig. 6c. Despite the two very different
parts of the image, the segmentation obtained with our SLT-
LoG method is very satisfactory.
IV. DISCUSSION AND CONCLUSION
We have proposed a novel and efficient vesicle segmenta-
tion method called SLT-LoG which involves an automatic
scale selection and a local threshold setting. After deter-
mining the optimal scale, a LoG operator is applied on
the images. The segmentation threshold is automatically and
locally set according to a given PFA value. Overall, SLT-LoG
outperforms state-of-the-art unsupervised methods. Except
the PFA which is in fact a detection sensitivity setting, the
only parameter of the method to be fixed is the size of
the local estimation windows W(p)or G(p). Its sensitivity
rapidly decreases when SNR increases. Our future work will
mainly focus on the automatic adaptation of the W(p)size.
This project is partially supported by R´
egion Bretagne (Brittany Council)
through a contribution to A. Basset’s Ph.D. student grant.
REFERENCES
[1] D. Axelrod. Total internal reflection fluorescence microscopy. Methods
in Cell Biology, 89:169–221, 2008.
[2] J. Boulanger, A. Gidon, C. Kervrann, and J. Salamero. A patch-based
method for repetitive and transcient event detection in fluorescence
imaging. PLoS One, 5(10):e13190, Oct. 2010.
[3] J. Boulanger, C. Kervrann, P. Bouthemy, P. Elbau, J.-B. Sibarita,
and J. Salamero. Patch-based non-local functional for denoising
fluorescence microscopy image sequences. IEEE Trans. Medical
Imaging, 29(2):442–453, Feb. 2010.
[4] D. S. Bright and E. B. Steel. Two-dimensional top hat filter for
extracting spots and spheres from digital images. J. Microscopy,
146(2):191–200, May 1987.
[5] F. C. Crow. Summed-area tables for texture mapping. ACM SIG-
GRAPH Comp. Graphics, 18(3):207–212, Jul. 1984.
[6] T. Lindeberg. Scale-space for discrete signals. IEEE Trans. Pattern
Analysis and Machine Intelligence, 12(3):234–254, Mar. 1990.
[7] T. Lindeberg. Feature detection with automatic scale selection. Int. J.
Comp. Vision, 30(2):79–116, Nov. 1998.
[8] E. Meijering, O. Dzyubachyk, and I. Smal. Methods for cell and
particle tracking. Elsevier, 2012.
[9] J.-C. Olivo-Marin. Extraction of spots in biological images using
multiscale products. Pattern Recog., 35(9):1989–1996, Sept. 2002.
[10] S. H. Rezatofighi, R. Hartley, and W. E. Hughes. A new approach for
spot detection in total internal reflection fluorescence microscopy. In
IEEE Int. Symp. Biomedical Imaging, ISBI’12, Barcelona, May 2012.
[11] D. Sage, F.R. Neumann, F. Hediger, S.M. Gasser, and M. Unser.
Automatic tracking of individual fluorescence particles: Application
to the study of chromosome dynamics. IEEE Trans. Image Process.,
14(9):1372–1383, Sep. 2005.
[12] I. Smal, M. Loog, W. J. Niessen, and E. H. W. Meijering. Quantitative
comparison of spot detection methods in fluorescence microscopy.
IEEE Trans. Medical Imaging, 29(2):282–301, Feb. 2010.
[13] P. Soille. Morphological image analysis: Principles and applications.
Springer, 2003.
[14] B. M. ter Haar Romeny. Front-end vision and multi-scale image
analysis. Computational Imaging and Vision. Springer, 2003.
[15] L. Vincent. Morphological grayscale reconstruction in image analysis:
Applications and efficient algorithms. IEEE Trans. Image Process.,
2(2):176–201, Apr. 1993.
[16] B. Zhang, M. J. Fadili, J.-L. Starck, and J.-C. Olivo-Marin. Multiscale
variance-stabilizing transform for mixed-Poisson-Gaussian processes
and its applications in bioimaging. In IEEE Int. Conf. Image Process.,
ICIP’07, San Antonio, Oct. 2007.
... La plupart de ces méthodes comprennent plusieurs paramètres devant être finement réglés pour obtenir de bons résultats [9,12]. Afin de réduire le nombre et la sensibilité des paramètres, nous avons développé une méthode de segmentation de vésicules avec sélection d'échelle automatique dénommée SLT-LoG et décrite dans [1]. Elle s'appuie sur un filtre laplacien de gaussienne, dont la variance est automatiquement sélectionnée dans un ensemble fini de valeurs prédéfinies. ...
... Sous l'hypothèse d'une distribution localement gaussienne des intensités de l'image, le seuil est déduit d'une probabilité de fausse alarme (PFA), choisie par l'utilisateur. Dans [1], la sélection d'échelle repose sur un filtre gaussien échantillonné. Lindeberg [6] a démontré que, dans le cadre des espaces multi-échelles, le rapport entre deux échelles successives de ce filtre doit être un entier impair. ...
... Lindeberg [6] a démontré que, dans le cadre des espaces multi-échelles, le rapport entre deux échelles successives de ce filtre doit être un entier impair. Un rapport de 3 avait ainsi été choisi dans [1]. Ceci se traduit par une estimation grossière de l'échelle, provoquant des erreurs de segmentation, lorsque de petits éléments sont très proches les uns des autres dans l'image. ...
... in the neighborhood of membrane events detected in the TIRFM sequence ofFig. 1. LoG we introduced in [15] for static images. It is based on the Laplacian of Gaussian (LoG) operator and proceeds in two steps. ...
... It is based on the Laplacian of Gaussian (LoG) operator and proceeds in two steps. First, the scale of the vesicles s * is automatically selected in a multiscale representation of the images as explained in [15]. To determine it, we use the first ten frames of the input sequence f , as it contains more spots than one frame of ∆. ...
... Secondly, appearing spots related to a fusion event are detected by thresholding the LoG of scale s * of every ∆(t). As described in [15], the threshold automatically adapts to local LoG statistics estimated in a sliding Gaussian window, whose size is not critical. Its radius is set to 60 px, which is a trade-off on background structure sizes in the processed images. ...
Conference Paper
Full-text available
Assessing the dynamics of plasma membrane diffusion processes in live cell fluorescence microscopy is of paramount interest to understand cell mechanisms. We propose a new method to detect vesicle fusion events, and estimate the associated diffusion coefficients in image sequences of total internal reflection fluorescence microscopy (TIRFM). In contrast to usual approaches, a diffusion coefficient is locally estimated for each detected fusing vesicle. We first detect the membrane fusion events and then select the diffusion configurations among them with a correlation test. To estimate the diffusion coefficient, a geometric model is fitted to the detected spot directly in the 2D+t subvolume. Quantitative results demonstrate the accuracy of the proposed method.
... Automatic selection of the detection scale is a challenging problem since the objects of interest may have different sizes or they may have the same size as the irrelevant objects in the background. Few methods of automatic scale selection [32][33][34] have been proposed recently. However, in the context of tissue microarrays, the diameter of assembled TMA cores is defined by the size of the needle used for extracting cores from paraffin tissue blocks. ...
... While the wavelet decomposition plays the role of a filtering which reduces the noise and enhances the objects of interest, a common way to detect objects is to threshold the filtered image -the wavelet decomposition of the input TMA image in our case. As depicted in [32], a global threshold is not appropriate to handle complex situations, especially when dealing with images acquired in fluorescence context because of their inhomogeneous background. To overcome this difficulty, we propose to define an adaptive threshold according to the local distribution of the wavelet decomposition ĵ u previously computed. ...
Article
Full-text available
Background: Over the last two decades, an innovative technology called Tissue Microarray (TMA), which combines multi-tissue and DNA microarray concepts, has been widely used in the field of histology. It consists of a collection of several (up to 1000 or more) tissue samples that are assembled onto a single support - typically a glass slide - according to a design grid (array) layout, in order to allow multiplex analysis by treating numerous samples under identical and standardized conditions. However, during the TMA manufacturing process, the sample positions can be highly distorted from the design grid due to the imprecision when assembling tissue samples and the deformation of the embedding waxes. Consequently, these distortions may lead to severe errors of (histological) assay results when the sample identities are mismatched between the design and its manufactured output. The development of a robust method for de-arraying TMA, which localizes and matches TMA samples with their design grid, is therefore crucial to overcome the bottleneck of this prominent technology. Results: In this paper, we propose an Automatic, fast and robust TMA De-arraying (ATMAD) approach dedicated to images acquired with brightfield and fluorescence microscopes (or scanners). First, tissue samples are localized in the large image by applying a locally adaptive thresholding on the isotropic wavelet transform of the input TMA image. To reduce false detections, a parametric shape model is considered for segmenting ellipse-shaped objects at each detected position. Segmented objects that do not meet the size and the roundness criteria are discarded from the list of tissue samples before being matched with the design grid. Sample matching is performed by estimating the TMA grid deformation under the thin-plate model. Finally, thanks to the estimated deformation, the true tissue samples that were preliminary rejected in the early image processing step are recognized by running a second segmentation step. Conclusions: We developed a novel de-arraying approach for TMA analysis. By combining wavelet-based detection, active contour segmentation, and thin-plate spline interpolation, our approach is able to handle TMA images with high dynamic, poor signal-to-noise ratio, complex background and non-linear deformation of TMA grid. In addition, the deformation estimation produces quantitative information to asset the manufacturing quality of TMAs.
... Automatic selection of the detection scale is a challenging problem since the objects of interest may have different sizes or they may have the same size as the irrelevant objects in the background. Few methods of automatic scale selection [Basset et al. 2014;Püspöki et al. 2015Püspöki et al. , 2016 have been proposed recently. However, in the context of tissue microarrays, the diameter of assembled TMA cores is defined by the size of the needle used for extracting cores from paraffin tissue blocks. ...
... While the wavelet decomposition plays the role of a filtering which reduces the noise and enhances the objects of interest, a common way to detect objects is to threshold the filtered image -the wavelet decomposition of the input TMA image in our case. As depicted in [Basset et al. 2014], a global threshold is not appropriate to handle complex situations, especially when dealing with images acquired in fluorescence context because of their inhomogeneous background. To overcome this difficulty, we propose to define an adaptive threshold according to the local distribution of the wavelet decomposition Ψ u previously computed. ...
Thesis
This thesis aims at developing dedicated methods for quantitative analysis of Tissue Microarray (TMA) images acquired by fluorescence scanners.We addressed these issues in biomedical image processing, including segmentation of objects of interest (i.e. tissue samples), correction of acquisition artifacts during the scanning process and improvement of acquired image resolution while taking into account imaging modality and scanner design.The developed algorithms allow envisaging a novel automated platform for TMA analysis, which is highly required in cancer research nowadays.On a TMA slide, multiple tissue samples which are collected from different donors are assembled according to a grid structure to facilitate their identification.In order to establish the link between each sample and its corresponding clinical data, we are not only interested in the localization of these samples but also in the computation of their array (row and column) coordinates according to the design grid because the latter is often very deformed during the manufacturing of TMA slides.However, instead of directly computing array coordinates as the existing approaches, we proposed to reformulate this problem as the approximation of the deformation of the theoretical TMA grid using ``thin plate splines'' given the result of tissue sample localization.We combined a wavelet-based detection and an ellipse-based segmentation to eliminate false alarms and thus improving the localization result of tissue samples.According to the scanner design, images are acquired pixel by pixel along each line, with a change of scan direction between two subsequent lines. Such scanning system often suffers from pixel mis-positioning (jitter) due to imperfect synchronization of mechanical and electronic components. To correct these scanning artifacts, we proposed a variational method based on the estimation of pixel displacements on subsequent lines.This method, inspired from optical flow methods, consists in estimating a dense displacement field by minimizing an energy function composed of a nonconvex data fidelity term and a convex regularization term. We used the half-quadratic splitting technique to decouple the original problem into two small sub-problems: one is convex and can be solved by a standard optimization algorithm, the other is non-convex but can be solved by a complete search. To improve the resolution of acquired fluorescence images, we introduced a method of image deconvolution by considering a family of convex regularizers.The considered regularizers are generalized from the concept of Sparse Variation which combines the $L_1$ norm and Total Variation (TV) to favors the co-localization of high-intensity pixels and high-magnitude gradient. The experiments showed that the proposed regularization approach produces competitive deconvolution results on fluorescence images, compared to those obtained with other approaches such as TV or the Schatten norm of Hessian matrix.
... However, since the selected threshold with respect to the mean and variance of the image may be inaccurate, it may not perform well on the micro-droplet detection. Besides, Basset et al. [18][19][20] proposed methods to select the optimal LoG scale or multiple scales corresponding to the different spot sizes in the image, but test results on fluorescent micro-droplet images proved the ineffectiveness of this method for the micro-droplet detection. As explained by Smal et al. [12], most current methods follow a common detection scheme, which consists of denoising the image, enhancing the spots and, finally, extracting the target spots in a binary map to further count the micro-droplets or estimate the positions. ...
Article
Full-text available
This paper developed and evaluated a quantitative image analysis method to measure the concentration of the nanoparticles on which alkaline phosphatase (AP) was immobilized. These AP-labeled nanoparticles are widely used as signal markers for tagging biomolecules at nanometer and sub-nanometer scales. The AP-labeled nanoparticle concentration measurement can then be directly used to quantitatively analyze the biomolecular concentration. Micro-droplets are mono-dispersed micro-reactors that can be used to encapsulate and detect AP-labeled nanoparticles. Micro-droplets include both empty micro-droplets and fluorescent micro-droplets, while fluorescent micro-droplets are generated from the fluorescence reaction between the APs adhering to a single nanoparticle and corresponding fluorogenic substrates within droplets. By detecting micro-droplets and calculating the proportion of fluorescent micro-droplets to the overall micro-droplets, we can calculate the AP-labeled nanoparticle concentration. The proposed micro-droplet detection method includes the following steps: (1) Gaussian filtering to remove the noise of overall fluorescent targets, (2) a contrast-limited, adaptive histogram equalization processing to enhance the contrast of weakly luminescent micro-droplets, (3) an red maximizing inter-class variance thresholding method (OTSU) to segment the enhanced image for getting the binary map of the overall micro-droplets, (4) a circular Hough transform (CHT) method to detect overall micro-droplets and (5) an intensity-mean-based thresholding segmentation method to extract the fluorescent micro-droplets. The experimental results of fluorescent micro-droplet images show that the average accuracy of our micro-droplet detection method is 0.9586; the average true positive rate is 0.9502; and the average false positive rate is 0.0073. The detection method can be successfully applied to measure AP-labeled nanoparticle concentration in fluorescence microscopy.
... Object detection can be performed by using any robust algorithms (e.g. see [19][20][21][22]). We have considered the method #10 in [23] based on structure tensors [24] and an optimal histogram based thresholding [25] since this combination of algorithms was able to reliably Given the initial set of objects, the trajectory x i,1: 1 , · · · , σ i,K } and the parameters θ i of object i are individually estimated using an iterative procedure described in the next section. ...
Article
Full-text available
Fluorescence lifetime is usually defined as the average nanosecond-scale delay between excitation and emission of fluorescence. It has been established that lifetime measurement yields numerous indications on cellular processes such as inter-protein and intra-protein mechanisms through fluorescent tagging and Förster resonance energy transfer (FRET). In this area, frequency domain fluorescence lifetime imaging microscopy (FD FLIM) is particularly well appropriate to probe a sample non-invasively and quantify these interactions in living cells. The aim is then to measure fluorescence lifetime in the sample at each location in space from fluorescence variations observed in a temporal sequence of images obtained by phase modulation of the detection signal. This leads to a sensitivity of lifetime determination to other sources of fluorescence variations such as intracellular motion. In this paper, we propose a robust statistical method for lifetime estimation on both background and small moving structures with a focus on intracellular vesicle trafficking.
Conference Paper
A model of two-type (or two-color) interacting random balls is introduced. Each colored random set is a union of random balls and the interaction relies on the volume of the intersection between the two random sets. This model is motivated by the detection and quantification of co-localization between two proteins. Simulation and inference are discussed. Since all individual balls cannot been identified, e.g. a ball may contain another one, standard methods of inference as likelihood or pseudolikelihood are not available and we apply the Takacs-Fiksel method with a specific choice of test functions.
Article
Microscopy imaging, including fluorescence microscopy and electron microscopy, has taken a prominent role in life science research and medicine due to its ability to investigate the 3D interior of live cells and organisms. A long-term research in bio-imaging at the sub-cellular and cellular scales consists then in inferring the relationships between the dynamics of macromolecules and their functions. In this area, image processing and analysis methods are now essential to understand the dynamic organization of groups of interacting molecules inside molecular machineries and to address issues in fundamental biology driven by advances in molecular biology, optics and technology. In this paper, we present recent advances in fluorescence and electron microscopy and we focus on dedicated image processing and analysis methods required to quantify phenotypes for a limited number but typical studies in cell imaging.
Article
Tissue core de-arraying is one of the most important steps in tissue microarray (TMA) image analysis. However, few solutions and mathematical frameworks are available. This paper presents a robust TMA de-arraying method adapted for digital images from classical optical and new fluorescent devices. The proposed algorithm is composed of three modules: (a) detection, (b) segmentation, and (c) array indexing. The detection of TMA cores is performed by local adaptive thresholding of isotropic wavelet transform coefficients. The segmentation component uses parametric ellipse to delineate the boundaries of potential tissue cores. Array indices of each core are computed by using thin-plate splines to estimate the deformation of the deposited core grid. Our method is appropriate for non-linear deformation and is able to quantify the deformation of TMA grids when compared to existing algorithms.
Conference Paper
Full-text available
Biological images acquired from fluorescence microscopy-based imaging techniques, such as total internal reflection fluorescence microscopy, generally contain hundreds of subcellular components, appearing in the images as bright spots. Therefore, the first step of analysis of these images usually involves the detection of these bright spots. In this paper, we propose a new approach for spot detection using the h-dome transform along with an adaptive mask obtained from regional information. Moreover, local gradient information is used in order to distinguish the spots from other structures. To evaluate the performance of our algorithm, we test it on synthetic images and also real TIRFM images and compare our results with those of two recent methods.
Article
Biological images acquired from fluorescence microscopy-based imaging techniques, such as total internal reflection fluorescence microscopy, generally contain hundreds of subcellular components, appearing in the images as bright spots. Therefore, the first step of analysis of these images usually involves the detection of these bright spots. In this paper, we propose a new approach for spot detection using the h-dome transform along with an adaptive mask obtained from regional information. Moreover, local gradient information is used in order to distinguish the spots from other structures. To evaluate the performance of our algorithm, we test it on synthetic images and also real TIRFM images and compare our results with those of two recent methods.
Article
Texture-map computations can be made tractable through use of precalculated tables which allow computational costs independent of the texture density. The first example of this technique, the “mip” map, uses a set of tables containing successively lower-resolution representations filtered down from the discrete texture function. An alternative method using a single table of values representing the integral over the texture function rather than the function itself may yield superior results at similar cost. The necessary algorithms to support the new technique are explained. Finally, the cost and performance of the new technique is compared to previous techniques.
Article
The top hat filter is a computer algorithm that extracts small, compact or rounded objects from digital images. Examples show application of the filter to micrographs and electron diffraction patterns.