Adaptive Multirate Data Acquisition of 3D Cell Images

Abstract

Not Available

Full text

Page 1

ADAPTIVE MULTIRATE DATA ACQUISITION OF 3D CELL IMAGES
T.E. Merryman1, J. Kovaˇ cevi´ c1,2, E. Garcia Osuna2and R.F. Murphy2,3
Carnegie Mellon University, Center for Bioimage Informatics
Departments of Electrical and Computer Engineering1, Biomedical Engineering2and Biological Sciences3
5000 Forbes Ave., Pittsburgh, PA 15213
<tadm@cmu.edu, jelenak@cmu.edu, eog@andrew.cmu.edu, murphy@cmu.edu>
ABSTRACT
We present an algorithm for efficient acquisition of fluorescence
microscopy data sets, a problem not addressed until now in the
literature. We do this as part of a larger system for protein classifi-
cation based on their subcellular location patterns, and thus strive
to maintain the achieved level of classification accuracy as much
as possible. This problem is similar to image compression but
unique due to additional restrictions, namely causality; we have
access only to the information that has been scanned up to that
point. While we do want to acquire fewer samples with as low
distortion as possible to achieve compression, our goal is to do so
while affecting the overall classification accuracy as little as possi-
ble. We achieve this by using an adaptive multiresolution scanning
scheme which samples the regions of the image area that hold the
most pertinent information. Our results show that we can achieve
significant compression which we can then use to increase either
time of space resolution of our data set, all while minimally affect-
ing the classification accuracy of the entire system.
1. INTRODUCTION
The motivation for finding an efficient way of acquiring cellular
images is great. In fluorescence microscopy, the confocal scan-
ning microscope is one of the most often used and it scans the
field pixel by pixel [1]. Photo-bleaching occurs each time the laser
used to excite the region being imaged focuses on a pixel. The
exposure time and laser intensity both play a major part in photo-
bleaching. By reducing the number of samples (maximum resolu-
tion locations) we need to scan, we save time which could be used
to speed up the acquisition process or in tradeoffs to reduce the
laser intensity (reduce the effects of photo-bleaching) and increase
the resolution in other dimensions [2]. One could reduce the num-
ber of samples acquired by using traditional sampling algorithms
but effects of aliasing would distort the approximation images [3].
We aim here for an adaptive, efficient algorithm that would scan
fewer pixel locations, while limiting the distortion and maintaining
as much information as possible about the distribution of fluores-
cence in the sample. This requires some means of evaluation, and
for this we build on prior work demonstrating that machine clas-
sifiers can recognize all mojor subcellular patterns in 3D images
of cultured cells with high accuracy [4, 5, 6]. We can thus com-
pare the classification accuracy for adaptively sampled images to
that for the original images to assess the degree of preservation of
image information content.
This work was supported in part by NSF-ITR grant EF-0331657.
The previoius work in this area focused on the process of re-
coveryratherthandifferentapproachesofdataacquisition[2,8,9].
Trying to efficiently acquire cellular images using adaptive sam-
pling methods is new.
We will work with 3D images (cell volumes) that have the
maximum resolution in each of the three dimensions. We will run
our multirate data acquisition algorithm using different input pa-
rameters on each of these data sets to simulate the acquisition from
the microscope and build a synthesized data set of adaptively sam-
pled images. We will then compare these images to the original
images and examine their rate-distortion curves to find where the
algorithm works optimally. We will do the same with standard
downsampling using bilinear interpolation. This rate-distortion
measure will give a general insight to the performance of the al-
gorithm. Although the rate-distortion gives us valuable informa-
tion, our goal is to eventually reduce the number of samples while
maintaining the classification accuracy of the system. We will thus
measure the compression ratio with respect to the classification ac-
curacy.
2. MOLECULAR IMAGING
Fig. 1. Image of a mitochondrial protein pattern.
Fig. 1 points to the properties of the data sets we will be work-
ing with. Observe how the image is predominantly dark with
mostly low pixel intensities. This observation leads us directly to
the most important assumption: High frequencies (rapid changes
in pixel intensities among neighboring pixels) will only occur in
a small percentage of the area contained within the image bound-
aries.
II - 1330-7803-8874-7/05/$20.00 ©2005 IEEEICASSP 2005
«¬

Page 2

Fig. 2. Images from a 3D sequence of actin protein pattern.
Fig. 2 shows that neighboring slices share similar patterns and
locations in the 3D image of actin.
With these observations in mind, we can make a strong argu-
ment for a multirate approach. When standard downsampling is
used and then interpolated back to the maximum resolution using
bilinear interpolation, all of the high-frequency content is not only
lost but is misrepresented as a result of aliasing. In the dark re-
gions of the images, only low frequencies are present. These low
frequencies can be captured with few samples. In contrast, areas
where high-intensity values are located, there are high frequencies
that need to be represented using a greater number of samples.
Using different sampling rates in acquiring the data set is called
multirate sampling. The images of subcellular protein locations
we examine in this work do not share the same location patterns
and thus the distribution of frequencies across the images varies.
This is the reason for an adaptive multirate approach.
3. MULTIRATE ACQUISITION OF CELLULAR IMAGES
The block diagram of our adaptive multirate algorithm is shown
in Fig. 3 and repeated for each slice of the 3D image. This is
the major component of the overall system that constructs an ap-
proximation to the original cellular data set. We now explain the
algorithm in more detail.
A cell to be scanned is placed on a confocal scanning micro-
scope. In the experiments we completed, we ran our algorithm on
imagesscannedbythemicroscopeatmaximumresolution[4]. Us-
ing these images we created a model for guiding the microscope’s
scanning protocol to implement our adaptive multirate sampling
algorithm.
Cellular field
Initialize probe locations
(method ’a’ or method ’b’)
Probe
Examine intensity
of probed location
Examine number
of probe locations
MR sampled image
Add probe
locations
Intensity greater
than threshold
Intensity less
than threshold
No probe locations remain
Probe locations remain
Fig. 3. Block diagram for the adaptive multirate sampling algo-
rithm.
Probe locations are initialized in one of two ways: The first is
to set up probe locations throughout the entire cellular field rect-
Fig.4. Thisgridgivesagraphicviewofhowthesamplingorprobe
locations are initialized. Each block represents a possible probe
location and the black blocks represent where a probe location has
been initialized.
angularaly at 8 or 16 primary units apart (primary unit refers to
the basic element of the maximum resolution). This can be seen in
Fig. 4. The grid in the figure represents all of the possible spaces
where samples can be drawn from the cellular field or in our sim-
ulation from the original maximum resolution image. Initializing
the probes in this fashion begins the sampling process assuming
that most of the area contained within the boundaries of the pho-
tographable region hold only low frequencies. The second way
in which the probe locations are initialized uses cellular location
knowledge that has already been acquired from previous slices in
the 3D sequence. Fig. 2 shows that cell locations in adjacent slices
are very similar. When stepping through the sampling process, it
is efficient to rule out areas unlikely to contain any pertinent infor-
mation. Fig. 5 shows how to initialize the probe locations if we can
guess where the cell protein structure is located. We will elaborate
on this in what follows.
These twomethodsforinitializingthe probe locationsare used
in different situations: the first method when there is no informa-
tion available about the location of the cell protein structure, while
the second one is used when information is available. To add more
robustness to the system, the first method can be used periodically
even when knowledge about the location is known.
When there are locations remaining to be probed, the micro-
scope will move to these locations and record the intensity of the
light emitted from the cellular field when exposed to the laser.
Probing occurs until there are no more locations left to be exam-
ined. In our simulation, a probe is represented by taking a sample
from the maximum resolution original image at a specific pixel
location.
Afteralocationisprobedandanintensityvalueisreturneditis
compared to a threshold. The threshold is set by the user and deter-
mines the sensitivity of the algorithm. A lower threshold will raise
the sensitivity and take more samples while a higher threshold will
regard more locations to be uninteresting. When a pixel’s intensity
is examined one of two things occurs: If the value is greater than
thethreshold, thenthevalueisstoredinwhatiscalledaforeground
image. The foreground image is an image that contains all of the
intensity values gathered from probes where the intensity exceeds
the threshold. After this value is stored, new probes are added to
the grid. The initial distance between probe locations is 8. New
areas to scan will be added as shown in Fig. 6. Probes are added
in the extended neighborhood of this pixel. The extended neigh-
II - 134
¬¬

Page 3

Fig.5. Thisgridgivesagraphicviewofhowthesamplingorprobe
locations are initialized. Each block represents a possible probe
location. Black blocks represent where a probe location has been
initialized. The oblique blob represents the assumed area where
the valuable cellular information is located. The probe locations
are only set up in the area where the cell is believed to be located.
borhood is (1.5) × (units away from current unit) from the current
location. In the figure, the distance between probe locations (black
pixels) is 8 units. In part (a), the gray pixels represent the probe lo-
cations added when the circled probe location reveals a value that
is greater than or equal to the threshold. This is a recursive process
and its next step is shown in part (b) of the same figure.
If the value returned from the probe at a certain location does
not exceed the threshold, then that value is also stored in what is
called the background image. The background image consists of
all the probed locations that are smaller than the threshold. When
all of the probe locations have been examined, the background
image is interpolated to the size of the original image. When all
probes are completed, the approximated image is constructed. The
approximated image is set equal to the background image where
the foreground image is without an intensity value.
(a)(b)
Fig. 6. (a) This grid shows the process for adding probe locations
(in gray). when probed value is greater than the threshold. (b) This
shows the recursive nature of the same process when a new probe
location exceeds the threshold and more probe locations are added
at a finer resolution.
To create a more efficient algorithm and to save even more
probes, we can use the information about the cell protein structure
location from the previous slice that has been approximated. This
is done by creating a mask. The mask examines the foreground
image of the neighboring slice. We convert the foreground image
into a binary image where 1 signifies where the foreground image
has positive intensities and 0 signifies no intensity. This image is
expanded by the morphological operation of dilation. This cre-
ates a mask that is larger than the area where the structure was
located in the previous slice and compensates for any growth or
movement of the cell. In the subsequent slice, we will only ini-
tialize the probe locations that lie in the region where the mask is
located. For 2D cellular images, the masking step is omitted. With
those images, the algorithm never assumes to have knowledge of
the protein location and always intializes the probe locations using
the first method of initialization described earlier in this section.
4. EXPERIMENTAL RESULTS
We synthesized the cellular field by using four sequences of im-
ages. Each of the sequences has 1024×1280×30 resolution. After
processingeachofthesequenceswecomparedthemtotheoriginal
sequences. We found the mean-squared error for different thresh-
olds which affect the number of samples taken. We also repeated
the process with standard downsampling and bilinear interpola-
tion and compared the results. Table 1 gives the optimal compres-
sion ratio and MSE for the four 3D data sets we tested. Note that
the compression ratios are close but the distortion is higher for
the standard downsampled approximations. Mean squared error is
given by:
MSE =
1
MNI
M
?
m=0
N
?
n=0
I
?
i=0
(Xmni− Ymni)2,
where X is the original sequence and Y is the approximated se-
quence whose values range from 0 to 255. M, N and I are param-
eters that describe the dimensions of the image.
DataMultirate
Compression
Ratio
34:1
56:1
122:1
113:1
Multirate
MSE
Standard
Comression
Ratio
36:1
49:1
121:1
100:1
Standard
MSE
Actin
Mito
Nucleus 1
Nucleus 2
14
4
14
13
18
19
22
30
Table 1. Compression ratios and MSE for four 3D data sets. The
results are given for the multirate algorithm and the standard algo-
rithm.
We next evaluated classification accuracy for images acquired
with the downsampling methods. We began by running the multi-
rate and standard downsampling algorithms on a bank of 500 2D
images of size 512×512 [7]. These images represent 10 distinct
classes (proteins). We then calculated the features based on SLF-
5 minus the DNA features for each of the approximated images
and used a Support Vector Machine (SVM) to classify[10]. Fig. 7
shows that the accuracy decline with standard downsampling is
more drastic than that of the multirate downsampling. Table 2(a)
gives the accuracy of the original images. Table 2(b) shows the
same detailed accuracy for multirate downsampling with a com-
pression ratio of 9:1. Table 2(c) shows the accuracy result with a
compressionratioof4:1. Notethat theoverallaccuracyisidentical
forparts(b)and(c); however, thestandarddownsamplingachieves
II - 135
¬¬

Page 4

this measure with more than twice the number of samples used by
the multirate approach.
24
Compression Ratio
68
78
80
82
84
86
Accuracy [%]
Multirate Algorithm
Standard Algorithm
Original (No Sampling)
Fig. 7. Classification accuracy of the original system (no sam-
pling), multirate algorithm, and standard algorithm.
5. CONCLUSIONS
We presented a scheme for adaptive multirate acquisition of 3D
cell images. We found that the intelligent acquisition of mul-
tirate downsampling outperforms standard downsampling. In a
rate-distortion sense, it out performs because it retains the high
frequencies and saves samples where low frequencies are present.
In a classification accuracy sense, the multirate algorithm also out-
performs standard downsampling as it retains a better classifica-
tion accuracy for higher compression rates than standard down-
sampling.
6. ACKNOWLDGEMENTS
We would like to thank Prof. Tsuhan Chen for valuable comments.
We would also like to thank Ting Zhao for the use of his feature
calculation code and SVM classifier code.
7. REFERENCES
[1] C. J. Cogswell, K. Carlsson (1994). “Three-dimensional mi-
croscopy: image acquisition and processing.” SPIE. Belling-
ham, Washington, USA.
[2] J. C. Bulinski, D. J. Odde, Bo. J. Howell, T. D. Salmon and
C. M. Waterman-Storer (2001). “Photobleaching and recov-
ery of ensconsin chimera in untreated cells. Rapid dynamics
of the microtubule binding of ensconsin in vivo.” Journal of
Cell Science. 114:3885-3897.
[3] M. Kostic (1999). “The art of signal sampling and alias-
ing: simulation with a LabVIEW virtual instrument –
what we see is not what it is!” NIWeek 99 Annual
Conference, National Instruments, Austin, TX, 1999.
www.kostic.niu.edu/NIWEEK99.htm
[4] M. Velliste, R.F. Murphy (2002). “Automated determination
of protein subcellular locations from 3D fluorescence micro-
scope images.” ” Proc 2002 IEEE Intl Symp Biomed Imaging
(ISBI 2002), pp. 867-870.
Original Image Accuracy (no sampling)
Output of Classifier
L2
0
2
0
6
80
0
4
0
18
2
Input
DNA
ER
Gia
Gpp
L2
Nuc
Mit
Act
Tfr
Tub
DNA
94
2
0
0
0
0
2
0
2
0
ER
2
82
0
0
0
2
0
0
2
8
Gia
0
0
84
8
0
0
0
0
0
0
Average accuracy: 82%
Accuracy with Multirate Algorithm
Gpp
0
0
10
80
2
0
0
0
0
0
Mit
0
2
0
0
0
82
0
0
8
8
Nuc
0
0
4
2
0
0
92
0
0
0
Act
0
0
0
0
0
0
0
100
2
0
TfR
4
0
2
2
16
6
2
0
56
12
Tub
0
12
0
2
2
10
0
0
12
70
(a)
Output of Classifier
L2
2
6
6
10
86
2
4
0
18
4
Input
DNA
ER
Gia
Gpp
L2
Nuc
Mit
Act
Tfr
Tub
DNA
82
4
0
0
0
0
4
0
2
0
ER
2
76
0
0
2
4
0
0
2
8
Gia
0
0
84
8
0
0
2
0
0
0
Gpp
0
0
4
76
2
0
0
0
2
2
Mit
0
0
0
0
0
78
0
0
4
4
Nuc
2
0
4
2
0
0
88
0
0
0
Act
0
0
0
0
0
0
0
100
2
0
TfR
12
2
2
2
10
6
2
0
54
6
Tub
0
12
0
2
2
10
0
0
16
76
(b)
Average accuracy: 80%, Compression Ratio: 9.8:1
Accuracy with Standard Algorithm
Output of Classifier
L2
0
0
0
6
78
2
2
0
16
2
Input
DNA
ER
Gia
Gpp
L2
Nuc
Mit
Act
Tfr
Tub
DNA
92
2
0
0
0
0
4
0
2
0
Average accuracy: 80%, Compression Ratio: 4:1
ER
2
88
0
0
4
6
0
0
2
8
Gia
0
0
82
8
0
0
2
0
0
0
Gpp
0
0
12
82
2
0
0
0
0
0
Mit
0
0
0
0
0
74
0
0
8
12
Nuc
0
0
4
0
0
0
90
0
0
0
Act
0
0
0
0
0
0
0
98
6
0
TfR
4
0
2
2
16
10
2
2
52
14
Tub
2
10
0
2
0
8
0
0
14
64
(c)
Table 2. Detailed classification accuracy for (a) no sampling, (b)
multirate sampling algorithm and (c) standard downsampling al-
gorithm.
[5] K. Huang, R. F. Murphy (2004). “Automated classification
of subcellular patterns in multicell images without segmen-
tation into single cells.” Proc 2004 IEEE Intl Symp Biomed
Imaging (ISBI 2004), pp. 1139-1142.
[6] X. Chen, R. F. Murphy (2004). “Robust classification of sub-
cellular location patterns in high resolution 3D fluorescence
microscope images.” Proceedings of the 26th Annual Inter-
national Conference of the IEEE Engineering in Medicine
and Biology Society, pp. 1632-1635.
[7] M. V. Boland, R. F. Murphy (2001). “A neural network clas-
sifier capable of recognizing the patterns of all major subcel-
lular structures in fluorescence microscope images of HeLa
cells.” Bioinformatics 17: pp. 1213-1223.
[8] K. Gonda, J. Fowler, N. Katoku-Kikyo, J. Haroldson J,
Wudel J, Kikyo N.(2003). “Reversible disassembly of so-
matic nucleoli by the germ cell proteins FRGY2a and
FRGY2b.” Nature Cell Biology 5, 205-210.
[9] E.A.J. Reits, J.J. Neefjes (2001). “From fixed to FRAP: mea-
suring protein mobility and activity in living cells.” Nature
Cell Biology 3, E145-147.
[10] K. Huang, M. Velliste, R. F. Murphy (2003). “Feature reduc-
tion for improved recognition of subcellular location patterns
in fluorescence microscope images.” Proc. SPIE 4962, 307-
318.
II - 136
¬«