Application of Renyi entropy for ultrasonic molecular imaging
M. S. Hughes, J. N. Marsh, J. M. Arbeit, R. G. Neumann, R. W. Fuhrhop, K. D. Wallace,
L. Thomas, J. Smith, K. Agyem, G. M. Lanza, and S. A. Wickline
Department of Medicine, Cardiovascular Division, Washington University School of Medicine,
Campus Box 8086, 660 South Euclide Avenue, St. Louis, Missouri 63110-1093
J. E. McCarthy
Department of Mathematics, Washington University in St. Louis, Cupples I Hall, One Brookings Drive,
St. Louis, Missouri 63130
?Received 18 November 2008; revised 12 February 2009; accepted 13 February 2009?
Previous work has demonstrated that a signal receiver based on a limiting form of the Shannon
entropy is, in certain settings, more sensitive to subtle changes in scattering architecture than
conventional energy-based signal receivers ?M. S. Hughes et al., J. Acoust. Soc. Am. 121, 3542–
3557 ?2007??. In this paper new results are presented demonstrating further improvements in
sensitivity using a signal receiver based on the Renyi entropy.
© 2009 Acoustical Society of America. ?DOI: 10.1121/1.3097489?
PACS number?s?: 43.60.Bf, 43.60.Lq, 43.60.Cg ?EJS?
In an earlier paper we reported on the comparison be-
tween a Shannon entropy analog, Hf, and more conventional
signal processing techniques, i.e., signal energy and its loga-
rithm as applied to beam formed radiofrequency ?rf? data.
Both analysis techniques were applied to data obtained in
neovasculature.1The comparison study was undertaken after
a preliminary conventional B-mode grayscale analysis of the
data was unable to detect changes in backscattered rf arising
from the accumulation of targeted nanoparticles in the
neovasculature in the insonified region. This result implied
that acoustic characterization of sparse collections of tar-
geted perfluorocarbon nanoparticles presented challenges
that might require the application of novel types of signal
processing. We were able to show that signal processing
based on a “moving window” Hfanalysis could distinguish
the difference in backscatter measured at 15 and 60 min and
?although it was not stressed? able to detect accumulation of
targeted nanoparticles 30 min post-injection. The signal en-
ergy, defined as the sum of squares of the signal amplitude
over the same moving window, was unable to distinguish
measurements made at any time during the 1 h experiment.
We stress that, although entropy-based techniques have a
long history for image enhancement and postprocessing of
reconstructed images, the approach we have taken in previ-
ous studies is different in that entropy is used directly as the
quantity defining the pixel values in the image. Specifically,
images were reconstructed by computing Hffor segments of
the individual rf A-lines that comprise a typical medical im-
age by applying a moving window, or “box-car,” analysis
facilitating local estimation of entropy values for regions
within the image.
All rf data are obtained by sampling a continuous func-
tion, y=f?t?, and subsequently using the sampled values to
compute its associated density function, wf?y?.
A. The function wf„y…
The density function wf?y? may be used to compute ei-
ther the entropy Hf, as in previous studies, or the Renyi
entropy as we do here. It corresponds to the density functions
used in statistical signal processing. From it, other math-
ematical quantities are subsequently derived ?e.g., mean val-
ues, variances, and covariances?.2–4While the density func-
tion is usually assumed to be continuous, infinitely
differentiable, and to approach zero at infinity in statistical
signal processing of random signals, in our application wf?y?
has ?integrable? singularities.
As in previous studies, we employ the convention that
the domain of f?t? is ?0,1?, so that, wf?y?, the density function
of f?t?, can be defined by the basic integral relation
Equation ?1? implies
wf??? = ?
either by breaking the integral into a sum over intervals of
monotonicity of f?t? ?“laps”? and changing variables, or by
choosing ??y? to be a Dirac delta function and using the
well-known expansion formula for a delta function of a
function.5All of our digitized waveforms f?t? are comprised
of at least one monotonic section, or “lap.” The lap bound-
aries are just the points t where f??t?=0. Within a lap, f?t?
has a well-defined inverse function so that Eq. ?2? may be
where N is the number of laps, gk?y? is the inverse of f?t? in
the kth-lap, and if y is not in the range of f?t? in the kth-lap,
gk??y? is taken to be 0.
J. Acoust. Soc. Am. 125 ?5?, May 2009© 2009 Acoustical Society of America31410001-4966/2009/125?5?/3141/5/$25.00
We also assume that all experimental waveforms f?t?
have a Taylor series expansion valid in ?0,1?. Then near a
time tksuch that f??tk?=0
y = f?t? = f?tk? +
2!f??tk??t − tk?2+ ¯ ,
tkis a lap boundary and on the left side of this point Eq. ?4?
may be truncated to second order and inverted to obtain
gk?y? ? tk??2?y − f?tk??/f??tk?,
?gk??y?? ? 1/?2f??tk??y − f?tk??.
The contribution to wf?y? from the right side of the lap
boundary, from gk+1?y?, is the same, so that the overall con-
tribution to wf?y? coming from the time interval around tkis
?gk??y?? ??2/?f??tk??y − f?tk???,
for 0?f?tk?−y?1 for a maximum at f?tk? and 0?y−f?tk?
?1 for a minimum. Thus, wf?y? has only a square root sin-
gularity ?we have assumed that tkis interior to the interval
?0,1?; if not, then the contributions to wfcome from only the
left or the right?. If additionally, f??tk?=0, then the square
root singularity in Eq. ?6? will become a cube-root singular-
ity, and so on, so that the density functions we consider will
have only integrable algebraic singularities.
Figure 1 illustrates, schematically, one possible type of
behavior possible in wf?y?: both discontinuities and algebraic
singularities ?indicated by arrows on the plots of wf?y??. Pro-
gressing from left to right in the figure illustrates how to
estimate qualitative features of wf?y? from f?t?. For instance,
the maxima in f?t? correspond to algebraic singularities in
wf?y?, plotted sideways in the middle panel to more clearly
indicate the relationship between its features and those of
f?t?. The rightmost panel shows wf?y? in a conventional lay-
out ?a rotated and flipped version of the plot in the middle
panel?. These plots show that the density functions possess
significantly different attributes from those usually consid-
ered in statistical signal processing.
The mathematical characteristics of the singularities are
important in order to guarantee the existence of the following
integral on which we base our analysis of signals in this
1 − rlog??
which is known as the Renyi entropy.6The physical signifi-
cance of the parameter r appearing in Eq. ?8? may be inter-
preted by analogy with statistical mechanics where the prob-
abilities wf?y? are given in terms of system energy levels
?with ?0being a physical constant?, and thermodynamic
quantities are derived from the partition function
Z =?e−?E?y?dy =?wf?y?−?/?0dy,
where ?=1/?kT?, with k being Boltzmann’s constant and T
being temperature.7From the equations we see that the Re-
nyi entropy, If?r?, is very similar to the partition function in
statistical mechanics and that the parameter r is analogous to
an inverse “temperature.” Moreover, If?r?→−Hf, as r→1,
using L’Hôpital’s rule, so that Ifis a generalization of Hfas
which previous studies have shown can be more sensitive to
subtle changes in scattering architecture than are more com-
monly used energy-based measures.1The purpose of this
study is to show that further sensitivity improvements may
be obtained using Ifat the suitable value of r.
For the density functions wf?y? encountered in our study,
If?r? is undefined for r?2. Moreover, as r→2−, the integral
appearing in Eq. ?8? will grow without bound due to the
singularities in the density function, wf?y? ?i.e., Eq. ?7??. The
behavior as r→2 is dominated by contributions from the
singularities. If the kth critical point is a minimum ?the argu-
ment for a maximum is similar? the contribution to the inte-
gral in Eq. ?8? is asymptotic to
?y − f?tk??
This is equal to
?y − f?tk??1−?/2dy,
2−???y − f?tk???/2
FIG. 1. A time-domain waveform, f?t?, with three critical points ?left?, and
its associated density function wf?y? which has three corresponding ?inte-
3142J. Acoust. Soc. Am., Vol. 125, No. 5, May 2009Hughes et al.: Application of Renyi entropy
where ak=?2/f??tk? ?for a maximum we have the asymptotic
This behavior is shown in the left panel of Fig. 2. More-
over, as shown in the right panel, it is possible that two
slightly different functions, f?t? and f?t?+??t?, where ? is
small, may have entropies, Hfand Hf+?that are close, as
shown in the figure, but whose Renyi entropies, If?r? and
If+??r?, diverge as r→2. If this amplification effect is not
dominated by noise, it may be exploited to distinguish subtly
different functions, such as those obtained from measure-
ments of backscattered ultrasound of targeted and nontar-
geted tissues. Our results show that this can happen in prac-
III. MATERIALS AND METHODS
A. Nanoparticles for molecular imaging
Across-section of the spherical liquid nanoparticles used
in our study is diagramed in Fig. 3. For in vivo imaging we
formulated nanoparticles targeted to ?v?3-integrins of
neovascularity in cancer by incorporating an “Arg-Gly-Asp”
mimetic binding ligand into the lipid layer. Methods devel-
oped in our laboratories were used to prepare perfluorocar-
bon ?perfluoro-octyl bromide, which remains in a liquid state
at body temperature and at the acoustic pressures used in this
monolayer.9,10The nominal sizes for each formulation were
measured with a submicron particle analyzer ?Malvern Zeta-
sizer, Malvern Instruments?. Particle diameter was measured
at 200?30 nm.
B. Animal model
The study was performed according to an approved ani-
mal protocol and in compliance with the guidelines of the
Washington University institutional animal care and use
The model used is the transgenic K14-HPV16 mouse in
which the ears typically exhibit squamous metaplasia, a pre-
cancerous condition, associated with abundant neovascula-
ture that expresses the ?v?3-integrin. Eight of these trans-
genic mice11,12were treated with 1.0 mg/kg intravenous of
either ?v?3-targeted nanoparticles ?n=4? or untargeted nano-
particles ?n=4? and imaged dynamically for 1 h using a re-
search ultrasound imager modified to store digitized rf wave-
forms acquired at 0, 15, 30, and 60 min time points. In both
targeted and untargeted cases, the mouse was placed on a
heated platform maintained at 37 °C, and anesthesia was
administered continuously with isoflurane gas ?0.5%?.
C. Ultrasonic data acquisition
A diagram of our apparatus is shown in Fig. 4. RF data
were acquired with a research ultrasound system ?Vevo 660,
Visualsonics, Toronto, Canada?, with an analog port and a
sync port to permit digitization. The tumor was imaged with
a 40 MHz single element “wobbler” probe and the rf data
corresponding to single frames were stored on a hard disk for
later off-line analysis. The frames ?acquired at a rate of 30
Hz? consisted of 384 lines of 4096 eight-bit words acquired
at a sampling rate of 500 MHz using a Gage CS82G digitizer
card ?connected to the analog-out and sync ports of the Vevo?
in a controller PC. Each frame corresponds spatially to a
region 0.8 cm wide and 0.3 cm deep.
FIG. 2. Left panel: A plot of If?r? ?left? showing that If?1?=−Hfand that
If?r? grows without bound as r→2. Right panel: Even though two similar
waveforms f?t? and f?t?+??t? may have nearly equal entropies, Hf, it is
possible that as r→2 their Renyi entropies may diverge.
FIG. 3. ?Color online? A cross-sectional diagram of the nanoparticles used
in our study.
FIG. 4. A diagram of the apparatus used to acquire rf data backscattered
from HPV mouse ears in vivo together with a histologically stained section
of the ear indicating portions where ?v?3-targeted nanoparticles could ad-
here and a fluorescent image demonstrating presence of targeted nanopar-
J. Acoust. Soc. Am., Vol. 125, No. 5, May 2009Hughes et al.: Application of Renyi entropy 3143
The wobbler transducer used in this study is highly fo-
cused ?3 mm in diameter? with a focal length of 6 mm and a
theoretical spot size of 80?1100 ?m ?lateral beam width
?depth of field at ?6 dB?, so that the imager is most sensi-
tive to changes occurring in the region swept out by the focal
zone as the transducer is “wobbled.” Accordingly, a gel
standoff was used, as shown in Fig. 4, so that this region
would contain the mouse ear.
A close-up view showing the placement of transducer,
gel standoff, and mouse ear is shown in the bottom of Fig. 4.
Superposed on the diagram is a B-mode gray scale image
?i.e., logarithm of the analytic signal magnitude?. Labels in-
dicate the location of skin ?top of image insert?, the structural
cartilage in the middle of the ear, and a short distance below
this, the echo from the skin at the bottom of the ear. Directly
above this is an image of a histological specimen extracted
from a human papilloma virus ?HPV? mouse model that has
been magnified 20 times to permit better assessment of the
thickness and architecture of the sites where ?v?3-targeted
nanoparticle might attach ?red by ?3staining?. Skin and tu-
mor are both visible in the image. On either side of the
cartilage ?center band in image?, extending to the dermal-
epidermal junction, is the stroma. It is filled with neoangio-
genic microvessels. These microvessels are also decorated
with ?v?3nanoparticles as indicated by the fluorescent im-
age ?labeled in the upper right of the figure? of a bisected ear
from an ?v?3-injected K14-HPV16 transgenic mouse ?Neu-
mann et al., unpublished?. It is in this region that the
?v?3-targeted nanoparticles are expected to accumulate, as
indicated by the presence of red ?3stain in the magnified
image of a histological specimen also shown in the image.
D. Ultrasonic data processing
Each of the 384 rf lines in the data was first upsampled
from 4096 to 8192 points, using a cubic spline fit to the
original data set in order to improve the stability of the ther-
modynamic receiver algorithms. Previous work has shown
benefit from increased input waveform length.13,14Next, a
moving window analysis was performed on the upsampled
data set using a rectangular window that was advanced in
0.064 ?s steps ?64 points?, resulting in 121 window posi-
tions within the original data set. This was done using both
continuous entropy, Hf, and Renyi entropy, If?1.99?, analysis
of the rf segments within each window in order to produce
an image ?either Hfor If?1.99?? for each time point in the
experiment. As described previously, the density function,
wf?y?, used to compute Hfand If?r? is computed using a
Fourier series representation.15For this study, where the de-
sire was to compute If?r? as near to its singular value as
possible, it was found that 16 384 terms were required for
accurate estimation. In order to complete computations in a
reasonable amount of time all calculations were performed
on a Linux cluster using open message passing interface.
E. Image processing
All rf data were processed off-line to reconstruct images
using information theoretic, either Hfor If?1.99?. Subse-
quently, a histogram of pixel values for the composite of the
0, 15, 30, and 60 min images was computed, either Hfor
If?1.99?. Image segmentation of each type of image, either
Hfor If?1.99?, at each time point in the experiment was then
performed automatically using its corresponding histogram
according to the following threshold criterion: The lowest
7% of pixel values were classified as “targeted” tissue, while
the remaining were classified as “untargeted” ?histogram
analysis was also performed using 10% and 13% thresholds,
with 7% having the best statistical separation between time
points?. The mean value of pixels classified as targeted was
computed at each time post-injection.
IV. RESULTS AND DISCUSSION
The results obtained after injection of targeted nanopar-
ticles, by either the If?1.99? or Hfreceivers, are shown in the
top and bottom panels of Fig. 5. Both panels compare the
growth, with time, of the change ?relative to 0 min? in mean
value of receiver output in the enhanced regions of images
obtained from all four of the animals in the targeted group.
Standard error bars are shown with each point. At 15 min the
change in mean value if If?1.99? is more than two standard
errors from zero, implying statistical significance at the 95%
level. As the bottom panel shows it is 30 min before Hfis
more than twice the standard error from zero.
The results obtained after injection of nontargeted nano-
particles, by either the If?1.99? or Hfreceivers, are shown in
the top and bottom panels of Fig. 6. Neither receiver exhibits
a statistically significant change in output over the course of
The value of 1.99 was chosen after an initial round of
numerical experimentation to assess numerical stability of
receiver output ?while also varying the number of terms re-
quired for the Fourier series reconstruction of wf?y?? versus
FIG. 5. A plot of average enhancement, i.e., change relative to value at time
0, obtained by analysis of If?1.99? ?top? and Hf?bottom? images from four
HPV mice injected with ?v?3-targeted nanoparticles.
3144J. Acoust. Soc. Am., Vol. 125, No. 5, May 2009Hughes et al.: Application of Renyi entropy
computation time. As the goal is ultimately to develop an
algorithm of clinical utility, the execution time of 1 week
required to compute the If?1.99? images for this study was
taken as an upper acceptable bound. Comparison of the data
in Figs. 5 and 6 show that If?1.99? is able to detect accumu-
lation of targeted nanoparticles in only half the time ?post-
injection? required by Hf. Pharmacokinetic dynamics would
lead us to expect the steady increase in targeted nanoparticles
in the region of insonification post-injection. Both plots of
Fig. 5 are consistent with this model; however, we may con-
clude that If?1.99? is more sensitive to their presence than Hf.
Future studies will concentrate on increasing the computa-
tional efficiency of our algorithm so that the region closer to
r=2 may be explored.
This study was funded by NIH Grant Nos. EB002168,
HL042950, and CO-27031 and NSF Grant No. DMS
0501079. The research was carried out at the Washington
University Department of Mathematics and the School of
1M. S. Hughes, J. E. McCarthy, J. N. Marsh, J. M. Arbeit, R. G. Neumann,
R. W. Fuhrhop, K. D. Wallace, D. R. Znidersic, B. N. Maurizi, S. L.
Baldwin, G. M. Lanza, and S. A. Wickline, “Properties of an entropy-
based signal receiver with an application to ultrasonic molecular imaging,”
J. Acoust. Soc. Am. 121, 3542–3557 ?2007?.
2R. S. Bucy and P. D. Joseph, Filtering for Stochastic Processes With
Applications to Guidance ?Chelsea, New York, NY, 1987?.
3U. Grenander and M. Rosenblatt, Statistical Analysis of Stationary Time
Series ?Chelsea, New York, NY, 1984?.
4N. Wiener, Extrapolation, Interpolation, and Smoothing of Stationary
Time Series: With Engineering Applications ?MIT, Cambridge, MA,
5R. N. Bracewell, The Fourier Transform and Its Applications ?McGraw-
Hill, New York, 1978?.
6T. M. Cover and J. A. Thomas, Elements of Information Theory ?Wiley-
Interscience, New York, 1991?.
7R. Tolman, The Principles of Statistical Mechanics ?Dover, New York,
8M. S. Hughes, J. N. Marsh, J. Arbeit, R. Neumann, R. W. Fuhrhop, G. M.
Lanza, and S. A. Wickline, “Ultrasonic molecular imaging of primordial
angiogenic vessels in rabbit and mouse models with ?v?3-integrin targeted
nanoparticles using information-theoretic signal detection: Results at high
frequency and in the clinical diagnostic frequency range,” Proceedings of
the 2005 IEEE Ultrasonics Symposium ?2005?.
9S. Flacke, S. Fischer, M. J. Scott, R. J. Fuhrhop, J. S. Allen, M. McLean,
P. Winter, G. A. Sicard, P. J. Gaffney, S. A. Wickline, and G. M. Lanza,
“Novel MRI contrast agent for molecular imaging of fibrin implications
for detecting vulnerable plaques,” Circulation 104, 1280–1285 ?2001?.
10G. M. Lanza, K. D. Wallace, M. J. Scott, W. P. Cacheris, D. R. Abend-
schein, D. H. Christy, A. M. Sharkey, J. G. Miller, P. J. Gaffney, and S. A.
Wickline, “A novel site-targeted ultrasonic contrast agent with broad bio-
medical application,” Circulation 94, 3334–3340 ?1996?.
11J. M. Arbeit, R. R. Riley, B. Huey, C. Porter, G. Kelloff, R. Lubet, J. M.
Ward, and D. Pinkel, “DFMO chemoprevention of epidermal carcinogen-
esis in k14-hpv16 transgenic mice,” Cancer Res. 59, 3610–3620 ?1999?.
12J. M. Arbeit, K. Mnger, P. M. Howley, and D. Hanahan, “Progressive
squamous epithelial neoplasia in k14-human papillomavirus type 16 trans-
genic mice,” J. Virol. 68, 4358–4368 ?1994?.
13M. Hughes, “Analysis of digitized waveforms using Shannon entropy,” J.
Acoust. Soc. Am. 93, 892–906 ?1993?.
14M. Hughes, “Analysis of digitized waveforms using Shannon entropy. II.
High-speed algorithms based on Green’s functions,” J. Acoust. Soc. Am.
95, 2582–2588 ?1994?.
15M. Hughes, “Analysis of ultrasonic waveforms using Shannon entropy,”
Proceedings of the 1992 IEEE Ultrasonics Symposium Vol. 2, pp. 1205–
FIG. 6. A plot of average enhancement, i.e., change relative to value at time
0, obtained by analysis of If?1.99? ?top? and Hf?bottom? images from four
HPV mice injected with nontargeted nanoparticles. Neither plot exhibits a
statistically significant change.
J. Acoust. Soc. Am., Vol. 125, No. 5, May 2009Hughes et al.: Application of Renyi entropy 3145