Content uploaded by Matthew Adam Turner
Author content
All content in this area was uploaded by Matthew Adam Turner on Apr 10, 2015
Content may be subject to copyright.
ABSTRACT
Experimental and Numerical Investigations of Novel Architectures Applied to Compressive
Imaging Systems
by
Matthew Adam Turner
A recent breakthrough in information theory known as compressive sensing is one com
ponent of an ongoing revolution in data acquisition and processing that guides one to acquire
less data yet still recover the same amount of information as traditional techniques, meaning
less resources such as time, detector cost, or power are required. Starting from these basic
principles, this thesis explores the application of these techniques to imaging. The ﬁrst labo
ratory example we introduce is a simple infrared camera. Then we discuss the application of
compressive sensing techniques to hyperspectral microscopy, speciﬁcally Raman microscopy,
which should prove to be a powerful technique to bring the acquisition time for such mi
croscopies down from hours to minutes. Next we explore a novel sensing architecture that
uses partial circulant matrices as sensing matrices, which results in a simpliﬁed, more robust
imaging system. The results of these imaging experiments lead to questions about the perfor
mance and fundamental nature of sparse signal recovery with partial circulant compressive
sensing matrices. Thus, we present the results of a suite of numerical experiments that show
some surprising and suggestive results that could stimulate further theoretical and applied
research of partial circulant compressive sensing matrices. We conclude with a look ahead
to adaptive sensing procedures that allow realtime, interactive optical signal processing to
further reduce the resource demands of an imaging system.
iii
First and foremost, I must thank my family still on this Earth, and those passed on to
the next life. I am deeply endebted to their unending encouragement, dedication, and love.
Indeed this achievement owes much to my parents who have always driven me to persevere.
Next, I must express my deepest appreciation to my research adviser Kevin F. Kelly who
encouraged me to apply for the NSF IGERT fellowship, provided space for me to work and
grow as a researcher, and supported my ventures out into side projects and allowing me to
ﬁnd my own research path. Thanks also to my Kelly Labmates, especially Lina Xu, Yun Li,
and Ting Sun from the camera lab. Cheers to Chad Byers for drinking coca tea and many
beers with me in Bogot´a. Thanks to Chaitra Rai for being a great friend who always had
an open ear. Thanks to Corey Slavonic for many good conversations on walks for coﬀee, at
lunch, or any of the many times I locked myself out of the oﬃce or lab.
Thanks very much to Woato Yin whose class taught me how important highdimensional
geometry is to understanding compressive sensing and sparse recovery. Indeed, his class
caused me to retreat at ﬁrst because it was so diﬃcult, but in that diﬃculty I regained a
grip and was able to view anew the beauty of geometry, which in high school ﬁrst attracted
me to mathematics.
Thanks to Richard Baraniuk for being such a positive character in the Rice Electrical and
Computer Engineering Department. Because of him we have an allstar lineup of speakers
every semester. His group members have been immensely helpful to me in the process of my
Master’s work.
I must also acknowledge Roger Moye of the Rice Shared Computing Grid who patiently
answered my questions on Unix programming, batch scripts, parallel computing, and for his
timely responses when MATLAB or something else was not working on the cluster. If not
for his help, Chapter 4 would not exist.
Finally, a very huge thank you to my wonderful friends who have also been bandmates,
mentors, roommates, and psychiatrists throughout these past four years. Life would be hell
without you.
Contents
Abstract ii
List of Illustrations vi
List of Tables x
1 Motivation and Introduction 1
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Introduction to the Mathematics of Compressive Sensing . . . . . . . . . . . 2
1.2.1 Matrix representation of a system of equations . . . . . . . . . . . . . 2
1.3 Infrared Imaging via Compressive Sampling . . . . . . . . . . . . . . . . . . 5
1.3.1 SinglePixel Camera General Setup . . . . . . . . . . . . . . . . . . . 5
1.3.2 Image Recovery with TVAL3 . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Other Applications and Outline of the Thesis . . . . . . . . . . . . . . . . . 13
2 Compressive Microscopy 15
2.1 Compressive Microscopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.1 Raster Scanning Microscope Systems . . . . . . . . . . . . . . . . . . 15
2.1.2 Compressive Microscopy Setup . . . . . . . . . . . . . . . . . . . . . 18
2.2 Raman Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 The Raman Eﬀect . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.2 Determining Chemical Structure from the Raman Spectrum of a
Material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.3 Raster Scanning Raman Microscopy . . . . . . . . . . . . . . . . . . . 26
2.2.4 Laserilluminated Compressive Sensing Microscope System . . . . . . 28
CONTENTS v
3 Circulant Matrices for Compressive Imaging 35
3.1 Theory of circulant matrices for imaging . . . . . . . . . . . . . . . . . . . . 35
3.1.1 Properties of circulant matrices . . . . . . . . . . . . . . . . . . . . . 35
3.2 Imaging with Partial Circulant Measurement Matrices . . . . . . . . . . . . 37
3.2.1 Imaging Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2.2 Formally Describing How to Build Φ from Φ
◦
. . . . . . . . . . . . . 41
3.3 Imaging Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.4 Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4 Experimental Investigation of Subsampled Circulant Matrices
for Compressive Sensing 49
4.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.1 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.2 Polytope Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.3 The Connection between Polytope Geometry and Convex Optimization 54
4.2 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.3 Phase Diagrams for Select n with Explanations . . . . . . . . . . . . . . . . 57
4.3.1 Basis Pursuit Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.3.2 Linear Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.3.3 Some Coherence Statistics . . . . . . . . . . . . . . . . . . . . . . . . 68
4.4 Discussion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5 Future Directions and Conclusion 71
5.1 Adaptive Sensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Bibliography 79
Illustrations
1.1 Detail of the DMD along with a photograph of a DMD displaying a 32 ×32
permuted WalshHadamard measurement vector, and a series of M
128 × 128 measurement vectors above it. . . . . . . . . . . . . . . . . . . . . 6
1.2 Illustration of imaging system with DMD detail . . . . . . . . . . . . . . . . 8
1.3 Infrared camera setup on the optical table . . . . . . . . . . . . . . . . . . . 10
1.4 n = 256 × 256 images of charcoalpainted “IR” behind acrylic paint as
imaged by our compressive imaging system with λ = 1450 nm . . . . . . . . 11
2.1 Raster Scanning Hyperspectral Microscopy Setup . . . . . . . . . . . . . . . 16
2.2 A simple addition of our singlepixel camera to a standalone Zeiss
microscope system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Reconstruction of n = 128 × 128 image of AF Test Target 1951 with δ = .95
and default TVAL3 parameters β, µ. . . . . . . . . . . . . . . . . . . . . . . 20
2.4 Method for determining best parameters for TVAL3 reconstruction. We see
that various choices for β, µ result not only in diﬀerent quality
reconstructions, but also diﬀerent reconstruction times (in seconds) for each,
as indicated below each reconstruction. . . . . . . . . . . . . . . . . . . . . . 21
2.5 The Raman spectrum of silicon dioxide substrate and graphite. A typical
measurement (compressive or not) in hyperspectral microscopy could be like
either one of these, or have two peaks together, or possibly contain many any
number of peaks corresponding to diﬀerent materials present in the sample. . 27
ILLUSTRATIONS vii
2.6 Visible light view of the graphite ﬂake on silicon substrate to be imaged via
the Raman eﬀect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.7 Visible light view of the graphite ﬂake on silicon substrate to be imaged via
the Raman eﬀect. The resolution is n = 33 × 33 . . . . . . . . . . . . . . . . 29
2.8 Raman image recovered from simulated compressive measurements for some
values of δ = m/n. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.9 Time for Rec PC to recover solution x
∗
as a function of 1  δ where δ is the
subsampling ratio. The dependence is nonlinear. . . . . . . . . . . . . . . . 31
2.10 Laserilluminated compressive sensing experimental setup . . . . . . . . . . . 32
2.11 Images of the smallest target on the AF Test Target 1951A. The bars are
2.2 µm wide. Images taken with 100x/.9NA Zeiss EC Epiplan/Neoﬂuoar
lens. Small ﬁeld of view, high magniﬁcation. . . . . . . . . . . . . . . . . . . 32
2.12 An alternative laserilluminated compressive sensing experimental setup . . . 33
3.1 ϕ
(1)
∈ R
1024
reshaped to 2D. White squares represent ϕ
(1)
k
= 1, black squares
represent ϕ
(1)
k
= 0, k = 1, . . . , n. . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2 Four copies of the seed vector ϕ
(1)
patterned onto an optical plate. By
shifting a selection mask (represented by the red box) to select one
measurement vector at a time, we generate, or ‘select,’ measurement basis
vectors from Φ
◦
, reshaped to 32 × 32. . . . . . . . . . . . . . . . . . . . . . . 39
3.3 By shifting one row or column of the mask at a time, we can generate all
n = N × N rows of a block circulant matrix Φ
◦
. The optical system is
identical to the DMDbased setup, where a lens L2 focuses an image of the
scene, represented by the arrow, onto the mask. The light that allows to
pass, corresponding to an ‘on’ pixel, or ϕ
ij
= 1, is collected by the lens L1
and directed towards the photodetector for measurement. . . . . . . . . . . . 40
ILLUSTRATIONS viii
3.4 Filled points in these plots indicate the location of the selection mask for
individual measurements in terms of row and column shifts. Thus, there are
more row shifts than column shifts for the sequential method and an equal
number of row and column shifts for the box method. The random path
shows some structure since the mask is only allowed to step one row or
column shift to generate the next measurement basis vector in the sequence,
and random is just that. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.5 Diﬀerence between taking measurement vectors from Φ
◦
sequentially (left
column) and according to a random path (right column) for a few
subsampling ratios. Note the reconstruction with random path measurement
vectors is relatively high quality even at a very low subsampling ratio δ.
Data acquired by Lina Xu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.6 Relative mean square error (normalized squared diﬀerence between
reconstructed image for a given δ and the one reconstructed with δ = 1) for
the four methods of generating the measurement basis Φ . . . . . . . . . . . 47
3.7 Time to solve the underlying optimization problem and recover an image for
various undersampling ratios, δ, for the four methods of generating the
measurement basis Φ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.1 The crosspolytope in three dimensions. There are six vertices, or 0faces,
twelve line segments, or 1faces, and eight 2faces, or what we commonly call
a face. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2 (BP) n = 32 × 32, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.3 (BP) n = 32 × 32, Ω
r
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.4 (BP) n = 32 × 32, Φ has Gaussian entries . . . . . . . . . . . . . . . . . . . . 60
4.5 (LP) n = 32 × 32, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.6 (LP) n = 32 × 32, Ω
r
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.7 (LP) n = 33 × 33, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
ILLUSTRATIONS ix
4.8 (LP) n = 31 × 31, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.9 (LP) n = 30 × 30, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.10 (LP) n = 16 × 16, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.11 (LP) n = 27 × 27, Ω
s
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.12 µ(Φ) for various resolutions and for random and sequential methods for a
series of values δ. Perhaps unexpectedly, the coherence for sequentialtype Φ
is lower than for randomtype, however the deviation from the mean is larger
for sequential than for random. . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.1 A series of 1bit adaptive measurements. An alternative description would
be a binary search, where on each measurement we split the portion of the
DMD where the ‘on’ pixel could be and ask ‘Which half of this active space
has the ‘on’ pixel?’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.2 Sequence of all log
2
(1024) = 10 32 ×32 binary adaptive measurement
vectors. The 0
th
measurement vector, where all the mirrors are ‘on’ is omitted. 75
5.3 Sequence of all 4 8 × 8 binary adaptive measurement vectors. The 0
th
measurement vector, where all the mirrors are ‘on’ is omitted. Theoretically
m ≈
p
log
2
n, however with only n = 64, we do not achieve this dramatic of
an improvement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.4 Sequence of all log
2
(log
2
1024) = 3 8 ×8, 2
2
i
grayscale levels for the i
th
measurement adaptive measurement vectors. The 0
th
measurement vector,
where all the mirrors are ‘on’ is omitted. . . . . . . . . . . . . . . . . . . . . 76
5.5 Results from sending 64 diﬀerent gray levels for the DMD to display along
with the actual measurement recorded at the DMD. We quantiﬁed the
spectrometer reading by ‘sum’ and ‘max’, ‘sum’ meaning we summed over
all wavelength bins and divided by the number of pixels, and for ‘max’ we
took the maximum value over all wavelength bins as we did for the
measurements presented in the rest of the chapter. . . . . . . . . . . . . . . 77
Tables
5.1 Table showing expected and actual measurement values for the i
th
measurement with the i gray levels scheme, taken as the maximum of the
peak of a spectrometer reading. . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2 Table showing expected and actual measurement values for the i
th
measurement, taken as the maximum of the peak of a spectrometer reading. 73
1
Chapter 1
Motivation and Introduction
1.1 Overview
The physical sciences, like the all aspects of society, are strained under the eﬀects of the
“data deluge,” which could be described as our species’ overwhelming ability to acquire
or create data coupled with our relatively weak skill in extracting useful information from
that data. In a raster scanning, hyperspectral microscope, such as a darkﬁeld, Raman, or
Fouriertransform infrared microscope, large amounts of data are collected, and oftentimes
acquisition is either timeintensive, or in the case of infrared microsocopy and spectroscopy,
monetarily expensive because exotic, nonsilicon based detectors are required. This thesis is
a description, exploration, and exposition, of how a new theory in signal processing, known
as compressed sensing, can be applied to microscopy and other imaging systems in order to
minimize these and other costs. Compressed sensing (CS) is a mathematical jewel itself. A
groundswell of mathematical and engineering work has risen exploring the implications and
theoretical applications of this theory. Based on the applications it has already found, CS
could prove to be one of the most useful developments in mathematics so far this century.
In this chapter, we introduce some notation we will need to describe compressive imaging
systems, followed by the introduction of our compressive infrared imaging system where we
will further illustrate the connection between the measurement formalism and the physical
camera system. We will see that compressive imaging reduces to solving an underdetermined
set of equations, unsolvable by elimination methods. In order to solve these equations, we
1.2. INTRODUCTION TO THE MATHEMATICS OF COMPRESSIVE SENSING 2
regularize the problem, in other words, we apply a priori information so that the deﬁcit of
information is suﬃciently reduced, and we recover an image as we would have otherwise.
1.2 Introduction to the Mathematics of Compressive Sensing
1.2.1 Matrix representation of a system of equations
In order to describe our measurement systems mathematically, we need to be able to ef
ﬁciently write large systems of equations. In the sequel, signal and image will be used
interchangably. Although there are philosophical diﬀerences, mainly that a signal can be
exactly known and recovered, but an image will always be an approximation to “true real
ity,” reality itself being an point of philosophical debate. Generally, we use the word ‘signal’
when referring to an arbitrary x ∈ R
n
and ‘image’ when referring speciﬁcally to a signal
approximated by an imaging system.
We now demonstrate how to compactly write large systems of equations in matrix nota
tion by means of a simple example. Consider the system of equations
3x
1
+ 2x
2
= −5
3x
1
− 9x
2
= 3
with x
1
, x
2
∈ R.
This is represented in matrix form by
y = Φx (1.1)
where
y =
−5
3
, x =
x
1
x
2
, and Φ =
3 2
3 −9
1.2. INTRODUCTION TO THE MATHEMATICS OF COMPRESSIVE SENSING 3
This allows us to eﬃciently describe an arbitrary number of equations and arbitrary
number of unknowns,
y ∈ R
m
y
1
y
2
.
.
.
y
m
=
ϕ
11
ϕ
12
. . . ϕ
1n
ϕ
21
ϕ
22
. . . ϕ
2n
.
.
.
.
.
.
.
.
.
.
.
.
ϕ
m1
ϕ
m2
. . . ϕ
mn
 {z }
Φ ∈ R
m×n
x
1
x
2
.
.
.
x
n
x ∈ R
n
(1.2)
is still just y = Φx. Using this matrix notation for a system of equations, we can describe
the acquisition of data that results in an image. For pixelarray or raster scan imaging,
Φ = I = δ
ij
, with δ
ij
=
1 if i = j
0 otherwise
,
or
I =
1 0 0 ··· 0
0 1 0 ··· 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 ··· 0 1
By simply reshaping the resulting vector y to be a rectangular 2D array, we recover an image
of the scene. Each row of Φ, which we denote ϕ
(i)
, probes the i
th
discretized point of x. If
we regard x as the scene to be imaged, note that x is not discrete until we impose some grid
on it. The value y
i
=
ϕ
(i)
, x
measures the brightness corresponding to each pixel. We call
ϕ
(i)
is the i
th
measurement vector. If we write a column of Φ as ϕ
c
i
∈ Φ = (ϕ
c
1
, . . . , ϕ
c
n
), then
1.2. INTRODUCTION TO THE MATHEMATICS OF COMPRESSIVE SENSING 4
we get the representation of the signal x,
y = ϕ
c
1
x
1
+ ϕ
c
2
x
2
+ . . . + ϕ
c
n
x
n
=
n
X
i=1
ϕ
c
i
x
i
. (1.3)
If Φ = I,
y
1
y
2
y
3
.
.
.
y
m
=
1
0
0
.
.
.
0
x
1
+
0
1
0
.
.
.
0
x
2
+ ··· +
0
0
.
.
.
0
1
x
n
. (1.4)
Thus we see that the measurements y may be viewed as a weighted sum of the coumns of Φ
where the weights are the discretized points of the scene, x.
In this work, compressive imaging is achieved through techniques deriving from transform
imaging, where more than one pixel is probed at a time, or, in other words, each measurement
vector ϕ
(i)
has many 1s in it. In the example to follow,
P
n
j=1
ϕ
(i)
j
=
n
2
, i = 1, . . . , m and
ϕ
(i)
∈ {0, 1}
n
. For compressive imaging, however, we do not have Φ ∈ R
n×n
, but instead
Φ ∈ R
m×n
with m < n. In other words we have less equations than unknowns. To quantify
the amount of undersampling, deﬁne the undersampling, or equivalently compression, ratio
δ =
m
n
.
The magic of compressive sensing is that we are still able to recover at least a good approx
imation to x, if not x exactly, from the underdetermined set of equations arising from our
measurements y = Φx. Before explaining how to recover an image from these underdeter
mined equations, here is an example of compressive imaging in action that also serves to
further solidify the notation to be used throughout the rest of this thesis.
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 5
1.3 Infrared Imaging via Compressive Sampling
As an example to prepare for the sequel in which we explore more advanced imaging systems,
we introduce the ‘Rice singlepixel camera.’ Speciﬁcally this camera is an infrared camera,
one of the useful applications of compressive sensing, especially for remote sensing and data
fusion applications of compressive sensing where detector cost is not the only one to be
mitigated. Infrared imaging has important applications in missile technologies as is well
known, and also for night vision and surveillance techniques. Predator drones, for example,
routinely acquire their targets for assassination with highly advanced infrared cameras [1]. A
more interesting application may be to combine multiple views for enhanced video sensing via
multiple compressive streams [2], and perhaps even do some sort of compressive data fusion
with radar imaging systems on drones to develop a full view of a scene, both indoors via
“throughthewall radar imaging” [3] and outdoors. A further bonus is that data collected
via compressive imaging is naturally encrypted as well as copmressed with no onboard
computing. This will become more clear once we better understand compressive imaging.
1.3.1 SinglePixel Camera General Setup
Here we begin our introduction of the infrared camera system by introducing the optical
element that displays the measurement vectors ϕ
(i)
. This is the digital micromirror device
(DMD) from Texas Instruments, Inc., shown in Figure 1.1 displaying one measurement vector
with an illustration of a series of m measurement vectors, again corresponding to the m rows
of the measurement vector Φ. The white pixels correspond to ϕ
ij
= 1, or we say this is an
‘on’ pixel, and black corresponds to ϕ
ij
= 0, or an ‘oﬀ’ pixel.
As in Figure 1.2, we focus our target scene, in this case the blue painted card as in
Figure 1.3, onto the digital micromirror device (DMD). The DMD, has 1024 × 768 mirrors
with a diagonal length of 13.6 µm, so the maximum resolution in pixels one can attain in
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 6
Figure 1.1 : Detail of the DMD along with a photograph of a DMD displaying a 32 × 32
permuted WalshHadamard measurement vector, and a series of M 128 × 128 measurement
vectors above it.
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 7
this setup is n = 768 × 1024. The size of the mirrors is compared with an ant leg in Figure
1.2. To acquire images at other resolutions, we operate mirrors in blocks to constitute one
pixel. The light from the scene is collected by a lens and focused onto the DMD. The DMD
displays reshaped rows of the measurement matrix Φ, the photodetector measures converts
light intensity to a voltage that is sent and stored on computer. The i
th
voltage measurement
serves as the inner product y
i
=
ϕ
(i)
, x
to yield the set of measurements y = Φx. An ‘on’
pixel, or ϕ
ij
= 1, directs light towards the detector, and an ‘oﬀ’ pixel directs light away from
the detector. The mirrors are ﬁxed to only ﬂip ±12
◦
away from parallel with the face of the
DMD. Thus, as in the transform imaging case described above, the DMD encodes the scene,
and the photodetector measures the total intensity of light reﬂected towards the detector
from the DMD for each measurement vector.
Our measurement matrix is a partial permuted Hadamard measurement matrix, which
we write as
Φ = R
Ω
P S
n
(1.5)
where
S
n
= f(H
n
) (1.6)
where H
n
is the Hadamard matrix of order n and f is a function on A ∈ R
n×n
, a
ij
being the
elements of A such that
f(a
ij
) =
1 if a
ij
= 1
0 otherwise
(1.7)
This follows, but is not identical to, the construction of “S matrices” in Harwit and Sloane [4].
The purpose is to allow us to use Hadamard matrices with a single detector, meaning our
measurement system can only implement measurement matrices Φ with elements ϕ
ij
∈
{0, 1}. P is an operator that permutes the columns of S
n
and R
Ω
selects a set of rows
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 8
Figure 1.2 : Illustration of imaging system with DMD detail
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 9
indexed by the entries of the set Ω. We take Ω = m which determines the number of
measurements we acquire, and traditionally we have set Ω to be m integers taken at random
from the set of natural numbers less than or equal to n without replacement. Compressive
sensing theory so far yields stronger guarantees for signal recovery when there is some element
of randomness in the measurement vectors, thus the random permutations of the columns
and then random selection of the rows [5, 6].
Once we have the measurement matrix Φ we sequentially display each row, or measure
ment vector, from Φ, collecting a measurement y
i
for each measurement vector. Once we
have displayed all m ϕ
(i)
to acquire the set of equations y = Φx, we employ the TVAL3 re
construction algorithm to recover an image. We implement this scheme as shown in Figure
1.3 and as illustrated in Figure 1.2.
As in Figure 1.2, we focus our target scene, in this case the blue painted card as in
Figure 1.3, onto the digital micromirror device (DMD). The blue square on the card is acrylic
paint. Below the acrylic paint are the letters “IR” in charcoal. We illuminate the card with
an array of 1450nm infrared light emitting diodes. Some of that light is able to penetrate
the acrylic paint to either reﬂect oﬀ the card or be absorbed by the charcoal letters. The
light reﬂecting from the scene is imaged onto the DMD, then collected by another lens and
the total intensity of light coming from the DMD is measured for m diﬀerent measurement
vectors by a Hamamatsu photodetector (model no. G6122) sensitive to wavelengths from
1–2 µm and peak sensitivity at λ = 1.95 µm.
The results of some compressive measurements for δ = .1 and δ = .075 are shown in
Figure 1.4.
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 10
Figure 1.3 : Infrared camera setup on the optical table
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 11
δ ≈ .1, m = 6550 δ ≈ .075, m = 4900
Figure 1.4 : n = 256 × 256 images of charcoalpainted “IR” behind acrylic paint as imaged
by our compressive imaging system with λ = 1450 nm
1.3.2 Image Recovery with TVAL3
After we acquire the measurements, y, the challenge is to recover an image x
∗
that represents
the scene x. To do so we regularize the problem, in this case by solving
(TV) min
x
n
X
i=1
kD
i
xk
2
subject to y = Φx (1.8)
where D
i
is a “local ﬁnitediﬀerence operator” such that D
i
x ∈ R
2
. Regularization is the
mathematical process of applying a priori information, in this case knowledge that the
solution x
∗
should have a small total variation (TV),
P
n
i=1
kD
i
xk
2
. This is one of many
possible methods of regularization. We will explore two alternative methods in Chapter 4.
To solve this we use TVAL3 of Li, Yin, and Zhang, which stands for total variation
minimization by augmented lagrangian and alternating direction algorithms [7]. TVAL3
traces its roots back to a seminal paper by Courant in 1943 on the quadratic penalty method
1.3. INFRARED IMAGING VIA COMPRESSIVE SAMPLING 12
[8]. Physicists will be familiar with the general principle of TVAL3, namely the augmented
Lagrangian, which is virtually identical to the Lagrangian formalism of dynamics. The
augmented Lagrangian method is used to solve the problem of Equation 1.8, (TV), as follows.
First, rewrite (TV) in the equivalent form
min
x,w
i
n
X
i=1
kw
i
k
2
subject to y = Φx and D
i
x = w
i
. (1.9)
The augmented Lagrangian for the rewritten problem is given by
L(x, λ, ν) =
n
X
i=1
kw
i
k
2
−ν
T
i
(D
i
x−w
i
)+
β
i
2
kD
i
x−w
i
k
2
2
−λ
T
(Φx−y)+
µ
2
kΦx−yk
2
2
. (1.10)
TVAL3 solves this problem through a socalled “alternating direction algorithm” developed
by Wang, Yang, Yin, and Zhang speciﬁcally designed to solve TV minimization problems
in imaging [9]. For further discussion of TVAL3 including the augmented Lagrangian for
malism, see [10] and [11]. Brieﬂy, the algorithm ﬁnds a new approximation for x given
Lagrange multipliers λ and ν, and then ﬁnds optimal Lagrange multipliers with the updated
x, which then are used in another iteration to ﬁnd a new approximation for x, and so on
until ∇L(x, λ, ν) < ε
tol
, where ε
tol
is a userdeﬁned tolerance parameter, which says that
L(x, λ, ν) has reached its global minimum, guaranteed by a global convergence theorem [11].
We used the TVAL3 algorithm with the default options for the coeﬃcients β
i
and µ in
Equation 1.10, and default stopping tolerance ε
tol
to obtain the 256 × 256 images in Figure
1.4. We achieve very high compression ratios, with subsampling ratios δ = .1 and δ = .075,
corresponding to m = 6550 and m = 4900 respectively.
1.4. OTHER APPLICATIONS AND OUTLINE OF THE THESIS 13
1.4 Other Applications and Outline of the Thesis
By way of the above example, it should be clear that compressive sensing is a powerful
method to reduce the resources required to acquire information. There are many applications
for compressive sensing outside of imaging. Much of the groundwork for compressive sensing
had been laid down before the breakthroughs by Cand`es, et al [5] and Donoho [6]. The
paradigm shift came out of the recognition that successful recovery of a signal depends on
a quantiﬁable dependence between how we acquire a signal and the structure of that signal.
Compressive sensing grew out of eﬀorts to ﬁnd the simplest accurate representation of a
signal.
Since CS guides one to acquire more information with less resources, it has found a wide
range of applications in science and engineering. Because measurements are already in a
compressed form no onboard compression is required. As such, there have been proposed
CS systems for astronomy [12] and hyperspectral remote sensing [13,14]. Other examples in
physics include radio astronomybased cosmology [15], and quantum state tomography [16],
which allows one to more accurately determine the state of a collection of, for example,
electron spins, which has applications in quantum computing.
Compressive sensing has also been applied to medicine and biology. CS holds much
promise to improve magnetic resonance imaging because of the greatly reduced acquisition
time it allows [17, 18]. Thus, when the patient is less able to control their movements, as
with children, or if time is of the essence as it often is in medicine, CS techniques could be
of immense beneﬁt.
Shental, et al [19], identify carriers of “rare variants” of disease via CS techniques applied
to group testing. Erlich, et al [20], applied a similar method to identify genetic disease
in Ashkenazi Jews, and then extended this work to “Compressed Genotyping” to identify
genetic variation of any sort in any number of individuals [21]. Machines known as DNA
1.4. OTHER APPLICATIONS AND OUTLINE OF THE THESIS 14
microarrays determine the sequence of base pairs in DNA. Dai, et al., [22] and Sheikh,
et al., [23] both explore the application of compressive sensing to DNA sequencing with
DNA microarrays. One of the most interesting applications of compressive sensing straddles
the disciplines of biology and physics. AlQuarishi and McAdams suggest a method for using
compressive sensing to eﬀectively learn a physical model for proteinDNA interactions, which
could have important applications in drug delivery, disease treatment, and fundamental
genomics [24].
In the sequel, we explore the application of compressive sensing to microscopic imaging
in Chapter 2, introduce a novel compressive imaging system based on circulant matrices in
Chapter 3, and then study numerically the eﬃcacy of circulant matrices for general com
pressive sensing in Chapter 4. The ﬁnal chapter will be dedicated to ﬁnal thoughts and
future directions with some preliminary data from adaptive sensing experiments, where the
i
th
measurement vector depends on the result y
i−1
=
ϕ
(i−1)
, x
. Although the work here is
dedicated to imaging, many of the results might have parallel applications or provide insight
into problems from the varied ﬁelds of application mentioned above, espeically Chapter 4.
15
Chapter 2
Compressive Microscopy
Now we turn our focus from novel imaging architectures and their characterization to con
crete applications to realworld imaging systems. Raman imaging is a prototypical example
of a measurement system that beneﬁts substantially from reducing the time required to ac
quire a Raman microscopic image. In our lab, and as is common in other laboratories as
well, highresolution raster scan Raman images require a few to tens of hours of acquisition
time. Each location must be probed individually, resulting in a measurement akin to that
described in . Below we will discuss in more detail the nature of acquiring data for Raman
microscopy.
2.1 Compressive Microscopy
In this section, we describe some various forms our microscope system could take, as well
as the measurement formalism we’ll need to describe them. To do that we ﬁrst introduce
the standard, noncompressive raster scanning system. Then we will introduce compressive
microscope systems, including a simple method to choose the best parameters for TVAL3,
ﬁrst introduced in Section 1.3.2, that can easily be extended to other imaging systems.
2.1.1 Raster Scanning Microscope Systems
A typical, simpliﬁed raster scanning microscope setup is shown in Figure 2.1. A laser is sent
through a beamsplitting mirror through the back of an objective lens which focuses the laser
light ideally to a diﬀractionlimited point, reﬂected and/or scattered light is collected by the
2.1. COMPRESSIVE MICROSCOPY 16
Figure 2.1 : Raster Scanning Hyperspectral Microscopy Setup
same objective lens, then directed back through the beamsplitter to either a photodetector
in the case of standard imaging, or a spectrometer for hyperspectral (i.e. beyond only three,
red, green, and blue, color channels) imaging. We use the same idealized measurement
formalism as before. In the simple raster scan imaging system we have the measurements,
y, given as
y = Ix, (2.1)
where I is the n × n identity matrix. As in a pixel array, each discretized point in the
scene is sampled by the measurement vector Φ = I, the measurement y
i
is proportional
to the number of photons registered by the detector during the acquisition time, t
acq
. In
reality it is stored on computer as the voltage reading from a photodetector caused by the
dislocation of electrons by incident photons. To physically sample each point in the image,
the sample stage moves in the “xy” directions (not to be confused with x and y vectors
2.1. COMPRESSIVE MICROSCOPY 17
from our measurement formalism) in discrete steps, the size of which determine the spatial
resolution of the image. All that is needed to recover an image from the set of measurements
y = Φx when Φ is the identity is a proper accounting of what point was illuminated when.
If instead we are performing hyperspectral microscopy, instead of y ∈ R
n
, we have y ∈
R
n×n
spec
, where n
spec
is the resolution of the spectrometer. The spectrometer works like a
prism to disperse the incoming light scattered or reﬂected by the sample, and then measure
how much light at a set of discrete wavelengths is present at each point. However, in fact
the light is dispersed by a diﬀraction grating, a reﬂective optical element cut with grooves.
Each wavelength of light reﬂects at a diﬀerent angle, causing the prismlike dispersion. A
highly rectangular CCD pixel array is calibrated so that the light striking a portion of it is
registered as, for example, λ = 632.5nm. The Ocean Optics USB4000 spectrometer we use
is sensitive to light from 2001100 nm with a Toshiba TCD1304AP Linear CCD array that
has a resolution of 3648 pixels, or 3648 possible wavelength bins. Although the spectrometer
is sensitive to light in this wide wavelength range, oftentimes a higher resolution in λ is
exchanged for a smaller range of wavelength values to be probed. This technique is useful for
Raman imaging, discussed below, but also for ﬂuorescence microscopy, a popular technique
used extensively in biological and medical research that allows for identiﬁcation of various
parts of a cell by functionalizing ﬂuorophores, proteins that absorb then emit at speciﬁc
wavelengths, to attach to speciﬁc parts of a cell. Before we discuss the Raman eﬀect and its
usefulness in more detail, we describe how we modify a whitelight illumination microscope
to employ compressive sensing, as well as some calibration data. We will introduce two
options for a microscope setup for laserillumination in Section ??.
2.1. COMPRESSIVE MICROSCOPY 18
Figure 2.2 : A simple addition of our singlepixel camera to a standalone Zeiss microscope
system.
2.1.2 Compressive Microscopy Setup
As introduced in the introduction, we will acquire measurements y = Φx with Φ being
a randomly permuted and subsampled Hadamard matrix as given in Equation 1.5. The
experimental setup is essentially the same, except our image is collected using microscope
optics. The simplest method for implementing compressive microscopy is to simply collect
light from a prebuilt microscope, as shown in Figure 2.3. Here, an image collected by the
internal optics of the microscope are projected out from the exit aperature, collected by
a lens, then sent to the DMD via a rotated mirror. Then an eyepiece collects the light
2.2. RAMAN IMAGING 19
corresponding to y
i
=
φ
(i)
, x
and focuses it down the photodetector, which, as discussed
previously, converts light intensity to an analog voltage signal, which is then converted to
digital at the analogtodigital converter (ADC), and saved on the computer.
Some calibration images of the standard Air Force test target AF1951 taken with this
setup are shown below. With δ = .95 and again the default parameters for β and µ we
acquired the image in Figure ??. The smallest bars are 2.2µm wide, which sets the ﬁeld
of view to about 40µm
2
. The image is fair quality, at best. In fact, especially in low
light situations, the default, or even the recommended range, of parameters is not optimal.
To determine the best choice of β and µ, we solve and plot a series of solutions x
∗
to the
optimization problem (TV) in Equation 1.8. Then, either using some mathematical heuristic
or simply by visual inspection, one may choose the optimal parameters.
2.2 Raman Imaging
2.2.1 The Raman Eﬀect
The Raman eﬀect was ﬁrst discovered by Indian physicist C.V. Raman in 1929. Brieﬂy, the
Raman eﬀect is an optical, quantum mechanical eﬀect that probes the vibrational properties
of a material, be it in the gas, liquid, or solid phase. As is common knowledge, some of the
light reﬂected from a material is the same wavelength as the incident light; the technical
term is Rayleigh scattering. However, if the material is ‘Ramanactive,’ there will also be
other wavelengths of light ‘reﬂected’ as well. More accurately, this light is scattered instead
of reﬂected, and we say it is Ramanscattered light. For readers familiar with ﬂuorescence,
this may sound like ﬂuorescence, but it is not the same eﬀect. The only true similarity is
that both eﬀects are mediated by the absorption of quanta of light, photons, by the negative
charge carriers of a material, electrons.
One of the most basic principles of physics, the conservation of energy, is the route by
2.2. RAMAN IMAGING 20
Figure 2.3 : Reconstruction of n = 128 × 128 image of AF Test Target 1951 with δ = .95
and default TVAL3 parameters β, µ.
2.2. RAMAN IMAGING 21
Figure 2.4 : Method for determining best parameters for TVAL3 reconstruction. We see that
various choices for β, µ result not only in diﬀerent quality reconstructions, but also diﬀerent
reconstruction times (in seconds) for each, as indicated below each reconstruction.
2.2. RAMAN IMAGING 22
which Raman scattering occurs. Photons of varying wavelength correspond to photons of
diﬀerent energy. A beam of light consisting of 100 ultraviolet photons is more energetic
than a beam of light with 1,000 infrared photons. However, in optics we would say that
the infrared beam is more intense, since there are more photons. The energy E of a single
photon is directly proportional to the frequency of the photon, ω,
E = ~ω (2.2)
In the Raman eﬀect, we have that a beam of incident light of a certain energy, i.e. ‘color’,
enters the material and light of a diﬀerent energy enters. Therefore, energy was lost or
gained. With the Raman eﬀect it is more likely to observe a lower energy, or redshifted,
photon, so let’s assume energy was lost. Where did this energy go?
The answer is that the light stimulated vibrations in the material. To understand this
better, let’s consider the diatomic molecule, say O
2
for example, as a pair of weights on a
spring. The nuclei are the weights, and the chemical bond (sharing of two electrons between
the nuclei) is the spring. The spring has a stiﬀness, K, corresponding to the strength of the
chemical bond. Assume that light, written as an electric ﬁeld
E(t) = E
0
cos(ωt + δ
k
), (2.3)
is incident on the molecule. In general the polarization vector of a diatomic molecule may
be written as
p = αE (2.4)
where α is the polarizability, or susceptibility to a change in polarization. In general it is an
arbitrary tensor, however in this case we assume it acts in only one dimension and we take
2.2. RAMAN IMAGING 23
it to only linear order,
α(x) = α(0) +
dα
dx
x=0
x (2.5)
We assume that x is the solution of a simple harmonic oscillator, so that
x(t) = A cos(ω
1
t) (2.6)
where
ω
1
=
s
K
µ
(2.7)
and µ is the reduced mass of the two nuclei. Combining Equations (2.3), (2.4), and (2.5),
we get
p(t) = α
0
E
0
cos(ωt + δ
k
) + Aα
0
(0)E
0
cos(ωt + δ
k
) cos(ω
1
t) (2.8)
Using a trigonometric identity we write
p(t) = α
0
E
0
cos(ωt + δ
k
) +
1
2
Aα
0
(0)E
0
{cos
(ω − ω
1
)t + δ
k
+ cos
(ω + ω
1
)t + δ
k
} (2.9)
Thus the polarization changes as a function of time, meaning that we have the acceleration of
charge with at two natural frequencies, ω, corresponding to the Rayleigh scattering (common
reﬂection) and the Raman scattered light at frequency ω
R
= ω ± ω
1
[25]. The minus sign
says that light is scattered at a frequency less than the incident frequency, so with energy
E = ~(ω − ω
1
).
Since in terms of color, this is a shift towards the ‘red’ end of the electromagnetic spectrum,
we call this photon redshifted. The plus sign corresponds to a blueshift, or equivalently a
2.2. RAMAN IMAGING 24
gain in energy for the incident photons,
E = ~(ω + ω
1
).
The photons redshifted via the Raman eﬀect are called Stokesshifted, and the blueshifted
photons are called antiStokes, after the famous English nineteenth century physicist Sir
George Stokes. There is a roughly 10
−6
chance that the incident photon will be Stokes
scatterd, and about a 10
−8
chance that an incident photon will be antiStokes scattered.
Thus, the fact that Raman originally observed this previously anomolous eﬀect with modest
optics and sunlight is quite amazing. Today, Raman spectroscopy and microscopy is per
formed with nearly singlefrequency laser light, an array of precision optics, and cooled CCD
spectrometers. Although it is such a weak eﬀect, it is a very powerful method for probing
the chemical structure of a sample, as we will see in the following section.
2.2.2 Determining Chemical Structure from the Raman Spectrum of a Material
The astute and informed reader will note that in order for the above derivation to be valid,
the ﬁrst derivative of the polarizability, α
0
(0) =
dα
dx

x=0
6= 0. This deﬁnes a socalled selection
rule for Raman scattering—the ﬁrst derivative of the polarizability with respect to space
must be nonzero. In the case of vibration of the O
2
molecule, α
0
(0) = 0 and so there
actually is no Raman scattering from vibrations of O
2
. However, one of the ﬁrst important
applications of Raman scattering was to show that there are other carbon dioxide, CO
2
,
had hitherto unknown vibrational modes, or in terms of Equation 2.9, unknown ω
1
[26, 27].
Our above derivation accounted only for vibrations in one dimension, but as also mentioned
above, polarizability, α is a tensor in general, so there could be nine total nonzero partial
ﬁrst derivatives with respect to space, and rotational modes are also allowed, so that if
an incident photon causes a molecule to rotate, that might also be reﬂected in the Raman
2.2. RAMAN IMAGING 25
spectrum through a blue or redshifted peak in the Raman spectrum.
The example of CO
2
is also important for us because before the Raman spectrum was
acquired for that molecule, its infrared (IR) spectrum, which also tells us about vibrational
modes, but for a diﬀerent set of selection rules, was known. The vibrational modes discovered
by Raman spectroscopy diﬀered from those discovered by infrared spectroscopy, but the
vibrations were on the same order of energy. This illustrates how Raman spectroscopy
allows one to use light in the visible part of the electromagnetic spectrum, with wavelength
λ ≈ 400 − 700nm, to probe energies in that would correspond to photons in the infrared,
corresponding to .8 µm ≤ λ ≤ 100 µm and larger. To detect such photons requires more
exotic light sources as well as more exotic detectors compared to visible light. Thus, Raman
spectroscopy provides a simpler, complementary, method for investigating the vibrations of
molecules.
Raman spectroscopy is not limited to probing vibrations of molecules. It is also possible
to determine the vibrational modes of solids, known as phonons, as in quanta of sound, just
like the photon is a quantum of light. We model a solid as a lattice of masses connected by
springs instead of just two or three masses connected by springs in the case of molecules in
either the gaseous or liquid state. When light is incident on the lattice, it either reﬂects as in
Rayleigh scattering, or it creates a phonon, which again is a vibration that travels through
the solid, and a lowerenergy phton. It is sensible to call this a quanta of sound, because it
is precisely vibrations of nuclei, transmitted through electronelectron interactions, that is
responsible for the majority of thermal conductivity and transmission of sound in materials.
Among the many important applications of Raman scattering in solids are stress and
strain analysis for silicon technologies and for investigating the properties of graphene and
carbon nanotubes, including how many layers of graphene are present in a graphene sample
and also how many layers comprise a nanotube or what diameters of nanotubes are present
2.2. RAMAN IMAGING 26
in a sample. We choose these two examples because, as a proof of concept of the applicability
of compressive sensing techniques to Raman microscopy, we show that compressive sensing
is indeed eﬀective for laserillumination based microscopy, and that through simulations on
actual Raman microscopy data, compressive sensing may be used to discriminate graphite
from silicon, which suggests it might be eﬀective for more complex samples.
2.2.3 Raster Scanning Raman Microscopy
Raman microscopy is a powerful experimental technique to determine the spatial distribution
of substances in a sample. We will show a simple example of this with experimental data
acquired on the commercial Renishaw Raman microscopy system, followed by two proposed
architectures for laserilluminated microscopy with the DMD. The example is graphite on
silicon dioxide, which is a toy model for a more interesting system, graphene on silicon
dioxide. Graphene is a single layer of carbon atoms in a 2D lattice, however the term graphene
is also used to describe more than one layer stacked on top of one another. It seems that
it is graphene until about ten layers, then graphene just becomes graphite. Graphite and
graphene share similar features in their Raman spectrum since both are carbon allotropes,
and in fact Raman spectroscopy is one method for determining whether a carbon sample
is singlelayer, doublelayer, or morelayered graphene or just simply graphite. Graphene is
a widelystudied material because of its novel conduction properties, strength, and overall
novelty, and was the subject of a recent Nobel prize in Physics. Silicon also has two prominent
peaks in its Raman spectrum, and both the Raman spectrum of silicon and of graphite are
shown in Figure 2.5. We focus only on the socalled ‘G’ peak of graphite with wavenumber
k ≈ 1590 cm
1
, and the wellknown silicon peak at k = 520 cm
1
.
We image the boxed region with graphite ﬂake shown in Figure 2.6 by sensing how intense
the Raman ‘G’ peak is. If the peak is not there, then we know that it is the silicon substrate
2.2. RAMAN IMAGING 27
Silicon Raman spectrum, k = 520 cm
1
peak Graphite Raman spectrum, ‘G’ peak
Figure 2.5 : The Raman spectrum of silicon dioxide substrate and graphite. A typical mea
surement (compressive or not) in hyperspectral microscopy could be like either one of these,
or have two peaks together, or possibly contain many any number of peaks corresponding
to diﬀerent materials present in the sample.
at that point, and if we observe the ‘G’ peak then we know it is graphite. By raster scanning
over the sample and acquiring a spectrum as in Figure 2.5, as explained in Section 2.1.1, we
can build a Raman image of the region of interest. The Raman image acquired by raster
scanning is shown in Figure 2.7. The brighter the pixel, the larger the maximum value of
the ‘G’ peak. To test the eﬃcacy of compressive imaging for Raman imaging, we simulate
compressive acquisition where Φ 6= I and Φ ∈ R
m×n
where m < n. Instead of having Φ be
randomly permuted WalshHadamard vectors, Φ are partial circulant matrices as explained
in Chapter 3. We hold oﬀ further discussion of the details of partial circulants for now. To
reconstruct an image from measurements taken with partial circulant Φ, we use the Rec PC
algorithm of Yin, et al., [28], also to be futher discussed in Chapter 3. We only need to say
here that we used default settings for userdeﬁned parameters and that the reconstruction
algorithm recovers the solution x
∗
according to the same problem (TV) in Equation 1.8.
We look at the reconstruction of x
∗
the compressive measurements for a few values of δ in
Figure 2.8. Data on convergence across a range of δ values is given in Figure 2.9. We see
2.2. RAMAN IMAGING 28
Figure 2.6 : Visible light view of the graphite ﬂake on silicon substrate to be imaged via the
Raman eﬀect.
that computational resources increase nonlinearly with δ, as expected.
2.2.4 Laserilluminated Compressive Sensing Microscope System
To ﬁnish this chapter, we present preliminary data from our home built copmressive sensing
microscope system, the optics on the table in Figure 2.10. With this setup we obtained the
images shown in Figure 2.11.
With the proper equipment, it is clearly possible to acquire Raman microscopic images
in a fraction of the time it would take with traditional methods. Because of some not yet
understood spectral phenomena arising from the diﬀractive properties of the DMD, it may
be better to construct the microscope as we have, shown in Figure 2.12, where instead of
implementing the measurement vectors by patterning light coming from the scene, the DMD
structures laser light sent to illuminate the scene, which is totally mathematically equivalent.
2.2. RAMAN IMAGING 29
Figure 2.7 : Visible light view of the graphite ﬂake on silicon substrate to be imaged via the
Raman eﬀect. The resolution is n = 33 × 33
2.2. RAMAN IMAGING 30
Figure 2.8 : Raman image recovered from simulated compressive measurements for some
values of δ = m/n.
2.2. RAMAN IMAGING 31
Figure 2.9 : Time for Rec PC to recover solution x
∗
as a function of 1  δ where δ is the
subsampling ratio. The dependence is nonlinear.
2.2. RAMAN IMAGING 32
Figure 2.10 : Laserilluminated compressive sensing experimental setup
AF Target Bars, n = 128 × 128 AF Target Bars, n = 256 × 256
Figure 2.11 : Images of the smallest target on the AF Test Target 1951A. The bars are
2.2 µm wide. Images taken with 100x/.9NA Zeiss EC Epiplan/Neoﬂuoar lens. Small ﬁeld of
view, high magniﬁcation.
2.2. RAMAN IMAGING 33
Figure 2.12 : An alternative laserilluminated compressive sensing experimental setup
2.2. RAMAN IMAGING 34
Studer, et al. [29], employ essentially the same setup for ﬂuorescence microscopy for biology.
35
Chapter 3
Circulant Matrices for Compressive Imaging
In this chapter we explore the use of circulant matrices for imaging. By employing measure
ment matrices Φ whose m rows are taken from an n × n circulant or blockcirculant matrix
we will denote Φ
◦
. We will deﬁne a circulant matrix mathematically, explain their utility,
and show how circulant matrices can result in a more versatile, eﬃcient imaging system.
Much of this material will also serve as an introduction for Chapter 4. The work presented
in Chapter 4 was motivated by the results of our imaging experiments presented in this
chapter.
3.1 Theory of circulant matrices for imaging
3.1.1 Properties of circulant matrices
Circulant matrices “underpin elementary harmonic analysis” (Aldrovandi, 2001 [30]) because
of their special relationship to the Fourier transform. This relationship enables us to more
carefully design our measurement matrices, but maintain a fast matrixvector multiply in
the form of the Fourier transform. To see why, let us explicitly write a circulant matrix.
A circulant matrix is a matrix C ∈ R
n×n
with entries t
i
∈ R, i = 0, 1, ··· , n − 1 such
3.1. THEORY OF CIRCULANT MATRICES FOR IMAGING 36
that
C =
t
0
t
n−1
t
n−2
··· t
1
t
1
t
0
t
n−1
··· t
2
t
2
t
1
t
0
··· t
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
t
n−1
t
n−2
t
n−3
··· t
0
. (3.1)
Such a matrix is also sometimes referred to as a convolution matrix. To see why, consider
c = (t
0
, t
n−1
, . . . , t
1
)
so that c ∈ R
n
. Deﬁne the (circular) convolution operator for vectors a, b ∈ R
n
to be
(a ∗ b)
k
=
n−1
X
i=0
a
i
b
k−i
, k = 0, 1, . . . , n − 1 (3.2)
then we see that for x ∈ R
n
Cx = c ∗ x (3.3)
This fact allows a fast matrixvector multiply on a binary computer via the fast Fourier
transform (FFT). Let F ∈ R
n×n
be the Fourier transform matrix
F
jl
=
1
√
n
e
2πi (j−1)(l−1)/n
(j, l = 1, 2, . . . , n).
Now apply the identity I = F
−1
F to the right hand side of Equation 3.3 and use the
convolution rule for the Fourier transform to obtain
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 37
Cx = c ∗ x
= F
−1
F (c ∗ x)
= F
−1
(F c)(Fx)
= F
−1
DF x
where D = diag(λ
1
, λ
2
, . . . , λ
n
) = diag(λ) is not only the Fourier transform of the vector c,
but also the eigenvectors of the matrix C, which reveals one more remarkable property of
circulant matrices, namely they are diagonalized by the Fourier transform, or, equivalently,
the eigenvectors of a circulant matrix are the columns of the Fourier matrix, F . For more
details see [30–32]. For an interesting application to machine multiplication for two numbers
with arbitrary digits and precision, see Knuth, 1981 [33].
So, the utility of circulant matrices for compressive sensing should now be clear. All we
need to do is deﬁne a seed vector c, compute its Fourier transform which gives us λ, and
perform two FFTs modulated by the entries of λ. Thus, the computation of the matrixvector
multiply Cx will not take O(n
2
) operations, but instead O(n log(n)) operations. Even just
for n = 128×128 = 16384, the resolution of the images we present below, we see a substantial
decrease in the number of operations since n
2
≈ 2.7 × 10
8
and n log(n) ≈ 2.3 × 10
5
, three
orders of magnitude diﬀerence. This savings is essential for an eﬃcient recovery algorithm.
3.2 Imaging with Partial Circulant Measurement Matrices
In this section we follow the structure of Chapter 1 and introduce the imaging system before
diving too deeply into the mathematics. The advantage we gain from circulants for imaging
is that we may pattern four copies of the ﬁrst row, or seed vector, ϕ
(1)
, of the measurement
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 38
Figure 3.1 : ϕ
(1)
∈ R
1024
reshaped to 2D. White squares represent ϕ
(1)
k
= 1, black squares
represent ϕ
(1)
k
= 0, k = 1, . . . , n.
matrix Φ onto an optical plate. An n = 32 × 32 example of ϕ
(1)
is shown in Figure 3.1.
By shifting the plate, we generate rows of a block circulant matrix, to be explained in more
detail below.
3.2.1 Imaging Setup
The imaging setup is identical to those introduced in the previous chapters, except now
instead of the DMD directing light towards or away from a photodetector via reﬂection, we
have a optical plate patterned with a mask to either allow the light to pass through or block
light from a pixel. This corresponds to ϕ
ij
∈ {1, 0}, again with a 1 being represented by
white in Figures 3.1 and 3.3, and furthermore, as before, ϕ
ij
= 1 is an element that allows
light to pass to the detector, and ϕ
ij
= 0 is an element that blocks light.
On a single optical plate we pattern four copies of ϕ
(1)
to make a 2N ×2N pixel grid. By
overlaying an N × N selection mask, shown in dashed green in Figure 3.2, we can generate
all n measurement vectors by moving the selection mask. Each selected N × N square
corresponds to a row of Φ
◦
, as illustrated in Figure 3.2. In practice, we would not actually
move the selection mask since this would also entail moving the photodetector. Instead we
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 39
move the optical plate itself, as illustrated in Figure 3.3.
ϕ
(1)
ϕ
(14)
ϕ
(32)
Figure 3.2 : Four copies of the seed vector ϕ
(1)
patterned onto an optical plate. By shifting
a selection mask (represented by the red box) to select one measurement vector at a time,
we generate, or ‘select,’ measurement basis vectors from Φ
◦
, reshaped to 32 × 32.
We deﬁne
Ω ⊆ {0, 1, . . . , N − 1} × {0, 1, . . . , N − 1} = Ω
◦
(3.4)
to represent the number of column shifts and row shifts of the optical plate used to acquire
the measurements y. If we acquire m measurements y
i
=
ϕ
(i)
, x
, then Ω = m. There are
four diﬀerent methods of creating Ω that we explore in this work,
1. Sequential: Starting with the selection mask in the lower left corner, shift the selection
mask one row at a time, N − 1 times. Then shift by one column and again perform
N − 1 row shifts. Repeat until m unique regions are selected by the selection mask.
This is illustrated in Figure 3.2.
2. Box: Do an equal number (≈ d
√
m e) of row and column shifts.
3. Random: Select m ω
i
∈ Ω
◦
at random.
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 40
column
shift
row
shift
photodetector
L1
L2
column
shift
row
shift
photodetector
L1
L2
Figure 3.3 : By shifting one row or column of the mask at a time, we can generate all
n = N × N rows of a block circulant matrix Φ
◦
. The optical system is identical to the
DMDbased setup, where a lens L2 focuses an image of the scene, represented by the arrow,
onto the mask. The light that allows to pass, corresponding to an ‘on’ pixel, or ϕ
ij
= 1, is
collected by the lens L1 and directed towards the photodetector for measurement.
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 41
4. Random Walk: Restrict the plate to shift one row or column at a time, but take the
step at random. The result is a random walk. At this time we do not optimize for
selfcrossings and require that m unique points are generated isntead of m steps taken.
3.2.2 Formally Describing How to Build Φ from Φ
◦
Recall the restriction operator R
Ω
from Section 1.3 that selects the rows indexed by the set
Ω, for example if Ω = {1, 2} then
R
Ω
A = R
Ω
a
11
a
12
··· a
14
a
21
a
22
··· a
24
.
.
.
.
.
.
.
.
.
.
.
.
a
41
a
42
··· a
44
= R
Ω
a
(1)
a
(2)
a
(3)
a
(4)
=
a
(1)
a
(2)
. (3.5)
Thus, we can compactly write building our measurement matrix Φ ∈ R
m×n
from a circulant
matrix Φ
◦
∈ R
n×n
as
Φ = R
Ω
Φ
◦
. (3.6)
In our application where Ω corresponds to coordinates of row and column shifts of the
optical plate, we cannot directly apply R
Ω
as above. Here instead the coordinates in Ω deﬁne
which row of the block circulant matrix Φ
◦
will be taken. To see how this works, let us write
the matrix
M =
a b c a b c
e f g e f g
h i j h i j
a b c a b c
e f g e f g
h i j h i j
, (3.7)
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 42
a 3 × 3 analogue of the pattern on the optical plate. In this toy example, then,
ϕ
(1)
=
a b c
e f g
h i j
,
which we have indicated by coloring it red in Equation 3.8. Of course ω
1
= (0, 0), no shifts.
If we implement the sequential method beginning with a column shift, the selection mask
would next select the red elements,
M =
a b c a b c
e f g e f g
h i j h i j
a b c a b c
e f g e f g
h i j h i j
, (3.8)
meaning
ϕ
(2)
=
b c a
f g e
i j h
and ω
2
= (0, 1). Continuing on we have
ϕ
(3)
=
c a b
g e f
j h i
.
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 43
If we want to reshape these as rows of Φ to implement y = Φx we would have
Φ =
a b c e f g h i j
b c a f g e i j h
c a b g e f j h i
.
Deﬁne
α =
a b c
b c a
c a b
, β =
e f g
f g e
g e f
, and γ =
h i j
i j h
j h i
Then
Φ =
α β γ
Continuing like this we can write the matrix of all reshaped measurement vectors generated
by such shifts as
Φ
◦
=
α β γ
γ α β
γ β α
. (3.9)
Just as C was called a convolution matrix in 1D, this matrix Φ
◦
is a 2D convolution matrix.
Φ
◦
is not circulant as with 1D, but block circulant. We still have
Φ
◦
x = F
−1
DF x
as discussed above, however now F and F
−1
are the 2D Fourier and inverse Fourier transform
[34].
3.2. IMAGING WITH PARTIAL CIRCULANT MEASUREMENT MATRICES 44
Sequetial Box
Random Path Random
Figure 3.4 : Filled points in these plots indicate the location of the selection mask for
individual measurements in terms of row and column shifts. Thus, there are more row shifts
than column shifts for the sequential method and an equal number of row and column shifts
for the box method. The random path shows some structure since the mask is only allowed to
step one row or column shift to generate the next measurement basis vector in the sequence,
and random is just that.
3.3. IMAGING RESULTS 45
3.3 Imaging Results
In this section we show results from imaging with each of those four methods. Our imaging
system is the same as in Section 1.3, except the illumination source is diﬀerent. We use a
broadband lamp here. The Hamamatsu detector is the same. We use the digital micromirror
device (DMD) to simulate the mask motions as a proofofconcept for this imaging scheme.
In order to recover an image from the measurements y = Φx, we use the Rec PC (PC
stands for ‘partial circulant’) algorithm of Yin, et al., [28]. Essentially it is the same as
TVAL3, but with some modiﬁcation, most notably to accommodate the circulant measure
ment matrices. One other diﬀerence is it oﬀers an explicit handling of both total variation
minimization and `
1
minimization, along with the usual ﬁdelity constraint. Rec PC ﬁnds x
∗
such that
(Rec PC) x
∗
= min
x
α
n
X
i=1
kD
i
xk +
µ
2
kΦx − yk
2
. (3.10)
For the images below, we set α = 10
−2
and µ = 1. Again, the operator D
i
∈ R
2×n
is a
discrete gradient operator.
As shown in Figure 3.5, random Ω outperforms the sequential one. Furthermore, on
comparison of all four methods introduced here, the relative error, deﬁned as
Relative Error =
kx
∗
δ=1
− x
∗
δ
k
2
2
kx
∗
δ=1
k
2
2
, (3.11)
where x
∗
δ=1
is the solution when m = n, or δ = 1, declines equally quickly for both random
and random walk, converging nonlinearly, while sequential and box converge approximately
linearly with increasing δ. Furthermore, it appears that the computational problem (Rec PC)
in Equation 3.10 is more diﬃcult for box or sequential patterns than the randomized patterns
since the algorithm takes longer to converge in these cases, as shown in Figure 3.7.
3.3. IMAGING RESULTS 46
Figure 3.5 : Diﬀerence between taking measurement vectors from Φ
◦
sequentially (left col
umn) and according to a random path (right column) for a few subsampling ratios. Note
the reconstruction with random path measurement vectors is relatively high quality even at
a very low subsampling ratio δ. Data acquired by Lina Xu.
3.3. IMAGING RESULTS 47
Figure 3.6 : Relative mean square error (normalized squared diﬀerence between reconstructed
image for a given δ and the one reconstructed with δ = 1) for the four methods of generating
the measurement basis Φ
Figure 3.7 : Time to solve the underlying optimization problem and recover an image for
various undersampling ratios, δ, for the four methods of generating the measurement basis
Φ
3.4. CONCLUSION AND DISCUSSION 48
3.4 Conclusion and Discussion
In this chapter we have demonstrated the feasibility and utility of circulant matrices for
imaging. Apparently, when the measurement vectors are chosen in a sequential fashion,
the recovery problem is more diﬃcult than if the measurement vectors are chosen with an
element of randomness. We may be tempted to generalize this phenomenon and say that
in all cases, a sequentiallybuilt Φ results in worse recovery than randomlygenerated Φ.
However results in Chapter 4 suggest this is not the case.
The dependence on shift type we see here may be due to the fact that our ﬁgure of
merit is the magnitude of the derivative, the total variation. When we minimize a cost
function that includes total variation, we are asking the recovery algorithm to maximize
piecewise constancy. Thus, we acquire more information with a new measurement only
if the measurement vector is not probing the same piecewise constant areas. By shifting
sequentially, we probe the same piecewise constant area with the same set of measurement
vector pixels ϕ
(i)
jk
= 1, and thus acquiring less new information per measurement for most
values of δ. The results shown in the next chapter support this claim.
49
Chapter 4
Experimental Investigation of Subsampled Circulant
Matrices for Compressive Sensing
As we saw in the previous chapter, there is a marked diﬀerence in signal recovery when the
measurement matrix Φ is subsampled via a restriction operator R
Ω
where the index set Ω is
chosen either randomly or sequentially. However in that chapter, we utilized a reconstruction
algorithm based on totalvariation (TV) minimization coupled with the usual leastsquares
condition. That sequentiallybuilt Φ gather less information per measurement for the ﬁrst
measurements, and thus require more measurements for commensurate performance with
randomlybuilt Φ is one possible and quite plausible explanation for this behavior. Plausible
because TV minimization ﬁnds images with the largest possible regions of small derivative,
so it favors piecewise constancy. An intuitive argument is that if a set of measurement
vectors sample identical regions of nearconstant light intensity from a scene, then it makes
sense that the measurements acquired would be somewhat redundant.
If this were the end of the story, then for `
1
minimization, which we reiterate serves as a
proxy for `
0
minimization, with strict equality constraints, we might expect that reconstruc
tion would not be sensitive to how we build Φ. In this chapter we show results that defy
that expectation. Furthermore these results suggest that the powerful precise undersampling
theorems proposed, developed, and derived for gaussian random measurement matrices by
Donoho and Tanner in a series of papers, [37–43] apply to partial circulant measurement
matrices as well.
Our results indicate that partial circulant or partial block circulant matrices might well
50
belong to what Donoho and Tanner call the universality class of gaussian matrices for phase
transitions with respect to signal recovery, or, as we will see shortly, phase transitions in
the number of faces of certain polytopes after projection by a measurement matrix ϕ. These
results show, perhaps surprisingly, that indeed, as Yin, et al., posited and demonstrated for
a few test cases [28], parti