Recent developments in coded aperture multiplexed imaging systems.
ABSTRACT We will review recent developments in coded aperture techniques for unconventional imaging systems. Specifically, we are interested in looking simultaneously in multiple directions using a common aperture. To accomplish this, we interleave several sparse sub-apertures that are pointed in different directions. The goal is to optimize the sub-apertures so that the point spread function (PSF) is well behaved, and resolution is preserved in the images. We will present an analysis of the underlying PSF design concept, as well as the necessary phase optimization techniques.
- SourceAvailable from: Christopher W. Slinger[Show abstract] [Hide abstract]
ABSTRACT: Coded aperture imaging has been used for astronomical applications for several years. Typical implementations used a fixed mask pattern and are designed to operate in the X-Ray or gamma ray bands. Recently applications have emerged in the visible and infra red bands for low cost lens-less imaging systems and system studies have shown that considerable advantages in image resolution may accrue from the use of multiple different images of the same scene - requiring a reconfigurable mask. Previously we reported on the early feasibility of realising such a mask based on polysilicon micr-opto-electromechanical systems (MOEMS) technology and early results in the visible and near IR bands. This employs interference effects to modulate incident light - achieved by tuning a large array of asymmetric Fabry-Perot optical cavities via an applied voltage whilst a hysteretic row/column addressing scheme is used to control the state of individual elements. In this paper we present transmission results from the target mid-IR band (3-5mum), compare them with theory and describe the scale up from a 3x3 proof-of-concept MOEMS microshutter array to a 560 x 560 element array (2cm x 2cm chip) with the associated driver electronics and embedded control - including aspects of electronic design, addressing control and integration. The resultant microsystem represents a core building block to realise much larger reconfigurable masks using a tiled approach with further integration challenges in the future.Proc SPIE 01/2008; 7096(Figure 1):70960D-70960D-8.
- [Show abstract] [Hide abstract]
ABSTRACT: Coded aperture imaging has been used for astronomical applications for several years. Typical implementations used a fixed mask pattern and are designed to operate in the X-Ray or gamma ray bands. Recently applications have emerged in the visible and infra red bands for low cost lens-less imaging systems and system studies have shown that considerable advantages in image resolution may accrue from the use of multiple different images of the same scene - requiring a reconfigurable mask. Previously we reported on the realization of a 2x2cm single chip mask in the mid-IR based on polysilicon micro-opto-electro-mechanical systems (MOEMS) technology and its integration with ASIC drive electronics using conventional wire bonding. The MOEMS architecture employs interference effects to modulate incident light - achieved by tuning a large array of asymmetric Fabry-Perot optical cavities via an applied voltage and uses a hysteretic row/column scheme for addressing. In this paper we present the latest transmission results in the mid-IR band (3-5mum) and report on progress in developing a scalable architecture based on a tiled approach using multiple 2 x 2cm MOEMS chips with associated control ASICs integrated using flip chip technology. Initial work has focused on a 2 x 2 tiled array as a stepping stone towards an 8 x 8 array.Proc SPIE 01/2009; 7468:74680E-74680E-9.
Recent Developments in coded aperture multiplexed imaging systems
A. Mahalanobis, C. Reyner, T. Haberfelde
Lockheed Martin Missiles and Fire Control, 5600 Sand Lake Road, Orlando, Florida, 32819
Mark Neifeld, University of Arizona
B.V.K. Vijaya Kumar, Carnegie Mellon University
We will review recent developments in coded aperture techniques for unconventional imaging
systems. Specifically, we are interested in looking simultaneously in multiple directions using a
common aperture. To accomplish this, we interleave several sparse sub-apertures that are pointed in
different directions. The goal is to optimize the sub-apertures so that the point spread function (PSF)
is well behaved, and resolution is preserved in the images. We will present an analysis of the
underlying PSF design concept, as well as the necessary phase optimization techniques.
The problem of wide area persistent surveillance can be addressed by designing a sensor that looks
down from a very high altitude, and continuously images over a large area. The challenge is to
achieve the resolution needed to detect and track objects of interest. Under the LACOSTE program,
we have developed a concept that employs interleaved sub-apertures with multiplex look directions
to achieve resolution while simultaneously maintaining wide area coverage. Another key constraint
that has guided our system design is the desire to use a minimum number of focal plane arrays
(FPAs) to reduce the cost as well as size, weight, and power requirements of the sensor.
The idea of using computational optics  and point spread function engineering  has been
reported by many researchers. These techniques offer powerful methods for designing and
optimizing non-conventional imaging systems. We use an image multiplexing approach that is
multi-scale hierarchical, and relies on image coding in the Fourier plan. The basic approach is
shown in Figure 1, although the exact dimensions and numbers of components vary with the design.
The overall Mask consists of several apertures, each of which is dedicated to address a section of
the ground. In turn, each aperture consists of several interleaved sub-apertures that address some
part of the assigned section at full resolution. Thus in principle, a small region of the ground can be
sensed with the FPA simply by looking through one such sub-aperture. However to gather sufficient
photons, it becomes necessary to simultaneously look through several sub-apertures and image
several sections of the ground on the FPA. The trick is to then demultiplex the different images 
and put them back together to create the large scene.
Visual Information Processing XVII, edited by Zia-ur Rahman, Stephen E. Reichenbach, Mark Allen Neifeld,
Proc. of SPIE Vol. 6978, 69780G, (2008) · 0277-786X/08/$18 · doi: 10.1117/12.781462
Proc. of SPIE Vol. 6978 69780G-1
2008 SPIE Digital Library -- Subscriber Archive Copy
Figure 1: In the basic approach, the ground is divided into several sections that are addressed by an array of
independent apertures. In turn each aperture has a set of interleaved sub-apertures that multiplex several
look directions on the same FPA. Image decoding techniques are used to separate the look directions and
reconstruct the original image
To achieve the image coding and simultaneously look in multiple directions, we need an active
device such as the “Eyelid” array  shown in Figure 2. These eyelids act as tiny shutters that can
be operated in groups to open or close different sub-apertures. The eyelid actuator consists of
alternating polymer and metal layers forming a curled film that is anchored to a substrate. The
device also contains a fixed conductor deposited on the substrate. Electrostatic actuation is achieved
by applying an electric field between the metal electrode in the flexible film stack and the fixed
conductor that unrolls the flexible film over the substrate surface. The curl in the device is caused
by thermal expansion differences between the metal and polymer layers in the thin film stack and
can be varied by changing the thin-film layer thicknesses.
The aperture is tiled with an array of eyelids that are divided into groups, each dedicated to look at a
different region on the ground. This is achieved by using interleaved beam-steering functions that
causes each group of eyelid to “look” at a unique direction as shown in Figure 3. The beam-steering
function can be realized using diffractives, various types of prisms, and other optical strategies that
offer different advantages in system trade-studies. The resulting images can be enhanced to remove
the effects of sparse aperture using well known techniques for image restoration .
The rest of the paper is organized as follows. Section 2 discusses the concept of interleaving bulk
prisms across an aperture to simultaneously look in multiple directions. Section 3 presents the
results of simulations that show it is possible to use small phase delays (shims) to optimize the PSF.
A summary and direction for future work are presented in Section 4.
Proc. of SPIE Vol. 6978 69780G-2
2. Beam Steering using Interleaved Prisms
At the heart of the system described in Section 1 is a set
of interleaved sparse apertures, each of which is
dedicated to look in a particular direction. Although
there are several methods for steering look directions, a
conceptually easy approach to point the optical axis in a
particular direction is to use bulk prisms. Consider the
case shown in Figure 3(a) where the aperture looks in
one of two directions (one at a time) using two different
bulk prisms. However, for the aperture to simultaneously
look in both directions, the prisms have to be interleaved
into a common surface as shown in Figure 3(b).
Essentially, complimentary sections of the two prisms
may be fitted together creating a surface that follows the
profile of one or the other prism. In Figure 3(b), we also
see that individual prism sections can be thought of as a small triangular micro-prism at the top, and
a rectangular base or shim whose height depends on the section of the bulk prism from which the
piece is obtained.
Figure 3: An aperture can be made to look separately in different directions using bulk prisms (a). The
interleaved arrangement in (b) allows the aperture to look in both directions simultaneously. The sections of
the bulk prisms in (b) can be thought of as a small mircro-prism on top of a shim.
The benefit of using a bulk prism is that over a narrow band of frequencies, the look direction is
nearly independent of the wavelength and no chromaticity effects are observed.. The problem of
interleaving bulk prisms is also conceptually simple, but unfortunately difficult to realize in practice
partly due to large differences between the heights of the shims of adjacent pieces. Therefore it is of
interest to reduce these while maintaining the quality of the PSF. This idea is shown in Figure 4
Figure 2: An example of a small array of
eyelids produced by RTI that are used to
dynamically control the sub-aperture and
its look directions.
Proc. of SPIE Vol. 6978 69780G-3
where segments of the bulk prism that all point in the same direction are replaced by identical
micro-prisms with considerably smaller shims than the original sizes based on the bulk prism’s
profile. The goal is to optimize the choice of the smaller shims so that resulting phase approximates
that of the original bulk prism. The trade however is that in making the shorter shims, chromaticity
effects become pronounced and must be factored into the analysis.
Figure 4: The sections of the bulk prism on the left are replaced by micro-prisms and smaller shims.
2.1. Optimizing phase
The question arises how best to optimize the choice of the smaller shims so that the effective phase
across the aperture behaves like that of the bulk prism. Let us assume that N sections of a bulk
prism are distributed randomly across the aperture. Each section of the bulk prism is modeled as a
micro-prism with slope η and a base of dimension a. We apply a weight
α which represents the
α is 1.0, although complex phase determined by the size of the attached shim. The magnitude of
particular sections of the aperture can be eliminated from the optimization by choosing the
α to be 0. Using n as the refractive index of the material, and λ as the wavelength,
the transmittance in the pupil plane (as a function of the spatial variable x ) is given by
( ) ()
The incoherent PSF is the magnitude square of the Fourier transform of ( ) x
of wavelengths, and is given by
where u is the spatial frequency variable in the far field approximation, and equivalently the
coordinates in the object plane. The image ( )
is modeled as the convolution of the incoherent
PSF and scene ( )
( )( ) ( )
The goal now is to select
α is such a manner that
over the range of wavelength between min
, integrated over a band
sin Eq 2
approximates the PSF of the bulk prism
Proc. of SPIE Vol. 6978 69780G-4
The problem can be formulated as a minimization of the mean square error (MSE) between the
ideal PSF and
represents the incoherent PSF of the aperture with the bulk prism
phase, we define the MSE metric
( )( )
and numerically minimize its value over all possible choices of
An insight to be gained from Eq. 2 is that the sinc function is due to the small micro-prism only, and
points in the desired look direction. However, its main-lobe is broad and cannot produce the desired
resolution in the image. On the other hand, the summation term is key to recovering resolution, and
so the phases of
α must be chosen such that this term converges to a narrow spot under the
mainlobe of the sinc envelope. The two terms then together form a well behaved PSF as we shall
see in following Section.
. Thus if ( )
3. Preliminary results
The simulations presented in this section were conducted in Matlab using the “fmincon” function to
carry out the optimization posed in Eq. (3). Approximately 400 random sections of a bulk prism
were used (i.e. j=1…400), over a narrow range of 0.2 microns in wavelengths. Figure 5(a) shows
the 2D random aperture pattern. In the absence of any pointing mechanism, the PSF of this random
sparse aperture would exhibit a well formed peak looking straight along the optical axis. Assume
now that a bulk prism with a slope towards the top left is placed across the aperture. The equivalent
PSF is shown in 5(b) which shows that the main lobe of the PSF (a narrow peak) has shifted to the
new look direction.
Figure 5: The random sparse aperture in (a) is combined with a bulk prism with a slope pointing to the
top left. The resulting “ideal” PSF is shown in (b)
Proc. of SPIE Vol. 6978 69780G-5