A Dual Light Stage.

Full text

Page 1

Eurographics Symposium on Rendering (2005)
Kavita Bala, Philip Dutré (Editors)
A Dual Light Stage
Tim Hawkins, Per Einarsson, and Paul Debevec
USC Institute for Creative Technologies
Abstract
We present a technique for capturing high-resolution 4D reflectance fields using the reciprocity property of light
transport. In our technique we place the object inside a diffuse spherical shell and scan a laser across its surface.
For each incident ray, the object scatters a pattern of light onto the inner surface of the sphere, and we photograph
the resulting radiance from the sphere’s interior using a camera with a fisheye lens. Because of reciprocity, the
image of the inside of the sphere corresponds to the reflectance function of the surface point illuminated by the
laser, that is, the color that point would appear to a camera along the laser ray when the object is lit from each
direction on the surface of the sphere. The measured reflectance functions allow the object to be photorealistically
rendered from the laser’s viewpoint under arbitrary directional illumination conditions. Since each captured re-
flectance function is a high-resolution image, our data reproduces sharp specular reflections and self-shadowing
more accurately than previous approaches. We demonstrate our technique by scanning objects with a wide range
of reflectance properties and show accurate renderings of the objects under novel illumination conditions.
Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Capturing Real-World
Data for Rendering
1. Introduction
Image-based relighting techniques simulate novel illumina-
tion on a subject based on images acquired in different basis
lighting conditions. Most commonly, the basis images of the
subject are taken under a discrete set of directional light-
ing conditions, and a linear combination of the basis images
formed to produce a rendering of the subject under novel
illumination. Distributing the lighting directions throughout
the sphere of incident illumination allows arbitrary distant
lighting environments to be simulated accurately.
In theory, this basic relighting process can reproduce the
full range of reflectance phenomena an object can exhibit
under distant illumination, including diffuse and specular re-
flection, self-shadowing, translucency, and caustics. In prac-
tice, however, the discretization of the incident lighting di-
rections limits the technique’s ability to accurately repro-
duce high-frequency reflectance characteristics: a shiny sur-
face reflecting a diffuse lighting environment can appear to
reflect many small light sources, and shadows cast by a mov-
ing virtual light source can appear to progress in a series of
steps rather than with continuous motion.
We present a novel technique for capturing reflectance
functions that exploits the reversibility of light transport, a
property known as reciprocity. Our device, which we call a
dual light stage, measures reflectance functions by revers-
ing the traditional roles of camera and light source. Where
a camera pixel would measure radiance along an incoming
ray of light R, we instead use a laser to send light out along
the reversed ray −R. Conversely, where a light source would
normally be placed to illuminate the object, we instead sense
the light radiating from the object toward the same direction.
We sense this reflected light by placing a diffuse spherical
surface around the object, photographing the image radiated
onto this sphere with a camera. While real cameras capture
many pixels in parallel, and real lighting conditions must be
appliedoneatatime,ourduallightstagereversesthesechar-
acteristics: for a virtual camera pixel corresponding to the
current laser ray, our camera captures the response of that
pixel to all illumination directions simultaneously. From the
captured images, which represent reflectance functions, we
can produce novel renderings of the object under arbitrary
distant illumination conditions. As expected, these images
appear to be acquired from the position of the laser beam,
rather than from the position of the camera sensing the ob-
ject’s reflectance functions.
2. Background and Related Work
2.1. Image-Based Relighting
From the additive nature of light, a rendering of a scene
under novel illumination can be created as a linear com-
bination of renderings under basis lighting conditions
[Hae92, NSD94]. [DHT∗00] used a light stage device with
c ? The Eurographics Association 2005.

Page 2

T. Hawkins, P. Einarsson, & P. Debevec / A Dual Light Stage
Figure 1: (a) A traditional light stage, where an object is progressively photographed as illuminated by a number of lighting
directions. (b) A dual light stage, where a laser is scanned across the object and the scattered light forms images on the inside
of a diffuse sphere. The reflectance function images are recorded by a camera with a fisheye lens that views the entire sphere.
a single spiraling light source to capture reflectance func-
tions of faces and objects with either 2048 or 8192 direc-
tions on the sphere, and showed relightings of the objects
using HDR image-based lighting environments acquired as
in [Deb98]. [KBMK01] used a robotic arm to move a light
source to different positions around an object to acquire re-
flectance functions of approximately 150 lighting directions.
[MGW01] captured reflectance functions of the upper hemi-
sphere of relatively diffuse objects, and created realistic real-
time point-source relightings based on a parabolic fit to the
reflectance function data. In our work, we capture similar
datasets, but we reverse the direction of the illumination to
capture reflectance functions at much higher resolution than
this previous work at over 100,000 pixels, and our imaging
process allows us to record the reflectance functions with-
out aliasing as a continuous image. As a result, we are able
to better reproduce high-frequency reflectance phenomena
such as sharp specularities, self-shadowing, and caustics. In
our work, we have used a relatively low spatial image res-
olution of approximately 200×200 pixels in favor of high
reflectance resolution of the to keep the datasets below a rea-
sonable size of 8GB.
[MPDW03] captures 6-dimensional reflectance fields that
represent an object’s response to spatially-varying illumina-
tion using a movable video projector, structured light pat-
terns, and a fixed camera viewpoint. However, the resolution
in each of the four lighting dimensions was necessarily rel-
atively limited. In our work, we capture only 4D reflectance
fields, but at significantly higher resolution than in previous
work.
2.2. Environment Matting
[ZWCS99, CZH∗00] has addressed the problem of captur-
ing high-resolution reflectance function behavior for light
emanating from the background behind an object. They do
this by analyzing the reflectance of structured light patterns
projected behind the object., and fit a parametric model of
the reflectance function in this region to achieve compelling
composited results for diffuse, specular, translucent, and re-
fractive materials. Our work using reversed lighting direc-
tions in effect yields non-parametric environment mattes
from the laser light scattering directly onto the area of the
sphere behind the object; as a result, our renderings appear
to have the background composited behind them. However,
our backgrounds are effectively much lower resolution than
those achieved in [ZWCS99, CZH∗00] due to the limited
fisheye image resolution.
2.3. Hybrid Techniques
[MPN∗02] combines the image-based visual hull techniques
techniques of [MBR∗00], the reflectance field acquistion
technique of [DHT∗00], and the environment matting tech-
nique of [ZWCS99] to capture high resolution parametric
reflectance functions and 3D geometry hull using LCD mon-
itors and a sparse set of light sources. [MPZ∗02] further
extended these techniques to apply to specularly reflective
and refractive objects. Our work has not focussed on three-
dimensional acquisition, but shows a unified approach to
environment matte and reflectance function acquisition by
imaging the complete sphere of scattered light from an ob-
ject with relatively detailed resolution.
c ? The Eurographics Association 2005.

Page 3

T. Hawkins, P. Einarsson, & P. Debevec / A Dual Light Stage
2.4. Spatially-Varying Reflectance Measurement
[LKG∗01] uses multiple illumination conditions and view-
points and a BRDF clustering technique to recover spatially
varying diffuse and specular reflectance behavior of objects
with known geometry. In [GLL∗04] the subsurface scatter-
ing properties of translucent objects are captured by illu-
minating the object with a dense set of laser points. For
each surface position, its impulse response is registered by
measuring the outgoing light intensity at every other surface
points. Our work resembles [GLL∗04] in that we also scan
a laser across the object surface and we record its light scat-
tering characteristics. Unlike [GLL∗04] we do not relate our
reflectance function images to object geometry and do not
render from new viewpoints or spatially-varying illumina-
tion. However, for single viewpoints and distant lighting, our
process simulate a wider range of reflectance properties in-
cluding diffuse and specular reflection.
2.5. Reciprocity
Reciprocity is a fundamental property of light transport that
is widely used in computer graphics. It is most commonly
used to refer to the invariance of the BRDF with respect to
exchanging the outgoing and incoming angle: the ratio of
the scattered light will be identical if the two directions are
interchanged.
fr(ωi→ ωo) = fr(ωo→ ωi)
(1)
However, reciprocity can be applied more generally to the
global light transport in any static scene. This is often re-
ferred to as the Helmholtz reciprocity principle [vH25], al-
though, as observed in [Vea98] and elsewhere, he stated only
a restricted form of the principle which was extended by oth-
ers [Ray00]. This principle states that any path of a light
beam is always reversible, and that the relative power loss
is the same for propagation in both directions. Rendering by
ray tracing implicitly assumes this more general reciprocity
condition.
[Vea96, Vea98] provide analyses of the limitations and
proper interpretation of reciprocity. [Pot04] also provides a
thorough overview.
Helmholtz reciprocity has been exploited for imaging in
a number of ways, including the flying spot scanners used
in early experimental television and in telecine systems,
and more recently in scanning optical microscopy, color
and range scanning [BRG92], and in the design of compact
scanning endoscopes [SSBR01]. Reciprocity has also been
used in recent years to aid stereo correspondence for non-
Lambertian surface reconstruction [MKZB01]. [ZBK02] ex-
ploits reciprocity for surface reconstruction in an approach
that jointly estimates accurate surface normals and stereo
correspondences.
In a similar spirit to our work, [SCG∗] demonstrates the
fact that cameras and video projectors are dual through reci-
procity. They show that spatially varying illumination effects
can be produced from multiple lighting directions using an
array of cameras to produce views from the viewpoint of
a video projector. Our work differs in that we use a laser
rather than a video projector allowing us to efficiently cap-
ture impulse response reflectance characteristics. While we
do not capture the effects of spatially varying illumination,
we are able to record directional illumination as a continu-
ous image, which removes the possibility of aliasing in the
reflectance functions.
3. Apparatus
Ourscanningsetup,seeninFigure2,consistsofa140cmdi-
ameter sphere, a high-speed color video camera with a fish-
eye lens, a 3-watt white laser, and a 2-axis galvanometer.
The sphere is constructed from two acrylic domes, which
are painted with 33% reflective diffuse grey primer on the
inside. The laser enters the sphere through a 13 cm diameter
hole somewhat above the equator of the sphere. Similarly,
we made a small observation hole at the top of the sphere for
the camera to view into the sphere, and a small hole in the
bottom of the sphere for a stand to hold the object we wish
to scan.
Our laser is a mixed gas argon-krypton ion laser with
strong spectral emission lines at 488, 512, and 635 nm, and
with several weaker spectral lines. The laser output power is
variable and at maximum power the laser emits 3 watts of
visible radiation. (This Class 4 laser is not eye safe, but the
fact that it immediately enters an enclosed sphere mitigates
the safety issue.) The laser appears white to the human eye,
and in conjunction with our color video camera allows us
to capture reflectance functions in color. However, the small
amount of energy in the yellow range of the spectrum yields
poor color discrimination between yellow, orange, and red.
White lasers with better spectral distributions are available,
such as that used in [GLL∗04].
Our spatial resolution is limited by the laser dot size,
which is one millimeter. Higher resolution could be achieved
by focusing the laser using additional optics.
Although we use a high-speed video camera and a high
power laser in our setup, this is not a necessity. The high
light output allows frames to be captured with short expo-
sures ranging from 20 µsec to 2 ms. In conjunction with the
high framerate of the video camera, this speeds up the cap-
ture process significantly. For slower scanning, a more cost
effective solution could use a less powerful laser and a low
cost machine vision camera.
4. Data Capture
To capture a reflectance field of an object, the galvanometer
scans the laser dot through about 200 horizontal scanlines
c ? The Eurographics Association 2005.

Page 4

T. Hawkins, P. Einarsson, & P. Debevec / A Dual Light Stage
(a) (b)(c) (d)
Figure 3: (a) A glass of wine illuminated by a laser spot, showing complex reflectance and scattering. (b)-(d) Three of the
captured reflectance functions for the glass of wine. These are fisheye images of the inside of the diffuse sphere, taken from
above. The object itself is visible in the center of each image. These central pixels are not valid reflectance function data, and
we mask them for subsequent processing. The large red swath in (b) corresponds to a caustic focussed through the wine.
Figure 2: (a) A dual light stage comprises a sphere, a laser,
a 2-axis galvanometer scanner, and a video camera. (b) The
top half of the sphere has been removed to show the object
inside. We can see the patterns of reflected light on the inside
of the sphere.
over the surface of the object. Approximately 200 times dur-
ing each scanline, the video camera captures an image of the
reflected irradiance on the inside of the sphere, representing
the reflectance function of a virtual pixel sampling the same
ray as the current laser ray. Examples of captured reflectance
function images for two different objects are seen in Fig-
ure 3. The typical resolution of 200 × 200 is appropriate
given the size of the scanned objects and the one millimeter
diameter of the laser beam. The resolution of the reflectance
functions themselves is determined by the resolution of the
video camera, which in our case is set to 384×384.
To cover the high dynamic range of the reflectance func-
tions we repeat each scan three times using different expo-
sure times. The three scans are processed into a single high
dynamic range dataset. This approach requires that the gal-
vanometers be very repeatable, which we found to be the
case. Each of the three scans takes approximately three min-
utes, for a total scan time of less than ten minutes. However,
because each scan represents 8 GB of data which must be
transferred to a hard drive, the actual elapsed time for a high
dynamic range scan is approximately one hour.
5. Geometric Mapping
The reflectance function images captured by the fisheye
camera record the amount of light scattered by the object
in all directions. Each pixel in the reflectance function cor-
responds to an outgoing direction from the center of the
sphere. To find this mapping we first model the fisheye lens
such that pixel (u,v) ∈ [−1,1]×[−1,1] maps to a vector
ω = (ωx,ωy,ωz) as follows:
θ = tan−1 −u
φ = 2sin−1?
ω = (sinθ cosφ, sinθ sinφ, −cosθ)
v
u2+v2
2
(2)
For each direction, we form a ray with origin at (0,1,0)
and direction (ωx,ωy,ωz) and intersect it with a unit sphere
to find the intersection (x,y,z) which corresponds to the re-
flected direction for pixel (u,v) relative to the center of the
c ? The Eurographics Association 2005.

Page 5

T. Hawkins, P. Einarsson, & P. Debevec / A Dual Light Stage
sphere:
(x,y,z) = (−2ωzωx,−2ωzωy,1−2ω2
z)
(3)
Finally, we rotate (x,y,z) to be in the laser’s coordinate
system. A lighting environment transformed to this coordi-
nate system is shown in Figure 4.
Figure 4: A light probe image in angular map format (left),
and our fisheye lens representation (right).
6. Inverse Rendering
Since our sphere is concave, light will reflect multiple times
inside the sphere before it goes towards the camera. In order
tointerpretthephotographsofthereflectedradianceontothe
sphere’s interior as reflectance functions, we need to subtract
the effect of these interreflections in the sphere. We remove
this indirect light from our data using an inverse rendering
technique, related to but simpler than those of [Mar98] and
[YDMH99].
We have found that the indirect light on the interior of a
sphere is actually uniform over the entire sphere, and can be
quickly computed and subtracted from each image to form
an accurate reflectance function. The irradiance from indi-
rect illumination E, at a surface point x, is the integral of
radiance L, over the whole surface area of the sphere:
E(x) =
?
ALcosθicosθo
r2
dA
(4)
As illustrated in Figure 5 , θiis the angle between the
incident light direction and the surface normal, θois the an-
gle between the exitant light direction and its correspond-
ing surface normal, and r is the distance between the two
points. From basic spherical geometry, we have θo= θiand
r = 2cosθ. It follows that the indirect irradiance is constant
for every point inside the sphere:
E =
?
A
L
4dA = π Lavg
(5)
where Lavgis the average radiance of the sphere. To com-
pute the corrected reflectance function R for pixel (x,y), we
subtract Lavgmultiplied by the Lambertian reflectance of the
sphere from each pixel:
R(x,y) = L(x,y)−ρdLavg
(6)
An example of a reflectance function before and after this
correction can be seen in Figure 6.
We note that the principle we are using to correct for in-
direct light also explains the success of integrating spheres
in producing extremely even illumination fields. The princi-
ple implies that when a light source illuminates a point on
the inside of a sphere, the radiosity of all other points inside
the sphere is constant and independent of the position of the
light source. Though no surface is perfectly Lambertian, we
found the Lambertian assumption was sufficient to remove
most of the indirect light from our reflectance functions. We
also assume that the scanned object’s contribution to indirect
lighting is negligible since our objects are relatively small
compared to the size of the sphere.
Figure 5: The geometry for inverse rendering.
(a)(b)
Figure 6: A reflectance function before (a) and after (b)
bounce light subtraction. Since the bounce light correction
is just a uniform subtraction, the features of the reflectance
function are unchanged, but it appears darker. Note that
these reflectance functions are shown at a bright exposure
to make the bounce light apparent.
c ? The Eurographics Association 2005.