DataPDF Available

VAST 2006 Paper1053 Photometric

Authors:
  • INSIGHT (Institute for the Study and Integration of Graphical Heritage Techniques)
The 7th International Symposium on Virtual Reality, Archaeology and Cultural Heritage
VAST (2006), pp. x – x
J. Paterson, K. Cai n
Efficient Field Capture of Epigraphy via Photometric Stereo
J. Paterson† and K. Cain
†Oxford 3D Te chnology
‡Institute for the Study and Integration of Graphical Heritage Techniques
Abstract
We describe a highly portable field technique for estimating surface normals, geometry and albedo from walls
and other areas of archaeological sites using limited sets of digital photographs. Surface geometry and albedo
are extracted from photometric calculations, yielding a complete model with estimated per-vertex colour. This
technique is demonstrated to be practical in pre-production for the digital planetarium film Maya Skies.
Categories and Subject Descriptors (according to ACM CCS) I.3.3 [Computer Graphics]: Modeling – Object
Scanning /Acquisition, Modeling – Appearance Modeling
1. Introduction
Photometric stereo has been shown to be an effective
method for capturing hi gh-resolution geomet ry and reflec-
tance properties of ancient inscriptions [EHD04]. Classical
photometric stereo [WOO80] requires a static camera pose
for each photograph acquired in a given image set, while
lighting varies. Our technique relaxes the fixed camera
constraint required by previous systems in order to de-
crease the time required to document epigraphic inscrip-
tions in situ. In practice, our technique obviates the need
for a camera tripod and multiple light sources during cap-
ture, which can simplify field work. We demonstrate that
efficient field capture for ~700cm2 regions can be com-
pleted in ~15 seconds, and that subsequent processing can
be completed quickly and with minimal user interaction.
2. System overview
Our technique offers a portable and convenient approach
to photometric capture. Figure 1 shows the key components
of the system. A standard digital still camera and flash unit
are rigidly attached using a simple aluminium boom
(shown right). This assures that geometrical relationship
between the camera and light source is fixed. The second
component is a light gathering frame which contains a
fiducial dot pattern and four cones (shown left). During
acquisition, the frame is positioned such that the surface
under inspection is visible through a central hole in the
frame.
The position and orientation of the camera-flash system is
varied during capture, yielding several photometric digital
images. Camera localization is derived from the observed
positions of the five fiducial dot markers in each image,
and incident lighting direction is inferred from the shadows
cast by the cones on the planar region of the frame. Knowl-
edge of the geometrical relationships between camera,
flash and frame allows photometric surface reconstruction
under a changing camera viewpoint. The frame also pro-
vides image-based light attenuation correction [PCF05].
Typically the surfaces under inspection are well modelled
by a Lambertian reflectivity assumption, although other
BRDF models could be applied [PCF05].
Figure 1: System equipment. Light gathering frame (left)
Camera and flash attached via boom (right).
3. Geometry and texture estimation
3.1 Image acquisition
Our subject in Figure 2 is a small region of the Venus
Platform, a structure in situ at the Maya site Chichén Itzá,
J. Paterson & K. Cain / Efficient Field Capture of Epigraphy via Photometric Stereo
Submitted to The 7th International Symposium of Virtual Reality, Archaeology and Cultural Heritage VAST (2006)
located in Yucatan, México. At Chichén Itzá, we acquired
image sets for twelve test regions during field work in
October 2005. These sample areas were selected to repre-
sent texture variations observed throughout Chichén Itzá.
The stone surfaces under study all feature a dominant
Lambertian component in their BDRF, and therefore prove
highly suitable for high quality reconstruction using
photometric stereo techniques.
Figure 2: Fiducial frame in-situ at Chichén Itzá (left).
Image data, showing fiducial markers (right).
3.2 Camera calibration and lens distortion correction
Camera calibration is a prerequisite to accurate optical
localization, deriving specified parameters, e.g. focal
length and a model of lens distortion. This is a well-
explored topic in the literature, with several approaches in
regular use. We have t ested geometric reconstructions
using images processed using 1) the Intel OpenCV library /
Bouget camera calibration toolbox and 2) the Rational
Function radial distortion model of [CLF05].
In the first case we rely on existing code and to compute
camera intrinsics using a sequence of checkerboard images
(Figure 3). Here, lens distortion is modelled with a low-
order radial polynomi al along with tangential distortion.
The softwar e provides methods for resampling an input
image to remove lens distortion.
The second approach instead models lens distortion ef-
fects using a rational function approach. Calibration can be
achieved using a single image of a scene containing
straight lines. Let x,y signify canonical image coordinates,
and U the distorted version as a homogeneous vector, then:
[]
T
yxyxyxAU 1
22
= (Eqn 1)
where A is a 3x6 parameter matrix.
Both techniques tested yielded acceptable results. Correc-
tion for lens distortion proved critical in achieving quality
reconstructions due to adverse effects of distortion on
camera localization.
Figure 3: Images used to compute camera intrinsics.
3.3 Image processing
Our end-user application “SurfaceImager” encapsulates
the complete reconstruction pipeline, incorporating means
for user input where necessary. Having loaded the photo-
metric images and calibration parameters, the user per-
forms segmentation, defining the target area (Figure 4a).
The fiducial markers on the frame are automatically identi-
fied and the camera pose estimated (Figure 4b). Finally, the
effective position of the flash bulb is determined by search-
ing for the shadows cast by the cones (Figure 4c), a task
achieved in a manner akin to tracing the shadow of the
gnomon on a traditional sundial. As the flash is fixed rela-
tive to the camera, information can be combined across
multiple images in a global optimization.
(a) (b) (c)
Figure 4: (a) Interactive image segmentation. (b) Auto-
matic fiducial detection. (c) Shadow tip localization.
4. Results - Geometry and texture estimation
Photometric stereo estimates geometry as a set of surface
normals, along with parameters specifying a model of
surface reflectivity. This is done by inverting the model
given a number of samples of surface intensity with known
incident lighting direction. With a Lambertian surface
model the camera observed surface intensity I is defined as:
LNI
r
r
.
ρ
= (Eqn 2)
Where ρ indicates surface reflectance, L
r
surface-
relative lighting direction and N
r
the surface normal.
Standard photometric stereo maintains a static camera
viewpoint. Photometric samples are acquired by varying
light direction and observing intensity change at a single
pixel. Denoting lighting direction and intensity at a given
pixel across the N photometric images with subscript, then:
[]
[]
NLLLIII T
N
T
N
r
r
L
r
r
L2121 = (Eqn 3)
Clearly this can be solved by premultiplication with the
pseduoinverse of the L matrix (this can be achieved in an
J. Paterson & K. Cain / Efficient Field Capture of Epigraphy via Photometric Stereo
Submitted to The 7th International Symposium of Virtual Reality, Archaeology and Cultural Heritage VAST (2006)
efficient manner, for example via SVD), providing an
estimated normal for each camera pixel.
To apply photometric techniques, our system first rectifies
the input images to a common viewpoint. This can be done
approximately using a planar perspective un-warping.
Figure 5 shows a set of input images along with their corre-
sponding rectified views. A 3D range map is then derived
by integrating the recovered surface normals.
Total image acquisition time for the images shown in
Figure 5 was ~15 seconds. Rectification and photometric
stereo processing steps for a mesh with dimensions
1024x592 were completed in ~6 seconds, using an AMD
x4600 processor running a 64-bit operating system.
(a) (b) (c) (d)
(e) (f)
Figure 5: (a) – (d) Input photometric images, captured
with varying camera and lighting direction (top row).
Rectified versions; note static viewpoint, moving illumina-
tion direction (bottom row). (e) Reconstructed 3D surface
with light ed col our al bedo. (f) Jet colouring scheme high-
lights detailed geometry.
A discernable bevel is seen at the boundary of the recon-
struction illustrated in Figure 5. This artefact is seen when,
for a given pixel, the light-gathering frame casts a shadow
onto the subject in one of the four input images. An ap-
proach in which shadowed pixels are excluded from
photometric stereo calculations in a reliable manner is
proposed in [CJ05]. In Figure 6, shadowed regions are
culled in the reconstructions shown.
(a) (b)
(c) (d)
Figure 6. (a) A second reconstructed 3D surface with
lighted colour albedo. (b) Jet colouring scheme highlights
detailed geometry. (c) – (d) Colour albedo and jet colour-
ing for a third example.
4. Downstream use in film production
Complete digital versions for specific structures at
Chichén Itzá are required for the educational planetarium
film Maya Skies. As the final rendered material will be
presented on hemispherical screens measuring up to 70’ in
diameter, subtle epigraphic details will be clearly visible to
planetaria viewers when geometry is placed near the scene
camera. Even distant regions of the scene may require high
levels of detail, since each rendered frame will be
4,096x4,096 pixels. Given the demands imposed by the
full-dome output format, high resolution geometry and
texture detail are highly desirable. However, since the
required models must include very detailed features of the
type seen in Figure 7, mesh simplification is clearly re-
quired prior to rendering.
Figure 7. Complex epigraphic detail seen at Chichén Itzá
demands geometric simplification for efficient rendering
while preserving the features seen above.
Sever al techniques have been proposed to achieve a high
level of detail at a relatively low geometric cost using
normal mapping, notably [CMSR98]. Using Sur-
faceImager, we export geometry with texture UV informa-
tion in .OBJ format, accompanied by an image texture map
in .BMP format. The output geometry is then decimated
into a low resolution triangle model for use as a normal
mapping basis. We compute the normal map using both
the original output geometry and t he decimated geometry;
results are seen in Figure 8.
J. Paterson & K. Cain / Efficient Field Capture of Epigraphy via Photometric Stereo
Submitted to The 7th International Symposium of Virtual Reality, Archaeology and Cultural Heritage VAST (2006)
(a) (b)
Figure 8: (a) Color albedo image from photometric stereo
calculations. (b) Normal map for the same region, com-
puted from SurfaceImager geometry.
Next, we load the decimated geometry into Alias Maya, the
application selected for lighting and rendering in Maya
Skies. Here, we apply the original color albedo map ex-
ported by SurfaceImager and the computed normal map to
the low resolution geometry. Results rendered from Maya
with a single light source are seen in Figure 9.
Figure 9. Recovered geometry and color albedo, lighted
and rendered in Alias Maya. A single light source (simu-
lating sunlight) varies in position with each frame.
Despite the high level of geometric simplification in the
decimated base model, the normal map effectively pre-
serves the details seen in the original photometric recon-
struction. We found that automated rapid synthesis of
normal maps from our photometric stereo approach was
practical, requiring ~12 seconds to compute maps at
1024x1024.
5. References
[CJ05] COLEMAN , E. AND JAIN, R.: Obtaining 3-
dimensional shape of textured and specular
surfaces using four source photometry. In Computer
Graphics and Image Processing, 18:309-328, 1982.
[CLF05] CLAUS, D. AND FITZGIBBON, A: A Rational Func-
tion Lens Distortion Model for General Cameras. In
Proceedings of the IEEE Conference on Computer Vi-
sion and Pattern Recognition (2005) 213-219.
[CMSR98] CIGNONI, P., MONTANI, C., SCOPIGNO, R. AND
ROCCHINI, C.: A general method for preserving attribute
values on simplified meshes. In IEEE Visualization 1998
59-66.
[EHD04] EINARSSON, P., HAWKINS , T. AND DEBEVEC, P.:
Photometric Stereo for Archeological Inscriptions. In
Proceedings of the SIGGRAPH 2004 Conference on
Sketches and Applications, ACM Press / ACM SIG-
GRAPH, Los Angeles, Cal iforni a, USA, August 2 004.
[PCF05] PATERSON, J. A., CLAUS, D. AND FITZGIBBON, A.:
BRDF and geometry capture from extended inhomoge-
neous samples using flash photography. In Computer
Graphics Forum (Special Eurographics Issue) vol. 4(3),
(2005) 383-391.
[WOO80] WOODHAM, R. J.: Photometric method for de-
termining surface orientation from multiple images.
Optical Engineering 19, 1, (1980) 139-144.
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
Many sophisticated solutions have been proposed to reduce the geometric complexity of 3D meshes. A problem studied less often is how to preserve on a simplified mesh the detail (e.g., color, high frequency shape detail, scalar fields, etc.) which is encoded in the original mesh. We present a general approach for preserving detail on simplified meshes. The detail (or high frequency information) lost after simplification is encoded through texture or bump maps. The original contribution is that preservation is performed after simplification, by building set of triangular texture patches that are then packed in a single texture map. Each simplified mesh face is sampled to build the associated triangular texture patch; a new method for storing this set of texture patches into a standard rectangular texture is presented and discussed. Our detail preserving approach makes no assumptions about the simplification process adopted to reduce mesh complexity and allows highly efficient rendering. The solution is very general, allowing preservation of any attribute value defined on the high resolution mesh. We also describe an alternative application: the conversion of 3D models with 3D static procedural textures into standard 3D models with 2D textures.
Article
Full-text available
Many sophisticated solutions have been proposed to reduce the geometric complexity of 3D meshes. A less studied problem is how to preserve on a simplified mesh the detail (e.g. color, high frequency shape detail, scalar fields, etc.) which is encoded in the original mesh. We present a general approach for preserving detail on simplified meshes. The detail (or high frequency information) lost after simplification is encoded through texture or bump maps. The original contribution is that preservation is performed after simplification, by building set of triangular texture patches that are then packed in a single texture map. Each simplified mesh face is sampled to build the associated triangular texture patch; a new method for storing this set of texture patches into a standard rectangular texture is presented and discussed. Our detail preserving approach makes no assumptions about the simplification process adopted to reduce mesh complexity and allows highly efficient rendering. The sol...
Article
A novel technique called photometric stereo is introduced. The idea of photometric stereo is to vary the direction of incident illumination between successive images, while holding the viewing direction constant. It is shown that this provides sufficient information to determine surface orientation at each image point. Since the imaging geometry is not changed, the correspondence between image points is known a priori. The technique is photometric because it uses the radiance values recorded at a single image location, in successive views, rather than the relative positions of displaced features. Photometric stereo is used in computer-based image understanding. It can be applied in two ways. First, it is a general technique for determining surface orientation at each image point. Second, it is a technique for determining object points that have a particular surface orientation. These applications are illustrated using synthesized examples.
Thephotometric stereo method is extended to four-source photometry for obtaining 3-dimensional shapes of visually textured and specular surfaces from intensity values. The table look-up approach (based on reflectance maps) for determining surface normals is shown to be of limited value in this context and a more direct method of computing these normals is used. Our method also eliminates the requirement of a preliminary system calibration procedure. The method is applied to real images as a test of its applicability.
Conference Paper
We introduce a new rational function (RF) model for radial lens distortion in wide-angle and catadioptric lenses, which allows the simultaneous linear estimation of motion and lens geometry from two uncalibrated views of a 3D scene. In contrast to existing models which admit such linear estimates, the new model is not specialized to any particular lens geometry, but is sufficiently general to model a variety of extreme distortions. The key step is to define the mapping between image (pixel) coordinates and 3D rays in camera coordinates as a linear combination of nonlinear functions of the image coordinates. Like a "kernel trick", this allows a linear algorithm to estimate nonlinear models, and in particular offers a simple solution to the estimation of nonlinear image distortion. The model also yields an explicit form for the epipolar curves, allowing correspondence search to be efficiently guided by the epipolar geometry. We show results of an implementation of the RF model in estimating the geometry of a real camera lens from uncalibrated footage, and compare the estimate to one obtained using a calibration grid.
Obtaining 3- dimensional shape of textured and specular surfaces using four source photometry A: A Rational Function Lens Distortion Model for General Cameras
  • References [ Cj05 ] Coleman
  • E And
  • R Jain
References [CJ05] COLEMAN, E. AND JAIN, R.: Obtaining 3- dimensional shape of textured and specular surfaces using four source photometry. In Computer Graphics and Image Processing, 18:309-328, 1982. [CLF05] CLAUS, D. AND FITZGIBBON, A: A Rational Function Lens Distortion Model for General Cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2005) 213-219.
Photometric Stereo for Archeological Inscriptions BRDF and geometry capture from extended inhomogeneous samples using flash photography
  • Ehd
  • P Einarsson
  • T Hawkins
  • And
  • P Debevec
  • Pcf
  • J A Paterson
  • D And
  • A Fitzgibbon
[EHD04] EINARSSON, P., HAWKINS, T. AND DEBEVEC, P.: Photometric Stereo for Archeological Inscriptions. In Proceedings of the SIGGRAPH 2004 Conference on Sketches and Applications, ACM Press / ACM SIG- GRAPH, Los Angeles, California, USA, August 2004. [PCF05] PATERSON, J. A., CLAUS, D. AND FITZGIBBON, A.: BRDF and geometry capture from extended inhomogeneous samples using flash photography. In Computer Graphics Forum (Special Eurographics Issue) vol. 4(3), (2005) 383-391.
Photometric Stereo for Archeological Inscriptions
  • P Einarsson
  • T Hawkins
  • P Debevec
EINARSSON, P., HAWKINS, T. AND DEBEVEC, P.: Photometric Stereo for Archeological Inscriptions. In Proceedings of the SIGGRAPH 2004 Conference on Sketches and Applications, ACM Press / ACM SIG-GRAPH, Los Angeles, California, USA, August 2004.
BRDF and geometry capture from extended inhomogeneous samples using flash photography
  • J A Paterson
  • D Claus
  • A Fitzgibbon
PATERSON, J. A., CLAUS, D. AND FITZGIBBON, A.: BRDF and geometry capture from extended inhomogeneous samples using flash photography. In Computer Graphics Forum (Special Eurographics Issue) vol. 4(3), (2005) 383-391.