ArticlePDF Available

Surface Reconstruction Algorithms for Detailed Close – Range Object Modelling

Authors:

Abstract and Figures

Nowadays 3D modeling is generally performed using image or range data. Range sensors are getting a quite common source of data for modeling purposes due to their speed and ability to capture millions of points. In this paper we report about two surface measurement algorithms for precise and detailed object reconstruction from terrestrial images. Photogrammetry has all the potentialities to retrieve the same details of an object that range sensors can achieve. Using advanced measurement techniques, which combine area-based and feature-based matching algorithms we are able to generate dense point clouds of complex and free-form objects, imaged in closely or widely separated images. Different examples are reported to show the potentiality of the methods and their applicability to different close-range data sets.
Content may be subject to copyright.
SURFACE RECONSTRUCTION ALGORITHMS FOR
DETAILED CLOSE-RANGE OBJECT MODELING
Fabio Remondino, Li Zhang
Institute of Geodesy and Photogrammetry, ETH Zurich, Switzerland
E-mail: fabio@geod.baug.ethz.ch
Web: http://www.photogrammetry.ethz.ch
KEY WORDS: Surface Reconstruction, Modeling, Precision, Matching
ABSTRACT
Nowadays 3D modeling is generally performed using image or range data. Range sensors are getting a quite common source of data
for modeling purposes due to their speed and ability to capture millions of points. In this paper we report about two surface
measurement algorithms for precise and detailed object reconstruction from terrestrial images. Photogrammetry has all the
potentialities to retrieve the same details of an object that range sensors can achieve. Using advanced measurement techniques,
which combine area-based and feature-based matching algorithms we are able to generate dense point clouds of complex and free-
form objects, imaged in closely or widely separated images. Different examples are reported to show the potentiality of the methods
and their applicability to different close-range data sets.
1. INTRODUCTION
Three-dimensional modeling from images is a great topic of
investigation in the research community, even if range sensors
are becoming more and more a common source and a good
alternative for generating 3D information quickly and precisely.
3D modeling of a scene should be meant as the complete
process that starts with the data acquisition and ends with a
virtual model in three dimensions visible interactively on a
computer. The interest in 3D modeling is motivated by a wide
spectrum of applications, such as animation, navigation of
autonomous vehicles, object recognition, surveillance,
visualization and documentation.
In the last years different solutions for image-based 3D
modeling have been developed. Most of the current reliable and
precise approaches are based on semi-automated procedures,
therefore the introduction of automated algorithms is a key goal
in the photogrammetric and vision communities. 3D modeling
methods can be classified according to the level of automation
or the required input data while their strength is reflected by the
variety of scene that can be processed and the level of detail
that can be reconstructed.
The common fully automated ‘shape from video’ framework
[e.g. Fitzibbon & Zisserman, 1998; Nister, 2001; Pollefeys et
al., 2004] requires good features in the images, very short
baseline and large overlap between consecutive frames,
requirements which are not always satisfied in practical
situations, due to occlusions, illumination changes and lack of
texture. So far, automated surface reconstruction methods, even
if able to recover complete 3D geometry of an object, reported
errors between 3% and 5% [Pollefeys et al., 1999], limiting
their use for applications requiring only nice-looking 3D
models. Furthermore, post-processing operations are generally
required, which means that user interaction is still needed.
Indeed the most impressive results are achieved with interactive
methods and taking advantage of the environment constraints,
in particular for architectural objects. For different applications,
such as cultural heritage documentation, semi-automated
methods are still preferred as smoothed results, missing details
or lack of accuracy are not accepted.
In this article we report about two surface matching algorithms
developed for the precise and detailed measurement and 3D
modeling of complex and free-form terrestrial objects, like pots,
reliefs, statues, façades, etc. Commercial photogrammetric
stations generally fail with tilted close-range images, therefore
the topic still need some developments. We will concentrate
only on the measurement of the object surface, assuming the
calibration and orientation of the images already performed. As
the network configurations that allow full and precise camera
calibration are usually very different from those used for scene
reconstruction, we generally first calibrate the camera using the
most appropriate set of images and afterwards recover the
orientation parameters of the scene’s images using the
calibration results. The orientation is generally performed by
means of a photogrammetric bundle adjustment, extracting the
required tie points with automated approaches [Remondino &
Ressl, 2006] or manual measurements.
The first surface measurement algorithm presented afterwards
matches the points in image-pairs, the second one works
simultaneously with many images. Both methods require some
seed points between the images at the beginning of the process,
to initialize it and improve the performances near surface
discontinuities. The seed points can be provided manually
(stereo or monocular measurements) or extracted automatically,
leading to a fully automated surface reconstruction method. The
number of seed points depends on the set of images, their
disparity and texture content. Starting from these seeds points, a
dense and robust set of correspondences covering the area of
interest is generated.
Our research aims to combine area-based and feature-based
matching techniques to recover complete and detailed 3D
surfaces. The methods can cope with depth discontinuity, wide
baselines, repeated pattern, occlusions and illumination
changes.
In the next sections, after an overview of image-based modeling
works and matching strategies, the two matching strategies are
described in details. Then some examples demonstrating the
potentialities of the algorithms and their applicability to
different close-range data sets are reported and discussed.
Results in form of 3D point clouds, shaded and textured 3D
models are shown.
2. 3D MODELING FROM IMAGES
Recovering a complete, detailed, accurate and realistic 3D
model from images is still a difficult task, in particular if
uncalibrated or widely separated images are used. Firstly
because the wrong camera parameters lead to inaccurate or
deformed results. Secondly because a wide baseline between
the images generally requires the user interaction in the
measurement phase.
The research activities in terrestrial image-based modeling can
be generally divided in area-based [e.g. Pollefeys et al., 2004]
and feature-based [e.g. Schmid & Zisserman, 2000] methods. A
more detailed classification of point-based methods is:
1. Approaches that try to get automatically a 3D model of the
scene from uncalibrated images (also called ‘shape from
video’ or ‘VHS to VRML’ or ‘Video-to-3D’). The fully
automated procedure widely reported in the vision
community [Fitzibbon & Zisserman, 1998; Pollefeys et al.,
1999; Nister 2001; Mayer, 2003] starts with a sequence of
images taken with an uncalibrated camera. The system then
extract interest points, sequentially match them across the
view-pairs and compute the camera parameters as well as
the 3D coordinates of the matched points using robust
techniques. This is done in a projective geometry
framework and is usually followed by a bundle adjustment.
A self-calibration, to compute the interior camera
parameters, is afterwards performed in order to obtain a
metric reconstruction, up to a scale, from the projective one.
The 3D surface model is then automatically generated by
means of dense depth maps on image pairs. See [Scharstein
& Szeliski, 2002] for a recent overview of dense stereo
correspondence algorithms. The key to the success of these
automated approaches is the very short interval between
consecutive images. Some approaches have been also
presented for the registration of widely separated views
[Pritchett & Zisserman, 1998; Matas et al., 2002; Xiao &
Shah, 2003; Lowe 2004] but their reliability and
applicability for automated image-based modeling of
complex objects is still not satisfactory, as they yield
mainly a sparse set of matched feature points. Dense
matching results under wide baseline conditions were
instead reported in [Megyesi & Chetverikov, 2004; Strecha
et al., 2004].
2. Approaches that perform an automated 3D reconstruction of
the scene from oriented images. The automated 3D
reconstruction is generally based on object constraints, like
verticality and perpendicularity [Werner & Zisserman,
2002; Van den Heuvel, 2003; Wilczkowiak et al., 2003] or
using the geometric epipolar constraint [Gruen et al., 2001].
3. Approaches that perform a semi-automated 3D
reconstruction of the scene from oriented images. The semi-
automated modeling rely on the human operator and
produced so far the most impressive results, in particular for
architectural objects [Debevec et al., 1996; El-Hakim, 2000,
2002; Gibson et al., 2002]. The interactive work consists of
the topology definition, segmentation, editing and 3D data
post-processing. The degree of automation increases when
certain assumptions about the object, such as
perpendicularity or parallel surfaces, can be introduced.
Manual measurements are also performed in some projects,
generally for complex architectural objects or in cultural
heritage documentations where highly precise and detailed
results are required [Gruen et al., 2004]. Manual measurements
are time consuming and provide for less dense 3D point clouds,
but have higher reliability compared to automated procedures.
3. MATCHING FOR SURFACE MEASUREMENTS
Image matching represents the establishment of
correspondences between primitives extracted from two or
more images. In its oldest form, image matching involved 4
transformation parameters (cross-correlation) and could already
provide for successful results [Foerstner, 1982]. Further
extensions considered a 6- and 8-parameters transformation,
leading to the well known non-linear Least Squares Matching
(LSM) estimation procedure [Gruen, 1985; Foerstner, 1986].
Gruen [1985] and Gruen & Baltsavias [1986] introduced the
Multi-Photo Geometrical Constraints into the image matching
procedure (MPGC) and integrated also the surface
reconstruction into the process. Then from image space, the
matching procedure was generalized to object space,
introducing the concept of ‘groundel’ or ‘surfel’ [Wrobel, 1987;
Helava, 1988].
Even if more than three decades have been devoted to the
image matching problem, nowadays some important limiting
factors still remain. A fully automated, precise and reliable
image matching method, adaptable to different image sets and
scene contents is not available, in particular for close-range
images. The limits stay in the insufficient understanding and
modeling of the undergoing processes (human stereo vision)
and the lack of appropriate theoretical measures for self-tuning
and quality control. The design of an image matcher should
take into account the topology of the object, the primitives used
in the process, the constraint used to restrict the search space, a
strategy to control the matching results and finally optimization
procedures to combine the image processing with the used
constraints. The correspondences between images are matched
starting from primitives (features and image intensity patterns)
and using similarity measures. Ideally we would like to find the
correspondences of every image pixel. But, in practice,
coherent collection of pixels and features are generally
matched.
A part from simple points, the extraction of feature lines (see
[Dhond & Aggarwal, 1989; Ziou & Tabbone, 1998] for a
review) is also a crucial step in the surface generation
procedure. Lines (edgel) provide more geometric information
than single points and are also useful in the surface
reconstruction (e.g. as breaklines) to avoid smoothing effects on
the object edges. Edge matching [Vosselman, 1992; Gruen &
Li, 1996; Schmid & Zisserman, 2000] establishes edge
correspondences over images acquired at different standpoints.
Similarity measures from the edges attributes (like length,
orientation and absolute gradient magnitude) are a key point for
the matching procedure. Unfortunately in close-range
photogrammetry, the viewpoints might change consistently;
therefore similarity measures are not always useful for edge
matching.
4. STEREO-PAIR SURFACE MEASUREMENT
The first developed algorithm is a stereo matcher with the
additional epipolar geometric constraint. The method was
firstly developed for the measurement of human body parts
[D’Apuzzo, 2003] and afterwards also applied to full human
body reconstruction [Remondino, 2004] and rock slopes
retrieval [Roncella et al., 2005]. It has been now extended to
include also edge matching. The main steps of the process are:
1. Image pre-processing: the images are processed with the
Wallis filter [Wallis, 1976] for radiometric equalization and
especially contrast enhancement. The filter enables a strong
enhancement of the local contrast by retaining edge details
and removing low-frequency information in the image. The
filter parameters are automatically selected analyzing the
image histogram.
2. Point matching: the goal is to produce a dense and robust set
of corresponding points between image-pairs. Starting from
few seed points well distributed in the images, the
automated process establishes correspondences by means of
LSM. The images are divided in polygonal regions
according to which of the seed point is closest. Starting
from the seed points, the automated process produce a
dense set of image correspondences in each polygonal
region by sequential horizontal and vertical shifts. One
image is used as template and the other as search image.
The algorithm matches correspondences in the
neighborhood of a seed point in the search image
(approximation point) by minimizing the sum of the squares
differences of the gray value between the two image
patches. If the orientation parameters of the cameras are
available, the epipolar geometric constraints between the
images can also be used in the matching process. Generally
two stereo-pairs (i.e. a triplet) are used: the matcher
searches the corresponding points in the two search images
independently and at the end of the process, the data sets
are merged to become triplets of matched 2D points.
3. Edge matching: the approach extracts line features based on
the edge detection and linking proposed in [Canny, 1986]
and [Henricsson & Heitger, 1994]. For each image, only the
edges longer than a certain threshold are kept. Afterwards
an edge matching is performed for each image pair of the
set. Firstly the middle points of the edges are matched,
providing a preliminary list of edge correspondences.
Afterwards, starting from the matched middle point, the
other points lying on the edge are matched in a propagative
way.
4. 3D Point cloud generation: the 2D matched points and edges
are transformed in 3D data by forward ray intersection,
using the camera exterior orientation parameters.
The developed matching process works on image pairs and
integrates the epipolar constraint in the least squares estimation,
limiting the patch in the search image to move along the
epipolar line. To evaluate the quality of the matching results,
different indicators are used: a posteriori standard deviation of
the least squares adjustment, standard deviation of the shift in x
and y directions and displacement from the start position in x
and y direction. Thresholds for these values are defined
manually for different cases, according to the level of texture in
image and to the type of template. The definition of the seed
points is generally crucial, in particular if discontinuities are
present on the surface. The matcher, working only with stereo-
pairs, is less robust than a multi-image strategy which takes into
account all the available and overlapping images at the same
time, but it is still able to provide for accurate and detailed 3D
surfaces.
5. MULTI-IMAGE SURFACE MEASUREMENT
The multi-image matching approach was originally developed
for the processing of the very high-resolution TLS Linear Array
images [Gruen & Zhang, 2003] and afterwards modified to
accommodate any linear array sensor [Zhang & Gruen, 2004;
Zhang, 2005]. Now it has been extended to process other image
data such as the traditional aerial photos or close-range images.
The multi-image approach uses a coarse-to-fine hierarchical
solution with an effective combination of several image
matching algorithms and automatic quality control. Starting
from the known calibration and orientation parameters, the
approach (Figure 1) essentially performs three mutually
connected steps:
1. Image pre-processing: the set of available images is proc-
essed combining an adaptive smoothing filter and the Wal-
lis filter [Wallis, 1976], in order to reduce the effects of the
radiometric problems such as strong bright and dark regions
and optimizes the images for subsequent feature extraction
and image matching. Furthermore image pyramids are gen-
erated.
2. Multiple Primitive Multi-Image (MPM) matching: this part is
the core of the all strategy for accurate and robust surface
reconstruction, utilizing a coarse-to-fine hierarchical match-
ing strategy. Starting from the low-density features in the
lowest resolution level of the image pyramid, the MPM
matching is performed with the aid of multiple images (two
or more), incorporating multiple matching primitives (fea-
ture points, grid points and edges) and integrating local and
global image information. The MPM approach consists of 3
integrated subsystems (Figure 1): the feature point extrac-
tion and matching, the edge extraction and matching (based
on edge geometric and photometric attributes) and the re-
laxation based relational matching procedure. Within the
pyramid levels, the matching is performed with an exten-
sion of the standard cross-correlation technique (Geometri-
cally Constrained Cross-Correlation -GC3-). The MPM
matching part exploits the concept of multi-image matching
guided from object space and allows reconstruction of 3D
objects by matching all available images simultaneously,
without having to match all individual stereo-pairs and
merge the results. Moreover, at each pyramid level, a TIN
is reconstructed from the matched features using the con-
strained Delauney triangulation method. The TIN is used in
the subsequent pyramid level for derivation of approxima-
tions and adaptive computation of some matching parame-
ters.
3. Refined matching: a modified Multi-Photo Geometrically
Constrained Matching (MPGC) and the Least Squares B-
Spline Snakes (LSB-Snakes) methods are used to achieve
potentially sub-pixel accuracy matches and identify some
inaccurate and possibly false matches. This is applied only
at the original image resolution level. The surface derived
from the previous MPM step provides well enough ap-
proximations for the two matching methods and increases
the convergence rate.
The main characteristics of the multi-image-based matching
procedure are:
Truly multiple image matching: the approach does not aim at
pure image-to-image matching but it directly seeks for im-
age-to-object correspondences. A point is matched simulta-
neously in all the images where it is visible and exploiting
the collinearity constraint, the 3D coordinates are directly
computed, together with their accuracy values.
Matching with multiple primitives: the method is a robust
hybrid image matching algorithms which takes advantage
of both area-based matching and feature-based matching
techniques and uses both local and global image
information. In particular, it combines an edge matching
method with a point matching method through a probability
relaxation based relational matching process.
Self-tuning matching parameters: they are automatically de-
termined by analyzing the results of the higher-level image
pyramid matching and using them at the current pyramid
level. These parameters include the size of the correlation
window, the search distance and the threshold values. The
adaptive determination of the matching parameters results
in higher success rate and less mismatches.
High matching redundancy: exploiting the multi-image con-
cept, highly redundant matching results are obtained. The
high redundancy also allows automatic blunder detection.
Mismatches can be detected and deleted through the analy-
sis and consistency checking within a small neighbourhood.
More details of the matching approach are reported in Zhang
[2005].
Figure 1: Workflow of the automated DSM generation approach. The
approach consists of 3 mutually connected components: the image pre-
processing, the multiple primitive multi-image (MPM) matching and
the refined matching procedure.
6. EXPERIMENTS
We have performed many tests on different close-range data
sets with the two surface reconstruction approaches. So far the
results are checked just visually, as no reference is available. In
the future an accuracy test should be performed. In the next
sections we report results from widely separated images,
untextured surfaces and detailed heritage objects. More
examples are reported in our homepage.
Test1. Three images of the main door of the S. Marco church in
Venice (Italy) are used. The image size is 2560x1920 pixels.
The triplet is acquired under a wide baseline (base-to-distance
ratio ~ 1:1.4) and very fine details are present on the object.
Both methods could correctly retrieve the surface details, as
shown in Figure 3. The stereo-pair strategy matched
approximately 590 000 points between the two pairs while the
multi-image matching recovered ca 700 000 points.
Figure 2: The three images of the church’s façade acquired under a large
baseline.
Test2. A very small pot (ca 3x4 cm) is modeled with the two
presented matching strategies. Six images, with a size of 1856 x
1392 pixels are used. The detailed results are shown in Figure 4
as textured and shaded surface model.
Test 3. The data set consists of 6 images of a Maya relief in
Edzna, Mexico. The object is approximately 4 meters long and
2 meters high. The images have different light conditions and
scales. Due to the frontal acquisition, the upper horizontal part
of the relief is not visible in the images, leading to some gaps in
the matching point results and some stretching effects in the
meshed model. Both methods could reconstruct all the details
of the heritage. The stereo-pairs approach (performed on 4
pairs) generated ca 860 000 points and 7 900 edges while with
the multi-image strategy a cloud of 1 940 000 points and 23 000
edges was produced. The results are shown in Figure 5.
Figure 3: Results of the stereo-pair matching method (above): recovered
3D point cloud, displayed with pixel intensity values and a particular of
the generated shaded model. Views of the shaded and textured surface
generated with the multi-image method (below).
Figure 4: Three (out of 6) image of the small pot (above). The surface
reconstructed with the stereo-pair matching and with the multi-photo
approach.
7. CONCLUSIONS AND OUTLOOK
We have presented two matching strategies for the precise
surface measurement and 3D reconstruction of complex and
detailed terrestrial objects. The stereo-pair approach constraints
the search of correspondences along the epipolar line while the
3D coordinates of points and matched edges are computed in a
second phase, using rejection criteria for the forward ray
intersection. The multi-image approach is more reliable and
precise but requires very accurate image orientation parameters
to exploit the collinearity constraint within the least squares
matching estimation. The maximum orientation errors in image
space should be less than 2-3 pixels.
The two approaches use points and edges to retrieve all the
surface details and they have both advantages and
disadvantages. They can be applied to short or wide baseline
images and can cope with scale changes, different illumination
conditions or repeated pattern. Employing the precise LSM
algorithm, they can recover sub-pixel accuracy matches. They
both need some seed points to initialize the matching procedure
and the number of seed points is strictly related to the image
texture and surface discontinuities.
The results so far achieved are promising but more tests have to
be performed as well as an accuracy assessment of the two
strategies. Photogrammetry has all the potentiality to retrieve
the same results (details) than range sensors. But to asses the
accuracy of the systems is not an easy task. Assessment on the
whole measured surface would require the two models to be in
the same reference systems or to set one model as reference and
transform the second one into the first reference system.
REFERENCES
Baltsavias, E.P., 1991: Multi-Photo geometrically constrained
matching. PhD Thesis, Institute of Geodesy and
Photogrammetry, ETH Zurich, Switzerland, 221 pages
D’Apuzzo, N., 2003: Surface Measurement and Tracking of
Human Body Parts from Multi Station Video Sequences. PhD
Thesis Nr. 15271, Institute of Geodesy and Photogrammetry,
ETH Zurich, Switzerland, 147 pages
Debevec, P., Taylor, C. and Malik, J., 1996: Modelling and
rendering architecture from photographs: a hybrid geometry
and image-based approach. ACM Proceedings of SIGGRAPH
’96, pp. 11-20
Dhond, U.R. and Aggarwal, J.K., 1989: Structure from Stereo.
IEEE Transaction on System, Man and Cybernetics, Vol. 19(6),
pp. 1489-1510
El-Hakim, S., 2000: A practical approach to creating precise
and detailed 3D models from single and multiple views.
IAPRS, 33(B5A), pp 122-129
El-Hakim, S., 2002: Semi-automated 3D reconstruction of
occluded and unmarked surfaces from widely separated views.
IAPRS, 34(5), pp. 143-148, Corfu, Greece
Fitzgibbon, A and Zisserman, A., 1998: Automatic 3D model
acquisition and generation of new images from video sequence.
Proceedings of European Signal Processing Conference, pp.
1261-1269
Foerstner, W., 1982: On the geometric precision of digital
correlation. International Archives of Photogrammetry, Vol.
24(3), pp.176-189
Foerstner, W., 1986: A feature based correspondence algorithm
for image matching. International Archives of Photogrammetry,
Vol. 26(3), Rovaniemi
Gibson, S., Cook, J. and Hubbold, R., 2002: ICARUS:
Interactive reconstruction from uncalibrated image sequence.
ACM Proceedings of SIGGRAPH’02, Sketches & Applications
Gruen, A., 1985: Adaptive least square correlation: a powerful
image matching technique. South African Journal of PRS and
Cartography, Vol. 14(3), pp. 175-187
Gruen, A. and Baltsavias, E., 1986: Adaptive least squares
correlations with geometrical constraints. Proc. of SPIE, Vol.
595, pp. 72-82
Gruen, A., and Li, H., 1996: Linear feature extraction with
LSB-Snakes from multiple images. IAPRS, Vol. 31(B3), pp.
266-272
Gruen, A., Zhang, L. and Visnovcova, J., 2001: Automatic
reconstruction and visualization of a complex Buddha Tower of
Bayon, Angkor, Cambodia. Proceedings 21.Wissenschaftlich-
Technische Jahrestagung der DGPF, pp. 289-301
Gruen, A., Zhang L., 2003. Automatic DTM Generation from
TLS data. In: Gruen/Kahman (Eds.), Optical 3-D Measurement
Techniques VI, Vol. I, ISBN: 3-906467-43-0, pp. 93-105.
Gruen, A., Remondino,F., Zhang, L., 2004: Photogrammetric
Reconstruction of the Great Buddha of Bamiyan, Afghanistan.
The Photogrammetric Record, Vol. 19(107)
Helava, U.V., 1988: Object-space least-squares correlation.
PE&RS, Vol. 54(6), pp. 711-714
Henricsson O. and Heitger F., 1994: The role of key-points in
finding contours. ECCV'94, Vol. 2, pp. 371-383
Lowe, D., 2004: Distinctive image features from scale-invariant
keypoints. International Journal of Computer Vision, Vol.
60(2), pp. 91-110
Matas, J., Chum, O., Urban, M. and Pajdla, T., 2002: Robust
wide baseline stereo from maximally stable extremal regions.
Proceedings of BMVC, pp. 384-393
Mayer, H., 2003: Robust orientation, calibration, and disparity
estimation of image triplets. 25th DAGM Pattern Recognition
Symposium (DAGM03), Number 2781, series LNCS,
Michaelis/Krell (Eds.), Magdeburg, Germany
Megyesi, Z. and Chetverikov, D., 2004: Affine propagation for
surface reconstruction in wide baseline stereo. Proc. ICPR
2004, Cambridge, UK
Nister, D., 2001: Automatic dense reconstruction from
uncalibrated video sequences. PhD Thesis, Computational
Vision and Active Perception Lab, NADA-KHT, Stockholm,
226 pages
Pollefeys, M., Koch, R. and Van Gool, L., 1999: Self-
calibration and metric reconstruction in spite of varying and
unknown internal camera parameters. IJCV, Vol. 32(1), pp. 7-
25
Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F.,
Cornelis, K., Tops, J. and Koch, R., 2004: Visual modeling
with a hand-held camera. IJCV, Vol. 59(3), pp. 207-232
Figure 5: Three (out of six) images of a Maya relief (above). Results of the stereo-pair matching (4 pairs) displayed as shaded model (with a close
view of the mesh) and as textured model (middle). Shaded model obtained with the multi-image matcher simultaneously run on 6 images (bottom).
Pritchett, P. and Zisserman, A. 1998: Matching and
reconstruction from widely separated views. 3D Structure
from Multiple Images of Large-Scale Environments, LNCS
1506
Remondino, F., 2004: 3D Reconstruction of Static Human
Body Shape from Image Sequence. Computer Vision and
Image Understanding, Vol. 93(1), pp. 65-85
Remondino, F. and Ressl, C., 2006: Overview and
experiences in automated markerless image orientation.
IAPRS, Comm. III Symposium, Bonn, Germany. In press.
Roncella, R., Forlani, G. and Remondino, F., 2005:
Photogrammetry for geological applications: automatic
retrieval of discontinuity in rock slopes. Videometrics VIII -
Beraldin, El-Hakim, Gruen, Walton (Eds), SPIE-IS&T
Electronic Imaging Vol. 5665, pp. 17-27
Scharstein, D. and Szeliski, R., 2002:. A taxonomy and
evaluation of dense two-frame stereo correspondence
algorithms. IJCV, 47(1/2/3): 7-42
Schmid, C. and Zisserman, A., 2000: The geometry and
matching of lines and curves over multiple views. IJCV, Vol.
40(3), pp. 199-233
Strecha, C., Tuytelaars, T. and Van Gool, L., 2003: Dense
Matching of Multiple Wide-baseline Views. IEEE
Proceedings of ICCV’03, Vol.2, pp. 1194-1201
Van den Heuvel, F., 2003: Automation in architectural
photogrammetry. PhD Thesis, Publication on Geodesy 54,
Netherlands Geodetic Commission
Vosselman, G., 1992: Relational matching. Lecture Notes in
Computer Science, No. 628, Springer Berlin, 190 pages
Wallis, R., 1976: An approach to the space variant
restoration and enhancement of images. Proc. of Symposium
on Current Mathematical Problems in Image Science, Naval
Postgraduate School, Monterey, CA
Werner, T. and Zisserman, A., 2002: New technique for
automated architectural reconstruction from photographs.
Proceedings 7th ECCV, Vol.2, pp. 541-555
Wilczkowiak, M., Trombettoni, G., Jermann, C., Sturm, P.
and Boyer, E., 2003: Scene modeling based on constraint
system decomposition techniques. IEEE Proceedings 9th
ICCV, pp. 1004-1010
Xiao, J. and Shah, M. 2003: Two-frame wide baseline
matching. IEEE Proceeding of 9th ICCV, Vol.1, pp. 603-610
Wrobel, B., 1987: Facet Stereo Vison (FAST Vision) – A
new approach to computer stereo vision and to digital
photogrammetry. Proc. of ISPRS Intercommission
Conference on ‘Fast Processing of Photogrammetric Data’,
Interlaken, Switzerland, pp. 231-258
Zhang, L., Gruen, A., 2004. Automatic DSM Generation
from Linear Array Imagery Data. IAPRS, Vol. 35(B3), pp.
128-133
Zhang, L., 2005: Automatic Digital Surface Model (DSM)
generation from linear array images. PhD Thesis Nr. 16078,
Institute of Geodesy and Photogrammetry, ETH Zurich,
Switzerland, 199 pages
Ziou, D. and Tabbone, S., 1998: Edge Detection Techniques
- An Overview. Journal of Pattern Recognition and Image
Analysis. Vol. 8, pp. 537-559
... Se probaron formas apropiadas de interacción entre los aspectos del método de modelado basado en rangos (modelado lasergramétrico -instrumentación óptica activa) y el método de modelado image-based (modelado fotogramétrico -técnica basada en sensores pasivos), cuya complementariedad también nos permitió estudiar efi cazmente la complejidad matérica y cromática de sus distintos componentes. El método de modelado range-based (Remondino, 2006) utiliza técnicas basadas en sensores activos, empleando, como se sabe, instrumentos (escáneres láser) que emiten señales electromagnéticas registradas por un sensor para derivar una medida de distancia (alcance). Junto con los datos topológicos, el sensor, por cada pulso láser emitido por el escáner, adquiere también un valor de refl ectancia que le permite distinguir materiales. ...
Article
El estudio se centra en el vínculo que las arquitecturas urbanas tejen a medida que se convierten en ciudades y propone un camino de exploración dirigido a obtener un modelo aumentativo de las características arquitectónicas, capaz de examinar las condiciones en el “estado” de la arquitectura, pero también sus latentes potenciales. La arquitectura es deconstruida, en sus múltiples fi guras, y reconstruida en su representación que revela su identidad heurística. La adopción de una metodología de encuesta integrada, experimentando con formas de interacción entre el modelado range-based y los métodos de modelado image-based, permitió investigar los espacios, llegando a modelos de retorno representativos. Los resultados configurativos permiten un reconocimiento oportuno de las características morfométricas y colorimétricas y hacen accesible la interrogación permanente de los datos incluso de forma remota, con el fi n de un uso inclusivo.
... Metodología, resultados y discusión La metodología de encuesta integrada adoptada hizo un uso sinérgico del método de range-based modeling (Remondino, 2006) y el método de image-based modeling (Apollonio, 2014), experimentando con su interacción rentable, en particular en referencia al estudio de la complejidad conformacional de los entornos y superfi cies exteriores del pabellón y la intrincada articulación de los recorridos aéreos y subterráneos (sistemas de escaleras, rampas, puentes levadizos, pasajes de conexión, etc.). También fue posible interesarse específi camente por la lectura de algunos materiales y texturas constitutivos. ...
Article
Los resultados del estudio integrado presentado tienen como objetivo apoyar una acción de intervención cuyas elecciones se basaron en un plan de análisis e investigaciones cognitivas en profundidad del conjunto monumental. A partir del Padiglione Cavaliere, se está llevando a cabo un proyecto de restauración y valorización que pretende devolver los espacios al sistema museístico del Castillo aragonés de Baia. La metodología de encuesta integrada adoptada hizo un uso sinérgico del método de range-based modeling y el método de image-based modeling, experimentando con su fructífera interacción, en particular en referencia a la lectura de la complejidad conformacional de los ambientes y superfi cies externas del edifi cio y la intrincada articulación de las rutas aéreas y subterráneas; se reservó un interés específi co para los paramentos de los muros de los baluartes que fortifi can el perímetro del Pabellón.
... The shooting operation was carried out with the exclusive use of a Nikon D5200 SLR camera and an AF-P DX NIKKOR 18-55mm VR wide-angle zoom lens. This photographic equipment is compatible with close-range photogrammetry processes (Remondino & Zhang, 2006), as the close-range shooting of the object guarantees the production of highly accurate and detailed threedimensional models. The method used, known as the 'multi-image approach' (Suwardhi et al., 2015;Murtiyoso et al., 2017), allows to obtain a digital model by collimating the points of the same object photographed from different positions, heights and distances. ...
... Edge extraction: Traditionally, line segments have been extensively used in the 2D space with simple gradient-based detectors like Sobel (Sobel, 1972) and LoG (Marr, 1980), to most sophisticated solutions as Canny (Canny, 1986), Rothwell (Rothwell et al., 1995), Edison (Meer and Georgescu, 2001) 3D reconstruction and linear segments: Line segments have been used in image registration tasks in photogrammetry already for a long time (Baillard et al., 1999). In the latest years, linear segment matching has been used in pairwise image matching (Wang et al., 2009;Zhang and Koch, 2014), as well as in SfM (Bertoli and Sturm, 2006;Micusik and Wildenauer, 2018) and SLAM algorithms (Hirose and Saito, 2012;Salaün et al., 2017;Zhou et al., 2019) for pose estimation and mapping or 3D reconstruction purposes (Remondino and Zhang, 2006). At the same time, matched linear segments are coupled with the SfM results as a less computational expensive alternative to the MVS reconstruction as in (Hofer et al., 2015;. ...
Article
Full-text available
Mesh models generated by multi view stereo (MVS) algorithms often fail to represent in an adequate manner the sharp, natural edge details of the scene. The harsh depth discontinuities of edge regions are eventually a challenging task for dense reconstruction, while vertex displacement during mesh refinement frequently leads to smoothed edges that do not coincide with the fine details of the scene. Meanwhile, 3D edges have been used for scene representation, particularly man-made built environments, which are dominated by regular planar and linear structures. Indeed, 3D edge detection and matching are commonly exploited either to constrain camera pose estimation, or to generate an abstract representation of the most salient parts of the scene, and even to support mesh reconstruction. In this work, we attempt to jointly use 3D edge extraction and MVS mesh generation to promote edge detail preservation in the final result. Salient 3D edges of the scene are reconstructed with state-of-the-art algorithms and integrated in the dense point cloud to be further used in order to support the mesh triangulation step. Experimental results on benchmark dataset sequences using metric and appearance-based measures are performed in order to evaluate our hypothesis.
... Based on the various implementations of the aforementioned fundamental steps, diverse methods have been proposed. Several approaches have been developed to measure the agreement between the pixels and find the best match, from local to semi-global [22] and global methods, from area or patch-based [23,24] to feature-based [25] or a combination of them [26]. The most important used criterion to find corresponding pixels is known as photo-consistency, which estimates the similarity of two (or more) pixels between two images [12]. ...
Article
Full-text available
The image-based 3D reconstruction pipeline aims to generate complete digital representations of the recorded scene, often in the form of 3D surfaces. These surfaces or mesh models are required to be highly detailed as well as accurate enough, especially for metric applications. Surface generation can be considered as a problem integrated in the complete 3D reconstruction workflow and thus visibility information (pixel similarity and image orientation) is leveraged in the meshing procedure contributing to an optimal photo-consistent mesh. Other methods tackle the problem as an independent and subsequent step, generating a mesh model starting from a dense 3D point cloud or even using depth maps, discarding input image information. Out of the vast number of approaches for 3D surface generation, in this study, we considered three state of the art methods. Experiments were performed on benchmark and proprietary datasets of varying nature, scale, shape, image resolution and network designs. Several evaluation metrics were introduced and considered to present qualitative and quantitative assessment of the results.
... Once the field work was completed, the restitution and elaboration of the three-dimensional model continued through the Agisoft Metashape software, a photogrammetric modeling software that through the featured recognition algorithms (SIFT) can automatically identify, in the frames, the key points (tie points). The processing takes place through four phases: Structure-from-motion (SfM), in which the software aligns the frames by determining the internal orientation parameters and generates the scattered cloud; Multiview Stereo Reconstruction (MVS), in which the software densifies the scattered cloud; Mesh reconstruction, where, through the meshing of the dense cloud, the software generates the model; Texturing, in which the software orthorectifies the images and projects them onto the mesh, returning the rendered model ( Figure 6) (Remondino, Zhang, 2006). ...
Article
Full-text available
The castle of Pescopagano, a small village located on the border between Basilicata and Campania, is a complex of great historical and landscape value, for the inseparable combination that binds its stones to the rock where it stands. Founded perhaps in the Byzantine times, but certainly renovated and built in its current forms between the 11th and 12th century, the castle had considerable military importance under Frederick II of Swabia. Seriously damaged by the earthquake of 1694, the fortress underwent a partial reconstruction, but ended up suffering further collapses caused by the Irpinia earthquake of 1980, such as to motivate the first interventions of securing and, above all, the application of the listing process. Today the castle is still largely in ruins and is only partially accessible thanks to a limited intervention on the paths. The present research aims at deepening the knowledge of the state of conservation, the damage mechanisms and the previous restoration interventions of the castle, in order to define possible strategies for its restoration and enhancement. The analysis work uses the most advanced laser scanning and drone detection systems, in order to document, as accurately as possible, the complex patrimonial system of the castle. Thanks to the combined use of these techniques, the objective is also to define methods that can be replicated in other contexts where the relationship between geomorphology and construction is so relevant that it jeopardizes the use of any other traditional survey system.
... In this step, corresponding pixels in two images are found. Methodologies have been conveniently categorized as window-based (local) and pixel-based (global) [14,[43][44][45][46][47][48]. With these initial adaptations, the expanding and filtering steps are repeated [41,49]. ...
Article
Full-text available
This research presents a novel method for automated construction progress monitoring. Using the proposed method, an accurate and complete 3D point cloud is generated for automatic outdoor and indoor progress monitoring throughout the project duration. In this method, Structured-from-Motion (SFM) and Multi-View-Stereo (MVS) algorithms coupled with photogrammetric principles for the coded targets’ detection are exploited to generate as-built 3D point clouds. The coded targets are utilized to automatically resolve the scale and increase the accuracy of the point cloud generated using SFM and MVS methods. Having generated the point cloud, the CAD model is generated from the as-built point cloud and compared with the as-planned model. Finally, the quantity of the performed work is determined in two real case study projects. The proposed method is compared to the Structured-from-Motion (SFM)/Clustering Multi-Views Stereo (CMVS)/Patch-based Multi-View Stereo (PMVS) algorithm, as a common method for generating 3D point cloud models. The proposed photogrammetric Multi-View Stereo method reveals an accuracy of around 99 percent and the generated noises are less compared to the SFM/CMVS/PMVS algorithm. It is observed that the proposed method has extensively improved the accuracy of generated points cloud compared to the SFM/CMVS/PMVS algorithm. It is believed that the proposed method may present a novel and robust tool for automated progress monitoring in construction projects.
... Initial applications of (CRP) in accident reconstruction often involved scene documentation, but more recently, it is used to quantify vehicle dimensions and crush damage (Randles et al., 2010). So far, automated surface reconstruction methods, even if able to recover complete 3D geometry of an object, reported errors between 3% and 5% (Pollefeys et al., 1999), limiting their use to applications requiring only nicelooking 3D models (Remondino and Zhang, 2006). Bethmann et al. (2010) presented a least-squares' matching algorithm using the plane projective transformation model and polynomial transformations to handle geometric distortions between the images with coded targets. ...
Article
Full-text available
The objective of this study was to introduce a new method for extracting as-built drawings of road intersections using cell phone camera through Close Range Photogrammetry (CRP). Three different cell phones of different resolutions (5MP, 8MP and 20MP) were used in this study to capture images for road intersections in Jordan. Calibration of cell phone cameras was done before the captured images were processed using iWitness and Agisoft software. For the purpose of accuracy assessment and result verification, 68 points were collected from the study area using differential GPS. The resultant as-built drawings extracted by this method were tested and compared with the collected GCPs from the field. Euclidian distance for different linear features in the 3D model was computed and compared with the extracted as-built drawing linear features. The results revealed that using 20MP, root mean square error RMSE in the x, y and z directions was 0.472m, 0.514m and 0.462m, respectively, which shows the feasibility of using such method for extracting road network intersection drawings. As the cell phone's resolution increased, the potential accuracy of the as-build drawings increased. This method of extracting as-built drawings using cell phones will open the door for efficient and practical future applications.
Chapter
Full-text available
Large area prospection with highly sensitive caesium magnetometers up to now has been one of the most successful geophysical prospecting methods in archaeology. The application of this method on pre-Hispanic cultures provides a perfect framework and has a high potential capacity for further development of magnetometry and archaeological prospection methods in general. Both shallow inclination as well as the low intensity of the geomagnetic field near the equator requires an adaption and modification of the caesium magnetometer. In the case of the geoglyphs of the Peruvian Atacama desert – a UNESCO World Heritage site since 1994 – the prospecting results are exemplified in detail. Magnetometry enables us to visualise not only the traces of numerous lightning strikes in the desert but also the traces of thus far unknown archaeological structures and older invisible lines beneath the multiphase trapezoidal geoglyphs. Magnetometry therefore turns from pure geophysics to a perfect archaeological tool for studying ancient sites without destruction.
Research
Full-text available
This represents one of several sections of "A Bibliography Related to Crime Scene Interpretation with Emphases in Geotaphonomic and Forensic Archaeological Field Techniques, Nineteenth Edition" (The complete bibliography is also included at ResearchGate.net.). This is the most recent edition of a bibliography containing resources for multiple areas of crime scene, and particularly outdoor crime scene, investigations. It replaces the prior edition and contains approximately 10,000 additional citations. As an ongoing project, additional references, as encountered, will be added to future editions. After it is secured, the first step toward processing a crime scene is assessing and recording its position in situ as well as its relationship to other sites affected by subject(s) and victim(s). This follows tenets shared by archaeology, geotaphonomy, and general criminalistics: Crime scenes do not occur in vacuums. They entail points of access, egress, and are influenced by immediate and neighboring environmental activities. Consider cases in which lone skulls are found by pedestrians. More often than not, those skulls were once attached to remains which lie nearby. By recording the environment or context from which the skull was recovered, (beyond its mere two dimensional position within a small grid), its peri- and postmortem history may become more clear. The routine recording of elevations and topographic features such as streams, flood plains, slopes, et cetera, near remains could provide the investigator, laboratory analyst, and ultimately a jury, with information related to taphonomic processes affecting those remains and explanations for their ultimate dispositions. Too often investigators take a myopic view toward crime scenes and concentrate only on the area around the primary piece of evidence. The experienced, well-trained, and practiced crime scene investigator or archaeologist realizes three integrated and expanding spheres of any investigation: the evidence or artifact, the incident, and the event. Toward expanding perspective into the incident and event spheres of investigation, this category contains resources which generally entail topics of search or survey techniques as well as mapping methodologies and equipment. Several of the articles also apply to logistics and preparation; however, the reader is also directed to the categories of “General Crime Scene and Death Scene Investigation” and "Excavation and Recovery Strategies" for additional references which contain discussions about logistics and planning. The most important component of planning for a forensic search or recovery is consideration of personal and team safety. Many sites suitable for the disposal of homicide victims pose significant risks to living individuals. Sites such as landfills and cesspools demonstrate one type of obvious hazard, while confined spaces such as wells, cisterns, or mine shafts represent another with less visible dangers such as carbon monoxide or structural collapse. Attention to safety extends beyond the field to laboratory and storage settings. Inadequate facilities, personal protection, or inappropriate handling of hazardous evidence or chemicals used to process evidence can put technicians and scientists at significant risk. For these reasons, this section of the bibliography begins with the topic of safety.
Article
Full-text available
CCD linear array sensors are widely used to acquire panchromatic and multispectral imagery for photogrammetric and remote sensing applications. The processing of this kind of images provides a challenge for algorithmic redesign and this opens the possibility to reconsider and improve many photogrammetric processing components. In addition, the basic capabilities of image matching techniques have so far not been fully utilized yet. This paper presents a matching procedure for automatic DSM generation from linear array imagery data. It can provide dense, precise and reliable results. The method uses a coarse-to-fine hierarchical solution with an effective combination of several image matching algorithms and automatic quality control. The DSMs are generated by combination of matching results of feature points, grid points and edges. Finally, a modified multi-photo geometrically constrained (MPGC) matching algorithm is employed to achieve sub-pixel accuracy for all the matched features with multi-image or multi-strip image data. The proposed approach in this paper has been applied to different areas with varying textures and terrain types. The accuracy tests are based on the comparison between the high quality DEMs / DSMs derived from airborne Laser Scanner or manual measurements and the automatic extracted DSMs. Results with STARIMAGER, IKONOS and SPOT5 HRS images are reported. We demonstrate with these experiments that our approach leads to good results.
Article
Full-text available
ABSTRACT The Adaptive Least Squares Correlation is a very potent and flexible technique for all kinds of data matching problems. Here its application to image matching is outlined. It allows for simultaneous radiometric corrections and local geometrical image shaping, whereby the system parameters are automatically assessed, corrected, and thus optimized during the least squares iterations. The various tools of least squares estimation can be favourably utilized for the assessment of the correlation quality. Furthermore, the system allows for stabilization and improvement of the correlation procedure through the simultaneous consideration of geometrical constraints, e.g. the collinearity condition. Some exciting new perspectives are emphasized, as for example multiphoto correlation, multitemporal and multisensor correlation, multipoint correlation, and simultaneous correlation/triangulation.
Conference Paper
Full-text available
The Adaptive Least Squares Correlation is a general and flexible technique for many different image matching problems. It allows for simultaneous local geometrical image shaping and radiometric corrections, whereby the system parameters are automatically assessed, corrected and thus optimized with respect to the specific signal during the least squares iterations. Precision and reliability measures can be developed to assess the quality of the match. A stabilization and improvement of the correlation procedure can be achieved through the simultaneous consideration of object point intersection conditions from conjugate rays and other sensor and object constraints. These geometrical constraints limit the search area size, reduce the number of alternatives, increase the precision and reliability of matching and provide simultaneously 3D-point positioning information. The method can be applied to correlation, object detection and measurement, and image tracking, while the 3D-information provided can be readily utilized for tasks requiring inspection, manipulation, object tracking and navigation. As an exciting new prospect the method can be applied to more than two images at a time. This paper outlines the basic concept of the technique.
Article
Full-text available
The Hindu and Buddhist monuments of Angkor in Cambodia are the legacy of highly developed Khmer empires. Well-known for their structural and surface complexity they constitute a great challenge to any attempt towards precise and detailed 3D measurements and modeling. This paper reports about a pilot project using modern techniques of analytical and digital photogrammetry to derive a photorealistic 3D model of one of the very complex towers of the famous Bayon temple of the ancient city of Angkor Thom. This high quality model will then be subject to visualization and animation. On occasion of a balloon photogrammetry mission over the Bayon temple of Angkor, the first author took a number of tourist-type terrestrial images with a Minolta Dynax 500si analogue SRL camera of one of the many Buddha-faced towers of Bayon. We aim at deriving automatically, after scanning of the images, a texture mapped 3D model of the very complex object. In a first step we have generated already such a model with a mixture of manual (phototriangulation) and automated procedures (image matching for surface reconstruction, editing for blunder removal, texture mapping, visualization and animation). This result has been presented in Visnovcova et al., 2001 and includes already
Article
Full-text available
Automated image orientation is still a key problem in close-range photogrammetry, in particular if wide baseline images are employed. Nowadays, within the image-based modeling pipeline, the orientation step is the one which could be fully and reliably automated, exploiting the potentiality of computer and image processing algorithms. In this paper, we summarize recent developments in this field and apply them in three different workflows to automatically extract markerless tie points from close- range images of different types (video sequence, large and wide baseline images). Furthermore we compare the results obtained from bundle block adjustment using the automatic tie points with the results obtained by manual measurements and show how the accuracies of the automatic tie point extraction can further be improved by including least squares matching techniques.