Content uploaded by Ewelina Rupnik
Author content
All content in this area was uploaded by Ewelina Rupnik on Jul 12, 2014
Content may be subject to copyright.
| INTERNATIONAL | FEBRUARY 20141616
Automated
Processing of
Oblique Imagery
CONNECTIVITY GRAPH TO COPE WITH TILTED VIEWS
Oblique multi-camera systems are rapidly maturing and expanding the market for
airborne technology and services. Because datasets of oblique images are large,
automated processing is a necessity. Here, the authors present a workfl ow for the
automated orientation of large oblique blocks using a connectivity graph and
discuss automated dense matching of oblique images.
the spotlight was on visualisation
and inspection of oblique images by
human operators. However, today’s
interest is focused on the metric
capabilities.
TILTED VERSUS VERTICAL
Tilted views are richer in content
compared to vertical views, but
they bring more occlusions and
degradation in similarity between
image features as well as signi cant
scale and illumination di erences.
Oblique images are also more prone to
hot spots and sun glints which arise
when looking towards or away from
the sun. Additionally, instabilities in
the ight trajectory are more harmful.
As a result, e cient execution of
large-scale projects requires reliable
solutions, automated extraction of
tie points and dense matching for 3D
reconstruction.
TIE POINTS
Interior and exterior parameters of
Francesco Nex holds a
PhD in geomatics from
the Politecnico of Turin
(Italy) and is now a
post-doc researcher at
3DOM-FBK. His research
interests lie in
information extraction
from imagery, data fusion and point cloud
processing.
franex@fbk.eu
Ewelina Rupnik
obtained her MSc in
geodesy and
cartography from the
University of Science
and Technology in
Krakow (Poland). She is
a researcher at
3DOM-FBK focusing on automated image
processing and interpretation.
rupnik@fbk.eu
Fabio Remondino obtained
his PhD in photogrammetry
from ETH Zurich and in 2007
moved to FBK Trento (Italy)
where he leads the 3D Optical
Metrology (3DOM) research
unit. His research interests
are automated data
processing and sensor and data integration. He is
president of EuroSDR Commission I on Sensors,
Primary Data Acquisition and Georeferencing.
remondino@fbk.eu
Oblique airborne multi-camera
systems are increasingly
complementing traditional vertical
views. Previously hidden façades and
building footprints are unveiled in
oblique views, which makes oblique
imagery useful for 3D city modelling
and cadastral purposes as well as
emergency rapid response and
scene interpretation. In the past,
GIM0214_Feat_Remondino 16 31-01-2014 13:45:40
FEATURE
FEBRUARY 2014 | INTERNATIONAL |
17
BY FABIO REMONDINO, EWELINA RUPNIK AND FRANCESCO NEX, ITALY
aerial images are commonly known
beforehand. Interior parameters are
retrieved through lab calibration and
exterior parameters are measured
directly with on-board sensors
(GNSS/IMU). Nevertheless, these
parameters are just approximates for
metric and automatic applications;
therefore, an adjustment in a least
squares sense has to be conducted.
Direct georeferencing of oblique
images without using ground control
points is still an issue to be resolved.
Experiences gained in processing
terrestrial images – which, just like
oblique aerial images, are convergent
and unordered – provided enough
insight to adopt and adjust the
terrestrial methodologies for use
in processing airborne oblique
imagery. e main obstacle is
the time-e cient generation of
putative correspondences between
overlapping images. Pairs, triplets or
larger sets of images with maximum
similarity are rst identi ed using
GNSS/IMU information and a
connectivity graph and subsequently
matched. A connectivity graph
expresses the spatial relationships
between the images, speeds
up the determination of image
correspondences and reduces the
number of outliers. e connectivity
between images is described in the
form of a graph of which the nodes
represent images and the edges
represent their relationships, i.e. two
images are linked with an edge if they
are spatially compatible. An image
pair has to ful l three conditions
in order to be connected: (1) their
footprints coincide by a given
percent, (2) cameras’ look directions
are similar or one of the cameras
is nadir, and (3) the number of
extracted homologous points exceeds
a threshold.
Series on Oblique Photogrammetry
This article is the second in a series on oblique photogrammetry, a joint initiative of
EuroSDR Comm. 1, Delft University of Technology and University of Twente (ITC). Edited by
Mathias Lemmens, the series is intended to cover concepts, applications and camera
systems currently available on the market. You are cordially invited to contribute. To do so,
feel free to contact the editorial manager at wim.vanwegen@geomares.nl or the senior
editor at m.j.p.m.lemmens@tudelft.nl.
Figure 1, Maltese-cross
obliques show
orientation failures if no
connectivity graph is
used (left) but with such
a graph (middle) the
correct parameters are
produced.
Figure 2, As Figure 1,
but here for fan-type
obliques.
GIM0214_Feat_Remondino 17 31-01-2014 13:45:40
FEATURE
FEBRUARY 2014 | INTERNATIONAL |
19
BUNDLE BLOCK ADJUSTMENT
e connectivity graph limits
the number of mismatches. But
the complexity of a network ’s
geometry of oblique image blocks
and the non-linearity of collinearity
equations require good initial
approximations of unknowns. e
Apero bundle adjustment software
the authors usually employ allows
a concatenation of direct methods
(spatial resection, essential matrix)
which avoids the need for precise
initial approximations and allows
unknown camera positions and 3D
object coordinates to be derived
(Figures 1 and 2). e bundle
adjustment of multi-camera images
must handle n di erent cameras with
di erent interior (IO) and exterior
orientation (EO) parameters. e
camera parameters can be retrieved
without constraints – each image
is oriented using an independent
EO for each acquisition – or with
constraints, which describe the
relative rotations and displacement
between cameras and are added to
the mathematical model, lowering
the number of unknowns and
stabilising the bundle solution.
Additionally, the IO of each camera
can be assumed known from a lab
calibration or simultaneously be
computed in the bundle solution
through self-calibration. e large
redundancy in oblique images
(Figure 3) helps to select the best
correspondences and achieves high
accuracy in 3D reconstructions.
DENSE MATCHING
Compared to vertical images, oblique
images give a deeper and more
complete description of urban areas,
allowing the extraction of denser
point clouds and more information
in the ‘smart city’ domain, with
façades and buildings completely
reconstructed (Figure 4). Mismatches
or wrong reconstructions can still
be present because of: (1) buildings,
roads or other objects having been
captured with di erent scales, (2)
presence of occluded areas, (3) depth
and image GSD changing more
suddenly than for vertical images,
and (4) the smaller intersection
angles and baseline between images
making point cloud generation
sensitive to noise. Higher overlaps
may overcome some of these issues,
although they increase the size of
datasets and the number of point
clouds.
CONCLUDING REMARKS
Traditional processing procedures
are being challenged. Because
oblique systems are still rather new,
however, many questions remain
open including: When should
oblique imagery be used? What
are its strengths and weaknesses?
What is the optimal acquisition
pattern for metric mapping? How
can illumination and scale changes
be dealt with? Which processing
software is reliable and e cient?
Methodologies need to be ne-tuned
in order to improve automated
processing, feature extraction and
scene interpretation, and for other
mapping purposes.
ACKNOWLEDGEMENTS
anks are due to Blom-CGR S.p.A.
(Parma, Italy) and VisionMap
(Israel) for providing the oblique
imagery.
FURTHER READING
- Gerke, M., and Xiao, J. (2013) Supervised and Unsupervised MRF-based 3D Scene
Classifi cation in Multiple-view Airborne Oblique Views. ISPRS Annals
of Photogrammetry, Remote Sensing and Spatial Information Sciences,
Vol. 2(3/W3), pp. 25-30, Antalya, Turkey.
- Lemmens, M. (2011) Digital Aerial Cameras. GIM International, Vol. 25(4). Micmac
(2013): http://logiciels.ign.fr/?-Micmac
- Nex., F., Rupnik, E., Remondino, F. (2013) Building Footprints Extraction from Oblique
Imagery. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information
Sciences, Vol. 2(3/W3), pp. 61-66, Antalya, Turkey.
- Rupnik, E., Nex, F., Remondino, F. (2013) Automatic Orientation of Large Blocks of
Oblique Images. Int. Archives of Photogrammetry, Remote Sensing and Spatial
Information Sciences, Vol. 40(1/W1), pp. 299-304, Hannover, Germany.
Figure 4, Colour-
coded dense point
cloud of an urban
area.
Figure 3, Redundancy
in the observations
of Maltese-cross
confi guration (left)
and fan.
GIM0214_Feat_Remondino 19 31-01-2014 13:45:41
