ArticlePDF Available

Unmanned Aerial Vehicle (UAV)-Based Assessment of Concrete Bridge Deck Delamination Using Thermal and Visible Camera Sensors: A Preliminary Analysis

Authors:

Abstract and Figures

Infrared and visible cameras were mounted on an unmanned aerial vehicle (UAV) to image bridge deck surfaces and map likely concrete delaminations. The infrared sensor was first tested on laboratory validation experiments, to assess how well it could detect and map delaminations under controlled conditions. Field tests on two bridge deck surfaces further extend the validation dataset to real-world conditions for heterogeneous concrete surfaces. Performance of the mapping instrument and algorithms were evaluated through receiver operating characteristic (ROC) curves, giving acceptable results. To improve the performance of the mapping by reducing the rate of false positives, i.e., areas wrongly mapped as being affected by delamination, visible images were jointly analyzed with the infrared imagery. The potential for expanding the method to include other products derived from the visible camera data, including high density 3D point locations generated by photogrammetric methods, promises to further improve the performance of the method, potentially making it a viable and more effective option compared to other platforms and systems for imaging bridge decks for mapping delaminations.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=urnd20
Download by: [71.13.208.198] Date: 28 July 2017, At: 06:19
Research in Nondestructive Evaluation
ISSN: 0934-9847 (Print) 1432-2110 (Online) Journal homepage: http://www.tandfonline.com/loi/urnd20
Unmanned Aerial Vehicle (UAV)-Based Assessment
of Concrete Bridge Deck Delamination Using
Thermal and Visible Camera Sensors: A
Preliminary Analysis
Rüdiger Escobar-Wolf, Thomas Oommen, Colin N. Brooks, Richard J. Dobson
& Theresa M. Ahlborn
To cite this article: Rüdiger Escobar-Wolf, Thomas Oommen, Colin N. Brooks, Richard J. Dobson
& Theresa M. Ahlborn (2017): Unmanned Aerial Vehicle (UAV)-Based Assessment of Concrete
Bridge Deck Delamination Using Thermal and Visible Camera Sensors: A Preliminary Analysis,
Research in Nondestructive Evaluation, DOI: 10.1080/09349847.2017.1304597
To link to this article: http://dx.doi.org/10.1080/09349847.2017.1304597
View supplementary material
Published online: 17 Apr 2017.
Submit your article to this journal
Article views: 35
View related articles
View Crossmark data
Unmanned Aerial Vehicle (UAV)-Based Assessment of
Concrete Bridge Deck Delamination Using Thermal and
Visible Camera Sensors: A Preliminary Analysis
Rüdiger Escobar-Wolf
a
, Thomas Oommen
a
, Colin N. Brooks
b
, Richard J. Dobson
b
,
and Theresa M. Ahlborn
a
a
Department of Geological and Mining Engineering and Sciences, Michigan Technological University,
Houghton Michigan, USA;
b
Michigan Technological Research Institute, Michigan Technological
University, Ann Arbor, Michigan, USA
ABSTRACT
Infrared and visible cameras were mounted on an unmanned
aerial vehicle (UAV) to image bridge deck surfaces and map likely
concrete delaminations. The infrared sensor was first tested on
laboratory validation experiments, to assess how well it could
detect and map delaminations under controlled conditions. Field
tests on two bridge deck surfaces further extend the validation
dataset to real-world conditions for heterogeneous concrete
surfaces. Performance of the mapping instrument and algorithms
were evaluated through receiver operating characteristic (ROC)
curves, giving acceptable results. To improve the performance of
the mapping by reducing the rate of false positives, i.e., areas
wrongly mapped as being affected by delamination, visible
images were jointly analyzed with the infrared imagery. The
potential for expanding the method to include other products
derived from the visible camera data, including high density 3D
point locations generated by photogrammetric methods, pro-
mises to further improve the performance of the method, poten-
tially making it a viable and more effective option compared to
other platforms and systems for imaging bridge decks for map-
ping delaminations.
KEYWORDS
Bridge-deck; delamination;
infrared; visible; UAV
1. Introduction
Concrete surfaces need to be periodically inspected to detect deterioration
and defects. Concrete pavement surfaces, including those on bridge decks
can develop a particular type of defect, called delamination [1,2]. The pre-
sence of delaminations will weaken the concrete, and eventually produce
spalling, leaving a hole in the concrete surface [2,3]. Common methods to
identify and map delamination areas involve both in situ and remote sensing
techniques. In situ methods, like hammer soundings and chain dragging
CONTACT Rüdiger Escobar-Wolf rpescoba@mtu.edu Department of Geological and Mining Engineering
and Sciences, Michigan Technological University, 630 Dow Environmental Sciences, 1400 Townsend Drive,
Houghton, MI 49931, USA.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/urnd.
Supplemental data for this article can be accessed on the publishers website.
RESEARCH IN NONDESTRUCTIVE EVALUATION
http://dx.doi.org/10.1080/09349847.2017.1304597
© 2017 American Society for Nondestructive Testing
tests, tend to be labor intensive, inspector subjective, and expensive; alter-
natively, delamination areas can be mapped using infrared remote sensing
[4]. The physical principle behind this method is the difference in heat
conduction between areas with delaminations and areas that do not have
delaminations [4,5]. As the concrete surface is exposed to a heat source, like
the natural sun-light, heat at the surface is transferred to the interior of the
concrete volume, whereas regions with delaminations will transfer heat less
efficiently causing the surface temperature to increase in those areas (see
Fig. 1). By mapping the differences in concrete surface temperatures, and
ignoring other sources of variation, it is possible to infer the likely location of
delaminations.
Temperature measurements of the concrete surface can be obtained remo-
tely using an infrared thermal camera. The electromagnetic radiation emitted
by a surface increases with the surfaces temperature, following the principles
of black body (or gray body, sensu stricto) radiance physics and Plancks law
[6]. For the purpose of mapping delaminations in many cases it is not
necessary to calculate actual temperatures with accuracy; having a map of
the relative temperatures is sufficient, and this in turn can be obtained from a
map of relative radiances. This approach has been applied successfully under
a variety of conditions, and using different platforms to deploy thermal
imaging instruments [4,7,8].
Recent studies have demonstrated several innovative applications of
Unmanned Aerial Vehicles (UAVs) to cope with disasters and conduct
infrastructure and resource monitoring. For example, UAV applications for
Tsunami recovery [9], community-based forest monitoring [10], collecting
traffic information [11], hyperspectral terrain mapping [12], unpaved road
Figure 1. Illustration of the heat flow through the concrete body as it is heated by a source
applied on its surface. Areas above delaminations will heat more than adjacent areas that do not
have such defects.
2R. ESCOBAR-WOLF ET AL.
assessment [13], and outdoor inspection of building facades [14]. Although
the potential applications of UAV abound, using UAVs have certain limita-
tions due to the weight of the sensors and the potential flight times that
hinder use in some fields [14,15].
In this study, the innovative application of deploying an infrared and optical
camera sensors from an UAV platform to perform bridge deck inspection to
detect likely delamination is presented. Compared with other platforms, UAVs
offer significant advantages for collecting infrared and optical images of con-
crete surfaces on roads, bridge decks, etc. From the altitude and at the speeds at
which UAVs can fly, a much larger area can be covered with high spatial
resolution compared with the area that would be covered by the same camera
mounted on a terrestrial vehicle, over the same amount of time. At the same
time, UAVs are much less expensive to operate than normal manned aircraft.
New rules proposed by the U.S. Federal Aviation Administration (FAA) in
February 2015 for small UAVs (those under 25 kg), and a FAA commercial
exemption process, make UAV deployments more feasible as an inspection
tool. To make this a viable application for bridge deck inspection, laboratory
and field testing are performed to demonstrate the capabilities of such coupled
sensor-UAV systems.
The motivation to use this kind of platform is to reduce cost and time for
the data acquisition and analysis. UAVs could potentially be deployed with-
out disrupting traffic, and due to the range of altitudes at which they could
fly, and depending on the sensor capabilities, bridge or road widths section
can be captured in a single frame, which would not be practically possible for
terrestrial platforms.
2. Methods
2.1. Infrared imaging sensor
A Tau 2 uncooled core instrument manufactured by FLIR® was used as the
thermal infrared imaging camera for the concrete surfaces. The Tau 2 is
sensitive to electromagnetic radiation in the ~8 to 15 µm spectral range, with
a 336 x 256 sensor (pixels) array of uncooled VOx Microbolometers. The
experiments were performed with a 13 mm (wide field of view) lens, result-
ing in total field of view angles of 25º x 19º, with an instantaneous field of
view (per pixel) of 1.3 mrad (~ 0.074º). The Tau 2 small size (13 x 19 mm),
low weight (< 70 g), and low power consumption (~1 W), is advantageous
when used on UAV platforms. Full specifications and other details on the
Tau 2 thermal camera can be found on the manufacturers website [16].
A TeAx ThermalCapture® module was used to capture and store the Tau
2 output as individual images, and stores them in a local USB memory for
further processing and analysis. The ThermalCapture® module is specifically
RESEARCH IN NONDESTRUCTIVE EVALUATION 3
designed for the Tau 2 instruments, and was chosen because of its light
weight (45 g), small size (60 x 54 x 15 mm), and low power consumption
(~2 W @ 5 V DC). The output images of the ThermalCapture module are
14 bit binary files, at rates up to ~1.2 frames per second. More information
on the ThermalCapture module can be found on the manufacturers web-
site [17].
To power the Tau 2 and ThermalCapture module, an EasyAcc® Xtra 12000
mAh power bank battery was used. This battery provides 5 V DC current, at
up to 2.1 A, through a USB interface. The battery is also compact (14.2 x 7.3
x 2.3 cm) and adds only 320 g to the weight of the system.
The Tau 2 is not temperature calibrated, but calibrated temperatures are
not needed for mapping delamination. The sensor in the camera produces an
output voltage that scales linearly with the radiance incident on the sensor.
The voltage is digitized within the system, to produce digital count values for
each pixel in the sensor array, which are stored in the binary frame files.
Post-processing of the data includes the conversion of the binary files to
standard 16 bit TIF format image files.
2.2. Optical imaging sensor
Additionally to the thermal infrared sensor, a readily available commercial
off-the-shelf (COTS) Nikon D800, digital single-lens reflex (DSLR) camera
was used to acquire high resolution imagery in the visible range of the
spectrum. The camera has a full-size (FX) complementary metal oxide
semiconductor (CMOS) sensor with 36.3 megapixel (mp) resolution with
frame rates of up to 4 frames per second (fps), capable of shooting TIF, JPEG,
and raw image files at up to 1/8,000 second shutter speed. It weighs about 1
kg, with a total of 1.5 kg with the selected Nikkor AF-S 50 mm f/1.4. The lens
was previously tested for an unpaved roads assessment study [13], as it is a
fast lens (one with a large aperture) capable of capturing more light than
typical zoom lenses and useful for photogrammetric data collection.
2.3. Unmanned aerial vehicle
The UAV platform used for this project was a Bergen hexacopter, a multi-
rotor platform commercially available from Bergen R/C Helicopters [21]. It
had previously been selected as a practical, relatively low cost (under US
$6,000), easy-to-operate, and stable platform suitable for collecting imagery
used to create 3D reconstructions from overlapping imagery with better than
2.5 cm resolution [13]. It has up to 30 minutes of flight time, depending on
configuration and weight load, and is more than capable of lifting the Nikon
sensor or Tau 2 sensor, with an approximately 4 kg load limit. With two
years of flight experience, its availability, and its known capability to lift
4R. ESCOBAR-WOLF ET AL.
sensors with the weights needed for this project, the Bergen hexacopter was
selected as the studys main UAV platform.
2.4. Laboratory testing and sensor validation
To assess the performance of the Tau 2 FLIR® and ThermalCapture system, a
series of controlled experiments were performed in the laboratory. The Tau 2
measurements were compared to those obtained by a calibrated FLIR® SC-
640 infrared camera [18], under simultaneous and identical measuring con-
ditions. Delaminations were simulated by building concrete slabs with
embedded pieces of Styrofoam, at variable depths, under the assumption
that the thermal conductivity of the Styrofoamis similar to that of air inside
a real delamination [19]. Figure 2 shows the designs of the concrete slabs
with the embedded Styrofoampieces. The concrete slabs were cured for a
minimum of 28 days before infrared imaging tests were performed.
Thermal imaging tests on the slab were performed by heating the concrete
surface for 15 minute intervals with an infrared radiation lamp, then turning
the lamp off to allow the concrete surface to cool down while acquiring images
over the latter time period. Thermal equilibration with the ambient temperature
was reached about 45 minutes after turning off the heating lamp. To map the
delamination areas in the laboratory experiments, a classification of pixels in
two categories, delamination and non-delamination, was performed by thresh-
old separation methods. Thresholds for the digital numbers representing the
surface radiance were chosen as percentile values from the cumulative
Figure 2. Concrete slab and simulated delaminations design [20].
RESEARCH IN NONDESTRUCTIVE EVALUATION 5
distribution. Datasets were evaluated by comparing moving pixel windows to
their local neighborhood. Classification performance levels were evaluated
through receiver operating characteristic (ROC) curves and parameters derived
from them. ROC curves describe the performance of the classification method
by graphically and quantitatively showing how the true positive rate (the ratio
of correctly classified instance to the total number of classified instances) varies
with relationship to the false positive rate (the ratio of falsely classified instances
to the total number of classified instances), as the threshold value in the
classification algorithm is changed. For further details on ROC analyses, see
Fawcett [20] and references therein. The Matlab® code used to do the classifica-
tion is given in the Supplement.
2.5. Field tests for the UAV and sensor system
The system was tested in the field on Merriman and Stark Road overpass
bridges, located on highway I-96 in Detroit, Michigan. The visible and
infrared sensors were mounted on the UAV, and flown separately over the
bridge decks for data acquisition of the deck surfaces. Sensors were pointing
vertically downward, and the UAV was flown at about 10 m above the bridge
deck surface. The instantaneous field of view (the pixel size at the bridge deck
surface) was between 1.3 and 1.4 cm for the Tau 2 sensor, and about 0.25 cm
(2.5 mm) for the Nikon D800 camera. Despite several UAV flight passes
along the length of the bridge decks in an attempt to acquire fully over-
lapping thermal imagery of the bridges, the actual coverage was not complete
due to a lack of real-time first-person-view of the infrared imagery acquisi-
tion during the operation of the UAV. However, the overlapping thermal
imagery did capture a majority of the deck surfaces at both field sites (see
Fig. 3). Further improvements to the system would include such first-person-
view control of the UAV, or alternatively, an automated UAV flight plan that
would ensure complete coverage of the target surface [11]. The heat source
for the bridge decks was natural solar heating as it is usually the case for such
studies [25,7,8].
To perform a validation test for the remote sensing data, the hammer
sounding technique was performed at seven locations on the Stark Road
Bridge to identify and roughly delimit potential delamination areas to help
understand the usefulness of the nominated likely delaminations obtained
through the UAV-based thermal imagery data. The SC640 FLIR sensor was
also used during the identification of delaminations to enhance the mapping.
Data processing was conducted using similar methods to those described
in Subsection 2.4 for the laboratory tests and for the infrared imagery.
Additionally, the high resolution visible imagery was used to generate a
digital elevation model (DEM), and corresponding orthorectified photo-
graphs. The georeferenced orthophotography was used to georeference the
6R. ESCOBAR-WOLF ET AL.
infrared images, using reflective marks and additional features that were
clearly visible in both the infrared and visible datasets, as tie-points. The
visible imagery was used jointly with the infrared images to enhance the
classification as explained in the next section.
3. Results and discussion
3.1. Laboratory testing and sensor validation
Laboratory experiments on the concrete slabs showed that the infrared sensor
can detect the delaminations for a variety of conditions. Visual inspection of
the infrared images revealed that the 10 x 10 cm delaminations were recog-
nizable as elevated temperature regions in both the uncalibrated Tau 2 and
the temperature calibrated SC 640 FLIR sensors, but the smaller 2.5 x 2.5 cm
delaminations are not. The limits of the delamination areas on the laboratory
concrete slabs appear sharper for shallower delamination depths, when the
temperature is higher, and become blurred when the delamination depth
increases and the temperature decreases (see Fig. 4); this is the expected
behavior for concrete delaminations [4]. Figure 4 shows the results for
different delamination depths (2.5 and 5 cm) and cooling times (15 and 40
minutes), for images acquired with both the SC 640 and the Tau 2 sensor.
Although the SC640 sensor shows smaller noise levels compared to the Tau 2
sensor, the sensitivity and overall performance of both sensors seem
comparable.
Figure 3. Infrared images (colorscale from black through red to yellow) overlaid on the high
resolution orthophotos obtained via the visible camera. The Merriman Road overpass bridge is
shown in (A), and the Stark Road overpass bridge is shown in (B).
RESEARCH IN NONDESTRUCTIVE EVALUATION 7
Mapping of delamination areas was performed by choosing thresholds for
the digital numbers (proxy for radiance) to classify pixels in two categories:
delamination and non-delamination area. The choice of a threshold value
can be based on the statistical properties of the pixel values dataset, and in
the case of the laboratory experiments, different percentiles of the distribu-
tion of pixel values were used. After obtaining poor results by applying single
thresholds to entire image datasets, moving windows of different sizes were
applied to the threshold values to more accurately capture the local variability
of the data.
Because accurate dimensions of the delaminations were known from the
laboratory experiment design and construction, the results obtained from the
infrared remote sensing mapping can be compared with the actual location
and the extent of delaminations. ROC analysis was used to assess the
performance of the mapping algorithms for different test conditions (e.g.,
moving window sizes, delamination depths, temperatures). To construct the
ROC curves, four parameters are needed, i.e., the numbers or true positives,
false positives, true negatives, and false negatives. True positives are pixels
classified as delamination by our method, that area real delaminations as
corroborated from the laboratory or field experiments. Similarly, false posi-
tives are pixels classified as delaminations by the method, but which are not
delaminations in reality. True negatives correspond to pixels classified as not
being delaminations, not being delamination piexles in reality, and false
negatives are pixels that have been classified as not belonging to the delami-
nation category, when in reality they are delaminations. Further details on
how to calculate the true positive rate, the false positive rate and their
Figure 4. Comparison between sensors. (A) and (C) are FLIR SC 640 images, and (B) and (D) are
Tau 2 images. The upper images (A and B) show delamination depths of 3.8 cm and cooling
times of 40 minutes, the lower images show delamination depths of 2.5 cm and cooling times of
15 minutes.
8R. ESCOBAR-WOLF ET AL.
interpretation in the ROC diagram are given by Fawcett [20]. ROC curves for
different cases are shown in Fig. 5, and the results are summarized in Table 1
for specific points along the curves. Under favorable conditions, e.g., shallow
(2.5 cm) delaminations and short (< 30 minutes) cooling times, and using a
wide enough moving window (100 pixels), the algorithm performs rela-
tively well, with Area Under the Curve (AUC) values > 0.8. The performance
degrades considerably when the delamination depth increases (e.g., 5 cm),
the cooling times are more prolonged (> 30 minutes), and the moving
window size decreases (e.g., 50 pixels); in such cases, the AUC values fall
below 0.6. Table 1 and Fig. 5 also show the results for two specific threshold
values, a less conservative value of 50th percentile and a more conservative
value of 90th percentile. The 50th percentile threshold results in higher false
positive rates (FPR
50
), and true positive rates (TPR
50
), as compared with the
corresponding values for the 90th percentile threshold (FPR
90
andTPR
90
),
but has a lower accuracy (Acc
50
) as compared with the accuracy results of
using the 90th percentile threshold (Acc
90
).
Figure 5. ROC curves for the laboratory concrete slab experiments. Delamination depths are
2.5 cm for (A) and (B), and 5.1 cm for (C) and (D). Moving window sizes are 50 x 50 pixels for (A)
and (C), and 150 x 150 pixels for (B) and (D). The different curve types correspond to the time in
minutes for the acquisition of the thermal image, after powering off the heating lamp. Circles
and triangles in each plot mark the 50th and 90th percentile threshold points.
RESEARCH IN NONDESTRUCTIVE EVALUATION 9
The ROC analysis confirms the results from visual observation of the
infrared images. The area under the ROC curve (AUC) is a measure of the
general performance of the classification method, in this case whether or not
the pixels that were classified as delamination or non-delamination, were so
in reality. AUC values can range from 0.5, corresponding to a completely
random classification without any informative power, to 1, in the case of a
perfect (no false positives, nor false negatives) classification. As the depth of
delamination becomes shallower and the experiment temperatures increase,
the AUC values increase correspondingly. But the AUC also increases with
the moving window size of the classification algorithm, highlighting that the
Table 1. ROC curve parameters for the laboratory concrete slab experiments.
D W T AUC FPR
50
TPR
50
Acc
50
FPR
90
TPR
90
Acc
90
2.5 50 18 0.73 0.53 0.87 0.49 0.06 0.17 0.89
2.5 50 28 0.70 0.54 0.82 0.49 0.07 0.18 0.88
2.5 50 34 0.67 0.54 0.77 0.48 0.06 0.16 0.89
2.5 50 41 0.64 0.53 0.72 0.48 0.07 0.15 0.88
2.5 100 18 0.90 0.59 0.98 0.44 0.06 0.54 0.92
2.5 100 28 0.88 0.60 0.97 0.43 0.06 0.52 0.91
2.5 100 34 0.84 0.62 0.94 0.42 0.05 0.41 0.92
2.5 100 41 0.78 0.61 0.89 0.42 0.06 0.33 0.91
2.5 150 18 0.94 0.69 0.99 0.36 0.05 0.70 0.94
2.5 150 28 0.93 0.68 0.99 0.36 0.07 0.74 0.92
2.5 150 34 0.89 0.71 0.97 0.33 0.05 0.57 0.93
2.5 150 41 0.80 0.71 0.95 0.33 0.06 0.37 0.91
3.8 50 19 0.68 0.53 0.83 0.50 0.08 0.12 0.85
3.8 50 25 0.64 0.53 0.76 0.49 0.08 0.12 0.85
3.8 50 40 0.59 0.54 0.67 0.48 0.08 0.12 0.84
3.8 100 19 0.82 0.58 0.91 0.46 0.07 0.47 0.88
3.8 100 25 0.77 0.59 0.87 0.45 0.07 0.40 0.88
3.8 100 40 0.68 0.60 0.78 0.43 0.09 0.31 0.86
3.8 150 19 0.86 0.67 0.93 0.39 0.07 0.66 0.90
3.8 150 25 0.81 0.67 0.90 0.38 0.07 0.55 0.90
3.8 150 40 0.70 0.68 0.82 0.37 0.09 0.40 0.87
5.1 50 7 0.62 0.53 0.71 0.49 0.08 0.14 0.85
5.1 50 9 0.60 0.54 0.67 0.48 0.07 0.13 0.85
5.1 50 15 0.59 0.54 0.65 0.48 0.08 0.13 0.85
5.1 50 20 0.57 0.54 0.63 0.48 0.07 0.12 0.85
5.1 50 23 0.56 0.54 0.62 0.48 0.08 0.13 0.85
5.1 100 7 0.75 0.59 0.87 0.45 0.09 0.33 0.86
5.1 100 9 0.72 0.60 0.84 0.44 0.07 0.29 0.87
5.1 100 15 0.71 0.60 0.83 0.44 0.07 0.28 0.87
5.1 100 20 0.69 0.61 0.80 0.43 0.06 0.24 0.88
5.1 100 23 0.66 0.61 0.78 0.43 0.07 0.23 0.87
5.1 150 7 0.76 0.66 0.91 0.39 0.10 0.39 0.86
5.1 150 9 0.75 0.69 0.89 0.36 0.07 0.35 0.88
5.1 150 15 0.75 0.69 0.88 0.36 0.07 0.36 0.88
5.1 150 20 0.73 0.71 0.87 0.34 0.05 0.29 0.89
5.1 150 23 0.69 0.70 0.84 0.35 0.07 0.26 0.87
Table 1 D is the delamination depth in cm. W is the analysis window width in pixels. T is the time in minutes
of acquisition of the thermal image since the IR lamp was shut down (a proxy for cooling). AUC is the area
under the ROC curve. FPR50, TPR50, and Accu50 are the false positive rate, true positive rate, and overall
accuracy at a threshold of 50th percentile. FPR90, TPR90, and Accu90 are the false positive rate, true
positive rate, and overall accuracy at a threshold of 90th percentile.
10 R. ESCOBAR-WOLF ET AL.
performance is not only dependent on the physical conditions of the test, but
also on the choice of the algorithm parameters during the analysis phase.
3.2. Field tests for the UAV and sensor system
The field test results from Merriman and Stark Roads further validate the
method, showing overall good agreement with independent determinations
of delamination areas. The thermographic analysis was performed on the
area where thermal images were available (see Fig. 3) and which covers most
of the bridge decks in both cases. Due to the rather limited extent and poor
resolution (defining the exact boundary of delamination areas) of the ham-
mer soundings, the validation of field experiments cannot be conducted as
rigorously as it was with the delaminations in the laboratory concrete slabs;
concrete coring would be needed for this, which was not available for this
project due to an impending deck repair project. However, a comparison
between the infrared-based delamination mapping, the hammer sounding,
and SC640-based mapping is still possible. The ROC curves and related
parameters were obtained for six areas on the Stark Road Bridge deck, for
which overlapping infrared imagery and hammer sounding data were avail-
able, as shown in Fig. 6 and Table 2. These show a good degree of agreement
between the field identified areas (via ground hammer sounding and SC649
imaging), and the UAV infrared imagebased mapping. The AUC values are
very high (> 0.95) for all tested cases, in part because the validation set is
Figure 6. ROC results for the field tests at the Merriman and Stark Road bridges. The six test
curves are tightly bundled and show a very good performance. Circles and triangles in each plot
mark the 50th and 90th percentile threshold points.
RESEARCH IN NONDESTRUCTIVE EVALUATION 11
small and consists of very obvious delamination areas; this may indicate a
bias in the sampling, and therefore, the apparently good performance has to
be interpreted in that context.
The concrete bridge deck surfaces show a much more complex structure in
the infrared images than the concrete slabs tested in the laboratory. A patchy
appearance is caused at least in part by the pattern of older vs. newer patches
of concrete from repairs performed over time (see Fig. 3). Although it is
assumed that changes in the concrete surface radiance in an infrared image
are due to difference in the surface temperature, the differences can also be
due to changes in the emissivity of the materials surface [4,8], and in that
case the validity of the thermal imaging method for mapping delaminations
may be compromised. This makes the classification more challenging.
However, results are improved by combining the infrared and visible imagery
through the methods described below.
3.3. Joint infrared and visible imagery analysis
One aim of the pixel classification method is to separate variations in
emissivity from actual delaminations. To improve the classification method,
visible imagery is included in the analysis. The Nikon D800 sensor detects
radiation in the visible part of the spectrum, specifically in three bands: red,
green, and blue (RGB). The color of concrete is usually close to gray, which
results in a high correlation of the RGB values, and hence a high information
redundancy in those bands; therefore, the three visible bands are joined into
a single averaged visible band.
The visible band is not sensitive to the delaminations, as is the case for
the infrared band, and can be used to detect potential false positives.
However, changes in the emissivity due to changes in the concrete surface
material (e.g., patches of different concrete) are likely to show up in the
visible bands. Therefore, cases of anomalies detected in the thermal and
visible bands may be due to changes in the surface material properties and
not due to delaminations, while anomalies detected in the infrared band,
but not on the visible bands, are much more likely to be caused by actual
delaminations.
Table 2. ROC parameter results for tests at the Merriman and Stark Road bridges.
Point AUC FPR
50
TPR
50
Acc
50
FPR
90
TPR
90
Acc
90
1 0.97 0.39 0.99 0.62 0.03 0.79 0.97
2 0.96 0.40 1.00 0.61 0.04 0.68 0.95
3 0.98 0.40 1.00 0.60 0.05 0.91 0.95
5 0.98 0.45 1.00 0.55 0.06 0.97 0.94
6 0.97 0.45 1.00 0.55 0.04 0.79 0.95
7 0.96 0.43 0.99 0.58 0.05 0.82 0.95
Table 2 Nomenclature is the same as in Table 1.
12 R. ESCOBAR-WOLF ET AL.
Figure 7 illustrates the rationale behind the mapping algorithm that
incorporates the visible bands in the analysis. Figure 7a shows the visible
image and Fig. 7b shows the infrared image of the same area. First, areas of
elevated temperature in the infrared image are flagged as potential delamina-
tion regions (Fig. 7c). Second, areas of potentially different emissivity are
mapped in the visible band, by applying a moving window method to high-
light pixels with values 1.5 times the standard deviation above the mean value
(Fig. 7d). Finally, the potential delamination areas obtained from the infrared
band is compared with areas that may have different emissivity, and obtained
from the visible band. Only pixels for which there appears to be no emissivity
changes are classified as potential delamination areas (Fig. 7e).
The method just described was applied to the concrete bridge deck for
which there was thermal image coverage (almost the entire bridge deck),
followed by post-processing of the mapped areas, including a low-pass
Figure 7. Illustration of the algorithm for detecting delaminations combining infrared and
thermal imager. See text for details.
RESEARCH IN NONDESTRUCTIVE EVALUATION 13
filtering, adjacency connectedness, and minimum area filtering of the classi-
fied pixels to discard individual isolated pixels. The results are shown in
Fig. 8. The area covered by thermal and visible imagery is 968 m
2
, and 14 m
2
were classified as delamination areas with this method, which is equivalent to
1.5% of the total area imaged with the infrared and visible sensors on that
particular concrete bridge deck. The Matlab® code used in the pixel classifi-
cation is given in the supplemental file.
Further work will be necessary to develop the method in more depth.
Controlled experiments using different types of surfaces and delaminations
could be used to test how well the joint thermal-optical method works. The
effects of heating cycles due to daily insolation variations could also be explored,
as this may be important for the application of the method. The method could
also be compared to other techniques in the field (e.g., high speed deck vehicle
scanners, GPR, etc.). The comparison to GPR could provide some insight into the
method, but the balance between the likely higher precision of the GPR versus the
potentially less expensive acquisition, operation, and analysis cost of the thermal
UAV would have to be weighed. More sophisticated analysis algorithms, includ-
ing machine learning and other data processing methods, could further improve
the delamination detection analysis.
4. Conclusions
The laboratory and field experiments using small infrared and DSLR visible
cameras mounted on a UAV show that the system can be effectively used to
map delamination areas on bridge decks. Laboratory tests were used to
validate the detection of delaminations from the images collected with a
small infrared sensor that can easily be deployed on UAVs. The field tests
on two concrete bridge decks were further used as a validation set, under
Figure 8. Final results of the delamination mapping method. Delaminations are show by blue
polygons on the visible (a) and infrared (b) imagery.
14 R. ESCOBAR-WOLF ET AL.
more challenging but realistic conditions of heterogeneous concrete surfaces,
at the larger scale of a real-world transportation infrastructure case. The
performance of the infrared delamination mapping system can be improved
by including visible imagery to reduce the rate of potential false positives, i.e.,
areas that are falsely mapped as delaminations but have a different surface
emissivity.
The UAV platform offers advantages, including maneuverability and
larger field of view per image, compared to traditional data acquisition
platforms for delamination mapping. However, several challenges must be
overcome to make this method fully operational for delamination mapping
purposes. Further testing and validation is necessary to build more robust
mapping algorithms. With the exception of the minimum area filtering
completed in the post-processing of the images, all the applied analysis
methods were pixel-based and have the disadvantage of taking into
account the spatial context in a marginal way, ignoring important char-
acteristics, like the shape and size of the mapped delaminations. Adapting
the method to include such spatially contextual information could improve
even more the performance. Although the visible bands were included in
the mapping algorithm, other products derived from the visible photogra-
phy, including 3D photogrammetric products, like digital elevation models
of the deck surface, would potentially improve the delamination mapping
method.
Acknowledgments
Comments by two anonymous reviewers greatly contributed to enhance the quality of this
paper.
Funding
This work was possible through funding by the Michigan Department of Transportation
under contract number 2013-0067, project number Authorization No. 1, OR number OR13-
008.
References
1. Guettala, A. and A. Abibsi. Materials and Structures 39(4):439446 (2006).
2. Masser, K. and W. Roddis. Journal of Transportation Engineering 116(5):583601
(1990).
3. Masliwec, T. Proceedings SPIE 0934, Thermosense X: Thermal Infrared Sensing for
Diagnostics and Control, 19. Orlando Technical Symposium. doi:10.1117/12.968482
(1988).
4. Washer, G., R. Fenwick, and N. Bolleni. Development of hand-held thermographic
inspection technologies. Missouri Department of Transportation. Organizational
Results Research Report. (2009).
RESEARCH IN NONDESTRUCTIVE EVALUATION 15
5. Manning, D. and F. Holt. Concrete International 2(11):3442 (1980).
6. Jense, J. Remote Sensing of the Environment: An Earth Resource Perspective. 2nd
Edition. Prentice-Hall (2007). Upper Saddle River, NJ, USA.
7. Vaghefi, K., R. Oats, D. Harris, T. Ahlborn, C. Brooks, K. Endsley, C. Roussi, R.
Shuchman, J. Burns, and R. Dobson. Journal of Bridge Engineering 17:886895 (2011).
8. Vaghefi, K., T. Ahlborn, D. Harris, and C. Brooks. Journal of Performance of
Constructed Facilities 29(4):04014102-1-8. doi: 10.1061/(ASCE)CF.1943-5509.0000465
(2013).
9. Murphy, R., K. L. Dreger, S. Newsome, J. Rodocker, B. Slaughter, R. Smith, E. Steimle,
T. Kimura, K. Makabe, K. Kon, H. Mizumoto, M. Hatayama, F. Matsuno, S. Tadokoro,
and O. Kawase. Journal of Field Robotics 29(5):819831 (2012).
10. Paneque-Gálvez, J., M. McCall, B. Napoletano, S. Wich, and L. Koh. Forests 5:1481
1507 (2014).
11. Liu, X. F., Z. R. Peng, L. Y. Zhang, and L. Li. J. Transport. Syst. Eng. Inform. Technol.12
(1):9197 (2012).
12. Lucieer, A., S. Robinson, D. Turner, S. Harwin, and J. Kelcey. International Archives of
the Photogrammetry, Remote Sensing and Spatial Information Sciences XXXIX-B1:429
433 (2012).
13. Dobson, R. J., T. Colling, C. Brooks, C. Roussi, M. K. Watkins, and D. Dean.
Transportation Research Record: Journal of the Transportation Research Board 2433
(1):108115 (2014).
14. Roca, D., S. Lagüela, L. Díaz-Vilariño, J. Armesto, and P. Arias. Automation in
Construction 36:128135 (2013).
15. Liu, P., A. Chen, Y. Huang, J. Han, J. Lai, S. Kang, T. Wu, M. Wen, and M. Tsai. Smart
Structures and Systems 13(6):10651094 (2014).
16. FLIR. Tau 2 Technical Specifications. http://www.flir.com/cvs/cores/view/?id=54717
(last accessed February 2015).
17. TeAx. ThermalCapture Technical Specifications http://www.teaxtec.de/index.php?id=
23&L=1 (last accessed February 2015).
18. FLIR. SC 640 Technical Specifications. http://www.flir.com/uploadedFiles/
Thermography_APAC/Products/Product_Literture/SC640_Datasheet%20APAC.pdf
(last accessed February 2015).
19. Vaghefi, K. , PhD dissertation. Michigan Technological University, 156 pp (2013).
20. Fawcett, T. Pattern Recognition Letters 27:861874 (2006).
21. Bergen R/C Helicopters. http://www.bergenrc.com/Multi.php (last accessed March
2015).
16 R. ESCOBAR-WOLF ET AL.
... The measurement distance was usually set as around 2 m to ensure sufficient image resolution for inspecting small defects, such as cracks [66,75]. When it comes to inspecting subsurface delamination, the measurement distance was set as around 10 m since the delamination usually covers a large area [73,76]. As for measuring bridge displacement, the measurement distance was usually set as around 3-7 m [77,78]. ...
... Most of the articles in 'Damage inspection' (44 out of 50) required a qualified pilot to operate the UAV for data collection, resulting in classification as Level 1 automation [65,66,68,[72][73][74][75][76]. For example, Wang et al. [100] manually flew a self-developed UAV and collected ...
... Instead of only using infrared images, fusion of infrared and RGB images can provide more robust results. Deck area [72] and surface defects [76] can be extracted from the RGB images to remove false positives caused by the background and surface defects in delamination detected from the infrared image. ...
Article
Full-text available
In recent years, Unmanned Aerial Vehicles (UAVs) have been increasingly used for bridge inspection. Although UAVs are expected to automate the whole bridge inspection process ranging from data collection and analysis to decision-making, most existing efforts still require substantial human intervention. There is currently no study that examines the state-of-the-art of automated UAV-enabled bridge inspection (UBI). This study conducts a systematic review of 115 journal articles published from 2007 to 2021 to understand the level of automation (LoA) of existing UBI approaches, highlight challenges and guide future research. The data collected was coded through a template analysis and assessed against a pre-designed LoA scale for UBI. Bibliometric and LoA analyses present in-depth insights into UBI both quantitatively and qualitatively. Furthermore, relevant challenges and future research opportunities towards fully automated UBI are discussed.
... For instance, one of the possible applications of the presented concept is multi-modal road inspection. Thermographic inspection can be used to detect delamination in concrete structures [89] or sinkholes in pavement roads [90]. However, for an automated process pipeline, it is required to extract regions of interest to ensure the accurate analysis of concrete or pavement structures. ...
Article
Full-text available
The drone-based multi-modal inspection of industrial structures is a relatively new field of research gaining interest among companies. Multi-modal inspection can significantly enhance data analysis and provide a more accurate assessment of the components’ operability and structural integrity, which can assist in avoiding data misinterpretation and providing a more comprehensive evaluation, which is one of the NDT4.0 objectives. This paper investigates the use of coupled thermal and visible images to enhance abnormality detection accuracy in drone-based multi-modal inspections. Four use cases are presented, introducing novel process pipelines for enhancing defect detection in different scenarios. The first use case presents a process pipeline to enhance the feature visibility on visible images using thermal images in pavement crack detection. The second use case proposes an abnormality classification method for surface and subsurface defects using both modalities and texture segmentation for piping inspections. The third use case introduces a process pipeline for road inspection using both modalities. A texture segmentation method is proposed to extract the pavement regions in thermal and visible images. Further, the combination of both modalities is used to detect surface and subsurface defects. The texture segmentation approach is employed for bridge inspection in the fourth use case to extract concrete surfaces in both modalities.
... Based on temperature distribution data, Omar [99] and Wells [100] completed a concrete bridge inspection using a UAS mounted with infrared thermography. Escobar-Wolf et al. [101] used IRT for undersurface delamination and deck hole detection, which generated thermal and visible images for a 968 m 2 area. Compared with direct contact hammer sounding, an overall accuracy of approximately 95% showed in 14 m 2 delamination. ...
Article
Full-text available
The increasing need for inexpensive, safe, highly efficient, and time-saving damage detection technology, combined with emerging technologies, has made damage detection by unmanned aircraft systems (UAS) an active research area. In the past, numerous sensors have been developed for damage detection, but these sensors have only recently been integrated with UAS. UAS damage detection specifically concerns data collection, path planning, multi-sensor fusion, system integration, damage quantification, and data processing in building a prediction model to predict the remaining service life. This review provides an overview of crucial scientific advances that marked the development of UAS inspection: underlying UAS platforms, peripherals, sensing equipment, data processing approaches, and service life prediction models. Example equipment includes a visual camera, a multispectral sensor, a hyperspectral sensor, a thermal infrared sensor, and light detection and ranging (LiDAR). This review also includes highlights of the remaining scientific challenges and development trends, including the critical need for self-navigated control, autonomic damage detection, and deterioration model building. Finally, we conclude with a brief discussion regarding the pros and cons of this emerging technology, along with a prospect of UAS technology research for damage detection.
... TIR&V images can provide thermal and reflectance attributes of ground objects. It is useful to register and synergistically use TIR&V images in many fields (Xiang et al., 2019), including precision agriculture (Messina and Modica, 2020), wildlife protection (Chrétien et al., 2015), infrastructure inspection (Escobar-Wolf et al., 2018), emergency rescue (Ambrosia et al., 2003), building facade thermal attribute mapping (Lin et al., 2019), and defect detection (Li et al., 2017). For example, Poblete et al. (2018) and Zhang et al. (2019) extracted the thermal attributes of pure vegetation while avoiding background influences from the registered TIR&V images. ...
Article
Automatic registration of unmanned aerial vehicle (UAV) thermal infrared and visible (TIR&V) images is fundamental for subsequent applications. However, few studies address this issue due to significant radiation gap, shape gap, and texture gap among TIR&V images. The area-based methods are not able to satisfy the accuracy and robustness of location at the same time, while the image pyramid-based methods are computationally expensive. To alleviate these problems, we proposed a so-called TWMM method for the registration of UAV TIR&V images taken by the camera equipped with both thermal infrared and visible sensors. TWMM is realized by combining Template matching with Weights, Multilevel local max-pooling, and Max index backtracking. TWMM consists of four steps: (1) computing similarity maps of the atomic patches using template matching with weights; (2) building pyramid similarity maps using multilevel local max-pooling; (3) deducing the corresponding points (CPs) from top to bottom using max index backtracking; and (4) eliminating outliers and estimating homography. Among the four steps, step 1 and step 2 are used to compute the similarity maps of patches with different sizes; step 3 and step 4 are used to deduce CPs and estimate homography with multiple similarity maps. TWMM was comprehensively evaluated with 600 UAV image pairs under four different scenes and also compared with current methods (i.e. SIFT, SURF, RIFT, RCB, TFeat, HardNet, RANSAC_Flow, HOPC, and CFOG). These image pairs have multiple features, i.e., different land covers, spatial resolutions, and illumination conditions, etc. Results indicate that TWMM achieves an 86.0% correct CP ratio (RCP) and a 96.0% correct matching rate (CMR) for all test images, which is a 15.1% improvement and 11.6% improvement, respectively, over the best state-of-the-art methods. TWMM also shows better robustness than other methods for weak-light images, achieving a 20.7% improvement in RCP and a 28.1% improvement in CMR. Therefore, TWMM is an effective and robust method for UAV TIR&V image registration and has good ability under different scenes.
... Unlike the aforementioned great number of visual applications, the IR inspection is still underexplored within the UAV applications, although IR images of bridge decks are already commonly used during conventional bridge inspections to obtain information on subsurface defects, like concrete delamination [103]. Some few researches that explore this nondestructive method are briefly described below: • Brooks et al. [8] and Escobar-Wolf et al. [88] used the FLIR Tau 2 IR camera, installed on the Bergen Hexacopter, to detect six delamination areas on the inspected bridge deck, where a total of seven delamination areas (all without visible signs) had been spotted before with a handheld FLIR SC 640 IR camera, whose presence were confirmed with a hammer test, i.e., only one delamination area was not found with the UAV IR imagery. Then, their proposed method improves the classification by also including the images collected by a Nikon D800 camera (also installed on the UAV) in the analysis; • Zink and Lovelace [103] tested the integrated IR camera from the Aeyron Skyranger UAV in one of the evaluated bridges, where the collected IR images clearly showed the thermal gradient at the deck beam locations; • Khan et al. [119] and Ellenberg et al. [120] performed experimental investigations on a mock-up concrete bridge, where a hexacopter based on DJI F550 frame (equipped with a GoPro Hero3 camera and a FLIR TAU2 IR camera) was used to map it using a multispectral approach, consisted of visual and IR imaging. ...
Article
Full-text available
Bridges and tunnels are large and complex structures that demand periodic inspections to assess their physical conditions. Although both have different designs and constructions from each other, a common problem they share is the drawbacks that their conventional inspections face. Moreover, conventional procedures not only are laborious, time-consuming, and costly, but also involve high and/or hard-to-reach places, often exposing the specialized inspectors to danger. To overcome these problems, the Unmanned Aerial Vehicle (UAV) is being explored to automate these inspections. Recently, the number of researches employing it within the civil infrastructure condition assessment has been growing in recent years, especially for the inspection of large and complex structures. Unlike the UAV-based bridge inspection that already has some review articles available in the literature, there are none yet for the tunnel inspection, to the best of authors' knowledge. Therefore, this article intends to conduct not only a review of the few UAV-based tunnel inspection researches available in the literature, but also an up-to-date review of UAV-based bridge inspection researches. Finally, the key challenges and future trends of the UAV-based inspection of these two structures are discussed, followed by the review conclusions.
... The use of sUAS systems also ensures rapid operation, accessibility, safety, and a unique view of a range of conditions, thereby providing useful information for timely decision-making. sUAS systems have also been used to map areas of likely concrete delamination, to generate unpaved road distress data for decision support systems, and to inspect bridge construction (9)(10)(11). ...
Article
Current practice for airport Pavement Management Program (PMP) inspection relies on visual surveys and manual interpretation of reports and sketches prepared by inspectors in the field to quantify pavement conditions using the Pavement Condition Index method set forth in ASTM D5340. In recent years, several attempts have been made, both by the industry and by airport operators, to use small Uncrewed (Unpersonned/Unmanned) Aircraft Systems (sUAS), or “drones,” to conduct various types of imaging and inspection of airport pavements. As part of a comprehensive study on the use of such sUAS to evaluate airfield pavement conditions, the objectives of this study were to assess the performance of various sUAS platforms and sensors in detecting and rating a subset of crack-based pavement distresses and to evaluate the use of a combination of different sUAS datasets to complement current methods used to support airport PMP. Two airports in Michigan were selected for sUAS data collection, and five sUAS platforms equipped with eight different sensors were flown at these airports at different altitudes to collect red, green, and blue (RGB) optical and thermal data at different resolutions. RGB orthophotos, digital elevation models, and thermal images were visually analyzed to study their usefulness in detecting and rating longitudinal and transverse cracks in flexible/asphalt pavements and longitudinal, transverse, and diagonal cracks, corner breaks, and durability cracks in rigid/concrete pavements. This study demonstrated the capability of using sUAS data in detecting and rating multiple crack-related distresses in both flexible and rigid airfield pavement systems.
... In recent years, with the rapid development of remote sensing technology with a higher spatio-temporal resolution, various remote sensing technologies have been widely used in deformation monitoring research of large-scale artificial structures. Examples include laser scanning technologies [3], distributed fiber optic sensing (DFOS) [4], and unmanned aerial vehicle measurements [5]. As a new type of Earth observation method, synthetic aperture radar interferometry (InSAR) technology can overcome the shortcomings of traditional observation methods. ...
Article
Full-text available
Ankang Airport is constructed on an expansive soil-fill platform in Shaanxi Province, Central China. Since its completion in 2020, it has suffered surface deformation caused by the consolidation and settlement of the fill layer and instability of the expansive soil slope. Exploring the special deformation law of expansive soil regions by remote sensing and analyzing the deformation characteristics of airports in mountainous areas have always been key issues in related disaster research. Based on the intensity and phase observation data of 37 Sentinel-1 synthetic aperture radar images, this study obtained the spatio-temporal distribution of the deformation of Ankang Airport from May 2020 to October 2021. First, phase optimization was performed on the original interferograms. Second, the persistent scatterer synthetic aperture radar interferometry (PS-InSAR) method was applied to extract the surface deformation information of Ankang Airport, and the accuracy was evaluated. Finally, the singular spectrum analysis method was introduced to jointly analyze the deformation information obtained by the InSAR technology in combination with geological and climatic data. The results show that the excavation area of Ankang Airport was basically stable, the filling area had obvious surface and uneven deformation, and the expansive soil fill slope exhibits deformation characteristics strongly related to slope, rainfall, and fill depth. The deformation was mainly caused by consolidation and settlement, supplemented by the expansion and shrinkage deformation of the expansive soil.
... Barrile et al. [15] stated that the use of UAV technology in infrastructure surveying spread recently in different applications. Different authors [16][17][18][19][20] have studied the need for a standard methodology and workflow for data acquisition and analysis. For example, the research completed for evaluating the bridge inspection quality in New York State in 2021 [21] concluded that some action should be taken to clarify the desired content of inspection reports. ...
Article
Full-text available
Many bridges and other structures worldwide present a lack of maintenance or a need for reha-bilitation. The first step in the rehabilitation process is to perform a bridge inspection to know the bridge′s current state. Routine bridge inspections are usually based only on visual recognition. In this paper, a methodology for bridge inspections in communication routes using images acquired by unmanned aerial vehicle (UAV) flights is proposed. This provides access to the upper parts of the structure safely and without traffic disruptions. Then, a standardized and systematized novel image acquisition protocol is applied for data acquisition. Afterwards, the images are studied by civil engineers for damage identification and description. Then, specific structural inspection forms are completed using the acquired information. Recommendations about the need of new and more detailed inspections should be included at this stage when needed. The suggested method-ology was tested on two railway bridges in France. Image acquisition of these structures was performed using an UAV for its ability to provide an expert assessment of the damage level. The main advantage of this method is that it makes it possible to safely accurately identify diverse damages in structures without the need for a specialised engineer to go to the site. Moreover, the videos can be watched by as many engineers as needed with no personal movement. The main objective of this work is to describe the systematized methodology for the development of bridge inspection tasks using a UAV system. According to this proposal, the in situ inspection by a specialised engineer is replaced by images and videos obtained from an UAV flight by a trained flight operator. To this aim, a systematized image/videos acquisition method is defined for the study of the morphology and typology of the structural elements of the inspected bridges. Addi-tionally, specific inspection forms are proposed for every type of structural element. The recorded information will allow structural engineers to perform a postanalysis of the damage affecting the bridges and to evaluate the subsequent recommendations.
... [7][8][9] Computer vision methods offer a promising approach to identifying the condition of infrastructure with inexpensive cameras installed on the UAV. For example, recent studies have demonstrated several innovative applications of UAVs equipped with cameras (e.g., optical, infrared) to conduct infrastructure monitoring such as delamination detection of concrete bridge decks, 10,11 modal analysis of a pedestrian suspension bridge, 12 and visual inspection of steel girder bridges. 13,14 In all of these applications, the UAV systems are primarily used as a mobile data collection platform to observe the system from afar and make no direct contact with the structure. ...
Article
Unmanned aerial vehicles (UAVs) equipped with onboard sensors empower end‐users to collect data within a wide range of civil engineering applications such as structural condition assessment. While UAVs have been used primarily as mobile sensing platforms to obtain imagery or other data, their mobility can also be used to deploy sensor networks. In this study, the feasibility of using an autonomous UAV to deploy wireless sensors in structural monitoring applications is proposed. While GPS‐based waypoint navigation is available for automating UAV flight operations, this approach does not provide the accuracy necessary for the precision placement of sensor payloads on structures. Computer vision‐based pose estimation is proposed to improve the accuracy of UAV localization for sensor placement. Variably sized fiducial markers integrated into a single pattern are applied to the surface of the structure and adopted as navigation and precision landing targets that identify sensor placement locations. Visual and inertial measurements are fused by means of a discrete Kalman filter to further increase the robustness of the relative position estimation algorithm that is included in the proportional‐integral‐derivative (PID) control law used for UAV landing. Outdoor experiments under realistic operational conditions are conducted to validate the proposed vision‐aided control of the UAV for sensor placement; the UAV is able to land on a predefined landing point within 10 cm. A UAV moving a wireless accelerometer to locations on a beam is used to experimentally show the validity of automating UAV sensor placement for modal analysis using reconfigurable sensor network topologies.
Article
Bridge decks need to be routinely inspected to ensure their serviceability, capacity, and safety under current traffic conditions. Traditionally, bridge deck inspection is performed on the ground by having inspectors either visually inspect surface conditions or interpret the acoustic feedback from hammer sounding or chain dragging to determine subsurface conditions. These traditional methods have many limitations, including but not limited to, expensive, labour-intensive, time-consuming, subjective, can exhibit a high degree of variability, requiring specialized staff on a regular basis, and unsafe. Recent advancements in remote sensing, especially small-uncrewed aircraft systems (S-UAS) based airborne imaging techniques and advanced image analysis techniques, have shown promise in improving current bridge deck inspection practices by providing an above-ground inspection method. This research explored the utility of S-UAS-based airborne imaging techniques and image processing techniques to develop a complete aerial data acquisition and analysis system to accurately detect and assess bridge deck wearing surface distresses in a timely and cost-effective manner. As part of the research project, a robust tool was also developed with the aim of being able to detect, extract, and map bridge deck wearing surface distresses with an adequate degree of accuracy while maximizing the ability to assist bridge inspectors with varying expertise. Research results revealed that the developed tool is able to effectively detect and map bridge deck wearing surface distresses at a high accuracy.
Conference Paper
Full-text available
This study is the first to use an Unmanned Aerial Vehicle (UAV) for mapping moss beds in Antarctica. Mosses can be used as indicators for the regional effects of climate change. Mapping and monitoring their extent and health is therefore important. UAV aerial photography provides ultra-high resolution spatial data for this purpose. We developed a technique to extract an extremely dense 3D point cloud from overlapping UAV aerial photography based on structure from motion (SfM) algorithms. The combination of SfM and patch-based multi-view stereo image vision algorithms resulted in a 2 cm resolution digital terrain model (DTM). This detailed topographic information combined with vegetation indices derived from a 6-band multispectral sensor enabled the assessment of moss bed health. This novel UAV system has allowed us to map different environmental characteristics of the moss beds at ultra-high resolution providing us with a better understanding of these fragile Antarctic ecosystems. The paper provides details on the different UAV instruments and the image processing framework resulting in DEMs, vegetation indices, and terrain derivatives.
Article
Full-text available
Data gathered through community-based forest monitoring (CBFM) programs may be as accurate as those gathered by professional scientists, but acquired at a much lower cost and capable of providing more detailed data about the occurrence, extent and drivers of forest loss, degradation and regrowth at the community scale. In addition, CBFM enables greater survey repeatability. Therefore, CBFM should be a fundamental component of national forest monitoring systems and programs to measure, report and verify (MRV) REDD+ activities. To contribute to the development of more effective approaches to CBFM, in this paper we assess: (1) the feasibility of using small, low-cost drones (i.e., remotely piloted aerial vehicles) in CBFM programs; (2) their potential advantages and disadvantages for communities, partner organizations and forest data end-users; and (3) to what extent their utilization, coupled with ground surveys and local ecological knowledge, would improve tropical forest monitoring. To do so, we reviewed the existing literature regarding environmental applications of drones, including forest monitoring, and drew on our own firsthand experience flying small drones to map and monitor tropical forests and training people to operate them. We believe that the utilization of small drones can enhance CBFM and that this approach is feasible in many locations throughout the tropics if some degree of external assistance and funding is provided to communities. We suggest that the use of small drones can help tropical communities to better manage and conserve their forests whilst benefiting partner organizations, governments and forest data end-users, particularly those engaged in forestry, biodiversity conservation and climate change mitigation projects such as REDD+.
Article
Full-text available
Civil engineers always face the challenge of uncertainty in planning, building, and maintaining infrastructure. These works rely heavily on a variety of surveying and monitoring techniques. Unmanned aerial vehicles (UAVs) are an effective approach to obtain information from an additional view, and potentially bring significant benefits to civil engineering. This paper gives an overview of the state of UAV developments and their possible applications in civil engineering. The paper begins with an introduction to UAV hardware, software, and control methodologies. It also reviews the latest developments in technologies related to UAVs, such as control theories, navigation methods, and image processing. Finally, the paper concludes with a summary of the potential applications of UAV to seismic risk assessment, transportation, disaster response, construction management, surveying and mapping, and flood monitoring and assessment.
Article
Unpaved roads make up roughly 33% of the road system within the United States and are vitally important to rural communities for transport of people and goods. Effective asset management of unpaved roads requires frequent inspections to determine the roads' condition and the appropriate preventive maintenance or rehabilitation. The major challenge with managing unpaved roads is low-cost collection of condition data that are compatible with a decision support system (DSS). The advent of cheap, reliable remote-sensing platforms such as unmanned aerial vehicles along with the development of commercial off-the-shelf image analysis algorithms provides a revolutionary opportunity to overcome these data volume and efficiency issues. By taking advantage of these technological leaps, a market-ready system to detect unpaved road distress data compatible with a DSS was developed. The system uses aerial imagery that can be collected from a remote-controlled helicopter or manned fixed-wing aircraft to create a three-dimensional model of sensed road segments. Condition information on potholes, ruts, washboarding, loss of crown, and float aggregate berms is then detected and characterized to determine the extent and severity of the distress. Once detection and analysis are complete, the data are imported into a DSS based on a geographic information system (Roadsoft) for use by road managers to prioritize preventive maintenance and rehabilitation efforts.
Article
Evaluating the condition of concrete bridge decks is an increasingly important challenge for transportation agencies and bridge inspection teams. Closing the bridge to traffic, safety, and time consuming data collection are some of the major issues during a visual or indepth bridge inspection. To date, several nondestructive testing technologies have shown promise in detecting subsurface deteriorations. However, the main challenge is to develop a data acquisition and analysis system to obtain and integrate both surface and subsurface bridge health indicators at higher speeds. Recent developments in imaging technologies for bridge decks and higher-end cameras allow for faster image collection while driving over the bridge deck. This paper will focus on deploying nondestructive imaging technologies such as the three-dimensional (3D) optical bridge evaluation system (3DOBS) and thermal infrared (IR) imagery on a bridge deck to yield both surface and subsurface indicators of condition, respectively. Spall and delamination maps were generated from the optical and thermal IR images. Integration of the maps into ArcGIS, a professional geographic information system (GIS), allowed for a streamlined analysis that included integrating and combining the results of the complimentary technologies. Finally, ground truth information was gathered through coring several locations on a bridge deck to validate the results obtained by nondestructive evaluation. This study confirms the feasibility of combining the bridge inspection results in ArcGIS and provides additional evidence to suggest that thermal infrared imagery provides similar results to chain dragging for bridge inspection. (C) 2014 American Society of Civil Engineers.
Article
This field report describes two deployments of heterogeneous unmanned marine vehicle teams at the 2011 Great Eastern Japan Earthquake response and recovery by the Center for Robot-Assisted Search and Rescue (USA) in collaboration with the International Rescue System Institute (Japan). Four remotely operated underwater vehicles (ROVs) were fielded in Minamisanriku and Rikuzentakata from April 18 to 24, 2011, for port clearing and victim recovery missions using sonar and video. The ROVs were used for multirobot operations only 46% of the time due to logistics. The teleoperated ROVs functioned as a dependent team 86% of the time to avoid sensor interference or collisions. The deployment successfully reopened the Minamisanriku New Port area and searched areas prohibited to divers in Rikuzentakata. The IRS-CRASAR team planned to return from October 18 to 28, 2011, with an unmanned aerial vehicle (UAV), an autonomous underwater vehicle (AUV), and an ROV to conduct debris mapping for environmental remediation missions. The intent was to investigate an interdependent strategy by which the UAV and AUV would rapidly conduct low-resolution scans identifying areas of interest for further investigation by the ROV. The UAV and AUV could not be used; however, the ROV was able to cover 80,000 m2 in 6 h, finding submerged wreckage and pollutants in areas previously marked clear by divers. The field work (i) showed that the actual and planned multirobot system configurations did not fall neatly into traditional taxonomies, (ii) identified a new measure, namely perceptual confidence, and (iii) posed five open research questions for multirobot systems operating in littoral regions. © 2012 Wiley Periodicals, Inc. © 2012 Wiley Periodicals, Inc.
Article
In this paper, the unmanned aerial vehicle (UAV) route planning problem is introduced to deploy the UAV for road traffic information collection. The scenario of using limited UAVs to detect road sections is considered, and a multi-objective optimization model is developed, which uses the number of the UAVs and UAV maximum cruise distance as constraints and aims to minimize the total cruise distance and maximize the number of detected road sections. A novel non-dominated sorting genetic algorithm for this problem is then proposed. The case study shows that the nearly optimal solution for planning UAV routes can be acquired effectively. Compared the obtained solution with the optimal feasible solution, the total cruise distance is reduced by 13.07% and the number of detected targets is increased by 41.67%. Finally, some issues on deploying UAVs for traffic information collection are discussed.摘要引入无人飞机作为城市道路固定交通检测设备的辅助手段,部署无人飞机进行道路交通信息采集,提出了无人飞机的路径规划问题。考虑了无人飞机数量有限,不足以对所有目标进行侦察的情形,建立了以总巡航距离最短、巡航目标数量最多的多目标优化模型,提出了可行路径的重组方法,构造了求解该问题的非支配排序遗传算法。案例分析结果表明:构造的算法可以求出无人飞机路径规划的近似最优解,与最优初始可行解相比,总巡航距离减少了13.07%,巡航目标数量增加了41.67%。最后,讨论了无人飞机在道路交通信息采集中可能面临的问题。
Article
A comprehensive data base is a critical component of a cost-effective bridge management program. Data is needed in order to assess the overall condition of the bridge network, establish priorities for repair, select the most appropriate methods for bridge repair, and prepare contract documents. The most practical and economic method of collecting this information is to use indirect measurement. techniques. Such non-destructive techniques do not directly measure the physical properties of the structure and the condition must be inferred using signal processing methods. A number of bridge testing techniques were investigated by the Ontario Ministry of Transportation over a period of almost ten years. The product of the studies is the Deck Assessment by Radar and Thermography (DART) system. As the acronym implies, DART utilises two basic systems: impulse radar and infrared thermography. This paper describes the main features of the DART infrared system. A theoretical two-dimensional heat transfer model for estimating temperature profiles in bridge decks is also presented. Actual temperature measurements are provided for comparison with the theoretical model.
Article
Traditional methods of bridge deck condition assessment are slow, labor-intensive, intrusive to traffic, and unreliable. Two new technologies, radar and infrared thermography, which have recently been introduced, show promise for producing rapid and accurate condition assessment for bridge decks. These technologies are being applied without the benefit of a firm physical understanding of their inherent capabilities and limitations. This paper discusses the physical principles upon which these techniques are based, and proposes simple physical models for the prediction of radar and infrared response to various bridge deck conditions. Parameter studies are carried out using these models to predict the radar and infrared response to moisture, chloride, delamination, and deck geometry. The model study results show the range of sensitivity and the inherent limitations of these two techniques. These results have led to the suggestion of a predictive technique that has been used in field studies of repaired and rehabilitated asphalt-overlaid decks. This technique has been shown to predict the area of deterioration to within 5% of total deck area.