Unmanned Aerial Vehicle (UAV)-Based Assessment of Concrete Bridge Deck Delamination Using Thermal and Visible Camera Sensors: A Preliminary Analysis

Abstract

Infrared and visible cameras were mounted on an unmanned aerial vehicle (UAV) to image bridge deck surfaces and map likely concrete delaminations. The infrared sensor was first tested on laboratory validation experiments, to assess how well it could detect and map delaminations under controlled conditions. Field tests on two bridge deck surfaces further extend the validation dataset to real-world conditions for heterogeneous concrete surfaces. Performance of the mapping instrument and algorithms were evaluated through receiver operating characteristic (ROC) curves, giving acceptable results. To improve the performance of the mapping by reducing the rate of false positives, i.e., areas wrongly mapped as being affected by delamination, visible images were jointly analyzed with the infrared imagery. The potential for expanding the method to include other products derived from the visible camera data, including high density 3D point locations generated by photogrammetric methods, promises to further improve the performance of the method, potentially making it a viable and more effective option compared to other platforms and systems for imaging bridge decks for mapping delaminations.

Full text

Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=urnd20
Download by: [71.13.208.198] Date: 28 July 2017, At: 06:19
Research in Nondestructive Evaluation
ISSN: 0934-9847 (Print) 1432-2110 (Online) Journal homepage: http://www.tandfonline.com/loi/urnd20
Unmanned Aerial Vehicle (UAV)-Based Assessment
of Concrete Bridge Deck Delamination Using
Thermal and Visible Camera Sensors: A
Preliminary Analysis
Rüdiger Escobar-Wolf, Thomas Oommen, Colin N. Brooks, Richard J. Dobson
& Theresa M. Ahlborn
To cite this article: Rüdiger Escobar-Wolf, Thomas Oommen, Colin N. Brooks, Richard J. Dobson
& Theresa M. Ahlborn (2017): Unmanned Aerial Vehicle (UAV)-Based Assessment of Concrete
Bridge Deck Delamination Using Thermal and Visible Camera Sensors: A Preliminary Analysis,
Research in Nondestructive Evaluation, DOI: 10.1080/09349847.2017.1304597
To link to this article: http://dx.doi.org/10.1080/09349847.2017.1304597
View supplementary material
Published online: 17 Apr 2017.
Submit your article to this journal
Article views: 35
View related articles
View Crossmark data

Unmanned Aerial Vehicle (UAV)-Based Assessment of
Concrete Bridge Deck Delamination Using Thermal and
Visible Camera Sensors: A Preliminary Analysis
Rüdiger Escobar-Wolf
a
, Thomas Oommen
a
, Colin N. Brooks
b
, Richard J. Dobson
b
,
and Theresa M. Ahlborn
a
a
Department of Geological and Mining Engineering and Sciences, Michigan Technological University,
Houghton Michigan, USA;
b
Michigan Technological Research Institute, Michigan Technological
University, Ann Arbor, Michigan, USA
ABSTRACT
Infrared and visible cameras were mounted on an unmanned
aerial vehicle (UAV) to image bridge deck surfaces and map likely
concrete delaminations. The infrared sensor was first tested on
laboratory validation experiments, to assess how well it could
detect and map delaminations under controlled conditions. Field
tests on two bridge deck surfaces further extend the validation
dataset to real-world conditions for heterogeneous concrete
surfaces. Performance of the mapping instrument and algorithms
were evaluated through receiver operating characteristic (ROC)
curves, giving acceptable results. To improve the performance of
the mapping by reducing the rate of false positives, i.e., areas
wrongly mapped as being affected by delamination, visible
images were jointly analyzed with the infrared imagery. The
potential for expanding the method to include other products
derived from the visible camera data, including high density 3D
point locations generated by photogrammetric methods, pro-
mises to further improve the performance of the method, poten-
tially making it a viable and more effective option compared to
other platforms and systems for imaging bridge decks for map-
ping delaminations.
KEYWORDS
Bridge-deck; delamination;
infrared; visible; UAV
1. Introduction
Concrete surfaces need to be periodically inspected to detect deterioration
and defects. Concrete pavement surfaces, including those on bridge decks
can develop a particular type of defect, called delamination [1,2]. The pre-
sence of delaminations will weaken the concrete, and eventually produce
spalling, leaving a hole in the concrete surface [2,3]. Common methods to
identify and map delamination areas involve both in situ and remote sensing
techniques. In situ methods, like hammer soundings and chain dragging
CONTACT Rüdiger Escobar-Wolf rpescoba@mtu.edu Department of Geological and Mining Engineering
and Sciences, Michigan Technological University, 630 Dow Environmental Sciences, 1400 Townsend Drive,
Houghton, MI 49931, USA.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/urnd.
Supplemental data for this article can be accessed on the publisher’s website.
RESEARCH IN NONDESTRUCTIVE EVALUATION
http://dx.doi.org/10.1080/09349847.2017.1304597
© 2017 American Society for Nondestructive Testing

tests, tend to be labor intensive, inspector subjective, and expensive; alter-
natively, delamination areas can be mapped using infrared remote sensing
[4]. The physical principle behind this method is the difference in heat
conduction between areas with delaminations and areas that do not have
delaminations [4,5]. As the concrete surface is exposed to a heat source, like
the natural sun-light, heat at the surface is transferred to the interior of the
concrete volume, whereas regions with delaminations will transfer heat less
efficiently causing the surface temperature to increase in those areas (see
Fig. 1). By mapping the differences in concrete surface temperatures, and
ignoring other sources of variation, it is possible to infer the likely location of
delaminations.
Temperature measurements of the concrete surface can be obtained remo-
tely using an infrared thermal camera. The electromagnetic radiation emitted
by a surface increases with the surface’s temperature, following the principles
of black body (or gray body, sensu stricto) radiance physics and Planck’s law
[6]. For the purpose of mapping delaminations in many cases it is not
necessary to calculate actual temperatures with accuracy; having a map of
the relative temperatures is sufficient, and this in turn can be obtained from a
map of relative radiances. This approach has been applied successfully under
a variety of conditions, and using different platforms to deploy thermal
imaging instruments [4,7,8].
Recent studies have demonstrated several innovative applications of
Unmanned Aerial Vehicles (UAVs) to cope with disasters and conduct
infrastructure and resource monitoring. For example, UAV applications for
Tsunami recovery [9], community-based forest monitoring [10], collecting
traffic information [11], hyperspectral terrain mapping [12], unpaved road
Figure 1. Illustration of the heat flow through the concrete body as it is heated by a source
applied on its surface. Areas above delaminations will heat more than adjacent areas that do not
have such defects.
2R. ESCOBAR-WOLF ET AL.

assessment [13], and outdoor inspection of building facades [14]. Although
the potential applications of UAV abound, using UAVs have certain limita-
tions due to the weight of the sensors and the potential flight times that
hinder use in some fields [14,15].
In this study, the innovative application of deploying an infrared and optical
camera sensors from an UAV platform to perform bridge deck inspection to
detect likely delamination is presented. Compared with other platforms, UAVs
offer significant advantages for collecting infrared and optical images of con-
crete surfaces on roads, bridge decks, etc. From the altitude and at the speeds at
which UAVs can fly, a much larger area can be covered with high spatial
resolution compared with the area that would be covered by the same camera
mounted on a terrestrial vehicle, over the same amount of time. At the same
time, UAVs are much less expensive to operate than normal manned aircraft.
New rules proposed by the U.S. Federal Aviation Administration (FAA) in
February 2015 for small UAVs (those under 25 kg), and a FAA commercial
exemption process, make UAV deployments more feasible as an inspection
tool. To make this a viable application for bridge deck inspection, laboratory
and field testing are performed to demonstrate the capabilities of such coupled
sensor-UAV systems.
The motivation to use this kind of platform is to reduce cost and time for
the data acquisition and analysis. UAVs could potentially be deployed with-
out disrupting traffic, and due to the range of altitudes at which they could
fly, and depending on the sensor capabilities, bridge or road widths section
can be captured in a single frame, which would not be practically possible for
terrestrial platforms.
2. Methods
2.1. Infrared imaging sensor
A Tau 2 uncooled core instrument manufactured by FLIR® was used as the
thermal infrared imaging camera for the concrete surfaces. The Tau 2 is
sensitive to electromagnetic radiation in the ~8 to 15 µm spectral range, with
a 336 x 256 sensor (pixels) array of uncooled VOx Microbolometers. The
experiments were performed with a 13 mm (wide field of view) lens, result-
ing in total field of view angles of 25º x 19º, with an instantaneous field of
view (per pixel) of 1.3 mrad (~ 0.074º). The Tau 2 small size (13 x 19 mm),
low weight (< 70 g), and low power consumption (~1 W), is advantageous
when used on UAV platforms. Full specifications and other details on the
Tau 2 thermal camera can be found on the manufacturer’s website [16].
A TeAx ThermalCapture® module was used to capture and store the Tau
2 output as individual images, and stores them in a local USB memory for
further processing and analysis. The ThermalCapture® module is specifically
RESEARCH IN NONDESTRUCTIVE EVALUATION 3

designed for the Tau 2 instruments, and was chosen because of its light
weight (45 g), small size (60 x 54 x 15 mm), and low power consumption
(~2 W @ 5 V DC). The output images of the ThermalCapture module are
14 bit binary files, at rates up to ~1.2 frames per second. More information
on the ThermalCapture module can be found on the manufacturer’s web-
site [17].
To power the Tau 2 and ThermalCapture module, an EasyAcc® Xtra 12000
mAh power bank battery was used. This battery provides 5 V DC current, at
up to 2.1 A, through a USB interface. The battery is also compact (14.2 x 7.3
x 2.3 cm) and adds only 320 g to the weight of the system.
The Tau 2 is not temperature calibrated, but calibrated temperatures are
not needed for mapping delamination. The sensor in the camera produces an
output voltage that scales linearly with the radiance incident on the sensor.
The voltage is digitized within the system, to produce digital count values for
each pixel in the sensor array, which are stored in the binary frame files.
Post-processing of the data includes the conversion of the binary files to
standard 16 bit TIF format image files.
2.2. Optical imaging sensor
Additionally to the thermal infrared sensor, a readily available commercial
off-the-shelf (COTS) Nikon D800, digital single-lens reflex (DSLR) camera
was used to acquire high resolution imagery in the visible range of the
spectrum. The camera has a full-size (FX) complementary metal oxide
semiconductor (CMOS) sensor with 36.3 megapixel (mp) resolution with
frame rates of up to 4 frames per second (fps), capable of shooting TIF, JPEG,
and raw image files at up to 1/8,000 second shutter speed. It weighs about 1
kg, with a total of 1.5 kg with the selected Nikkor AF-S 50 mm f/1.4. The lens
was previously tested for an unpaved roads assessment study [13], as it is a
fast lens (one with a large aperture) capable of capturing more light than
typical zoom lenses and useful for photogrammetric data collection.
2.3. Unmanned aerial vehicle
The UAV platform used for this project was a Bergen hexacopter, a multi-
rotor platform commercially available from Bergen R/C Helicopters [21]. It
had previously been selected as a practical, relatively low cost (under US
$6,000), easy-to-operate, and stable platform suitable for collecting imagery
used to create 3D reconstructions from overlapping imagery with better than
2.5 cm resolution [13]. It has up to 30 minutes of flight time, depending on
configuration and weight load, and is more than capable of lifting the Nikon
sensor or Tau 2 sensor, with an approximately 4 kg load limit. With two
years of flight experience, its availability, and its known capability to lift
4R. ESCOBAR-WOLF ET AL.

sensors with the weights needed for this project, the Bergen hexacopter was
selected as the study’s main UAV platform.
2.4. Laboratory testing and sensor validation
To assess the performance of the Tau 2 FLIR® and ThermalCapture system, a
series of controlled experiments were performed in the laboratory. The Tau 2
measurements were compared to those obtained by a calibrated FLIR® SC-
640 infrared camera [18], under simultaneous and identical measuring con-
ditions. Delaminations were simulated by building concrete slabs with
embedded pieces of Styrofoam™, at variable depths, under the assumption
that the thermal conductivity of the Styrofoam™is similar to that of air inside
a real delamination [19]. Figure 2 shows the designs of the concrete slabs
with the embedded Styrofoam™pieces. The concrete slabs were cured for a
minimum of 28 days before infrared imaging tests were performed.
Thermal imaging tests on the slab were performed by heating the concrete
surface for 15 minute intervals with an infrared radiation lamp, then turning
the lamp off to allow the concrete surface to cool down while acquiring images
over the latter time period. Thermal equilibration with the ambient temperature
was reached about 45 minutes after turning off the heating lamp. To map the
delamination areas in the laboratory experiments, a classification of pixels in
two categories, delamination and non-delamination, was performed by thresh-
old separation methods. Thresholds for the digital numbers representing the
surface radiance were chosen as percentile values from the cumulative
Figure 2. Concrete slab and simulated delaminations design [20].
RESEARCH IN NONDESTRUCTIVE EVALUATION 5

distribution. Datasets were evaluated by comparing moving pixel windows to
their local neighborhood. Classification performance levels were evaluated
through receiver operating characteristic (ROC) curves and parameters derived
from them. ROC curves describe the performance of the classification method
by graphically and quantitatively showing how the true positive rate (the ratio
of correctly classified instance to the total number of classified instances) varies
with relationship to the false positive rate (the ratio of falsely classified instances
to the total number of classified instances), as the threshold value in the
classification algorithm is changed. For further details on ROC analyses, see
Fawcett [20] and references therein. The Matlab® code used to do the classifica-
tion is given in the Supplement.
2.5. Field tests for the UAV and sensor system
The system was tested in the field on Merriman and Stark Road overpass
bridges, located on highway I-96 in Detroit, Michigan. The visible and
infrared sensors were mounted on the UAV, and flown separately over the
bridge decks for data acquisition of the deck surfaces. Sensors were pointing
vertically downward, and the UAV was flown at about 10 m above the bridge
deck surface. The instantaneous field of view (the pixel size at the bridge deck
surface) was between 1.3 and 1.4 cm for the Tau 2 sensor, and about 0.25 cm
(2.5 mm) for the Nikon D800 camera. Despite several UAV flight passes
along the length of the bridge decks in an attempt to acquire fully over-
lapping thermal imagery of the bridges, the actual coverage was not complete
due to a lack of real-time first-person-view of the infrared imagery acquisi-
tion during the operation of the UAV. However, the overlapping thermal
imagery did capture a majority of the deck surfaces at both field sites (see
Fig. 3). Further improvements to the system would include such first-person-
view control of the UAV, or alternatively, an automated UAV flight plan that
would ensure complete coverage of the target surface [11]. The heat source
for the bridge decks was natural solar heating as it is usually the case for such
studies [2–5,7,8].
To perform a validation test for the remote sensing data, the hammer
sounding technique was performed at seven locations on the Stark Road
Bridge to identify and roughly delimit potential delamination areas to help
understand the usefulness of the nominated likely delaminations obtained
through the UAV-based thermal imagery data. The SC640 FLIR sensor was
also used during the identification of delaminations to enhance the mapping.
Data processing was conducted using similar methods to those described
in Subsection 2.4 for the laboratory tests and for the infrared imagery.
Additionally, the high resolution visible imagery was used to generate a
digital elevation model (DEM), and corresponding orthorectified photo-
graphs. The georeferenced orthophotography was used to georeference the
6R. ESCOBAR-WOLF ET AL.

infrared images, using reflective marks and additional features that were
clearly visible in both the infrared and visible datasets, as tie-points. The
visible imagery was used jointly with the infrared images to enhance the
classification as explained in the next section.
3. Results and discussion
3.1. Laboratory testing and sensor validation
Laboratory experiments on the concrete slabs showed that the infrared sensor
can detect the delaminations for a variety of conditions. Visual inspection of
the infrared images revealed that the 10 x 10 cm delaminations were recog-
nizable as elevated temperature regions in both the uncalibrated Tau 2 and
the temperature calibrated SC 640 FLIR sensors, but the smaller 2.5 x 2.5 cm
delaminations are not. The limits of the delamination areas on the laboratory
concrete slabs appear sharper for shallower delamination depths, when the
temperature is higher, and become blurred when the delamination depth
increases and the temperature decreases (see Fig. 4); this is the expected
behavior for concrete delaminations [4]. Figure 4 shows the results for
different delamination depths (2.5 and 5 cm) and cooling times (15 and 40
minutes), for images acquired with both the SC 640 and the Tau 2 sensor.
Although the SC640 sensor shows smaller noise levels compared to the Tau 2
sensor, the sensitivity and overall performance of both sensors seem
comparable.
Figure 3. Infrared images (colorscale from black through red to yellow) overlaid on the high
resolution orthophotos obtained via the visible camera. The Merriman Road overpass bridge is
shown in (A), and the Stark Road overpass bridge is shown in (B).
RESEARCH IN NONDESTRUCTIVE EVALUATION 7

Mapping of delamination areas was performed by choosing thresholds for
the digital numbers (proxy for radiance) to classify pixels in two categories:
delamination and non-delamination area. The choice of a threshold value
can be based on the statistical properties of the pixel values dataset, and in
the case of the laboratory experiments, different percentiles of the distribu-
tion of pixel values were used. After obtaining poor results by applying single
thresholds to entire image datasets, moving windows of different sizes were
applied to the threshold values to more accurately capture the local variability
of the data.
Because accurate dimensions of the delaminations were known from the
laboratory experiment design and construction, the results obtained from the
infrared remote sensing mapping can be compared with the actual location
and the extent of delaminations. ROC analysis was used to assess the
performance of the mapping algorithms for different test conditions (e.g.,
moving window sizes, delamination depths, temperatures). To construct the
ROC curves, four parameters are needed, i.e., the numbers or true positives,
false positives, true negatives, and false negatives. True positives are pixels
classified as delamination by our method, that area real delaminations as
corroborated from the laboratory or field experiments. Similarly, false posi-
tives are pixels classified as delaminations by the method, but which are not
delaminations in reality. True negatives correspond to pixels classified as not
being delaminations, not being delamination piexles in reality, and false
negatives are pixels that have been classified as not belonging to the delami-
nation category, when in reality they are delaminations. Further details on
how to calculate the true positive rate, the false positive rate and their
Figure 4. Comparison between sensors. (A) and (C) are FLIR SC 640 images, and (B) and (D) are
Tau 2 images. The upper images (A and B) show delamination depths of 3.8 cm and cooling
times of 40 minutes, the lower images show delamination depths of 2.5 cm and cooling times of
15 minutes.
8R. ESCOBAR-WOLF ET AL.

interpretation in the ROC diagram are given by Fawcett [20]. ROC curves for
different cases are shown in Fig. 5, and the results are summarized in Table 1
for specific points along the curves. Under favorable conditions, e.g., shallow
(2.5 cm) delaminations and short (< 30 minutes) cooling times, and using a
wide enough moving window (≥100 pixels), the algorithm performs rela-
tively well, with Area Under the Curve (AUC) values > 0.8. The performance
degrades considerably when the delamination depth increases (e.g., 5 cm),
the cooling times are more prolonged (> 30 minutes), and the moving
window size decreases (e.g., 50 pixels); in such cases, the AUC values fall
below 0.6. Table 1 and Fig. 5 also show the results for two specific threshold
values, a less conservative value of 50th percentile and a more conservative
value of 90th percentile. The 50th percentile threshold results in higher false
positive rates (FPR
50
), and true positive rates (TPR
50
), as compared with the
corresponding values for the 90th percentile threshold (FPR
90
andTPR
90
),
but has a lower accuracy (Acc
50
) as compared with the accuracy results of
using the 90th percentile threshold (Acc
90
).
Figure 5. ROC curves for the laboratory concrete slab experiments. Delamination depths are
2.5 cm for (A) and (B), and 5.1 cm for (C) and (D). Moving window sizes are 50 x 50 pixels for (A)
and (C), and 150 x 150 pixels for (B) and (D). The different curve types correspond to the time in
minutes for the acquisition of the thermal image, after powering off the heating lamp. Circles
and triangles in each plot mark the 50th and 90th percentile threshold points.
RESEARCH IN NONDESTRUCTIVE EVALUATION 9

The ROC analysis confirms the results from visual observation of the
infrared images. The area under the ROC curve (AUC) is a measure of the
general performance of the classification method, in this case whether or not
the pixels that were classified as delamination or non-delamination, were so
in reality. AUC values can range from 0.5, corresponding to a completely
random classification without any informative power, to 1, in the case of a
perfect (no false positives, nor false negatives) classification. As the depth of
delamination becomes shallower and the experiment temperatures increase,
the AUC values increase correspondingly. But the AUC also increases with
the moving window size of the classification algorithm, highlighting that the
Table 1. ROC curve parameters for the laboratory concrete slab experiments.
D W T AUC FPR
50
TPR
50
Acc
50
FPR
90
TPR
90
Acc
90
2.5 50 18 0.73 0.53 0.87 0.49 0.06 0.17 0.89
2.5 50 28 0.70 0.54 0.82 0.49 0.07 0.18 0.88
2.5 50 34 0.67 0.54 0.77 0.48 0.06 0.16 0.89
2.5 50 41 0.64 0.53 0.72 0.48 0.07 0.15 0.88
2.5 100 18 0.90 0.59 0.98 0.44 0.06 0.54 0.92
2.5 100 28 0.88 0.60 0.97 0.43 0.06 0.52 0.91
2.5 100 34 0.84 0.62 0.94 0.42 0.05 0.41 0.92
2.5 100 41 0.78 0.61 0.89 0.42 0.06 0.33 0.91
2.5 150 18 0.94 0.69 0.99 0.36 0.05 0.70 0.94
2.5 150 28 0.93 0.68 0.99 0.36 0.07 0.74 0.92
2.5 150 34 0.89 0.71 0.97 0.33 0.05 0.57 0.93
2.5 150 41 0.80 0.71 0.95 0.33 0.06 0.37 0.91
3.8 50 19 0.68 0.53 0.83 0.50 0.08 0.12 0.85
3.8 50 25 0.64 0.53 0.76 0.49 0.08 0.12 0.85
3.8 50 40 0.59 0.54 0.67 0.48 0.08 0.12 0.84
3.8 100 19 0.82 0.58 0.91 0.46 0.07 0.47 0.88
3.8 100 25 0.77 0.59 0.87 0.45 0.07 0.40 0.88
3.8 100 40 0.68 0.60 0.78 0.43 0.09 0.31 0.86
3.8 150 19 0.86 0.67 0.93 0.39 0.07 0.66 0.90
3.8 150 25 0.81 0.67 0.90 0.38 0.07 0.55 0.90
3.8 150 40 0.70 0.68 0.82 0.37 0.09 0.40 0.87
5.1 50 7 0.62 0.53 0.71 0.49 0.08 0.14 0.85
5.1 50 9 0.60 0.54 0.67 0.48 0.07 0.13 0.85
5.1 50 15 0.59 0.54 0.65 0.48 0.08 0.13 0.85
5.1 50 20 0.57 0.54 0.63 0.48 0.07 0.12 0.85
5.1 50 23 0.56 0.54 0.62 0.48 0.08 0.13 0.85
5.1 100 7 0.75 0.59 0.87 0.45 0.09 0.33 0.86
5.1 100 9 0.72 0.60 0.84 0.44 0.07 0.29 0.87
5.1 100 15 0.71 0.60 0.83 0.44 0.07 0.28 0.87
5.1 100 20 0.69 0.61 0.80 0.43 0.06 0.24 0.88
5.1 100 23 0.66 0.61 0.78 0.43 0.07 0.23 0.87
5.1 150 7 0.76 0.66 0.91 0.39 0.10 0.39 0.86
5.1 150 9 0.75 0.69 0.89 0.36 0.07 0.35 0.88
5.1 150 15 0.75 0.69 0.88 0.36 0.07 0.36 0.88
5.1 150 20 0.73 0.71 0.87 0.34 0.05 0.29 0.89
5.1 150 23 0.69 0.70 0.84 0.35 0.07 0.26 0.87
Table 1 D is the delamination depth in cm. W is the analysis window width in pixels. T is the time in minutes
of acquisition of the thermal image since the IR lamp was shut down (a proxy for cooling). AUC is the area
under the ROC curve. FPR50, TPR50, and Accu50 are the false positive rate, true positive rate, and overall
accuracy at a threshold of 50th percentile. FPR90, TPR90, and Accu90 are the false positive rate, true
positive rate, and overall accuracy at a threshold of 90th percentile.
10 R. ESCOBAR-WOLF ET AL.

performance is not only dependent on the physical conditions of the test, but
also on the choice of the algorithm parameters during the analysis phase.
3.2. Field tests for the UAV and sensor system
The field test results from Merriman and Stark Roads further validate the
method, showing overall good agreement with independent determinations
of delamination areas. The thermographic analysis was performed on the
area where thermal images were available (see Fig. 3) and which covers most
of the bridge decks in both cases. Due to the rather limited extent and poor
resolution (defining the exact boundary of delamination areas) of the ham-
mer soundings, the validation of field experiments cannot be conducted as
rigorously as it was with the delaminations in the laboratory concrete slabs;
concrete coring would be needed for this, which was not available for this
project due to an impending deck repair project. However, a comparison
between the infrared-based delamination mapping, the hammer sounding,
and SC640-based mapping is still possible. The ROC curves and related
parameters were obtained for six areas on the Stark Road Bridge deck, for
which overlapping infrared imagery and hammer sounding data were avail-
able, as shown in Fig. 6 and Table 2. These show a good degree of agreement
between the field identified areas (via ground hammer sounding and SC649
imaging), and the UAV infrared image–based mapping. The AUC values are
very high (> 0.95) for all tested cases, in part because the validation set is
Figure 6. ROC results for the field tests at the Merriman and Stark Road bridges. The six test
curves are tightly bundled and show a very good performance. Circles and triangles in each plot
mark the 50th and 90th percentile threshold points.
RESEARCH IN NONDESTRUCTIVE EVALUATION 11

small and consists of very obvious delamination areas; this may indicate a
bias in the sampling, and therefore, the apparently good performance has to
be interpreted in that context.
The concrete bridge deck surfaces show a much more complex structure in
the infrared images than the concrete slabs tested in the laboratory. A patchy
appearance is caused at least in part by the pattern of older vs. newer patches
of concrete from repairs performed over time (see Fig. 3). Although it is
assumed that changes in the concrete surface radiance in an infrared image
are due to difference in the surface temperature, the differences can also be
due to changes in the emissivity of the material’s surface [4,8], and in that
case the validity of the thermal imaging method for mapping delaminations
may be compromised. This makes the classification more challenging.
However, results are improved by combining the infrared and visible imagery
through the methods described below.
3.3. Joint infrared and visible imagery analysis
One aim of the pixel classification method is to separate variations in
emissivity from actual delaminations. To improve the classification method,
visible imagery is included in the analysis. The Nikon D800 sensor detects
radiation in the visible part of the spectrum, specifically in three bands: red,
green, and blue (RGB). The color of concrete is usually close to gray, which
results in a high correlation of the RGB values, and hence a high information
redundancy in those bands; therefore, the three visible bands are joined into
a single averaged visible band.
The visible band is not sensitive to the delaminations, as is the case for
the infrared band, and can be used to detect potential false positives.
However, changes in the emissivity due to changes in the concrete surface
material (e.g., patches of different concrete) are likely to show up in the
visible bands. Therefore, cases of anomalies detected in the thermal and
visible bands may be due to changes in the surface material properties and
not due to delaminations, while anomalies detected in the infrared band,
but not on the visible bands, are much more likely to be caused by actual
delaminations.
Table 2. ROC parameter results for tests at the Merriman and Stark Road bridges.
Point AUC FPR
50
TPR
50
Acc
50
FPR
90
TPR
90
Acc
90
1 0.97 0.39 0.99 0.62 0.03 0.79 0.97
2 0.96 0.40 1.00 0.61 0.04 0.68 0.95
3 0.98 0.40 1.00 0.60 0.05 0.91 0.95
5 0.98 0.45 1.00 0.55 0.06 0.97 0.94
6 0.97 0.45 1.00 0.55 0.04 0.79 0.95
7 0.96 0.43 0.99 0.58 0.05 0.82 0.95
Table 2 Nomenclature is the same as in Table 1.
12 R. ESCOBAR-WOLF ET AL.

Figure 7 illustrates the rationale behind the mapping algorithm that
incorporates the visible bands in the analysis. Figure 7a shows the visible
image and Fig. 7b shows the infrared image of the same area. First, areas of
elevated temperature in the infrared image are flagged as potential delamina-
tion regions (Fig. 7c). Second, areas of potentially different emissivity are
mapped in the visible band, by applying a moving window method to high-
light pixels with values 1.5 times the standard deviation above the mean value
(Fig. 7d). Finally, the potential delamination areas obtained from the infrared
band is compared with areas that may have different emissivity, and obtained
from the visible band. Only pixels for which there appears to be no emissivity
changes are classified as potential delamination areas (Fig. 7e).
The method just described was applied to the concrete bridge deck for
which there was thermal image coverage (almost the entire bridge deck),
followed by post-processing of the mapped areas, including a low-pass
Figure 7. Illustration of the algorithm for detecting delaminations combining infrared and
thermal imager. See text for details.
RESEARCH IN NONDESTRUCTIVE EVALUATION 13

filtering, adjacency connectedness, and minimum area filtering of the classi-
fied pixels to discard individual isolated pixels. The results are shown in
Fig. 8. The area covered by thermal and visible imagery is 968 m
2
, and 14 m
2
were classified as delamination areas with this method, which is equivalent to
1.5% of the total area imaged with the infrared and visible sensors on that
particular concrete bridge deck. The Matlab® code used in the pixel classifi-
cation is given in the supplemental file.
Further work will be necessary to develop the method in more depth.
Controlled experiments using different types of surfaces and delaminations
could be used to test how well the joint thermal-optical method works. The
effects of heating cycles due to daily insolation variations could also be explored,
as this may be important for the application of the method. The method could
also be compared to other techniques in the field (e.g., high speed deck vehicle
scanners, GPR, etc.). The comparison to GPR could provide some insight into the
method, but the balance between the likely higher precision of the GPR versus the
potentially less expensive acquisition, operation, and analysis cost of the thermal
UAV would have to be weighed. More sophisticated analysis algorithms, includ-
ing machine learning and other data processing methods, could further improve
the delamination detection analysis.
4. Conclusions
The laboratory and field experiments using small infrared and DSLR visible
cameras mounted on a UAV show that the system can be effectively used to
map delamination areas on bridge decks. Laboratory tests were used to
validate the detection of delaminations from the images collected with a
small infrared sensor that can easily be deployed on UAVs. The field tests
on two concrete bridge decks were further used as a validation set, under
Figure 8. Final results of the delamination mapping method. Delaminations are show by blue
polygons on the visible (a) and infrared (b) imagery.
14 R. ESCOBAR-WOLF ET AL.

more challenging but realistic conditions of heterogeneous concrete surfaces,
at the larger scale of a real-world transportation infrastructure case. The
performance of the infrared delamination mapping system can be improved
by including visible imagery to reduce the rate of potential false positives, i.e.,
areas that are falsely mapped as delaminations but have a different surface
emissivity.
The UAV platform offers advantages, including maneuverability and
larger field of view per image, compared to traditional data acquisition
platforms for delamination mapping. However, several challenges must be
overcome to make this method fully operational for delamination mapping
purposes. Further testing and validation is necessary to build more robust
mapping algorithms. With the exception of the minimum area filtering
completed in the post-processing of the images, all the applied analysis
methods were pixel-based and have the disadvantage of taking into
account the spatial context in a marginal way, ignoring important char-
acteristics, like the shape and size of the mapped delaminations. Adapting
the method to include such spatially contextual information could improve
even more the performance. Although the visible bands were included in
the mapping algorithm, other products derived from the visible photogra-
phy, including 3D photogrammetric products, like digital elevation models
of the deck surface, would potentially improve the delamination mapping
method.
Acknowledgments
Comments by two anonymous reviewers greatly contributed to enhance the quality of this
paper.
Funding
This work was possible through funding by the Michigan Department of Transportation
under contract number 2013-0067, project number Authorization No. 1, OR number OR13-
008.
References
1. Guettala, A. and A. Abibsi. Materials and Structures 39(4):439–446 (2006).
2. Masser, K. and W. Roddis. Journal of Transportation Engineering 116(5):583–601
(1990).
3. Masliwec, T. Proceedings SPIE 0934, Thermosense X: Thermal Infrared Sensing for
Diagnostics and Control, 19. Orlando Technical Symposium. doi:10.1117/12.968482
(1988).
4. Washer, G., R. Fenwick, and N. Bolleni. Development of hand-held thermographic
inspection technologies. Missouri Department of Transportation. Organizational
Results Research Report. (2009).
RESEARCH IN NONDESTRUCTIVE EVALUATION 15

5. Manning, D. and F. Holt. Concrete International 2(11):34–42 (1980).
6. Jense, J. Remote Sensing of the Environment: An Earth Resource Perspective. 2nd
Edition. Prentice-Hall (2007). Upper Saddle River, NJ, USA.
7. Vaghefi, K., R. Oats, D. Harris, T. Ahlborn, C. Brooks, K. Endsley, C. Roussi, R.
Shuchman, J. Burns, and R. Dobson. Journal of Bridge Engineering 17:886–895 (2011).
8. Vaghefi, K., T. Ahlborn, D. Harris, and C. Brooks. Journal of Performance of
Constructed Facilities 29(4):04014102-1-8. doi: 10.1061/(ASCE)CF.1943-5509.0000465
(2013).
9. Murphy, R., K. L. Dreger, S. Newsome, J. Rodocker, B. Slaughter, R. Smith, E. Steimle,
T. Kimura, K. Makabe, K. Kon, H. Mizumoto, M. Hatayama, F. Matsuno, S. Tadokoro,
and O. Kawase. Journal of Field Robotics 29(5):819–831 (2012).
10. Paneque-Gálvez, J., M. McCall, B. Napoletano, S. Wich, and L. Koh. Forests 5:1481–
1507 (2014).
11. Liu, X. F., Z. R. Peng, L. Y. Zhang, and L. Li. J. Transport. Syst. Eng. Inform. Technol.12
(1):91–97 (2012).
12. Lucieer, A., S. Robinson, D. Turner, S. Harwin, and J. Kelcey. International Archives of
the Photogrammetry, Remote Sensing and Spatial Information Sciences XXXIX-B1:429–
433 (2012).
13. Dobson, R. J., T. Colling, C. Brooks, C. Roussi, M. K. Watkins, and D. Dean.
Transportation Research Record: Journal of the Transportation Research Board 2433
(1):108–115 (2014).
14. Roca, D., S. Lagüela, L. Díaz-Vilariño, J. Armesto, and P. Arias. Automation in
Construction 36:128–135 (2013).
15. Liu, P., A. Chen, Y. Huang, J. Han, J. Lai, S. Kang, T. Wu, M. Wen, and M. Tsai. Smart
Structures and Systems 13(6):1065–1094 (2014).
16. FLIR. Tau 2 Technical Specifications. http://www.flir.com/cvs/cores/view/?id=54717
(last accessed February 2015).
17. TeAx. ThermalCapture Technical Specifications http://www.teaxtec.de/index.php?id=
23&L=1 (last accessed February 2015).
18. FLIR. SC 640 Technical Specifications. http://www.flir.com/uploadedFiles/
Thermography_APAC/Products/Product_Literture/SC640_Datasheet%20APAC.pdf
(last accessed February 2015).
19. Vaghefi, K. , PhD dissertation. Michigan Technological University, 156 pp (2013).
20. Fawcett, T. Pattern Recognition Letters 27:861–874 (2006).
21. Bergen R/C Helicopters. http://www.bergenrc.com/Multi.php (last accessed March
2015).
16 R. ESCOBAR-WOLF ET AL.