Content uploaded by Vitor S. Martins
Author content
All content in this area was uploaded by Vitor S. Martins on Aug 09, 2022
Content may be subject to copyright.
Remote Sensing of Environment 280 (2022) 113203
Available online 8 August 2022
0034-4257/© 2022 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Deep learning high resolution burned area mapping by transfer learning
from Landsat-8 to PlanetScope
V.S. Martins
a
, D.P. Roy
a
,
b
,
*
, H. Huang
a
, L. Boschetti
c
, H.K. Zhang
d
, L. Yan
a
a
Center for Global Change and Earth Observations, Michigan State University, East Lansing, MI 48824, USA
b
Department of Geography, Environment, and Spatial Sciences, Michigan State University, East Lansing, MI 48824, USA
c
College of Natural Resources, University of Idaho, Moscow, ID 83844, USA
d
Geospatial Sciences Center of Excellence, Department of Geography and Geospatial Sciences, South Dakota State University, Brookings, SD 57007, USA
ARTICLE INFO
Edited by Marie Weiss
Keywords:
PlanetScope
Landsat
Burned area mapping
Deep learning
Transfer learning
Fire
ABSTRACT
High spatial resolution commercial satellite data provide new opportunities for terrestrial monitoring. The recent
availability of near-daily 3 m observations provided by the PlanetScope constellation enables mapping of small
and spatially fragmented burns that are not detected at coarser spatial resolution. This study demonstrates, for
the rst time, the potential for automated PlanetScope 3 m burned area mapping. The PlanetScope sensors have
no onboard calibration or short-wave infrared bands, and have variable overpass times, making them challenging
to use for large area, automated, burned area mapping. To help overcome these issues, a U-Net deep learning
algorithm was developed to classify burned areas from two-date Planetscope 3 m image pairs acquired at the
same location. The deep learning approach, unlike conventional burned area mapping algorithms, is applied to
image spatial subsets and not to single pixels and so incorporates spatial as well as spectral information. Deep
learning requires large amounts of training data. Consequently, transfer learning was undertaken using pre-
existing Landsat-8 derived burned area reference data to train the U-Net that was then rened with a smaller
set of PlanetScope training data. Results across Africa considering 659 PlanetScope radiometrically normalized
image pairs sensed one day apart in 2019 are presented. The U-Net was rst trained with different numbers of
randomly selected 256 ×256 30 m pixel patches extracted from 92 pre-existing Landsat-8 burned area reference
data sets dened for 2014 and 2015. The U-Net trained with 300,000 Landsat patches provided about 13% 30 m
burn omission and commission errors with respect to 65,000 independent 30 m evaluation patches. The U-Net
was then rened by training on 5,000 256 ×256 3 m patches extracted from independently interpreted Plan-
etScope burned area reference data. Qualitatively, the rened U-Net was able to more precisely delineate 3 m
burn boundaries, including the interiors of unburned areas, and better classify “faint” burned areas indicative of
low combustion completeness and/or sparse burns. The rened U-Net 3 m classication accuracy was assessed
with respect to 20 independently interpreted PlanetScope burned area reference data sets, composed of 339.4
million 3 m pixels, with low 12.29% commission and 12.09% omission errors. The dependency of the U-Net
classication accuracy on the burned area proportion within 3 m pixel 256 ×256 patches was also examined,
and patches <6.5% burned were less accurately classied. A regression analysis between the proportion of 30 m
grid cells classied as burned against the proportion labelled as burned in the 3 m reference maps showed high
agreement (r
2
=0.91, slope =0.93, intercept <0.001), indicating that the commission and omission errors
largely compensate at 30 m resolution.
1. Introduction
Wildland res have substantial effects on terrestrial ecosystems and
greenhouse gas emissions with a recent apparent surge of destructive
res causing social disruption and economic costs (Balch et al., 2017;
Bowman et al., 2020; Ward et al., 2020). Satellite data have been used
for several decades to monitor re, by detecting the locations of actively
burning res (Wooster et al., 2021) and by mapping the spatial extent of
the area affected by re, usually referred to as the “burned area”
(Chuvieco et al., 2019). Many burned area products and algorithms have
* Corresponding author at: Center for Global Change and Earth Observations, Michigan State University, East Lansing, MI 48824, USA.
E-mail address: roydavi1@msu.edu (D.P. Roy).
Contents lists available at ScienceDirect
Remote Sensing of Environment
journal homepage: www.elsevier.com/locate/rse
https://doi.org/10.1016/j.rse.2022.113203
Received 1 February 2022; Received in revised form 19 July 2022; Accepted 27 July 2022
Remote Sensing of Environment 280 (2022) 113203
2
been developed for a variety of sensors, primarily using optical wave-
length time series to detect the spectral changes induced by re using
per-pixel rule-based or supervised classication approaches. There is no
consensus burned area mapping algorithm, and algorithms are sensor
specic. In the initial remote sensing era, satellite based burned area
mapping used one or a small number of medium spatial resolution
Landsat images (Hall et al., 1980; Chuvieco and Congalton, 1988; L´
opez
García and Caselles, 1991) or coarse spatial resolution Advanced Very
High-Resolution Radiometer (AVHRR) data that are suboptimal for
burned area mapping (Giglio and Roy, 2020). The most widely used
systematically generated re products are derived from Moderate Res-
olution Imaging Spectroradiometer (MODIS) data acquired by NASA's
Terra and Aqua satellites that have dedicated re monitoring capabil-
ities (Justice et al., 2002; Giglio et al., 2018). However, the MODIS 500
m burned area product and other coarse spatial resolution burned area
products do not detect small and spatially fragmented burns (Laris,
2005; Roy et al., 2019). It is well established that re effects are non-
permanent, for example, a recent global analysis with MODIS data
found that the median burned area persistence time was only 29 days
(Melchiorre and Boschetti, 2018). This and cloud and optically thick
smoke obscuration mean that near-daily satellite observations, such as
provided by MODIS or by combination of data from the Landsat and
Sentinel-2 sensors, are required to reliably map on a systematic basis the
area burned and the date of burning (Giglio et al., 2018; Roy et al.,
2019). Since the advent of free and open access Landsat and Sentinel-2
data (Zhu et al., 2019), 20–30 m burned area products have been
derived using time series algorithms applied over large areas including
algorithms using data from Landsat combined with MODIS active re
detections (Boschetti et al., 2015), Sentinel-2 combined with MODIS
active re detections (Roteta et al., 2019), only Landsat (Hawbaker
et al., 2017), and both Landsat and Sentinel-2 (Roy et al., 2019). The
availability of high spatial resolution (<10 m) satellite data provided by
the commercial sector provides new opportunities for detailed burned
area mapping and for assessment of post-re damage and burn severity
(Arnett et al., 2015; Dragozi et al., 2016; Warner et al., 2017; Vander-
hoof et al., 2018). However, satellite data with high spatial and high
temporal resolution are only recently becoming available. Notably,
Planet Labs Inc. has deployed a constellation of PlanetScope CubeSats
that provide multispectral data at 3–4 m spatial resolution (Planet Team,
2021) with a 30 h global median average revisit interval (Roy et al.,
2021).
This study demonstrates, for the rst time, the potential for auto-
mated PlanetScope 3 m burned area mapping. Recently we illustrated
qualitatively the potential of PlanetScope imagery for mapping small
and spatially fragmented burns (Roy et al., 2019) and a number of other
studies have demonstrated the utility of PlanetScope data for wildre
damage assessment (Michael et al., 2018; Chung et al., 2020). The high
spatial and temporal resolution observations provided by the Planet-
Scope constellation may enable mapping of small and spatially frag-
mented burns, not detected at moderate or coarse spatial resolution, and
may provide data needed to assess the accuracy of coarser resolution
burned area products (Roy et al., 2019). However, there are several
challenges: (i) the PlanetScope sensors acquire imagery in only the
visible and near infrared (NIR) (and also red-edge in the latest sensor
generation), and not in the short-wave infrared (SWIR) that is useful for
burned area mapping (Pereira et al., 1999; Roy et al., 2005a; Bastarrika
et al., 2011; Huang et al., 2016), (ii) imagery are acquired with quite
different overpass times and so solar geometry and bi-directional
reectance effects (Huang and Roy, 2021; Roy et al., 2021) that may
impact the ability to map burns reliably in single images and particularly
when multi-temporal images are used (Trigg et al., 2005; Roy et al.,
2019), (iii) radiometric inconsistencies may occur among images due to
relatively low sensor calibration accuracy and spectral response function
differences among the PlanetScope sensor generations (Houborg and
McCabe, 2018; Huang and Roy, 2021). To help overcome these issues, a
deep learning approach is described that classies burned areas detected
in spatially coincident two-date PlanetScope image pairs that have been
relatively radiometrically normalized. The deep learning approach,
unlike conventional burned area mapping algorithms, is applied to
image spatial subsets and not to single pixels and so incorporates spatial
as well as spectral information.
Deep learning is increasingly used for satellite image classication
(Zhu et al., 2017; Ma et al., 2019). Convolutional neural networks (CNN)
were developed to explore spatial relationships to classify natural im-
ages into a single class (LeCun et al., 2015). The pixels in an image may
be classied independently by application of a trained CNN to image
spatial sub-sets, referred to as to patches, to classify the central patch
pixel. This process can be applied to patches translated across the image
but is computationally expensive and can result in smoothed classied
feature boundaries (Zhang et al., 2018; Martins et al., 2020). The U-Net
architecture is a form of fully convolutional network that classies every
patch pixel and is less affected by these issues (Ronneberger et al.,
2015). The U-Net is the deep learning architecture used in this study for
these reasons and because it is widely used for satellite image classi-
cation (Xu et al., 2018; Stoian et al., 2019; Brandt et al., 2020). The U-
Net has not been used to map burned areas in PlanetScope data, but has
been used recently for burned area mapping with lower spatial resolu-
tion satellite data. Recently, Knopp et al. (2020) used U-Net to classify
10 m burned areas in 110 ×110 km post-re Sentinel-2 images located
in Italy, United Kingdom, and Spain, with a training dataset composed of
~2600 256 ×256 10 m Sentinel-2 patches and visually interpreted class
labels. Independent reference patches were used to validate the results
with a high burn class F1-score >0.86. The F1-score, also known as the
Dice coefcient, is the harmonic mean of the user's and producer's ac-
curacies, sometimes referred to as the precision and recall, respectively
(Congalton and Green, 2019). Pinto et al. (2020) implemented a U-Net
architecture with a Long Short-Term Memory layer to map burned areas
using 7 months of near daily 750 m Visible Infrared Imaging Radiometer
Suite (VIIRS) images resampled to 0.01◦for sub-national regions in
California, Portugal, Brazil, Mozambique, and Australia. The U-Net was
trained with 2000 128 ×128 0.01◦pixel VIIRS patches labelled using
the 500 m NASA MODIS burned area product. The results were
compared with a variety of burned area products with variable burn
class F1 scores from 0.582 to 0.920. Recent research has used other deep
learning approaches to classify burned areas using Sentinel-2 and syn-
thetic aperture radar Sentinel-1C-band data. Sentinel-1 data can pene-
trate clouds and smoke (Torres et al., 2012) but many sensing and
environmental factors affect the C-band backscatter (Beaudoin et al.,
1990; Pulliainen et al., 1996) and, for example, the change in back-
scatter pre- and post-re can be comparable to changes in moisture
content and seasonal phenology in savannas (Mathieu et al., 2019).
Belenguer-Plomer et al. (2021) applied two CNN-based architectures
independently to Sentinel-1, Sentinel-2, and to both datasets together,
degraded to a common 40 m resolution to map burned areas in 110 ×
110 km tiles in different biomes. The results were validated with 30 m
Landsat burned/unburned area maps, and provided 0.35 to 0.64 F1
values that varied with land cover and the input data type. Similarly,
Zhang et al. (2021a) trained a CNN with native 20 m Sentinel-1 and 20
m Sentinel-2 data to map burned areas in 12 re events in California.
The results were validated with PlanetScope 3 m reference data gener-
ated by thresholding NDVI difference maps and they reported F1 values
from 0.77 to 0.96.
Supervised classication requires signicant amounts of training
data that are time-consuming and expensive to collect (Wulder et al.,
2018) and in particular this is a bottleneck for deep learning (Najafabadi
et al., 2015). One strategy to mitigate this issue is to use transfer
learning, i.e., to train a classier using pre-existing training data and
then apply it to classify a different data set (Tuia et al., 2016; Nogueira
et al., 2017; Zou and Zhong, 2018). The “black box” nature of deep
learning algorithms precludes understanding of their internal decisions,
but their generalization ability is known to enable repurposing for other
classication tasks and data types (Oquab et al., 2014; Hu et al., 2015;
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
3
Marmanis et al., 2015). Usually, supervised classiers become less
reliable when used to classify satellite imagery acquired in space/time
further from the conditions that the training data were collected under,
or when applied to imagery that have different sensor characteristics or
pre-processing. Despite this, studies have demonstrated successful
transfer learning among different satellite sensors with broadly similar
spectral characteristics (Wurm et al., 2019; Mateo-García et al., 2020).
In this paper, a transfer learning approach is implemented by using a
pre-existing 30 m Landsat-8 burned area data set to train a U-Net that is
then rened with interpreted 3 m PlanetScope images. The research is
demonstrated using 659 PlanetScope two-date image pairs distributed
across Africa, which is the continent with the most burning (Giglio et al.,
2018) and where we have experience developing and validating satellite
burned area products (Roy and Boschetti, 2009; Roy et al., 2005a,
2005b, 2019). The rened U-Net is used to classify two-date PlanetScope
image pairs, mapping the 3 m pixels that burned between the image
acquisition dates. The results are validated with 20 visually interpreted
burned area maps derived from 3 m PlanetScope image pairs acquired
one day apart across Africa.
2. Data
2.1. Pre-existing Landsat-8 burned area reference data
The Africa portion (black squares, Fig. 1) of the Landsat-8 burned
area reference data set generated to assess the accuracy of the Collection
6 NASA MODIS 500 m burned area product (Boschetti et al., 2019) was
used to train the U-Net. At each location, the data set is composed of a
visually interpreted 30 m burned area map and the two Landsat-8
Operational Land Imager (OLI) atmospherically corrected images
sensed 16 days apart that were used to generate it. The Landast-8 image
pairs were selected between March 1st 2014 to March 19th 2015
following a probability-based sampling protocol using a stratied
random sampling in both time and space (Boschetti et al., 2016). The
Landsat-8 image pairs were interpreted visually following the Commit-
tee on Earth Observation Satellites (CEOS) protocol (Boschetti et al.,
2009) to label 30 m pixels as either burned (i.e., locations burned be-
tween the two image acquisition dates), unburned (i.e., locations that
did not burn between the two image acquisition dates), or unmapped (i.
e., locations that were unobserved due to clouds or shadows in one or
both images, or that could not be interpreted unambiguously). A total of
92 30 m burned area reference maps, and their associated Landsat-8
surface reectance image pairs, located at 78 unique Landsat
path/row locations were selected. All of them were selected to contain at
Fig. 1. Locations of the 92 Landsat 30 m burned area reference data derived from Landsat image pairs sensed 16 days apart at 78 unique Landsat path/rows (black
squares) and of the 659 PlanetScope 3 m image pairs sensed one day apart (circles, colored to denote the number of pairs).
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
4
least one burned 30 m pixel. They occur in each of the ve African bi-
omes (tropical savanna, temperate savanna, deserts and xeric shrub-
lands, tropical forest, Mediterranean) dened by Olson et al. (2001).
This pre-existing Landsat-8 burned area reference data set has several
benets for this study: (i) they were derived with high accuracy needed
for training purposes, (ii) the sampling methodology used to select the
data ensures a range of burned and unburned conditions, and (iii) the
Landsat-8 OLI has high radiometric and geometric quality (Roy et al.,
2014) and similar visible and NIR bands as the PlanetScope sensors
(Table 1) which is helpful for transfer learning from Landsat to
PlanetScope.
2.2. PlanetScope data
The rst two sensor generations, PlanetScope-0 (Dove-Classic) and
PlanetScope-1 (Dove-R), that acquire 12-bit blue, green, red, NIR radi-
ance, were used as only a minority of the third generation (SuperDove)
images were available when this research was initiated. Atmospherically
corrected 3 m orthorectied images (Level 3B), each typically covering
25 km ×11.5 km (PlanetScope-0) or 25 km ×23 km (PlanetScope-1),
were used. The data are atmospherically corrected with the 6SV radia-
tive transfer model, the same used to correct the Landsat-8 OLI data
(Vermote et al., 2016), with MODIS derived aerosol information (Planet
Team, 2021). The PlanetScope 3 m Usable Data Mask (UDM2), that la-
bels each pixel as clear, cloud, cloud shadow, haze, snow, missing, or
suspect due to saturation or downlink errors (Planet Team, 2021), was
used. When the UDM2 was not available, the Usable Data Mask (UDM)
that has fewer quality ags (cloudy or suspect ags) was used. The
shorter wavelength blue bands available on PlanetScope and Landsat-8
OLI were not used because atmospheric correction errors are signi-
cantly greater at blue wavelengths (Fraser and Kaufman, 1985; Ju et al.,
2012) which makes them less reliable for burned area mapping (Roy
et al., 2019).
A total of 659 PlanetScope 3 m image two-date pairs (dots, Fig. 1)
acquired November 2018 to December 2019, i.e., in different years to
the Landsat burned area reference data, were used. Only PlanetScope
two-date image pairs that met the following criteria were used: (i) fall
within or intersected the 78 unique Landsat path/row locations, (ii)
have ≥25 km
2
common overlapping spatial area and ≥75% high-quality
pixels (cloud- and shadow-free, no missing or suspect pixels) in both
images (as labelled by the UDM2 or UDM), (iii) have <3 m pixel
misregistration assessed by visual comparison of the two images in each
pair, (iv) be able to be relatively radiometrically normalized (Section
3.2.1), (v) be composed of images acquired one day apart and with solar
zenith angle difference <5◦. Criterion (i) was to leverage the transfer
learning with similar landscape characteristics in both Landsat and
PlanetScope datasets, (ii) - (iv) were to ensure that a meaningful amount
and good quality PlanetScope image pair data were classied, and (v)
was to reduce unwanted bidirectional reectance effects as the images
are acquired with different overpass times and so different solar zenith
angles (Huang and Roy, 2021).
3. Methodology
3.1. Overview
The methodology was implemented in the following manner: (i) a
relative radiometric normalization was applied to each PlanetScope 3 m
image pair to minimize non-surface change differences associated with
the variable image quality, (ii) two independent subsets of PlanetScope
image pairs were interpreted into 3 m burned area reference data, (iii)
two independent sets of 30 m pixel patches were extracted from the pre-
existing Landsat-8 burned area reference data, one set was used to train
the U-Net and the other to demonstrate the U-Net performance applied
to Landsat data and to derive the optimal U-Net training size, (iv) the
Landsat trained U-Net was rened with 3 m patches extracted from the
rst set of the PlanetScope burned area reference data to improve the
applicability of the U-Net for the PlanetScope classication, (v) the
rened U-Net was used to classify all 3 m PlanetScope image pairs, (vi)
the classication accuracy was evaluated using the second set of Plan-
etScope burned area reference data, including an accuracy assessment of
the scale discrepancy between the Landsat and PlanetScope imagery.
These steps, and the U-Net architecture, are described below in detail.
3.2. Pre-processing and generation of PlanetScope and Landsat reference
data
3.2.1. Planetscope image pair relative radiometric normalization
The selected PlanetScope image pairs were sensed one day apart with
<5◦solar zenith difference, and so differences between the images,
other than surface changes due to any burning, were due to sensor
bandwidth and spectral response differences (if the images were sensed
by different PlanetScope sensor generations), calibration differences,
and residual atmospheric correction errors (Huang and Roy, 2021). To
minimize non-surface change differences, a relative radiometric
normalization approach was applied independently to the red, green
and NIR surface reectance in each pair. The image in each pair with the
smallest aerosol optical depth (AOD) (reported in the image metadata)
was selected as the independent image and the other as the dependent
image. For each band a Theil-Sen regression was used to estimate a
linear model between the independent and the dependent image. The
Theil-Sen estimator is less sensitive to outliers (Theil, 1950; Sen, 1968)
and so the spectral normalization of no-change pixels becomes more
reliable as pixels with abrupt surface change are often treated as “out-
liers” (Schott et al., 1988; Olthof et al., 2005). The regression was
derived using corresponding pixel surface reectance values sampled
every 15 pixels east-west and north-south directions, discarding pixels
that were agged as cloud or cloud shadow in either image. The Pearson
correlation coefcient (r) between the band pixel samples was derived,
and if r <0.5 for any of the three bands, or if the number of pixels
considered was <10,000, then the image pair was discarded. Otherwise,
the Theil-Sen regression coefcients (slope and intercept terms) were
applied to all the surface reectance pixel values in the dependent image
to normalize the image.
3.2.2. Generation of PlanetScope 3 m burned area reference data and
extraction of 3 m burned area patch data that were used to rene the
Landsat trained U-Net
PlanetScope 3 m burned area reference data were derived for 40 of
the 659 radiometrically normalized image pairs. The 40 pairs were
selected across all ve Africa biomes and included 13 of the 17 Inter-
national Geosphere-Biosphere Programme (IGBP) land cover classes
dened in the Collection 6500 m MODIS land cover product for 2019
(Sulla-Menashe et al., 2019), with a diversity of burning conditions.
Each of the 40 pairs were interpreted visually to label 3 m pixels as
either burned (i.e., the pixel burned between the two image acquisition
dates), unburned (i.e., the pixel did not burn between the two acquisi-
tions), or unmapped (i.e., surface was unobserved due to clouds or
shadows in one or both images, or could not be interpreted unambigu-
ously). This was time consuming due to the complexity of the landscape
and the subtle patterns of burning that can be captured in PlanetScope
imagery (Roy et al., 2019). Therefore, to help speed up the interpreta-
tion, the second image acquisition was segmented into burned and
Table 1
The Landsat-8 30 m OLI and PlanetScope-0 and PlanetScope-1 3 m bands (and
bandwidths, units in nm) used in this study.
Green Red NIR
Landsat-8 OLI 530–590 (60) 640–670 (30) 850–880 (30)
PlanetScope-0 500–590 (90) 590–670 (80) 780–860 (80)
PlanetScope-1 547–585 (38) 650–682 (32) 846–888 (42)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
5
unburned classes using the Mean-Shift segmentation algorithm (Coma-
niciu and Meer, 2002) applied to the green, red, and NIR bands. The
image was over-segmented to provide many segments per burn, so that
they could then be rapidly checked, edited, and interactively merged on-
screen. This over-segmentation reects standard practice (Pavlidis and
Liow, 1990; Blaschke et al., 2014). Segments with new burn pixels
occurring between the two image dates were selected and labelled as
burned. A second interpreter was responsible for quality control through
visual inspection, and if needed, further renement of the burned area
edges. The resulting 40 pairs of interpreted PlanetScope imagery were
split into two halves - twenty pairs were used to validate the nal
PlanetScope burned/unburned classication (Section 3.4) and the other
twenty were used to rene the U-Net model (Section 3.3.3) using 5000
extracted 256 ×256 3 m patches.
The U-Net trained with 256 ×256 30 m Landsat training patches was
rened using 256 ×256 3 m patches extracted from the 20 PlanetScope
burned area reference data. Note that 256 ×256 pixel patch dimensions
are often used for U-Net remote sensing studies (Wagner et al., 2019;
Brandt et al., 2020). A total of 5000 256 ×256 3 m patches were used to
rene the Landsat trained U-Net. Specically, from each of the 20
PlanetScope burned area reference data sets, 250 256 ×256 3 m patches
composed of a predictor variable (a numeric code to denote burned or
unburned) and six explanatory variables (green, red, and NIR surface
reectance values from each of the two PlanetScope images sensed one
day apart) were randomly extracted. In this random selection process,
any patches containing 3 m pixels labelled as unmapped (i.e., surface
was unobserved due to clouds or shadows in one or both PlanetScope
images, or that could not be interpreted unambiguously) were rejected.
3.2.3. Generation of Landsat-8 30 m burned area patch data
A large number of 256 ×256 30 m pixel patches were extracted
independently from each of the 92 pre-existing Landsat-8 burned area
reference data sets (Section 2.1). The majority, termed Landsat training
patches, were used to train the U-Net (Section 3.3.1) and a minority,
termed Landsat evaluation patches, were used to derive the optimal
Landsat-8 burned area patch training set size (Section 3.3.2). Each patch
30 m pixel was composed of six explanatory variables and a predictor
variable. The six explanatory variables were the red, green and NIR
surface reectance values of the two Landsat-8 OLI images used to
derive the Landsat burned area reference data, and the predictor vari-
able was a binary code to denote if the pixel was burned or unburned as
dened by the Landsat burned area reference map. Pixels labelled as
unmapped in the Landsat burned area reference map (i.e., because they
were unobserved due to clouds or shadows in one or both Landsat im-
ages) were labelled unburned. This is not an issue because the nal U-
Net classications of the PlanetScope imagery are post-processed using
Fig. 2. Example 256 ×256 30 m Landsat training (654 red squares) and evaluation (73 gray squares) patches extracted from pre-existing Landsat-8 burned area
reference data derived from Landsat-8 OLI images acquired October 6 and October 22 2014 over Gauteng Province, South Africa. The orange pixels show the 30 m
pixels labelled in the Landsat-8 burned area reference data as burned (i.e., locations that burned between the two Landsat image acquisition dates), black shows
pixels that were unburned (i.e., locations that did not burn between the two image acquisition dates) or were unmapped (i.e., locations that were unobserved due to
clouds or shadows in one or both images, or that could not be interpreted unambiguously). (For interpretation of the references to colour in this gure legend, the
reader is referred to the web version of this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
6
the PlanetScope UDM2/UDM cloud and shadow mask (Section 3.3.4).
Fig. 2 shows a typical example of the training (red) and evaluation
(gray) patches that were extracted. The patches were selected randomly
from each Landsat-8 burned area reference data set but constrained to
ensure that (i) each patch contained one or more 30 m burned pixels, (ii)
the majority of the extracted patches were designated as training
patches and the remainder as evaluation patches, and (iii) the training
and evaluation patches did not overlap spatially to ensure that they were
independent. The total number of extracted patches varied among the
Landsat-8 burned area reference data sets (being dependent on the
number and spatial arrangement of the reference data 30 m burned
pixels). The patch selection process was as follows. First, the 30 m
burned pixels were spatially clustered into n
burned
burned areas, each
composed of spatially connected burned pixels found by 8-neighbor
adjacent pixel clustering. Then the geographic centroids of each of the
n
burned
burned areas were derived. Next, candidate extracted patches
were dened by 256 ×256 30 m pixel boxes centered on each
geographic centroid. All the boxes that overlapped spatially were
identied and given a unique group membership number {1, 2, …,
n
groups
≤n
burned
}
.
Then the training and evaluation patches were
selected randomly from different groups to ensure that the training and
evaluation patches did not overlap spatially. This random selection was
undertaken iteratively until 80–90% of all the boxes were selected as
training patches and the remainder were assigned as evaluation patches.
Sometimes, due to the spatial topology of the 30 m pixel burned areas,
the 80–90% allocation was not possible. In this case, the largest group of
boxes was divided into two or more independent groups by removing up
to 50 boxes that overlapped. This allowed the generation of more non-
overlapping groups. The above selection procedure was able to pro-
vide a large number of Landsat training and evaluation patches from
each Landsat-8 burned area reference data set. For example, in Fig. 2 a
total of 654 training (red) and 73 evaluation (gray) patches were
extracted.
3.3. U-Net deep learning
3.3.1. U-Net architecture and training with Landsat-8 burned area patch
data
The U-Net uses the generative nature of a convolutional autoencoder
to learn delicate boundaries with a multiscale architecture to provide
different levels of data abstraction that are used to classify each patch
pixel (Ronneberger et al., 2015). The architecture consists of three
sections. i.e., an encoder, a bottleneck, and a decoder. The encoder (or
contracting path, Fig, 3 left side) explores higher semantic information
by reducing in steps the spatial resolution of the input patch data, while
the decoder (or expanding path, Fig. 3 right side) recovers the spatial
information.
The U-Net is composed of a sequence of building blocks (Fig. 3
dashed boxes) containing two convolutional layers. Each convolutional
layer consists of kernels, dened by i ×i ×r real valued matrices, termed
kernel weights, and an associated single real valued bias, where i ×i are
the kernel spatial dimensions and r is the number of spectral bands in the
input image patch for the rst convolutional layer and for subsequent
layers is equal to the number of feature maps in the previous layer. When
the U-net is trained, the kernel weights and the bias values, collectively
termed the network coefcients, are dened. In the U-Net classication
process a series of square two-dimensional real valued matrices, termed
feature maps, are generated (Fig. 3 gray lled rectangles). The core
operation in the convolutional layer is a spatial convolution of the kernel
matrix weights with the input patch data (in the rst layer) or previous
Fig. 3. The U-Net architecture used to classify 256 ×256 pixel patches (two dates of green, red, and NIR surface reectance) into burned or unburned classes. Each
building block (dashed black boxes) contains two convolutional layers that are used to generate feature maps (gray lled rectangles). The number of feature maps is
specied above each gray lled rectangle and the feature map spatial dimensions are dened on the left side. The horizontal gray arrows show skip connections used
to copy feature maps from the encoder (light gray rectangles) to their decoder block counterpart. The orange and red vertical arrows show the maximum pooling and
transposed convolution operations, respectively. (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of
this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
7
feature map (in subsequent layers), followed by addition of the bias term
to generate the feature map. Typically, every feature map value is non-
linearly transformed and in this study the rectied linear unit (ReLU)
function was used so that after the transformation any negative values
were set to zero and positive values remained unchanged (Glorot et al.,
2011).
The U-Net encoder section (Fig. 3 left side) consists of several
contraction blocks. Each block takes an input that applies two convo-
lution layers followed by a maximum pooling step. The maximum
pooling calculates the maximum value of each 2 ×2 adjacent pixel re-
gion of the feature map to create a downsampled feature map. The
number of kernels (and so feature maps) after each contraction block
doubles so that the architecture can learn complex structures effectively.
The U-Net bottleneck is the bottom-most layer and mediates between
the encoder and the decoder. It uses two convolution layers without any
pooling operation. The U-Net decoder section (Fig. 3 right side) consists
of as many expansion blocks as there are encoder contraction blocks.
Each expansion block takes an input that applies two convolution layers
preceded by a transpose convolution step. The transpose convolution
creates an upsampled feature map by inserting a column and a row of
zeros between each row and column of the feature map to increase the
feature map size and then applying a 2 ×2 convolution kernel with
learnable weights. Skip connections (Fig. 3 horizontal gray arrows) are
used to copy encoder feature maps (Fig. 3 light gray rectangles) to their
decoder block counterparts to ensure that the features learned while
contracting the image are used to reconstruct the image. In the last
decoder block (top right, Fig. 3) there are an additional n convolutional
layers (and so feature maps), where n is the number of desired classes,
that are ltered to provide the nal classication result.
Fig. 3 shows the U-Net structure used to classify each 256 ×256 pixel
patch composed of two dates of green, red, and NIR surface reectance
into burned and unburned classes. Similar to most U-Net land cover
classication studies, 3 ×3 kernels were used. Following common
practice, 32 kernels for each of the two convolutional layers in the rst
encoder block (Wagner et al., 2019; Brandt et al., 2020) were imple-
mented but with an additional block in the encoder and decoder (ve
instead of four) to explore deeper abstraction features. This provided
blocks with 32, 64, 128, 256, 512, and 1024 kernels per convolutional
layer that generated square feature maps with side dimensions of 256,
128, 64, 32, 16 and 8 pixels respectively. The last decoder block has an
additional two layers that generate 256 ×256 burned and unburned
feature maps.
The extracted Landsat training patches (Section 3.2.3) were used to
train the U-Net to derive the network coefcients. A conventional
gradient descent method was used to iteratively update the coefcients
by minimizing a loss function. The network coefcients were initialized
randomly, then with each iteration they were updated by adding the
coefcient gradient values of the loss function perturbed by a small
amount. The training was undertaken in two steps: “mini-batches” of
training data were passed in the forward propagation through the
network, and then, the estimated error between the predicted and
training data class labels was used to update the coefcients during the
backpropagation (Rumelhart et al., 1986; LeCun et al., 1990). This
process can be described:
L(θ) = − 1
m∑
m
i=1
(ci×logci+ (1−ci) × log(1−ci) ) (1)
where L is the binary loss function, θ are the network coefcients, m is
the number of training patches in the mini-batch, c is a vector of the
patch pixel training data class labels, and
c is a corresponding vector of
the U-Net predicted class condences (real numbers from 0 to 1). The
backpropagation algorithm computes the partial derivatives, i.e.,
∂
L(θ)/
∂
θ, of the loss function (LeCun et al., 2015), and the network
coefcients are updated by gradient descent after each interaction
(Bengio, 2012) as:
θ(t+1)=θ(t)−
η∂
L(θ)
∂
θ(2)
where θ are the network coefcients, t is an iteration step equivalent to a
single forward mini-batch of training data introduced to the network,
and
η
is the learning rate dened by a small (<1) positive real number.
An epoch of iterations is completed when all the training patches are
used, and several epochs are needed to update the network coefcients
until a satisfactory classication performance is obtained.
The U-Net was implemented with the TensorFlow 2.2.0-Keras
framework (Abadi et al., 2016) and congured with a conventional bi-
nary loss function and using Adam optimization of the gradient descent
with default settings (learning rate
η
=0.001, β
1
=0.9, β
2
=0.999,
ε
=
10
−8
) (Kingma and Ba, 2015). Conventional batch normalization was
applied to normalize the feature maps, using the mean and standard
deviation of the feature maps derived from each mini-batch of training
samples, to reduce overtting and vanishing gradient issues (Ioffe and
Szegedy, 2015). The training was undertaken over 75 epochs with a 128
mini-batch size. In practice, the number of epochs is evaluated during
the training progress, too few epochs may result in poor classication
accuracy and too many may lead to overtting (Liu et al., 2008). In this
study, using the optimal Landsat-8 burned area patch training data set
(described in the next section), 75 epochs were used because the training
overall accuracy converged and did not improve >0.05% over the last
10 epochs.
3.3.2. Landsat U-Net burned/unburned classication and selection of
optimal Landsat-8 burned area patch training set size
The U-Net classication denes for each patch pixel a burn con-
dence value:
c(i,j) = 1/(1+e−alast(i,j))(3)
where
c(i, j) is the burn condence (a real number from 0 to 1), (i, j) is
the patch pixel location, and a
last
(i, j) is the burn class feature map
generated in the last decoder block (Fig. 3 top right). The burn con-
dence map was converted into a binary burned/unburned class value by
application of a condence threshold as:
b(i,j) = {1,If c(i,j) ≥ c0
0,otherwise (4)
where b(i, j) is the binary classication (1 =burned, 0 =unburned),
c(i,
j) is the burn condence dened as Eq. (3), and c
0
is the condence
threshold nominally set as 0.5.
Deep learning performance typically increases with more training
data but at increasing computational cost (Garcia-Garcia et al., 2018;
Zhang et al., 2021b). In general, if increasing amounts of training data
provides reduced classication accuracy then the model is overtting
and if the classication accuracy continues to increase as training data
are added then the model is still undertting. Therefore, in this study,
experiments were undertaken to assess the impact of using an increasing
proportion (0.0125, 0.0625, 0.125, 0.1875, 0.25, 0.5, 0.75, 1.0) of the
Landsat training patches to obtain an optimal set that provided a stable
and accurate classication of the Landsat evaluation patch data. Spe-
cically, the U-Net was trained independently with each training pro-
portion selected randomly without replacement from all the Landsat
training patches, and each trained U-Net was then used to classify all the
Landsat evaluation patches. The six explanatory variables (three bands
of Landsat surface reectance, for two dates) in each Landsat evaluation
patch were U-Net classied and the results compared with the evalua-
tion patch predictor values (i.e., burned and unburned categories) to
populate a burned/unburned confusion matrix. Note that the Landsat
training and evaluation patches do not overlap (Fig. 2) and so the
training patches used to train each U-Net never include the evaluation
patches. Conventional overall classication accuracy and burn class
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
8
omission and commission error statistics were derived from the confu-
sion matrices (Foody, 2002; Boschetti et al., 2019). The optimal set of
training patches was selected as the one that provided high overall
classication accuracy and a good balance between burn commission
and omission errors. In these experiments c
0
was set as 0.5, i.e., the
default threshold to convert the condence to a binary class value.
3.3.3. U-Net renement with PlanetScope burned area reference patch data
Initial classication results derived by applying the Landsat trained
U-Net to the radiometrically normalized PlanetScope image pairs pro-
vided encouraging results (illustrated in Section 4). However, recent
transfer learning research has demonstrated that augmenting, often
termed ne-tuning, the training with a small set of training data derived
from the imagery that will be classied (referred to as the target imag-
ery) can improve the classication performance. There are a number of
approaches including (i) replacement of the last convolutional layer by a
new one with randomly initialized weights that are trained using target
imagery training data (Agrawal et al., 2014; Campos et al., 2017), (ii)
freezing some convolution layers and re-training the model on the
remaining layers using target imagery training data (Nogueira et al.,
2017; Lima and Marfurt, 2020), or, (iii) undertaking additional training
of the trained network layers using target image training data and a
small learning rate (Tremblay et al., 2018; Cetinic et al., 2018; Wurm
et al., 2019). In this study, approach (iii) was adopted because it enables
tuning of the network coefcients with a relatively small target training
data set and without overtting (Maggiori et al., 2016; Nogueira et al.,
2017). The U-Net trained with the optimal proportion of Landsat
training patches was further trained with the 5000 PlanetScope 256 ×
256 3 m patches extracted from the 20 PlanetScope burned area refer-
ence data sets developed for this purpose (Section 3.2.2). The U-Net was
further trained with 25 epochs and a 1e
−5
learning rate. The resulting U-
Net is hereafter referred to as the rened U-Net.
3.3.4. PlanetScope burned/unburned classication using the rened U-Net
The rened U-Net was applied to classify each of the 659 radio-
metrically normalized PlanetScope 3 m image pairs. Each pair had a ≥
25 km
2
common overlapping spatial area and so the rened U-Net was
applied to 256 ×256 3 m pixel subsets translated systematically every
Fig. 4. Frequency distributions of the burned area proportion (expressed as a percentage of the 256 ×256 30 m pixel patch) of the 400,000 Landsat training and the
65,000 Landsat evaluation patches. Histogram bin widths equal 1%.
Fig. 5. Sensitivity analysis of the U-Net Landsat classication accuracy with respect to the number of Landsat training patches used to classify the 65,000 Landsat
evaluation patches.
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
9
192 pixels across the common area so that the subsets overlapped by
25% (i.e., by 64 3 m pixels) in the x and y image axes. An overlap of this
magnitude is often used because the U-Net classication performance
may be reduced nearer the sub-set borders (Ronneberger et al., 2015;
Derksen et al., 2019). The rened U-Net was applied to each subset pixel
to derive a burn condence value (Eq. (3)) that was thresholded (Eq. (4))
into a burned or unburned class. Classied pixels falling within 16 pixels
of each subset border were discarded (Freudenberg et al., 2019) and
overlapping areas among the different classied subsets were combined
by voting (Li et al., 2019) to keep burn classied pixel values. Pixels
labelled as cloud, cloud shadow, missing or suspect in the UDM2/UDM
of either image in the pair were reclassied as “unmapped”. Thus, the
PlanetScope classication had three classes: burned, unburned, and
unmapped.
The nal classication results were generated using an optimal
condence threshold (c
o
). The threshold was determined by thresh-
olding the burn condence values generated by application of the
rened U-Net to the 20 PlanetScope burned area reference data sets used
to rene the U-Net (Section 3.3.3) with different c
o
values (from 0.1 to
0.9, in steps of 0.05). The optimal c
o
value was selected as the one that
provided high overall classication accuracy and a balance between the
commission and omission errors. Pixels that were reclassied as “un-
mapped” based on the UDM2/UDM or labelled as “unmapped” in the
interpreted 20 burned area maps were ignored. The classied 659
PlanetScope image pairs were examined visually for quality assessment
purposes. This was undertaken because burned area classication errors,
such as stripes at input-image granule boundaries and anomalous
burning patterns (for examples, see Humber et al. (2019)), may remain
undetected by the burned area accuracy assessment that was undertaken
using a limited number of PlanetScope validation reference data sets
(Section 3.4). Summary statistics of the classied burned area pro-
portions were also derived for each of the 659 image pairs to check that
unfeasible classied burned area proportions did not occur.
3.4. PlanetScope burned area accuracy assessment
The classied PlanetScope image pairs were compared with the 20
validation PlanetScope burned area reference data sets (Section 3.2.2).
Recall that the 20 sets were selected across Africa and were different
from those used to rene the U-Net. As before, the overall classication
accuracy and the burn omission and commission errors were derived
from a burned/unburned confusion matrix, ignoring unmapped 3 m
Fig. 6. Example Landsat-8 NIR-Red-Green surface reectance 30 m images acquired 16 days apart on (a) October 6 2014, and (b) October 22 2014, (c) the cor-
responding Landsat evaluation patch data, (d) the U-Net classication of the two Landsat-8 images. Results for three 128 ×128 30 m pixel subsets selected within
three Landsat evaluation patches located in the north, west, and south of Fig. 2. The U-Net was trained with 300,000 Landsat training patches. (For interpretation of
the references to colour in this gure legend, the reader is referred to the web version of this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
10
pixels.
The factor of ten scale discrepancy between the Landsat and Plan-
etScope imagery is potentially an issue for the transfer learning, as the
spatial arrangement of burns in 30 m and 3 m pixel 256 ×256 patches
may be different. Our expectation is that the substantial Landsat training
data set used, and the U-Net renement with PlanetScope imagery, may
mitigate this issue. However, to examine this, the confusion matrix
based accuracy measures were derived considering 256 ×256 3 m
patches grouped by the proportion that was interpreted as burned. The
patches were extracted from the 20 validation PlanetScope burned area
reference data sets and categorized into eight proportion groups with the
same number of patches per group. The same number of patches per
group were extracted so that the accuracy results could be meaningfully
compared among groups. For each group, the patches were extracted at
random but ensuring that the patches did not overlap spatially, and that
each contained at least one reference mapped burns so that both omis-
sion and commission errors could be assessed. The number of patches
that could be extracted that met these criteria was limited, and a total of
36 patches per group were selected.
In addition, the burned proportions dened in 30 ×30 m grid cells in
the classied and the validation 3 m burned area maps were compared
by linear regression, considering only grid cells with <50% unmapped
data. The linear regression coefcient of determination (r
2
) and the
regression slope and intercept terms summarize the precision and the
accuracy of the classied burned area proportions respectively (Roy and
Boschetti, 2009; Boschetti et al., 2019). If the 3 m errors of omission and
commission compensate each other in the 30 m grid cells, then the r
2
and slope term will be close to unity, and the intercept will be close to
zero, indicating high precision and accuracy of the classied burned
area proportions.
Fig. 7. (a) PlanetScope surface reectance spectral scatterplots comparing the dependent image sensed June 22 2019 (Fig. 8a) and the independent image sensed
June 23 2019 (Fig. 8b); surface reectance spectral histograms before (b) and after (c) the relative radiometric normalization.
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
11
4. Results
4.1. Landsat-8 burned area training and evaluation data
A total of 400,000 and 65,000 Landsat training and evaluation
patches, respectively, were extracted from the 92 pre-existing Landsat-8
burned area reference data sets. Fig. 4 shows histograms of the patch
burned area proportions. The training and evaluation histograms are
similar, which indicates that the extraction methodology provided
representative but independent sets of training and evaluation patches.
The histograms are positively skewed (a greater frequency of small
values); 90% of the patches have burned area proportions <20% and <
19% and the most frequent proportions are 2% and 1%, in the training
and evaluation datasets, respectively. <1% of the patches have burned
area proportion >50% in either dataset.
4.2. Selection of optimal Landsat-8 burned area patch training set size
and example U-Net Landsat burned/unburned classication results
Fig. 5 presents the results of the experiment designed to select an
optimal amount of Landsat training data for the U-Net classication. An
increasing proportion of the Landsat training patches was used to train
the U-Net and classify the 65,000 Landsat evaluation patches. Eight
proportions were considered, using from 5000 to all 400,000 of the
training patches. The overall classication accuracies were all >97.11%.
The commission errors were relatively stable and varied by 3.3% among
the eight sets of classications but the omission errors decreased
monotonically with the number of training patches from 22.47% (5000
training patches) to 13.15% (300,000 training patches) and then
increased slightly to 13.86% (400,000 training patches). The small in-
creases in omission error (0.64%) and commission error (0.54%) from
300,000 to 400,000 training patches indicates adequate training data
representation provided by the 300,000 training patches. Further, the
most balanced omission and commission errors were provided with
Fig. 8. Example PlanetScope 3 m image pair (NIR-Red-Green surface reectance) sensed on (a) June 22nd 2019 (48.37
◦solar zenith, 0.22 AOD) and (b) June 23rd
2019 (48.4◦solar zenith, 0.12 AOD), near Chitembo, Angola (13.37◦S, 16.64◦E). U-Net classication of the image pair (c) before and (d) after relative radiometric
normalization. The U-Net was trained with 300,000 Landsat training patches. (For interpretation of the references to colour in this gure legend, the reader is
referred to the web version of this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
12
300,000 training patches. Consequently, the U-Net model trained with
300,000 training patches was selected for the rest of this study.
Fig. 6 shows examples of U-Net Landsat classication results for 128
×128 30 m pixel subsets of three Landsat evaluation patches. Recall that
the Landsat evaluation and training patches do not overlap (Fig. 2) and
so the 300,000 training patches extracted across Africa to train the U-
Net did not use the evaluation data. The relatively high degree of cor-
respondence between the burned areas dened in the Landsat evalua-
tion patches (Fig. 6c) and the U-Net classications (Fig. 6d) is
encouraging. Notably, the U-Net is able to handle variable pre- to post-
re reectance changes, including quite subtle changes (Fig. 6 bottom
row), and delineate complex burn boundaries. The Figs. 5 and 6 results
indicate that the U-Net can be used to classify burned areas present in
Landsat imagery and demonstrates the U-Net applicability on the source
data (Landsat) before the PlanetScope transfer learning.
4.3. Illustration of the need for PlanetScope relative radiometric
normalization
Fig. 7 illustrates the PlanetScope relative radiometric normalization
process for an image pair, illustrated in the top row of Fig. 8, with a 75.5
km
2
common area acquired over central Angola. The rst image
(Fig. 8a) contains older burns in the center-south, with new burns along
the S.E. edge and in the center of the second image that was sensed a day
later (Fig. 8b). The Theil–Sen regression was implemented with the
reectance of the rst image as the dependent variable and the reec-
tance of the second image as the independent variable because the rst
image had higher AOD (0.22) than the second (0.12). Fig. 7(a) shows
scatterplots comparing the red, green and NIR surface reectance for the
two images and the Theil–Sen regressions (solid black lines). The new
burns are exhibited by lower reectance in the second (independent)
image, particularly in the NIR, but, as the new burns occupied a minority
of the overlapping image area they did not affect the Theil–Sen regres-
sion. The regression slopes and offsets are not unity and zero, respec-
tively. This is likely because of residual atmospheric correction errors
and sensor calibration differences between the two images, and not due
to band-pass and bi-directional reectance effects (Huang and Roy,
2021) as both were sensed by the same PlanetScope-1 sensor generation
and with similar (~48◦) solar zenith angles. Fig. 7(b) and 7(c) show
surface reectance histograms before and after the relative radiometric
normalization, respectively, illustrating how the normalization makes
the reectance for the dependent image more similar to the independent
image.
The relative radiometric normalization improved the PlanetScope
classication results or had no impact. For example, Fig. 8 shows the
application of the U-Net trained with 300,000 Landsat patches, to the
two PlanetScope images before (Fig. 8c) and after (Fig. 8d) normaliza-
tion. In this example there is an evident reduction in the large number of
commission burn errors (Fig. 8c) with only the new burns classied in
the normalized results (Fig. 8d). The narrow intermittent lines of com-
mission errors apparent in the unnormalized classication results
(Fig. 8c) are likely due to the PlanetScope detector array boundaries
(diagonal lines parallel to the image swath edges) and sensitivity to the
burned area majority voting process in 256 ×256 3 m patch overlap
areas (horizontal and vertical lines). These results also illustrate the
utility of the transfer learning i.e., using a Landsat trained U-Net to
classify the PlanetScope imagery, that is developed in the following
sections.
4.4. Selection of the rened U-Net optimal burn condence threshold and
illustrative PlanetScope burned/unburned classication results
The rened U-Net was derived by further training the U-Net with
5000 3 m patches extracted from 20 PlanetScope burned area reference
data sets (Section 3.3.3). Fig. 9 presents the results of the experiment to
select the rened U-Net optimal burn condence threshold value. This
was undertaken by application of the rened U-Net to classify 20 pairs of
PlanetScope images and burned area reference data considering a range
of c
o
values (Section 3.3.4). Using a higher c
o
value will reduce the
likelihood of a pixel being classied as burned. Thus, as expected, the
omission and commission errors monotonically increase and decrease,
respectively, with c
o
. However, this variation is not symmetrical around
c
o
=0.5. The overall classication accuracies were >99.4% for all
threshold values and the most balanced omission and commission er-
rors, 5.76% and 5.14% respectively, were obtained for c
o
=0.75.
Consequently, 0.75 was selected as the optimal threshold value. These
results were derived using the 20 PlanetScope burned area reference
data sets used to rene the U-Net, and so they do not constitute an in-
dependent validation, which is presented in the following section.
Fig. 10 illustrates the improved classication results provided by the
rened U-Net (Fig. 10d) compared to that provided by the U-Net
(Fig. 10c). The top row of Fig. 10 shows a 5.6 ×6.1 km subset of a
Fig. 9. Sensitivity analysis of the Rened U-Net classication accuracy with respect to the condence threshold (c
o
) used to classify the 20 PlanetScope burned area
reference data sets used to rene the U-Net. The 20 PlanetScope image pairs were radiometrically normalized.
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
13
PlanetScope image pair sensed a day apart falling in South Africa near
the Botswanan border. This pair was selected because it captures an
extensive biomass burning event that covers a large number of 3 m
pixels (extensive black areas evident across the center and east of the
second image, Fig. 10b) and because it includes burns with linear and
curved edges (delimited by roads and eld boundaries), burns with
irregular edges, and faint burns that are indicative of low combustion
completeness that are challenging to detect (Roy et al., 2019). Notably,
comparing Fig. 10c and d, the rened U-Net was able to more precisely
delineate the different burned boundaries including the boundaries of
interior unburned areas. In addition, the rened U-Net was able to better
classify the extensive faint burned areas occurring near the northern
image edge (just west of 25.94◦E) and in the central south. These
qualitative examples demonstrate the relevance of the ne-tuning pro-
cedure to provide improved transferability and burn classication
accuracy.
4.5. Africa PlanetScope burned-unburned classication and validation
results
The rened U-Net was used to classify the 659 pairs of radiometri-
cally normalized PlanetScope images. The classied pairs contained a
diversity of burns with individual burns ranging from single 3 m pixels
to extensive burns that covered up to 16.6% of the common classied
Fig. 10. Example radiometrically normalized PlanetScope 3 m image pair (NIR-Red-Green surface reectance) sensed on (a) September 19th 2019 (40.8
◦solar
zenith, 0.18 AOD) and (b) September 20th 2019 (37.9◦solar zenith, 0.12 AOD), over a common area of 163 km
2
, near Mahikeng, South Africa (25.84◦S, 25.92◦E),
and the classication results derived (c) using the U-Net trained with Landsat training patches, (d) using the rened U-Net. (For interpretation of the references to
colour in this gure legend, the reader is referred to the web version of this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
14
image area, with a median of 0.09%. Figs. 11 and 12 show illustrative
classication results in different countries for eight of the 20 Planet-
Scope image pairs that were used to derive 3 m burned area reference
maps for the validation (Section 3.2.2). Fig. 11 shows examples with a
high proportion of correctly classied pixels (green) relative to the
burned area reference maps, and Fig. 12 shows examples with larger
proportions of omission (red) and commission (yellow) errors. Small
spatial subsets (from 0.18 km
2
to 7.85 km
2
) with variable dimensions
are illustrated to capture different burn patterns in detail.
In Fig. 11, the Zambian, Ethiopian and South African examples
illustrate the capability of the rened U-Net to preserve inner unburned
islands, and to correctly classify only the new burns evident in the sec-
ond image. The Madagascar example is over an extensive low combus-
tion completeness burn, that are often hard to detect, but was classied
by the rened U-Net. These examples are quite representative with
relatively small proportions of omission and commission errors that
occur typically along the burn boundaries.
Fig. 12 presents detailed examples of omission and commission er-
rors. It is well established that surface changes that are spectrally similar
to burned areas (e.g. ooding, rapid vegetation senescence, shadows,
snow melt) may be misclassied as burned (Roy et al., 2005a). The
Morocco example has commission errors due to agricultural harvesting
that is difcult to differentiate reliably from burning (Hall et al., 2016;
Giglio et al., 2018). The Chad example shows an area of riparian vege-
tation with high NIR and low visible surface reectance (red tones)
bisected by two narrow river channels that drain into a lake (cyan
tones). The commission errors are atypical of the 659 PlanetScope image
pair classication results, as water bodies were normally correctly
classied as unburned. However, in this case the water was sediment
laden, indicated by monotonically increasing PlanetScope reectance
with wavelength and by high (~0.2 to >0.3) NIR reectance, and the
reectance dropped in the second image in every band over the water.
The commission errors occur in the narrow river channels and nearby
pools, and not the lake, perhaps because of their small dimensions that
were captured by the U-Net training. The Angola and Cote d'Ivoire ex-
amples show typical omission errors that occur because the change in
reectance between the two images was too small to be detected.
The validation results for the 20 pairs and their locations are shown
in Fig. 13. The results were derived by per-pixel comparison of the
rened U-Net classication and the 3 m burned area reference map
(Section 3.3.4). The burn commission errors range from 1.1% to 76.1%
(median 13.1%) and the omission errors range from 3.3% to 42.0%
(median 13.8%). The worst results were at the Morocco site which had
the smallest number of interpreted burned pixels among the 20 sites,
with a high proportion of omission errors around the edges of spatially
fragmented burns, with commission errors due to agricultural harvest-
ing (example shown in Fig. 12a). Recall that image pairs with ≥25 km
2
common area and ≥75% high-quality pixels (cloud- and shadow-free,
no missing or suspect pixels) were used. The image pairs in Morocco,
Cote d'Ivoire and Uganda had the smallest interpreted burned areas,
2129, 5082, and 4585 3 m pixels, respectively, corresponding to <0.5%
of their common areas. The sites in South Africa and Nigeria had the
Fig. 11. Examples illustrating rened U-Net burned area classication results. The radiometrically normalized PlanetScope 3 m images (NIR-Red-Green surface
reectance) were sensed one day apart in 2019. The bottom row illustrates the 3 m classication and burned area reference maps derived for the validation together,
showing correctly classied pixels (green), pixels that should have been classied as burned but were not (red), and pixels that were incorrectly classied as burned
(yellow). (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
15
largest interpreted burned areas, 229,930 and 221,007 3 m pixels,
respectively, corresponding to <1.5% of their common image areas. The
overall classication accuracy for each image pair (not reported in
Fig. 13) was always high (≥99.65%) which is expected as only a mi-
nority of each reference map was burned.
Table 2 presents the confusion matrix generated considering all 20
site results together, composed of 339.4 million 3 m pixels. A total of
1522,97 pixels were classied as burned and 1,519,464 pixels were
interpreted as burned in the reference maps. The overall classication
accuracy was 99.89% which as noted above reects the predominance of
the unburned pixels and so is not a particularly useful accuracy measure
(Stehman, 1997; Boschetti et al., 2004). The burn commission and
omission errors were 12.29% and 12.09%, respectively, indicting a
satisfactory per-pixel classication with balanced omission and com-
mission errors.
Fig. 14 shows per-pixel omission and commission derived consid-
ering 256 ×256 3 m patches extracted randomly from the 20 validation
pairs and grouped together by the patch proportion interpreted as
burned. Eight proportion groups with 36 patches per group were
extracted. The omission and commission errors decrease with the patch
proportion interpreted as burned. The omission errors were 46% when
the patches were >0 to 0.1% burned and fell to about 10% when the
patches were >6.5% burned. The commission errors were 54% when
the patches were >0 to 0.1% burned and fell to about 6% when the
patches were >15.8% burned. These results indicate that the rened U-
Net classication of small burns is less reliable than larger ones. How-
ever, this nding is not denitive because patches with higher burned
proportions can contain several disconnected small burns. For example,
although the patch area is comparable to those illustrated in Figs. 12 and
13, patches may contain several different burns, for example, evident in
Fig. 11a.
Fig. 15 shows a density plot of the proportion of 30 m grid cells (0–1)
classied as burned against the proportion labelled as burned in the 3 m
reference maps. The common areas of the 20 validation data sets were
considered, >3.39 million 30 m grid cells. The rainbow logarithmic
colour scale represents the frequency of cells with the same x-axis and y-
axis proportion values. The values are predominantly clustered around
the 1:1 line, reecting the high degree of correspondence between the
classied and reference data, and indicating that the errors of commis-
sion and omission compensate at 30 m resolution. Specically, the low
dispersion of the data, quantied by the coefcient of determination (r
2
=0.91) is indicative of the precision of the classication, whereas the
slope of the regression line close to unity (0.93) and intercept close to
zero (<0.001) reect its accuracy (Roy and Boschetti, 2009; Boschetti
et al., 2016). It should be noted that the relatively high density of cells
along the edges of the plot, which indicates the presence of 30 m cells
with large omission or commission errors, is primarily due to the
omission and commission of small patches discussed above with respect
to Fig. 14.
Fig. 12. Examples illustrating rened U-Net burned area classication results with errors. The radiometrically normalized PlanetScope 3 m images (NIR-Red-Green
surface reectance) were sensed one day apart in 2019. The bottom row illustrates the 3 m classication and burned area reference maps derived for the validation
together, showing correctly classied pixels (green), pixels that should have been classied as burned but were not (red), and pixels that were incorrectly classied as
burned (yellow). (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
16
5. Discussion and conclusion
The near daily global coverage provided by the PlanetScope
constellation opens new opportunities for high spatial resolution ana-
lyses (Roy et al., 2021), including burned area mapping. However, the
sensors have no onboard calibration or SWIR bands, and are sensed with
variable solar geometry, thus adding challenges to their use for large
area, automated, burned area mapping. To help overcome these issues, a
U-Net deep learning algorithm was used to classify burned areas from
two-date Planetscope image pairs acquired at the same location. The U-
Net architecture is trained with and applied to image patches and so
incorporates spatial as well as spectral information. Given the signicant
effort required to develop a comprehensive burned area Planetscope
training data set, a transfer learning approach using pre-existing Land-
sat-8 derived burned area reference data (Boschetti et al., 2019) was
instead developed.
Initial experiments using solely Landsat data conrmed that deep
learning can be used to classify burned areas (Figs. 5 and 6). A con-
ventional U-Net, trained with 300,000 Landsat-8 256 ×256 30 m
patches, provided ~13% Landsat 30 m burn omission and commission
errors that were quantied with 65,000 independent 256 ×256 30 m
evaluation patches. As both the evaluation and classication data were
30 m resolution these results do not constitute a formal validation. The
standard protocol for burned area validation emphasizes the importance
of interpreted maps with ner spatial resolution than the classied
product (Boschetti et al., 2009; Roy et al., 2019). Further work to
investigate deep learning for Landsat burned area mapping is recom-
mended and moderate resolution burned area mapping research, cited
Fig. 13. Locations of the 20 3 m burned area reference maps used to validate the rened U-Net PlanetScope classication, and the resulting commission (blue) and
omission (red) percentage errors. (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)
Table 2
Confusion matrix summarizing the rened U-Net classication accuracy derived
from the 20 validation pairs (Fig. 13), matrix elements are shown as 3 m pixel
counts and as percentages (in parentheses).
Reference
Classied Burned Unburned Total
Burned 1,335,779
(0.39%)
187,198
(0.06%)
1522,977
(0.45%)
Unburned 183,685
(0.05%)
337,686,901
(99.50%)
337,870,586
(99.55%)
Total 1,519,464
(0.44%)
337,874,099
(99.56%)
339,393,563
(100.00%)
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
17
in the introduction, is ongoing.
A total of 659 PlanetScope 3 m image pairs sensed one day apart,
with ≥25 km
2
common area, with ≥75% high-quality pixels, and sensed
with solar zenith difference <5◦, were selected across Africa. It is well
established that reliable burned area mapping requires stable reec-
tance time series, particularly when different sensor data are utilized
(Roy et al., 2019). Atmospherically corrected and orthorectied (Level
3B) PlanetScope images were used but they do not have the same degree
of temporal reectance stability compared to more expensive earth
observation systems such as Landsat (Houborg and McCabe, 2018).
Consequently, each PlanetScope image pair was independently radio-
metrically normalized. A straightforward relative radiometric
normalization was applied based on a regression-based adjustment of
the bands in the image with the higher aerosol optical depth (AOD) to
the bands in the other image. This assumes that the lower AOD image
has less aerosol contamination and so is more reliably atmospherically
corrected (Ju et al., 2012; Doxani et al., 2018). Other normalization
methods could be used. However, we found that the normalization
unambiguously improved the transfer learning classication results
(Fig. 8). Regardless of whether the data are radiometrically normalized
or not, surface changes that are spectrally and spatially similar to burned
areas may be misclassied as burned (Fig. 12). Future work to reduce
burn commission errors is recommended, for example, based on time
series deep learning, which is evolving rapidly (Masolele et al., 2021; Xu
et al., 2018), with temporal constraints such as those used by the NASA
MODIS and combined Landsat Sentinel-2 burned area product genera-
tion algorithms (Roy et al., 2019).
The implementation of a transfer learning approach presented
several challenges. The Landsat-8 and PlanetScope imagery have
different spatial (30 m vs. 3 m) and temporal (16 vs. 1 day) resolutions,
different radiometric quality and spectral response functions (Houborg
and McCabe, 2018; Huang and Roy, 2021), and the Landsat-8 and
PlanetScope data were acquired in different years (2014–2015 vs.
2019). Despite these issues, the comprehensive training with 300,000
Landsat patches extracted across Africa likely captured a wide range of
burned and unburned conditions, and the common features of burned
areas, such as the drop in reectance post-re, likely helped a sensor
agnostic U-Net burned/unburned representation. Previous transfer
learning studies have highlighted the benets of ne-tuning the trained
network (Volpi and Tuia, 2016; Nogueira et al., 2017; Li et al., 2018).
Consequently, in this study, further training of the Landsat derived U-
Net model was undertaken using 20 PlanetScope burned area reference
data sets. This involved additional training of the Landsat trained
network layers with a small learning rate (Tremblay et al., 2018; Cetinic
et al., 2018; Wurm et al., 2019). The resulting rened U-Net more pre-
cisely delineated burned boundaries and better detected faint burned
areas (Fig. 10).
It is well established that statistically rigorous burned area product
validation requires comparison with multi-date higher spatial resolution
data that have been interpreted independently into burned area refer-
ence maps, and that have been collected with a suitable sampling
strategy allowing for the estimation of accuracy metrics and relative
standard errors (Padilla et al., 2015; Boschetti et al., 2016, 2019).
However, there are no higher spatial resolution (i.e., <3 m) image pairs
Fig. 14. Sensitivity analysis of the Rened U-Net classication accuracy with respect to burned area proportion. Results shown for eight proportion groups composed
of 36,256 ×256 3 m patches per group, that were randomly selected from the 20 validation pairs.
Fig. 15. Density plot of burned area proportions between the rened U-Net 3 m
classied results and the 3 m reference data, within 30 m ×30 m cells,
considering the 20 validation data sets. The point density distribution is
calculated using a 25 ×25 quantization of the plot axes, and is displayed with a
rainbow logarithmic colour scale. The solid line shows the ordinary least
squares regression of the plotted data. The dashed 1:1 line is shown for refer-
ence. Only grid cells with <50% unmapped area were considered.
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
18
acquired coincident with the PlanetScope image pair classication re-
sults reported in this study. Consequently, 20 PlanetScope image pairs,
different to the 20 used to rene the U-Net, were visually interpreted to
delineate 3 m burn reference maps. They were located across Africa and
provide a Stage 2 validation (dened as in Boschetti et al., 2016) as they
encompass a range of burned conditions but were not selected following
a probability sampling design. In the future, the synoptic availability of
very high spatial resolution commercial satellite data is anticipated that
may provide a new source of burned area reference data, although sig-
nicant effort is required to ensure the quality of the interpreted burned
area reference maps.
The rened U-Net PlanetScope classication accuracy was rst
assessed using conventional 3 m per-pixel confusion matrix-based met-
rics. Balanced commission and omission errors of 12.29% and 12.09%,
respectively, with variability among the 20 validation locations (Fig. 13)
were observed. This overall performance is comparable or better than
documented for Landsat and Sentinel-2 burned area mapping research
(Bastarrika et al., 2011; Padilla et al., 2015; Roteta et al., 2019; Roy
et al., 2019) which as discussed earlier are sensors better suited for
burned area mapping, although, meaningful comparison with other
studies is difcult because of the different validation protocols, spatial
coverage, and sensor resolutions that were employed. An experiment to
examine the dependency of the U-Net classication accuracy on the
burned area proportion within 3 m pixel 256 ×256 patches (Fig. 14)
indicated that classication of patches with smaller burned area pro-
portions was less reliable than larger proportions. This may be related to
the scale difference between the Landsat training patches and the
patches used to classify the PlanetScope imagery, although natural
processes often exhibit spatial self-similarity over a range of scales
(Pentland, 1984), and we often observed burned areas with similar
spatial arrangements at Landsat 30 m and PlanetScope 3 m resolution.
Object-based accuracy measures (Persello and Bruzzone, 2010; Yan and
Roy, 2014) were not used in this study as they are unreliable for objects
comprised of only a small number of pixels (Jaccard, 1901; Tetteh et al.,
2021) and reliable clustering of burned pixels into burned objects is not
trivial (Roy et al., 2019).
The per-pixel confusion-matrix results cannot discriminate between
errors of omission and commission due to misclassication of entire
burned or unburned patches, and errors due to the misclassication of
isolated pixels in the vicinity of correctly classied burned patches. In-
depth examination of these issues is beyond the scope of this paper.
The regression analysis between proportion of area burned in coarser
resolution cells showed however that the classication results, when
aggregated at 30 m resolution, are both precise (as indicated by the high
coefcient of determination) and accurate (as indicated by the regres-
sion line small intercept and slope close to unity). This implies that the
omission and commission errors largely compensate at 30 m resolution,
thus suggesting that the proposed classication methodology would be
suitable for the generation of independent high resolution reference data
for the validation of moderate spatial resolution continental and global
burned area products. The results of this study indicate that systematic
burned area mapping with PlanetScope imagery is possible, providing
local, near daily, assessment of re propagation, and mapping of small
and spatially fragmented burned areas.
CRediT authorship contribution statement
V.S. Martins: Methodology, Software, Formal analysis, Visualiza-
tion, Writing – original draft, Writing – review & editing. D.P. Roy:
Conceptualization, Methodology, Writing – original draft, Writing –
review & editing. H. Huang: Visualization, Writing – original draft. L.
Boschetti: Writing – original draft. H.K. Zhang: Writing – original draft,
Writing – review & editing. L. Yan: Writing – original draft.
Declaration of Competing Interest
The authors declare that they have no known competing nancial
interests or personal relationships that could have appeared to inuence
the work reported in this paper.
Data availability
Data will be made available on request.
Acknowledgments
This research was funded by the NASA Advancing Collaborative
Connections for Earth System Science (ACCESS) Program Grant
80NSSC21M0023, the NASA Land Cover/Land Use Change Multi-Source
Land Imaging Science Program Grant NNX15AK94, and by the U.S.
Geological Survey Landsat science team (grant 140G0119C0009).
Planet Inc. are thanked for provision of the PlanetScope imagery used in
this study that were made available through the NASA Commercial
Smallsat Data Acquisition (CSDA) program.
References
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Zheng, X., 2016.
Tensorow: A system for large-scale machine learning. In: 12th Symposium on
Operating Systems Design and Implementation. Savannah, GA, USA, pp. 265–283.
Agrawal, P., Girshick, R., Malik, J., 2014. Analyzing the performance of multilayer
neural networks for object recognition. In: Fleet, D., Pajdla, T., Schiele, B.,
Tuytelaars, T. (Eds.), European Conference on Computer Vision. Springer. Cham,
Switzerland, pp. 329–344.
Arnett, J.T., Coops, N.C., Daniels, L.D., Falls, R.W., 2015. Detecting forest damage after a
low-severity re using remote sensing at multiple scales. Int. J. Appl. Earth Obs.
Geoinf. 35, 239–246.
Balch, J.K., Bradley, B.A., Abatzoglou, J.T., Nagy, R.C., Fusco, E.J., Mahood, A.L., 2017.
Human-started wildres expand the re niche across the United States. Proc. Natl.
Acad. Sci. 114 (11), 2946–2951.
Bastarrika, A., Chuvieco, E., Martin, M.P., 2011. Mapping burned areas from Landsat
TM/ETM+data with a two-phase algorithm: balancing omission and commission
errors. Remote Sens. Environ. 115 (4), 1003–1012.
Beaudoin, A., Le Toan, T., Gwyn, Q.H.J., 1990. SAR observations and modeling of the C-
band backscatter variability due to multiscale geometry and soil moisture. IEEE
Trans. Geosci. Remote Sens. 28 (5), 886–895.
Belenguer-Plomer, M.A., Tanase, M.A., Chuvieco, E., Bovolo, F., 2021. CNN-based
burned area mapping using radar and optical data. Remote Sens. Environ. 260,
112468.
Bengio, Y., 2012. Practical recommendations for gradient-based training of deep
architectures. In: Montavon, G., Orr, G.B., Müller, K.R. (Eds.), Neural Networks:
Tricks of the Trade. Springer, Berlin, Germany, pp. 437–478.
Blaschke, T., Hay, G.J., Kelly, M., Lang, S., Hofmann, P., Addink, E., Tiede, D., 2014.
Geographic object-based image analysis–towards a new paradigm. ISPRS J.
Photogramm. Remote Sens. 87, 180–191.
Boschetti, L., Flasse, S.P., Brivio, P.A., 2004. Analysis of the conict between omission
and commission in low spatial resolution dichotomic thematic products: the Pareto
Boundary. Remote Sens. Environ. 91 (3–4), 280–292.
Boschetti, L., Roy, D.P., Justice, C.O., 2009. International global burned area satellite
product validation protocol. In: CEOS-CalVal (Ed.), Part I—Production and
Standardization of Validation Reference Data. Committee on Earth Observation
Satellites, USA, pp. 1–11.
Boschetti, L., Roy, D.P., Justice, C.O., Humber, M.L., 2015. MODIS–Landsat fusion for
large area 30m burned area mapping. Remote Sens. Environ. 161, 27–42.
Boschetti, L., Stehman, S.V., Roy, D.P., 2016. A stratied random sampling design in
space and time for regional to global scale burned area product validation. Remote
Sens. Environ. 186, 465–478.
Boschetti, L., Roy, D.P., Giglio, L., Huang, H., Zubkova, M., Humber, M.L., 2019. Global
validation of the collection 6 MODIS burned area product. Remote Sens. Environ.
235, 111490.
Bowman, D.M., Kolden, C.A., Abatzoglou, J.T., Johnston, F.H., van der Werf, G.R.,
Flannigan, M., 2020. Vegetation res in the Anthropocene. Nat. Rev. Earth Environ.
1 (10), 500–515.
Brandt, M., Tucker, C.J., Kariryaa, A., Rasmussen, K., Abel, C., Small, J., Fensholt, R.,
2020. An unexpectedly large count of trees in the West African Sahara and Sahel.
Nature 587 (7832), 78–82.
Campos, V., Jou, B., Giro-i-Nieto, X., 2017. From pixels to sentiment: ne-tuning CNNs
for visual sentiment prediction. Image Vis. Comput. 65, 15–22.
Cetinic, E., Lipic, T., Grgic, S., 2018. Fine-tuning convolutional neural networks for ne
art classication. Expert Syst. Appl. 114, 107–118.
Chung, M., Han, Y., Kim, Y., 2020. A framework for unsupervised wildre damage
assessment using VHR satellite images with PlanetScope data. Remote Sens. 12 (22),
3835.
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
19
Chuvieco, E., Congalton, R.G., 1988. Mapping and inventory of forest res from digital
processing of TM data. Geocart. Int. 3 (4), 41–53.
Chuvieco, E., Mouillot, F., van der Werf, G.R., San Miguel, J., Tanase, M., Koutsias, N.,
Giglio, L., 2019. Historical background and current developments for mapping
burned area from satellite earth observation. Remote Sens. Environ. 225, 45–64.
Comaniciu, D., Meer, P., 2002. Mean shift: a robust approach toward feature space
analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24 (5), 603–619.
Congalton, R.G., Green, K., 2019. Assessing the Accuracy of Remotely Sensed Data:
Principles and Practices, , Third edition346. CRC press. Boca Raton, FL, USA.
Derksen, D., Inglada, J., Michel, J., 2019. A metric for evaluating the geometric quality of
land cover maps generated with contextual features from high-dimensional satellite
image time series without dense reference data. Remote Sens. 11 (16), 1929.
Doxani, G., Vermote, E., Roger, J.C., Gascon, F., Adriaensen, S., Frantz, D.,
Vanhellemont, Q., 2018. Atmospheric correction inter-comparison exercise. Remote
Sens. 10 (2), 352.
Dragozi, E., Gitas, I.Z., Bajocco, S., Stavrakoudis, D.G., 2016. Exploring the relationship
between burn severity eld data and very high resolution GeoEye images: the case of
the 2011 Evros wildre in Greece. Remote Sens. 8 (7), 566.
Foody, G.M., 2002. Status of land cover classication accuracy assessment. Remote Sens.
Environ. 80 (1), 185–201.
Fraser, R.S., Kaufman, Y.J., 1985. The relative importance of aerosol scattering and
absorption in remote sensing. IEEE Trans. Geosci. Remote Sens. 5, 625–633.
Freudenberg, M., N¨
olke, N., Agostini, A., Urban, K., W¨
org¨
otter, F., Kleinn, C., 2019. Large
scale palm tree detection in high resolution satellite images using U-Net. Remote
Sens. 11 (3), 312.
Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-
Gonzalez, P., Garcia-Rodriguez, J., 2018. A survey on deep learning techniques for
image and video semantic segmentation. Appl. Soft Comput. 70, 41–65.
Giglio, L., Roy, D.P., 2020. On the outstanding need for a long-term, multi-decadal,
validated and quality assessed record of global burned area: caution in the use of
advanced very high resolution radiometer data. Sci. Remote Sens. 2, 100007.
Giglio, L., Boschetti, L., Roy, D.P., Humber, M.L., Justice, C.O., 2018. The collection 6
MODIS burned area mapping algorithm and product. Remote Sens. Environ. 217,
72–85.
Glorot, X., Bordes, A., Bengio, Y., 2011. Deep sparse rectier neural networks. In:
Gordon, G.J., Dunson, D.B., Dudík, M. (Eds.), Proceedings of the Fourteenth
International Conference on Articial Intelligence and Statistics. Fort Lauderdale,
FL, USA, pp. 315–323.
Hall, D.K., Ormsby, J.P., Johnson, L., Brown, J., 1980. Landsat digital analysis of the
initial recovery of burned tundra at Kokolik river, Alaska. Remote Sens. Environ. 10,
263–272.
Hall, J.V., Loboda, T.V., Giglio, L., McCarty, G.W., 2016. A MODIS-based burned area
assessment for Russian croplands: mapping requirements and challenges. Remote
Sens. Environ. 184, 506–521.
Hawbaker, T.J., Vanderhoof, M.K., Beal, Y.J., Takacs, J.D., Schmidt, G.L., Falgout, J.T.,
Dwyer, J.L., 2017. Mapping burned areas using dense time-series of Landsat data.
Remote Sens. Environ. 198, 504–522.
Houborg, R., McCabe, M.F., 2018. A cubesat enabled spatio-temporal enhancement
method (cestem) utilizing planet, landsat and modis data. Remote Sens. Environ.
209, 211–226.
Hu, F., Xia, G.S., Hu, J., Zhang, L., 2015. Transferring deep convolutional neural
networks for the scene classication of high-resolution remote sensing imagery.
Remote Sens. 7 (11), 14680–14707.
Huang, H., Roy, D.P., 2021. Characterization of Planetscope-0 Planetscope-1 surface
reectance and normalized difference vegetation index continuity. Sci. Remote Sens.
3, 100014.
Huang, H., Roy, D.P., Boschetti, L., Zhang, H.K., Yan, L., Kumar, S.S., Li, J., 2016.
Separability analysis of Sentinel-2A multi-spectral instrument (MSI) data for burned
area discrimination. Remote Sens. 8 (10), 873.
Humber, M.L., Boschetti, L., Giglio, L., Justice, C.O., 2019. Spatial and temporal
intercomparison of four global burned area products. Int. J. Digit. Earth 1–25.
Ioffe, S., Szegedy, C., 2015. Batch normalization: Accelerating deep network training by
reducing internal covariate shift. In: Bach, F., Blei, D. (Eds.), International
Conference on Machine Learning. PMLR. Lille, France, pp. 448–456.
Jaccard, P., 1901. ´
Etude comparative de la distribution orale dans une portion des
Alpes et des Jura. Bull. Soc. Vaudoise Sci. Nat. 37, 547–579.
Ju, J., Roy, D.P., Vermote, E., Masek, J., Kovalskyy, V., 2012. Continental-scale
validation of MODIS-based and LEDAPS Landsat ETM+atmospheric correction
methods. Remote Sens. Environ. 122, 175–184.
Justice, C.O., Giglio, L., Korontzi, S., Owens, J., Morisette, J.T., Roy, D., Kaufman, Y.,
2002. The MODIS re products. Remote Sens. Environ. 83 (1–2), 244–262.
Kingma, D.P., Ba, J., 2015. Adam: A method for stochastic optimization. In: Bengio, Y.,
LeCun, Y. (Eds.), 3rd International Conference on Learning Representations. San
Diego, CA, USA.
Knopp, L., Wieland, M., R¨
attich, M., Martinis, S., 2020. A deep learning approach for
burned area segmentation with Sentinel-2 data. Remote Sens. 12 (15), 2422.
Laris, P.S., 2005. Spatiotemporal problems with detecting and mapping mosaic re
regimes with coarse-resolution satellite data in savanna environments. Remote Sens.
Environ. 99 (4), 412–424.
LeCun, Y., Boser, B.E., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W.E.,
Jackel, L.D., 1990. Handwritten digit recognition with a back-propagation network.
In: Touretzky, D. (Ed.), Advances in Neural Information Processing Systems. Morgan
Kaufmann Publishers Inc, Denver, CO, United States, pp. 396–404.
LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521 (7553), 436–444.
Li, Y., Zhang, H., Xue, X., Jiang, Y., Shen, Q., 2018. Deep learning for remote sensing
image classication: a survey. Wiley Interdiscip. Rev. Data Min. Knowledge Discov.
8 (6), e1264.
Li, Z., Shen, H., Cheng, Q., Liu, Y., You, S., He, Z., 2019. Deep learning based cloud
detection for medium and high resolution remote sensing images of different sensors.
ISPRS J. Photogramm. Remote Sens. 150, 197–212.
Lima, R.P., Marfurt, K., 2020. Convolutional neural network for remote-sensing scene
classication: transfer learning analysis. Remote Sens. 12 (1), 86.
Liu, Y., Starzyk, J.A., Zhu, Z., 2008. Optimized approximation algorithm in neural
networks without overtting. IEEE Trans. Neural Netw. 19 (6), 983–995.
L´
opez García, M.J., Caselles, V., 1991. Mapping burns and natural reforestation using
thematic mapper data. Geocart. Int. 1, 31–37.
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., Johnson, B.A., 2019. Deep learning in remote
sensing applications: a meta-analysis and review. ISPRS J. Photogramm. Remote
Sens. 152, 166–177.
Maggiori, E., Tarabalka, Y., Charpiat, G., Alliez, P., 2016. Convolutional neural networks
for large-scale remote-sensing image classication. IEEE Trans. Geosci. Remote Sens.
55 (2), 645–657.
Marmanis, D., Datcu, M., Esch, T., Stilla, U., 2015. Deep learning earth observation
classication using ImageNet pretrained networks. IEEE Geosci. Remote Sens. Lett.
13 (1), 105–109.
Martins, V.S., Kaleita, A.L., Gelder, B.K., da Silveira, H.L., Abe, C.A., 2020. Exploring
multiscale object-based convolutional neural network (multi-OCNN) for remote
sensing image classication at high spatial resolution. ISPRS J. Photogramm. Remote
Sens. 168, 56–73.
Masolele, R.N., De Sy, V., Herold, M., Marcos, D., Verbesselt, J., Gieseke, F., Martius, C.,
2021. Spatial and temporal deep learning methods for deriving land-use following
deforestation: a pan-tropical case study using Landsat time series. Remote Sens.
Environ. 264, 112600.
Mateo-García, G., Laparra, V., L´
opez-Puigdollers, D., G´
omez-Chova, L., 2020.
Transferring deep learning models for cloud detection between Landsat-8 and Proba-
V. ISPRS J. Photogramm. Remote Sens. 160, 1–17.
Mathieu, R., Main, R., Roy, D.P., Naidoo, L., Yang, H., 2019. The effect of surface re in
Savannah systems in the Kruger National Park, South Africa, on the backscatter of C-
band Sentinel-1 images. Fire 2 (3), 37.
Melchiorre, A., Boschetti, L., 2018. Global analysis of burned area persistence time with
MODIS data. Remote Sens. 10 (5), 750.
Michael, Y., Lensky, I.M., Brenner, S., Tchetchik, A., Tessler, N., Helman, D., 2018.
Economic assessment of re damage to urban forest in the wildland–urban interface
using planet satellites constellation images. Remote Sens. 10 (9), 1479.
Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R.,
Muharemagic, E., 2015. Deep learning applications and challenges in big data
analytics. J. Big Data 2 (1), 1.
Nogueira, K., Penatti, O.A., Dos Santos, J.A., 2017. Towards better exploiting
convolutional neural networks for remote sensing scene classication. Pattern
Recogn. 61, 539–556.
Olson, D.M., Dinerstein, E., Wikramanayake, E.D., Burgess, N.D., Powell, G.V.,
Underwood, E.C., Kassem, K.R., 2001. Terrestrial ecoregions of the world: a new
map of life on EarthA new global map of terrestrial ecoregions provides an
innovative tool for conserving biodiversity. BioScience 51 (11), 933–938.
Olthof, I., Pouliot, D., Fernandes, R., Latifovic, R., 2005. Landsat-7 ETM+radiometric
normalization comparison for northern mapping applications. Remote Sens.
Environ. 95 (3), 388–398.
Oquab, M., Bottou, L., Laptev, I., Sivic, J., 2014. Learning and transferring mid-level
image representations using convolutional neural networks. In: Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724.
Columbus, OH, USA.
Padilla, M., Stehman, S.V., Ramo, R., Corti, D., Hantson, S., Oliva, P., Chuvieco, E., 2015.
Comparing the accuracies of remote sensing global burned area products using
stratied random sampling and estimation. Remote Sens. Environ. 160, 114–121.
Pavlidis, T., Liow, Y.T., 1990. Integrating region growing and edge detection. IEEE Trans.
Pattern Anal. Mach. Intell. 12, 225–233.
Pentland, A.P., 1984. Fractal-based descriptions of natural scenes. IEEE Trans. Pattern
Anal. Mach. Intell. 6, 661–674.
Pereira, J.M., S´
a, A.C., Sousa, A.M., Silva, J.M., Santos, T.N., Carreiras, J.M., 1999.
Spectral characterisation and discrimination of burnt areas. In: Remote Sensing of
Large Wildres. Springer, Berlin, Heidelberg, pp. 123–138.
Persello, C., Bruzzone, L., 2010. A novel protocol for accuracy assessment in
classication of very high resolution images. IEEE Trans. Geosci. Remote Sens. 48
(3), 1232–1244.
Pinto, M.M., Libonati, R., Trigo, R.M., Trigo, I.F., DaCamara, C.C., 2020. A deep learning
approach for mapping and dating burned areas using temporal sequences of satellite
images. ISPRS J. Photogramm. Remote Sens. 160, 260–274.
Planet Team, 2021. Planet Imagery Product Specications. February 2021, Available
online. https://assets.planet.com/docs/Planet_Combined_Imagery_Product_Specs_l
etter_screen.pdf. Last Accessed: 1/18/2022.
Pulliainen, J.T., Mikhela, P.J., Hallikainen, M.T., Ikonen, J.P., 1996. Seasonal dynamics
of C-band backscatter of boreal forests with applications to biomass and soil
moisture estimation. IEEE Trans. Geosci. Remote Sens. 34 (3), 758–770.
Ronneberger, O., Fischer, P., Brox, T., 2015. U-net: Convolutional networks for
biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M.,
Frangi, A.F. (Eds.), Medical Image Computing and Computer-Assisted Intervention.
Springer, Munich, Germany, pp. 234–241.
Roteta, E., Bastarrika, A., Padilla, M., Storm, T., Chuvieco, E., 2019. Development of a
Sentinel-2 burned area algorithm: generation of a small re database for sub-
Saharan Africa. Remote Sens. Environ. 222, 1–17.
V.S. Martins et al.
Remote Sensing of Environment 280 (2022) 113203
20
Roy, D.P., Boschetti, L., 2009. Southern Africa validation of the MODIS, L3JRC, and
GlobCarbon burned-area products. IEEE Trans. Geosci. Remote Sens. 47 (4),
1032–1044.
Roy, D.P., Jin, Y., Lewis, P.E., Justice, C.O., 2005a. Prototyping a global algorithm for
systematic re-affected area mapping using MODIS time series data. Remote Sens.
Environ. 97 (2), 137–162.
Roy, D.P., Frost, P.G.H., Justice, C.O., Landmann, T., Le Roux, J.L., Gumbo, K.,
Davies, D., 2005b. The Southern Africa Fire Network (SAFNet) regional burned-area
product-validation protocol. Int. J. Remote Sens. 26 (19), 4265–4292.
Roy, D.P., Wulder, M.A., Loveland, T.R., Woodcock, C.E., Allen, R.G., Anderson, M.C.,
Zhu, Z., 2014. Landsat-8: science and product vision for terrestrial global change
research. Remote Sens. Environ. 145, 154–172.
Roy, D.P., Huang, H., Boschetti, L., Giglio, L., Yan, L., Zhang, H.H., Li, Z., 2019. Landsat-
8 and Sentinel-2 burned area mapping- a combined sensor multi-temporal change
detection approach. Remote Sens. Environ. 231, 111254.
Roy, D.P., Huang, H., Houborg, R., Martins, V.S., 2021. A global analysis of the temporal
availability of PlanetScope high spatial resolution multi-spectral imagery. Remote
Sens. Environ. 264, 112586.
Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning representations by back-
propagating errors. Nature 323 (6088), 533–536.
Schott, J.R., Salvaggio, C., Volchok, W.J., 1988. Radiometric scene normalization using
pseudoinvariant features. Remote Sens. Environ. 26 (1), 1–16.
Sen, P.K., 1968. Estimates of the regression coefcient based on Kendall’s tau. J. Am.
Stat. Assoc. 63 (324), 1379–1389.
Stehman, S.V., 1997. Selecting and interpreting measures of thematic classication
accuracy. Remote Sens. Environ. 62 (1), 77–89.
Stoian, A., Poulain, V., Inglada, J., Poughon, V., Derksen, D., 2019. Land cover maps
production with high resolution satellite image time series and convolutional neural
networks: adaptations and limits for operational systems. Remote Sens. 11 (17),
1986.
Sulla-Menashe, D., Gray, J.M., Abercrombie, S.P., Friedl, M.A., 2019. Hierarchical
mapping of annual global land cover 2001 to present: the MODIS collection 6 land
cover product. Remote Sens. Environ. 222, 183–194.
Tetteh, G.O., Gocht, A., Erasmi, S., Schwieder, M., Conrad, C., 2021. Evaluation of
Sentinel-1 and Sentinel-2 feature sets for delineating agricultural elds in
heterogeneous landscapes. IEEE Access 9 (116702-116719).
Theil, H., 1950. A rank-invariant method of linear and polynomial. Mathematics 392,
387–392.
Torres, R., Snoeij, P., Geudtner, D., Bibby, D., Davidson, M., Attema, E., Rostan, F., 2012.
GMES Sentinel-1 mission. Remote Sens. Environ. 120, 9–24.
Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., Bircheld, S.,
2018. Training deep networks with synthetic data: Bridging the reality gap by
domain randomization. In: Conference on Computer Vision and Pattern Recognition
Workshops. IEEE Computer Society, Salt Lake City, UT, USA, pp. 969–977.
Trigg, S.N., Roy, D.P., Flasse, S.P., 2005. An in situ study of the effects of surface
anisotropy on the remote sensing of burned savannah. Int. J. Remote Sens. 26 (21),
4869–4876.
Tuia, D., Persello, C., Bruzzone, L., 2016. Domain adaptation for the classication of
remote sensing data: an overview of recent advances. IEEE Geosci. Remote Sens.
Mag. 4 (2), 41–57.
Vanderhoof, M.K., Burt, C., Hawbaker, T.J., 2018. Time series of high-resolution images
enhances efforts to monitor post-re condition and recovery, Waldo Canyon re,
Colorado, USA. Int. J. Wildland Fire 27 (10), 699–713.
Vermote, E., Justice, C., Claverie, M., Franch, B., 2016. Preliminary analysis of the
performance of the Landsat 8/OLI land surface reectance product. Remote Sens.
Environ. 185, 46–56.
Volpi, M., Tuia, D., 2016. Dense semantic labeling of subdecimeter resolution images
with convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 55 (2),
881–893.
Wagner, F.H., Sanchez, A., Tarabalka, Y., Lotte, R.G., Ferreira, M.P., Aidar, M.P.,
Aragao, L.E., 2019. Using the U-net convolutional network to map forest types and
disturbance in the Atlantic rainforest with very high resolution images. Remote Sens.
Ecol. Conserv. 5 (4), 360–375.
Ward, M., Tulloch, A.I., Radford, J.Q., Williams, B.A., Reside, A.E., Macdonald, S.L.,
Watson, J.E., 2020. Impact of 2019–2020 mega-res on Australian fauna habitat.
Nat. Ecol. Evol. 4 (10), 1321–1326.
Warner, T.A., Skowronski, N.S., Gallagher, M.R., 2017. High spatial resolution burn
severity mapping of the New Jersey Pine Barrens with WorldView-3 near-infrared
and shortwave infrared imagery. Int. J. Remote Sens. 38 (2), 598–616.
Wooster, M.J., Roberts, G.J., Giglio, L., Roy, D.P., Freeborn, P.H., Justice, C.O.,
Ichoku, C.M., Schroeder, W., Boschetti, L., Davies, D.K., Smith, A.M.S., Csiszar, I.,
Frost, P., Setzer, A., Zhang, T., de Jong, M.C., Johnston, J.M., Xu, W., Schmidt, C.,
Strydom, T., Ellision, L., Vadrevu, K.P., Tanpipat, V., San-Miguel-Ayanz, J.,
Sparks, A.M., McCarty, J.L., Nguyen, H., 2021. Satellite remote sensing of active
res: history and current status, applications and future requirements. Remote Sens.
Environ. 267, 112694.
Wulder, M.A., Coops, N.C., Roy, D.P., White, J.C., Hermosilla, T., 2018. Land cover 2.0.
Int. J. Remote Sens. 39 (12), 4254–4284.
Wurm, M., Stark, T., Zhu, X.X., Weigand, M., Taubenb¨
ock, H., 2019. Semantic
segmentation of slums in satellite images using transfer learning on fully
convolutional neural networks. ISPRS J. Photogramm. Remote Sens. 150, 59–69.
Xu, Y., Wu, L., Xie, Z., Chen, Z., 2018. Building extraction in very high resolution remote
sensing imagery using deep learning and guided lters. Remote Sens. 10 (1), 144.
Yan, L., Roy, D.P., 2014. Automated crop eld extraction from multi-temporal web
enabled Landsat data. Remote Sens. Environ. 144, 42–64.
Zhang, C., Sargent, I., Pan, X., Li, H., Gardiner, A., Hare, J., Atkinson, P.M., 2018. An
object-based convolutional neural network (OCNN) for urban land use classication.
Remote Sens. Environ. 216, 57–70.
Zhang, Q., Ge, L., Zhang, R., Metternicht, G.I., Du, Z., Kuang, J., Xu, M., 2021a. Deep-
learning-based burned area mapping using the synergy of Sentinel-1&2 data. Remote
Sens. Environ. 264, 112575.
Zhang, C., Bengio, S., Hardt, M., Vinyals, O., 2021b. Understanding deep learning (still)
requires rethinking generalization. Commun. ACM 64 (3), 107–115.
Zhu, X.X., Tuia, D., Mou, L., Xia, G.S., Zhang, L., Xu, F., Fraundorfer, F., 2017. Deep
learning in remote sensing: a comprehensive review and list of resources. IEEE
Geosci. Remote Sens. Mag. 5 (4), 8–36.
Zhu, Z., Wulder, M.A., Roy, D.P., Woodcock, C.E., Hansen, M.C., Radeloff, V.C.,
Scambos, T.A., 2019. Benets of the free and open Landsat data policy. Remote Sens.
Environ. 224, 382–385.
Zou, M., Zhong, Y., 2018. Transfer learning for classication of optical satellite image.
Sens. Imaging 19 (1), 6.
V.S. Martins et al.
