No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
CNN-Based Semantic Change Detection in Satellite Imagery
Abstract
Timely disaster risk management requires accurate road maps and prompt damage assessment. Currently, this is done by volunteers manually marking satellite imagery of affected areas but this process is slow and often error-prone. Segmentation algorithms can be applied to satellite images to detect road networks. However, existing methods are unsuitable for disaster-struck areas as they make assumptions about the road network topology which may no longer be valid in these scenarios. Herein, we propose a CNN-based framework for identifying accessible roads in post-disaster imagery by detecting changes from pre-disaster imagery. Graph theory is combined with the CNN output for detecting semantic changes in road networks with OpenStreetMap data. Our results are validated with data of a tsunami-affected region in Palu, Indonesia acquired from DigitalGlobe.
... Labor-intensive tasks, such as the mapping of road networks in aerial images, are becoming inexpensive with deep learning [8]. For damage assessment in timely disaster risk management, an adequate deep-learning-based framework has been proposed [9]. Many aerial image datasets, such as [10], are presented with fine annotations and readily available for semantic segmentation. ...
Aerial images are important for monitoring land cover and land resource management. An aerial imaging source which keeps its position at a higher altitude, and which has a considerable duration of airtime, employs wireless communications for sending images to relevant receivers. An aerial image must be transmitted from the image source to a ground station where it can be stored and analyzed. Due to transmission errors, aerial images which are received from an image transmitter contain distortions which can affect the quality of the images, causing noise, color shifts, and other issues that can impact the accuracy of semantic segmentation and the usefulness of the information contained in the images. Current semantic segmentation methods discard distorted images, which makes the available dataset small or treats them as normal images, which causes poor segmentation results. This paper proposes a deep-learning-based semantic segmentation method for distorted aerial images. For different receivers, distortions occur differently, and by considering the receiver specificness of the distortions, the proposed method was able to grasp the acceptability for a distorted image using semantic segmentation models trained with large aerial image datasets to build a combined model that can effectively segment a distorted aerial image which was received by an analog image receiver. Two combined deep learning models, an approximating model, and a segmentation model were trained combinedly to maximize the segmentation score for distorted images. The results showed that the combined learning method achieves higher intersection-over-union (IoU) scores than the results obtained by using only a segmentation model.
With the acceleration of urban expansion, urban change detection (UCD), as a significant and effective approach, can provide the change information with respect to geospatial objects for dynamic urban analysis. In recent years, through the use of machine learning and artificial intelligence, change detection methods have gradually developed from the traditional pixel-based comparison methods in the 1980s to data-driven deep learning methods. Deep learning methods have huge advantages in the application of remote sensing big data, by virtue of their huge feature extraction and expression capabilities. Many change detection datasets have been released to meet the requirements of deep learning. However, the existing datasets suffer from three bottlenecks: (1) the volume of the datasets is small, which can easily cause overfitting; (2) most datasets have a spatial resolution of meters, making it difficult to detect changes in small objects because there are multi-scale objects in urban areas; and (3) most of the datasets have been designed for binary change detection (BCD), and lack semantic annotation, so that they cannot be used to obtain the direction of the change or to analyze the type of change, for further application in urban areas. Therefore, it is difficult to apply these datasets to detect large-scale urban semantic changes in complex environments. To address these issues, a large-scale ultra high resolution (0.1 m) UCD dataset for deep learning based BCD and semantic change detection (SCD) is introduced in this article, which is named the Hi-UCD dataset. We selected an area of 102 m² in Tallinn, the capital of Estonia, as the study area. There are a total of 40800 pairs of 512 × 512 patches, nine types of land cover and 48 types of semantic change in the Hi-UCD dataset. We developed three metrics–binary consistency, change area consistency and no-change area consistency–to evaluate the semantic consistency of the SCD methods from different aspects. A comprehensive analysis and investigation is provided in this article after we benchmarked this dataset using deep learning methods for BCD and SCD. We also found that the unchanged samples can help distinguish the changed area, and HRNet used as the backbone to construct a multi-task model can perform well in the Hi-UCD dataset. Meanwhile, the visualization results obtained with the Hi-UCD test set, which is a large geographic area covering 54 km², are shown to reflect the real-world urban application scenarios. The experimental results show that the Hi-UCD dataset is a challenging yet useful benchmark dataset, which can be used for analyzing large-scale refined urban changes.
Change detection of the newly constructed areas (NCAs) is important for urban development. The advances of remote sensing and deep learning algorithms promotes the high precision of the research work. In this study, we firstly constructed a high-resolution labels for change detection based on the GF-2 satellite images, and then applied five deep learning models of change detection, including STANets (BASE, BAM, and PAM), SNUNet (Siam-NestedUNet), and BiT (Bitemporal image Transformer) in the Core Region of Jiangbei New Area of Nanjing, China. The BiT model is based on transformer, and the others are based on CNN (Conventional Neural Network). Experiments have revealed that the STANet-PAM model generally performs the best in detecting the NCAs, and the STANet-PAM model can obtain more detailed information of land changes owing to its pyramid spatial-temporal attention module of multiple scales. At last, we have used the five models to analyze urbanization processes from 2015 to 2021 in the study area. Hopefully, the results of this study could be a momentous reference for urban development planning.
Satellite images are an extremely valuable resource in the aftermath of natural disasters such as hurricanes and tsunamis where they can be used for risk assessment and disaster management. In order to provide timely and actionable information for disaster response, in this paper a framework utilising segmentation neural networks is proposed to identify impacted areas and accessible roads in post-disaster scenarios. The effectiveness of pretraining with ImageNet on the task of aerial image segmentation has been analysed and performances of popular segmentation models compared. Experimental results show that pretraining on ImageNet usually improves the segmentation performance for a number of models. Open data available from OpenStreetMap (OSM) is used for training, forgoing the need for time-consuming manual annotation. The method also makes use of graph theory to update road network data available from OSM and to detect the changes caused by a natural disaster. Extensive experiments on data from the 2018 tsunami that struck Palu, Indonesia show the effectiveness of the proposed framework. ENetSeparable, with 30% fewer parameters compared to ENet, achieved comparable segmentation results to that of the state-of-the-art networks.
Road vector database plays an important role in post-earthquake relief, rescue and reconstruction. However, due to data privacy policy, it is difficult for general users to obtain high-precision and complete vector data of road network. The OpenStreetMap (OSM) project provides an open-source, global free road dataset, but there are inevitable geo-localization/projection errors, which will lead to large errors in hazard survey analysis. In this paper, we proposed a road centerline correction method using post-earthquake aerial images. Under the constraint of the vector road map (OpenStreetMap), we rectified the centerline by the context feature and spectral gradient feature of post-event images automatically. The experiment implemented on 0.5 m/pixel post-event aerial images of Haiti, 2010, showed that the completeness and extraction quality of proposed method were over 90% and 80% without any manual intervention. © 2019 Institute of Seismology, China Earthquake Administration, etc.
We present the DeepGlobe 2018 Satellite Image Understanding Challenge, which includes three public competitions for segmentation, detection, and classification tasks on satellite images (Figure 1). Similar to other challenges in computer vision domain such as DAVIS[21] and COCO[33], DeepGlobe proposes three datasets and corresponding evaluation methodologies, coherently bundled in three competitions with a dedicated workshop co-located with CVPR 2018.
We observed that satellite imagery is a rich and structured source of information, yet it is less investigated than everyday images by computer vision researchers. However, bridging modern computer vision with remote sensing data analysis could have critical impact to the way we understand our environment and lead to major breakthroughs in global urban planning or climate change research. Keeping such bridging objective in mind, DeepGlobe aims to bring together researchers from different domains to raise awareness of remote sensing in the computer vision community and vice-versa. We aim to improve and evaluate state-of-the-art satellite image understanding approaches, which can hopefully serve as reference benchmarks for future research in the same topic. In this paper, we analyze characteristics of each dataset, define the evaluation criteria of the competitions, and provide baselines for each task.
Typhoons and hurricanes cause extensive damage to coast cities annually, demanding urban authorities to take effective actions in disaster response to reduce losses. One of the first priority in disaster response is to identify and clear road obstacles, such as fallen trees and ponding water, and restore road transportation in a timely manner for supply and rescue. Traditionally, identifying road obstacles is done by manual investigation and reporting, which is labor intensive and time consuming, hindering the timely restoration of transportation. In this work, we propose RADAR, a low-cost and real-time approach to identify road obstacles leveraging large-scale vehicle trajectory data and heterogeneous road environment sensing data. First, based on the observation that road obstacles may cause abnormal slow motion behaviors of vehicles in the surrounding road segments, we propose a cluster direct robust matrix factorization (CDRMF) approach to detect road obstacles by identifying the collective anomalies of slow motion behaviors from vehicle trajectory data. Then, we classify the detected road obstacles leveraging the correlated spatial and temporal features extracted from various road environment data, including satellite images and meteorological records. To address the challenges of heterogeneous features and sparse labels, we propose a semi-supervised approach combining co-training and active learning (CORAL). Real experiments on Xiamen City show that our approach accurately detects and classifies the road obstacles during the 2016 typhoon season with precision and recall both above 90%, and outperforms the state-of-the-art baselines.
In the aftermath of a disaster, there is an urgent need for base maps to support relief efforts, especially in developing countries. In response to this, the OpenStreetMap project has been leveraged to produce maps of disaster-affected areas in a collaborative way. However, there has been little investigation aimed at explaining the collaborative mapping activity itself. This study presents an exploratory case study on how the collaborative mapping activities that followed the Nepal Earthquake in 2015 were coordinated and structured, i.e. how volunteers were organized, and what were the main outcomes of their activity in the context of disaster management. The results show that a large number of remote contributors spread across the world carried out concerted efforts to support the relief work. Moreover, coordination mechanisms were used by local actors to share their knowledge with remote mappers, and, hence, to improve the accuracy of the map.
Road tracing is a promising technique to increase the efficiency of road mapping. In this paper a new road tracing algorithm is presented. Road positions are computed by matching the average grey value profile of a reference road segment with profiles taken from the image. The road parameters are estimated by the recursive Kaiman filter. By utilizing the prediction step of the Kaiman filter the road tracer is able to continue following the road despite temporary failures of the profile matching that are due to road crossings, exits and cars.
Road tracing is a promising technique to increase the efficiency of road mapping. In this paper a new road tracing algorithm is presented. Road positions are computed by matching the average grey value profile of a reference road segment with profiles taken from the image. The road parameters are estimated by the recursive Kalman filter. By utilising the prediction step of the Kalman filter the road tracer is able to continue following the road despite temporary failures of the profile matching that are due to road crossings, exits and cars.
This paper describes how multisource satellite data and efficient image analysis may successfully be used to conduct rapid-mapping tasks in the domain of disaster and crisis-management support. The German Aerospace Center (DLR) has set up a dedicated crosscutting service, which is the so-called "Center for satellite-based Crisis Information" (ZKI), to facilitate the use of its Earth-observation capacities in the service of national and international response to major disaster situations, humanitarian relief efforts, and civil security issues. This paper describes successful rapid satellite mapping campaigns supporting disaster relief and demonstrates how this technology can be used for civilian crisis-management purposes. During the last years, various international coordination bodies were established, improving the disaster-response-related cooperation within the Earth-observation community worldwide. DLR/ZKI operates in this context, closely networking with public authorities (civil security), nongovernmental organizations (humanitarian relief organizations), satellite operators, and other space agencies. This paper reflects on several of these international activities, such as the International Charter Space and Major Disasters, describes mapping procedures, and reports on rapid-mapping experiences gained during various disaster-response applications. The example cases presented cover rapid impact assessment after the Indian Ocean Tsunami, forest fires mapping for Portugal, earthquake-damage assessment for Pakistan, and landslide extent mapping for the Philippines
We propose a novel approach for rapid segmentation of flooded buildings by fusing multiresolution, multisensor, and multitemporal satellite imagery in a convolutional neural network. Our model significantly expedites the generation of satellite imagery-based flood maps, crucial for first responders and local authorities in the early stages of flood events. By incorporating multitemporal satellite imagery, our model allows for rapid and accurate post-disaster damage assessment and can be used by governments to better coordinate medium- and long-term financial assistance programs for affected areas. The network consists of multiple streams of encoder-decoder architectures that extract spatiotemporal information from medium-resolution images and spatial information from high-resolution images before fusing the resulting representations into a single medium-resolution segmentation map of flooded buildings. We compare our model to state-of-the-art methods for building footprint segmentation as well as to alternative fusion approaches for the segmentation of flooded buildings and find that our model performs best on both tasks. We also demonstrate that our model produces highly accurate segmentation maps of flooded buildings using only publicly available medium-resolution data instead of significantly more detailed but sparsely available very high-resolution data. We release the first open-source dataset of fully preprocessed and labeled multiresolution, multispectral, and multitemporal satellite images of disaster sites along with our source code.
Constructing digital pedestrian maps of the disaster affected areas is a vital requirement for post-disaster relief operation. Digital maps help disaster management agencies in decision making and mobilizing the manpower and resources in the disaster affected areas. However, due to the intermittent connectivity after large scale natural disasters, the existing web based digital mapping systems remain inaccessible in the disaster affected areas. Moreover, the road networks in such areas change drastically because of disaster (for instance waterlogging due to flood, structural collapse or incidental destructions like a landslide). Hence, existing analog (paper based) route maps of such areas become obsolete. In this work, we propose Post-Disaster Map Builder, a crowdsenced digital pedestrian map construction system over smartphone based DTN. The proposed system generates digital pedestrian maps of the disaster affected areas using battery powered mobile handheld devices. Here, trajectory traces collected by the volunteers are periodically shared among the other volunteers of the disaster affected area through smartphone based DTN. Pedestrian maps of the disaster affected areas are gradually constructed in the mobile handheld devices of the volunteers by combining the collected traces over time. The proposed system is thoroughly evaluated through simulation and testbed implementation. The results reveal that our proposed system can automatically construct digital pedestrian maps of disaster affected areas with high accuracy at the cost of marginal delay.
There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at
http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net
.
We present a probabilistic representation of network structures in images. Our target application is the extraction of urban roads from aerial images. Roads appear as thin, elongated, partially curved structures forming a loopy graph, and this complex layout requires a prior that goes beyond standard smoothness and co-occurrence assumptions. In the proposed model the network is represented as a union of 1D paths connecting distant (super-)pixels. A large set of putative candidate paths is constructed in such a way that they include the true network as much as possible, by searching for minimum cost paths in the foreground (road) likelihood. Selecting the optimal subset of candidate paths is posed as MAP inference in a higher-order conditional random field. Each path forms a higher-order clique with a type of clique potential, which attracts the member nodes of cliques with high cumulative road evidence to the foreground label. That formulation induces a robust PN-Potts model, for which a global MAP solution can be found efficiently with graph cuts. Experiments with two road data sets show that the proposed model significantly improves per-pixel accuracies as well as the overall topological network quality with respect to several baselines. © 2015 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS).
Rectified activation units (rectifiers) are essential for state-of-the-art
neural networks. In this work, we study rectifier neural networks for image
classification from two aspects. First, we propose a Parametric Rectified
Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU
improves model fitting with nearly zero extra computational cost and little
overfitting risk. Second, we derive a robust initialization method that
particularly considers the rectifier nonlinearities. This method enables us to
train extremely deep rectified models directly from scratch and to investigate
deeper or wider network architectures. Based on our PReLU networks
(PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012
classification dataset. This is a 26% relative improvement over the ILSVRC 2014
winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass
human-level performance (5.1%, Russakovsky et al.) on this visual recognition
challenge.
We introduce Adam, an algorithm for first-order gradient-based optimization
of stochastic objective functions. The method is straightforward to implement
and is based an adaptive estimates of lower-order moments of the gradients. The
method is computationally efficient, has little memory requirements and is well
suited for problems that are large in terms of data and/or parameters. The
method is also ap- propriate for non-stationary objectives and problems with
very noisy and/or sparse gradients. The method exhibits invariance to diagonal
rescaling of the gradients by adapting to the geometry of the objective
function. The hyper-parameters have intuitive interpretations and typically
require little tuning. Some connections to related algorithms, on which Adam
was inspired, are discussed. We also analyze the theoretical convergence
properties of the algorithm and provide a regret bound on the convergence rate
that is comparable to the best known results under the online convex
optimization framework. We demonstrate that Adam works well in practice when
experimentally compared to other stochastic optimization methods.
In recent years, due to an increasing number of extreme meteorological events potentially related to climate change, a growing attention has been paid to the operational use of satellite remote sensing applied to emergency management applications. This is mainly due to the large and timely availability of different types of remotely sensed data—as well as geospatial information acquired in the field—which may be potentially exploited in the different phases of the disaster management cycle. This paper is mainly focused on the use of remotely sensed data in emergency mapping applications supporting the crisis response phase, including natural disasters as well as humanitarian crisis. The main relevant initiatives related to emergency mapping operational services will be briefly described. For the main disaster types the suitable imagery, their relevant technical features and the main processing techniques generally adopted in an operational framework will be described. Known limitations of a satellite-based approach will be described and potential complementary geomatics techniques (including emerging technologies) to overcome the aforementioned limits will be discussed.
Improving on Our ExplanationIntellectual Impact and LegacyFurther ReadingReferences
In this paper, we present work on automatic road extraction from high-resolution aerial imagery taken over urban areas. In order to deal with the high complexity of this type of scenes, we integrate detailed knowledge about roads and their context using explicitly formulated scale-dependent models. The knowledge about how and when certain parts of the road and context model are optimally exploited is expressed by an extraction strategy.The key feature of the presented approach is the integral treatment of three essential issues of object extraction in complex scenes. (1) Specific parts of the road model and extraction strategy are automatically adapted to the respective contextual situation. (2) The extraction incorporates components for self-diagnosis that internally evaluate hypotheses indicating their relevance for further processing. (3) Multiple views on the scene are utilized in different ways. Redundancies in the extraction are exploited, occlusions are predicted and obviated, and a 3D object description is generated.The results achieved with our approach show that a stringent realization of these issues enables the extraction of roads even if their appearance is heavily affected by other objects. Based on an external evaluation of the results, we discuss advantages but also remaining deficiencies of this approach.
Reliably extracting information from aerial imagery is a difficult problem with many practical applications. One specific
case of this problem is the task of automatically detecting roads. This task is a difficult vision problem because of occlusions,
shadows, and a wide variety of non-road objects. Despite 30 years of work on automatic road detection, no automatic or semi-automatic
road detection system is currently on the market and no published method has been shown to work reliably on large datasets
of urban imagery. We propose detecting roads using a neural network with millions of trainable weights which looks at a much
larger context than was used in previous attempts at learning the task. The network is trained on massive amounts of data
using a consumer GPU. We demonstrate that predictive performance can be substantially improved by initializing the feature
detectors using recently developed unsupervised learning methods as well as by taking advantage of the local spatial coherence
of the output labels. We show that our method works reliably on two challenging urban datasets that are an order of magnitude
larger than what was used to evaluate previous approaches.
This paper presents an automated approach to finding main roads in
aerial images. The approach is to build geometric-probabilistic models
for road image generation. We use Gibbs distributions. Then, given an
image, roads are found by MAP (maximum a posteriori probability)
estimation. The MAP estimation is handled by partitioning an image into
windows, realizing the estimation in each window through the use of
dynamic programming, and then, starting with the windows containing high
confidence estimates, using dynamic programming again to obtain optimal
global estimates of the roads present. The approach is model-based from
the outset and is completely different than those appearing in the
published literature. It produces two boundaries for each road, or four
boundaries when a mid-road barrier is present
SpaceNet: a remote sensing dataset and challenge series
- A Van Etten
- D Lindenbaum
- T M Bacastow
The Huge, Unseen Operation Behind the Accuracy of Google Maps
- G Miller