Conference PaperPDF Available

Progressive Label Refinement-Based Distribution Adaptation Framework for Landslide Detection


Abstract and Figures

Efficient and accurate landslide detection is of great significance for an emergency response to geological disasters. However, detecting landslides from remote sensing images faces two challenges: small objects and class imbalance, and distribution inconsistency. In this paper, the progressive label refinement-based distribution adaptation for the landslide detection framework was proposed. The scale promotion, Lovasz loss, and online hard example mining strategy are adopted to alleviate the class imbalance, and the separated normalization and pseudo label refinement were proposed to encode the statistical inconsistency for reducing the distribution differences between the training and validation/testing data. The proposed framework has a significant potential for the large-scale global typical natural disaster monitoring rapidly from multi-sensor remote sensing imagery and ranking first place in the validation (F1-score=80.41%) and test leaderboard (F1-score=74.54%) in the LandSlide4Sense competition.
Content may be subject to copyright.
Progressive Label Refinement-Based Distribution
Adaptation Framework for Landslide Detection
Hengwei Zhao
,Junjue Wang
,Yang Pan
,Ailong Ma
,Xinyu Wang
and Yanfei Zhong
1State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University 430074, China
2School of Remote Sensing and Information Engineering, Wuhan University 430074, China
Ecient and accurate landslide detection is of great signicance for an emergency response to geological disasters. However,
detecting landslides from remote sensing images faces two challenges: small objects and class imbalance, and distribution
inconsistency. In this paper, the progressive label renement-based distribution adaptation for the landslide detection
framework was proposed. The scale promotion, Lovasz loss, and online hard example mining strategy are adopted to alleviate
the class imbalance, and the separated normalization and pseudo label renement were proposed to encode the statistical
inconsistency for reducing the distribution dierences between the training and validation/testing data. The proposed
framework has a signicant potential for the large-scale global typical natural disaster monitoring rapidly from multi-sensor
remote sensing imagery and ranking rst place in the validation (F1-score=80.41%) and test leaderboard (F1-score=74.54%) in
the LandSlide4Sense competition.
Landslide detection, Small objects and class imbalance, Distribution inconsistency, Progressive label renement
1. Introduction
Landslide is a worldwide destructive natural phe-
nomenon, usually following an earthquake or heavy rain-
fall, where thousands of small to medium-sized ground
movements occur [
]. Landslides bring serious harm
to society and the economy. Remote sensing technol-
ogy oers the possibility of rapid and large-area land
cover monitoring [
], and the detection of globally
distributed landslides from multi-source, multi-spectral
remote sensing images using machine learning and com-
puter vision algorithms facilitates rapid response and
management of landslide-generated disasters.
In the early stage of the research, the methods for
identifying landslides from remote sensing images were
mostly semi-automatic two-stage methods: extracting
discriminative features of landslides through expert
knowledge rstly, and then using SVM or RF for clas-
sication [
]. Although using expert knowledge to con-
struct discriminative features is transparent and exible,
CDCEO 2022: 2nd Workshop on Complex Data Challenges in Earth
Observation, July 25, 2022, Vienna, Austria
*Corresponding author.
These authors contributed equally.
$ (H. Zhao);
(J. Wang); (Y. Pan); (A. Ma);
(X. Wang); (Y. Zhong) (H. Zhao); (J. Wang)
0000-0001-5878-5152 (H. Zhao); 0000-0002-9500-3399 (J. Wang);
0000-0002-6190-4340 (Y. Pan); 0000-0003-3692-6473 (A. Ma);
0000-0002-0493-3954 (X. Wang); 0000-0001-9446-5850 (Y. Zhong)
©2022 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
ISSN 1613-0073
CEUR Workshop Proceedings (
it is laborious, time-consuming, and subjective [
]. Deep
learning-based methods make fully automated one-stage
landslide extraction possible, and these methods are well-
reviewed in [
]. However, most of these methods were
validated in small local regions, and the performance
of these models applied directly to a new region in an
emergency is unclear. To promote the development of
the landslide detection eld, the LandSlide4Sense com-
petition was held and a large landslide dataset, which
was collected from diverse geographical regions, is pub-
licly available to help develop a new landslide detection
algorithm [
]. Landslides detection from large remote
sensing imagery will encounter the following two prob-
lems: 1) Small objects and class imbalance and 2)
Distribution inconsistency.
As shown in Figure.1, small objects and class imbalance
are the rst challenges of landslide detection. In the real
scene, the landslide will have some small branches or
the landslide itself is relatively small in area, and as a
result, there will be a serious imbalance in the number
of pixels between the landslide and the background, as
shown in the Figure.1(b) where the number of pixels in
the background is 49 times that of the landslide in the
training set. Small objects and class imbalance will lead
to the problem of lower recall scores.
As shown in Figure.2, distribution inconsistency is the
second challenge of landslide detection. The mean and
standard deviation values of the training, validation, and
testing sets are counted band-by-band and displayed in
the Figure.2, where the histogram is the mean and the
error bars are the standard deviation. Because landslide
remote sensing images are collected from diverse geo-
graphical regions, there are signicant inconsistencies in
(a) Sentinel-2 imagery with
(b) Ratio of the number of pix-
Figure 1: Small objects and class imbalance problem in land-
slide detection. The ratio is the number of landslide pixels
to the number of background pixels in the LandSlide4Sense
training set.
Figure 2: Distribution inconsistency between the training set
and validation set. The histogram is the mean of the data and
the error bars are the standard deviation of the data.
the means and standard deviation values of the training
and validation/test sets, which pose a great challenge to
the generality of the landslide detection algorithm.
This paper proposes a progressive label renement
based on a distribution adaptation landslide detection
framework to overcome the above problems. The pro-
posed framework achieves F1-score=74.54% in the test
leaderboard of the LandSlide4Scence challenge.
2. Method
To address the two challenges above, this paper proposed
the progressive label renement framework for domain
statistics adaptation, including data preprocessing, model
ensemble, model training, model inference, and pseudo
label renement. The overview of the proposed algo-
rithm is shown in Figure. 3.
2.1. Data Preprocessing
Because the small landslide areas account for few pixels
and represent weak features, we take scale promotion
to resize the original images (128
128 pixels) into 512
512 pixels. Besides, random ip, rotation, and color
perturbation are adopted for data augmentation. As we
stacked multi-spectral, DEM, and Slope data as inputs,
the color perturbation is only applied to spectral data.
As the training and validation (testing) data have in-
consistent statistics, the mean values of the pixel values
of the data in the source and target domains are signi-
cantly dierent. Separated normalization is proposed to
reduce the statistical dierence between two domains,
which takes dierent mean and standard deviation values
to normalize the data in the source and target domains,
respectively. The mean and standard deviation values
were calculated from train and validation/test sets, respec-
tively. Separate normalization is similar to cross-sensor
normalization [
], but the domain-specic statistical nor-
malization is performed in the input of the model.
2.2. Model Ensemble and Model Training
Several advanced networks are selected for the ensem-
bling, including Swin-Transformer [
], EcientNetV2 [
and SegFormer [
]. The SegFormer adopts multilayer per-
ceptron (MLP) for the decoder and the other networks
adopt U-Decoder for resolution restoring. SegFormer uti-
lizes self-attention operations to t landslides of variant
shapes as well as the MLP to enhance the dicult sample
To further increase the generalization capability of
the model across dierent domains, the batch normaliza-
tions in the network are replaced with the cross-sensor
to automatically encode the statistical
inconsistency during the training [3].
As for model optimization, Lovasz loss [
] and on-
line hard example mining strategy were adopted to ad-
dress the class imbalance problem, and Soft-cross entropy
loss [
] was adopted to counteract the negative eects
of noisy labels in the pseudo labels.
2.3. Model Inference and Progressive
Label Refinement
In the inference phase of the model, the average of the
probability values output by the above three models is
taken as the nal inference result.
To further align the distributions of the two domains,
the progressive label renement is designed to improve
the pseudo-labels. Based on the model prediction, the
pseudo labels can be generated from the best models in
round, using the threshold of 0.7. As for the
round, the source samples come from the train set and the
target samples are test images with pseudo labels. The
Figure 3: Progressive label refinement-based distribution adaptation for landslide detection.
pseudo-label generation and domain-adaptation training
perform iteratively, progressively rening the test labels.
3. Challenge Results
The data used in LandSlide4Scence [
] are collected from
diverse geographical regions, which consists of training,
validation, and test sets containing 3799, 245, and 800
image patches, respectively. Each image patch is a com-
posite of 14 bands that include: multi-spectral data from
Sentinel-2 (B1-B12), slope data from ALOS PALSAR, and
DEM from ALOS PALSAR. All bands in the competition
dataset are resized to the resolution of about 10m per
pixel. The image patches have the size of 128
128 pixels
and are labeled pixel-wise. We set batch size as
each model was trained for 20𝑘steps.
The last round renement results on the validation
leaderboard are shown in Table 1. Compared with the
baseline, separate normalization signicantly improved
the accuracy. The selected advanced networks were re-
ned with several rounds and we ensemble them to ob-
tain the highest F1-sorce=80.41%.
The results from the best models serve as a baseline and
achieve F1-sorce=73.07% on the test leaderboard. Similar
to the validation development, the label renement was
continuously performed on the test set. The test results
in Table. 2show that the performances of the models are
progressively improved as the round increases. Round3
obtains the best result F1-sorce=74.54%.
Table 1
Results on the validation leaderboard. SN: Separated Normal-
ization. SLO: So cross entropy loss+Lovasz loss+OHEM. SP:
Scale promotion.
Refinement SN SLO SP Model F1(%)
x1. ResNet 63.52
x1. ResNet 72.71
x1. ResNet 73.20
x1. ResNet 75.29
x2.5 ResNet 75.98
x2.5 EfV2. 76.24
x2.5 SegFormer 76.57
x2.5 Swin. 76.78
Round2 x4 SegFormer 78.38
Swin. 77.98
Round3 x4 EfV2. 79.37
SegFormer 79.91
Round4 x4
EfV2. 80.06
HRNet-OCR 80.17
SegFormer 80.11
Swin. 80.34
Ensemble 80.41
4. Conclusion
By analyzing the Landslide4Sense dataset, we conclude
two challenges in landslide detection including (1) small
objects and class imbalance; (2) distribution inconsis-
tency. Hence, the progressive label renement-based dis-
Table 2
Results on the test leaderboard
Rfinement Model F1(%)
Baseline Ensemble 73.07
Round1 Ensemble 73.62
Round2 Ensemble 74.03
Round3 Ensemble 74.54
tribution adaptation for landslide detection framework
was proposed. Through multiple rounds of pseudo-label
optimization and separately normalization, the perfor-
mance of the model continues to improve. Our solu-
tion ranked rst place on the Landslide4Sense challenge.
In the future, we will extend the framework into multi-
temporal images for landslide monitoring.
Thanks to the Institute of Advanced Research in Articial
Intelligence for organizing this competition. This work
was supported by National Natural Science Foundation
of China under Grant No.42071350, No.42171336 and
No.42101327, in part by the Fundamental Research Funds
for the Central Universities under Grant 2042021kf0070,
and LIESMARS Special Research Funding.
P. Lima, S. Steger, T. Glade, N. Tilch, L. Schwarz,
A. Kociu, Landslide susceptibility mapping at na-
tional scale: A rst attempt for austria, in: M. Mikos,
B. Tiwari, Y. Yin, K. Sassa (Eds.), Advancing Culture
of Living with Landslides, Springer International
Publishing, Cham, 2017, pp. 943–951.
H. Zhao, Y. Zhong, X. Wang, X. Hu, C. Luo, M. Boitt,
R. Piiroinen, L. Zhang, J. Heiskanen, P. Pellikka,
Mapping the distribution of invasive tree species
using deep one-class classication in the tropical
montane landscape of kenya, ISPRS Journal of Pho-
togrammetry and Remote Sensing 187 (2022) 328–
344. URL:
article/pii/S0924271622000715. doi:
J. Wang, A. Ma, Y. Zhong, Z. Zheng, L. Zhang,
Cross-sensor domain adaptation for high spatial
resolution urban land-cover mapping: From
airborne to spaceborne imagery, Remote
Sensing of Environment 277 (2022) 113058.
article/pii/S0034425722001729. doi:
S. Tavakkoli Piralilou, H. Shahabi, B. Jari-
hani, O. Ghorbanzadeh, T. Blaschke, K. Gho-
lamnia, S. R. Meena, J. Aryal, Landslide de-
tection using multi-scale image segmentation
and dierent machine learning models in the
higher himalayas, Remote Sensing 11 (2019).
O. Ghorbanzadeh, Y. Xu, P. Ghamisi, M. Kopp,
D. Kreil, Landslide4Sense: Reference Benchmark
Data and Deep Learning Models for Landslide De-
tection, arXiv e-prints (2022) arXiv:2206.00515.
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin,
B. Guo, Swin transformer: Hierarchical vision trans-
former using shifted windows, in: Proceedings of
the IEEE/CVF International Conference on Com-
puter Vision, 2021, pp. 10012–10022.
M. Tan, Q. Le, Ecientnetv2: Smaller models and
faster training, in: International Conference on
Machine Learning, PMLR, 2021, pp. 10096–10106.
E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Al-
varez, P. Luo, Segformer: Simple and ecient de-
sign for semantic segmentation with transformers,
Advances in Neural Information Processing Sys-
tems 34 (2021).
M. Berman, A. R. Triki, M. B. Blaschko, The lovász-
softmax loss: A tractable surrogate for the opti-
mization of the intersection-over-union measure in
neural networks, in: Proceedings of the IEEE con-
ference on computer vision and pattern recognition,
2018, pp. 4413–4421.
C. Szegedy, V. Vanhoucke, S. Ioe, J. Shlens, Z. Wo-
jna, Rethinking the Inception Architecture for Com-
puter Vision, arXiv e-prints (2015) arXiv:1512.00567.
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Some invasive tree species threaten biodiversity and cause irreversible damage to global ecosystems. The key to controlling and monitoring the propagation of invasive tree species is to detect their occurrence as early as possible. In this regard, one-class classification (OCC) shows potential in forest areas with abundant species richness since it only requires a few positive samples of the invasive tree species to be mapped, instead of all the species. However, the classical OCC method in remote sensing is heavily dependent on manually designed features, which have a limited ability in areas with complex species distributions. Deep learning based tree species classification methods mostly focus on multi-class classification, and there have been few studies of the deep OCC of tree species. In this paper, a deep positive and unlabeled learning based OCC framework -ITreeDet-is proposed for identifying the invasive tree species of Eucalyptus spp. (eucalyptus) and Acacia mearnsii (black wattle) in the Taita Hills of southern Kenya. In the ITreeDet framework, an absNegative risk estimator is designed to train a robust deep OCC model by fully using the massive unlabeled data. Compared with the state-of-the-art OCC methods, ITreeDet represents a great improvement in detection accuracy, and the F1-score was 0.86 and 0.70 for eucalyptus and black wattle, respectively. The study area covers 100 km 2 of the Taita Hills, where, according to our findings, the total area of eucalyptus and black wattle is 1.61 km 2 and 3.24 km 2 , respectively, which represent 6.78% and 13.65% of the area covered by trees and forest. In addition, both invasive tree species are located in the higher elevations, and the extensive spread of black wattle around the study area confirms its invasive tendency. The maps generated by the use of the proposed algorithm will help local government to develop management strategies for these two invasive species.
Full-text available
Landslides represent a severe hazard in many areas of the world. Accurate landslide maps are needed to document the occurrence and extent of landslides and to investigate their distribution, types, and the pattern of slope failures. Landslide maps are also crucial for determining landslide susceptibility and risk. Satellite data have been widely used for such investigations-next to data from airborne or unmanned aerial vehicle (UAV)-borne campaigns and Digital Elevation Models (DEMs). We have developed a methodology that incorporates object-based image analysis (OBIA) with three machine learning (ML) methods, namely, the multilayer perceptron neural network (MLP-NN) and random forest (RF), for landslide detection. We identified the optimal scale parameters (SP) and used them for multi-scale segmentation and further analysis. We evaluated the resulting objects using the object pureness index (OPI), object matching index (OMI), and object fitness index (OFI) measures. We then applied two different methods to optimize the landslide detection task: (a) an ensemble method of stacking that combines the different ML methods for improving the performance, and (b) Dempster-Shafer theory (DST), to combine the multi-scale segmentation and classification results. Through the combination of three ML methods and the multi-scale approach, the framework enhanced landslide detection when it was tested for detecting earthquake-triggered landslides in Rasuwa district, Nepal. PlanetScope optical satellite images and a DEM were used, along with the derived landslide conditioning factors. Different accuracy assessment measures were used to compare the results against a field-based landslide inventory. All ML methods yielded the highest overall accuracies ranging from 83.3% to 87.2% when using objects with the optimal SP compared to other SPs. However, applying DST to combine the multi-scale results of each ML method significantly increased the overall accuracies to almost 90%. Overall, the integration of OBIA with ML methods resulted in appropriate landslide detections, but using the optimal SP and ML method is crucial for success.
Conference Paper
Full-text available
Numerous publications that addressing landslide susceptibility were published over the past decades, also due to an increasing demand of spatial information regarding potentially endangered areas. However, studies that provide an overview on landslide susceptibility at national scale are still scarce. This research presents a first attempt to generate a national scale landslide susceptibility map for Austria based on statistical techniques. Binary logistic regression has been applied to delineate susceptible areas using three different predictor sets. The initial predictor set relates to topographic variables only (model A), and was gradually expanded with the factors geology (model B) and land cover (model C). The Area Under the Receiver Operating Characteristic Curve (AUROC) was used to validate the predictions by means of a k-fold cross-validation. The obtained acceptable prediction performances (mean AUROC of model A: 0.76, B: 0.81 and C: 0.82) suggest a relatively high predictive performance of all models. However, during this study, several limitations of the conducted analysis (e.g. limited landslide data, bias propagation, overoptimistic performance estimates) became evident. The main drawbacks and further steps towards a more reliable representation of landslide susceptibility at national scale are discussed.
Conference Paper
We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perceptron (MLP) decoders. SegFormer has two appealing features: 1) SegFormer comprises a novel hierarchically structured Transformer encoder which outputs multiscale features. It does not need positional encoding, thereby avoiding the interpolation of positional codes which leads to decreased performance when the testing resolution differs from training. 2) SegFormer avoids complex decoders. The proposed MLP decoder aggregates information from different layers, and thus combining both local attention and global attention to render powerful representations. We show that this simple and lightweight design is the key to efficient segmentation on Transformers. We scale our approach up to obtain a series of models from SegFormer-B0 to Segformer-B5, which reaches much better performance and efficiency than previous counterparts.For example, SegFormer-B4 achieves 50.3% mIoU on ADE20K with 64M parameters, being 5x smaller and 2.2% better than the previous best method. Our best model, SegFormer-B5, achieves 84.0% mIoU on Cityscapes validation set and shows excellent zero-shot robustness on Cityscapes-C. Code is available at:
Urban land-cover information is essential for resource allocation and sustainable urban development. Recently, deep learning algorithms have shown promising results in land-cover mapping with high spatial resolution (HSR) imagery. However, the limitation of the annotation and the divergence of the multi-sensor images always challenge the transferability of deep learning, thus hindering city-level or national-level mapping. In this paper, we propose a scheme to leverage small-scale airborne images with labels (source) for unlabeled large-scale spaceborne image (target) classification. Considering the sensor characteristics, a Cross-Sensor Land-cOVEr framework, called LoveCS, is introduced to address the difficulties of the spatial resolution inconsistency and spectral differences. As for the structural design, cross-sensor normalization is proposed to automatically learn sensor-specific normalization weights, thereby narrowing the spectral differences hierarchically. Furthermore, a dense multi-scale decoder is proposed to effectively fuse the multi-scale features from different sensors. As for the model optimization, self-training domain adaptation is adopted, and multi-scale pseudo-labeling is proposed to reduce the scale divergence brought by the spatial resolution inconsistency. The effectiveness of LoveCS was tested on data from the three cities of Nanjing, Changzhou, and Wuhan in China. The comprehensive results all show that LoveCS is superior to the existing domain adaptation methods in cross-sensor tasks, and has good generalizability. Compared with the existing land-cover products, the obtained results have the highest accuracy and spatial resolution (1.0 m). Overall, LoveCS provides a new perspective for large-scale land-cover mapping based on limited HSR images. The code is available at
  • O Ghorbanzadeh
  • Y Xu
  • P Ghamisi
  • M Kopp
  • D Kreil
O. Ghorbanzadeh, Y. Xu, P. Ghamisi, M. Kopp, D. Kreil, Landslide4Sense: Reference Benchmark Data and Deep Learning Models for Landslide Detection, arXiv e-prints (2022) arXiv:2206.00515. arXiv:2206.00515.
Efficientnetv2: Smaller models and faster training
  • M Tan
  • Q Le
M. Tan, Q. Le, Efficientnetv2: Smaller models and faster training, in: International Conference on Machine Learning, PMLR, 2021, pp. 10096-10106.
  • C Szegedy
  • V Vanhoucke
  • S Ioffe
  • J Shlens
  • Z Wojna
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the Inception Architecture for Computer Vision, arXiv e-prints (2015) arXiv:1512.00567. arXiv:1512.00567.