Xiao Xiang Zhu

Xiao Xiang Zhu
Technische Universität München | TUM

Prof. Dr.-Ing. habil.

About

754
Publications
220,579
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
17,610
Citations
Citations since 2017
572 Research Items
15978 Citations
201720182019202020212022202301,0002,0003,0004,000
201720182019202020212022202301,0002,0003,0004,000
201720182019202020212022202301,0002,0003,0004,000
201720182019202020212022202301,0002,0003,0004,000
Introduction
I am passionate about Space and Math. My research interests are Earth observation (EO), signal processing, machine learning and data science, with a special application focus on global urban mapping. Concrete recent topics are: - AI for EO - explorative model-based signal processing algorithms for EO - multisensory data fusion - Big Data Analytics – from knowledge discovery, HPC to geoscientific applications - harvesting unconventional geodata sources, e.g. social media and NewSpace.
Additional affiliations
April 2018 - present
German Aerospace Center
Position
  • Head of Department
July 2015 - present
Technische Universität München
Position
  • Professor
May 2015 - June 2015
The University of Tokyo
Position
  • Visiting Scientist

Publications

Publications (754)
Article
Solar power is a clean and renewable energy source. Promoting solar technology can not only offer all people affordable, reliable, and modern energy, but also mitigate energy-related emissions and pollutants. This significantly contributes to sustainable development goals. Aerial imagery can provide a cost-effective way for large-scale rooftop sola...
Article
Urban land use on a building instance level is crucial geo-information for many applications yet challenging to obtain. Steet-level images are highly suited to predict building functions as the building façades provide clear hints. Social media image platforms contain billions of images, including but not limited to street perspectives. This study...
Article
Full-text available
Although deep learning techniques have brought unprecedented accuracy to automatic building extraction, several main issues still constitute an obstacle to effective and practical applications. The industry is eager for higher accuracy and more flexible data usage. In this paper, we present a co-learning framework applicable to building extraction...
Article
Full-text available
High-resolution satellite images can provide abundant, detailed spatial information for land cover classification, which is particularly important for studying the complicated built environment. However, due to the complex land cover patterns, the costly training sample collections, and the severe distribution shifts of satellite imageries caused b...
Preprint
The domain adaptation (DA) approaches available to date are usually not well suited for practical DA scenarios of remote sensing image classification, since these methods (such as unsupervised DA) rely on rich prior knowledge about the relationship between label sets of source and target domains, and source data are often not accessible due to priv...
Article
Full-text available
In Earth observation, multimodal data fusion is an intuitive strategy to break the limitation of individual data. Complementary physical contents of data sources allow comprehensive and precise information retrieval. With current satellite missions, such as ESA Copernicus programme, various data will be accessible at an affordable cost. Future appl...
Article
Full-text available
Remote sensing (RS) image scene classification has obtained increasing attention for its broad application prospects. Conventional fully-supervised approaches usually require a large amount of manually-labeled data. As more and more RS images becoming available, how to make full use of these unlabeled data is becoming an urgent topic. Semi-supervis...
Conference Paper
Quantum machine learning is the synergy between quantum computing resources and machine learning methods. In particular, quantum machine learning refers to quantum algorithms promising to compute some machine learning methods and optimization problems (polynomially) faster than conventional algorithms. Quantum algorithms for computing any problems...
Article
As unconventional sources of geo-information, massive imagery and text messages from open platforms and social media form a temporally quasi-seamless, spatially multiperspective stream, but with unknown and diverse quality. Due to its complementarity to remote sensing data, geo-information from these sources offers promising perspectives, but harve...
Conference Paper
Automatic label generation systems, which are capable to generate huge amounts of labels with limited human efforts, enjoy lots of potential in the deep learning era. These easy-to-come-by labels inevitably bear label noises due to a lack of human supervision and can bias model training to some inferior solutions. However, models can still learn so...
Conference Paper
Discovering ancient agricultural terraces in desert regions is important for the monitoring of long-term climate changes on the Earth's surface. However, traditional ground surveys are both costly and limited in scale. With the increasing accessibility of aerial and satellite data, machine learning techniques bear large potential for the automatic...
Article
Full-text available
Many current deep learning approaches make extensive use of backbone networks pre-trained on large datasets like ImageNet, which are then fine-tuned to perform a certain task. In remote sensing, the lack of comparable large annotated datasets and the wide diversity of sensing platforms impedes similar developments. In order to contribute towards th...
Conference Paper
Surface melt on the Antarctic Ice Sheet is an important climate indicator, yet the spatial scale of modeling and observing surface melt is insufficient to capture crucial details and understand local processes. High-resolution climate models could provide a solution, but they are computationally expensive and require finetuning for some model param...
Article
Full-text available
Finding sparse solutions of underdetermined linear systems commonly requires the solving of L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularized least squares minimization problem, which is also known as the basis pursuit denoising (BPDN). They are computationally expensive since the...
Article
Full-text available
Obtaining a dynamic population distribution is key to many decision-making processes such as urban planning, disaster management and most importantly helping the government to better allocate socio-technical supply. For the aspiration of these objectives, good population data is essential. The traditional method of collecting population data throug...
Preprint
Self-supervised pre-training bears potential to generate expressive representations without human annotation. Most pre-training in Earth observation (EO) are based on ImageNet or medium-size, labeled remote sensing (RS) datasets. We share an unlabeled RS dataset SSL4EO-S12 (Self-Supervised Learning for Earth Observation - Sentinel-1/2) to assemble...
Article
Full-text available
The scientific outcomes of the 2022 Landslide4Sense (L4S) competition organized by the Institute of Advanced Research in Artificial Intelligence (IARAI) are presented here. The objective of the competition is to automatically detect landslides based on large-scale multiple sources of satellite imagery collected globally. The 2022 L4S aims to foster...
Preprint
Full-text available
As unconventional sources of geo-information, massive imagery and text messages from open platforms and social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality. Due to its complementarity to remote sensing data, geo-information from these sources offers promising perspectives, but harv...
Article
Full-text available
Ethics is a central and growing concern in all applications utilizing Artificial Intelligence (AI). Earth Observation (EO) or Remote Sensing (RS) research relies heavily on both Big Data and AI or Machine Learning (ML). While this reliance is not new, with increasing image resolutions and the growing number of EO/RS use-cases that have a direct imp...
Preprint
Full-text available
Automated crop-type classification using Sentinel-2 satellite time series is essential to support agriculture monitoring. Recently, deep learning models based on transformer encoders became a promising approach for crop-type classification. Using explainable machine learning to reveal the inner workings of these models is an important step towards...
Article
Full-text available
Many of the laminar-turbulent flow localisation techniques are strongly dependent upon expert control even-though determining the flow distribution is the prerequisite for analysing the efficiency of wing & stabiliser design in aeronautics. Some recent efforts have dealt with the automatic localisation of laminar-turbulent flow but they are still i...
Preprint
Full-text available
Earth observation, aiming at monitoring the state of planet Earth using remote sensing data, is critical for improving our daily lives and living environment. With an increasing number of satellites in orbit, more and more datasets with diverse sensors and research domains are published to facilitate the research of the remote sensing community. In...
Article
The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover. However, the large domain gap between optical and SAR images as well as the severe speckle noise of SAR images may cause significant interference in SAR-based cloud removal, resulting in performance degener...
Chapter
Training semantic segmentation models with few annotated samples has great potential in various real-world applications. For the few-shot segmentation task, the main challenge is how to accurately measure the semantic correspondence between the support and query samples with limited training data. To address this problem, we propose to aggregate th...
Preprint
Full-text available
Unmanned aerial vehicles (UAVs) are widely applied for purposes of inspection, search, and rescue operations by the virtue of low-cost, large-coverage, real-time, and high-resolution data acquisition capacities. Massive volumes of aerial videos are produced in these processes, in which normal events often account for an overwhelming proportion. It...
Preprint
Full-text available
Unmanned aerial vehicles (UAVs) are now widely applied to data acquisition due to its low cost and fast mobility. With the increasing volume of aerial videos, the demand for automatically parsing these videos is surging. To achieve this, current researches mainly focus on extracting a holistic feature with convolutions along both spatial and tempor...
Article
An accurate parameterization of glacier calving is essential for understanding glacier dynamics and constraining ice-sheet models. The increasing availability and quality of remote sensing imagery opens the prospect of a continuous and precise mapping of relevant parameters such as calving front locations. However, it also calls for automated and s...
Chapter
Multi-Target Domain Adaptation (MTDA) is a recently popular powerful setting in which a single classifier is learned for multiple unlabeled target domains. A popular MTDA approach is to sequentially adapt one target domain at a time. While only one pass is made through each target domain, the adaptation process for each target domain may consist of...
Preprint
Full-text available
The scientific outcomes of the 2022 Landslide4Sense (L4S) competition organized by the Institute of Advanced Research in Artificial Intelligence (IARAI) are presented here. The objective of the competition is to automatically detect landslides based on large-scale multiple sources of satellite imagery collected globally. The 2022 L4S aims to foster...
Article
Super-resolution is an essential task in remote sensing. It can enhance low-resolution remote sensing images and benefit downstream tasks such as building extraction and small object detection. However, existing remote sensing image super-resolution methods may fail in many real-world scenarios because they are trained on synthetic data generated...
Preprint
High-resolution satellite images can provide abundant, detailed spatial information for land cover classification, which is particularly important for studying the complicated built environment. However, due to the complex land cover patterns, the costly training sample collections, and the severe distribution shifts of satellite imageries, few stu...
Preprint
Full-text available
In Earth observation, multimodal data fusion is an intuitive strategy to break the limitation of individual data. Complementary physical contents of data sources allow comprehensive and precise information retrieve. With current satellite missions, such as ESA Copernicus programme, various data will be accessible at an affordable cost. Future appli...
Preprint
Full-text available
Hyperparameter optimization (HPO) is a well-studied research field. However, the effects and interactions of the components in an HPO pipeline are not yet well investigated. Then, we ask ourselves: can the landscape of HPO be biased by the pipeline used to evaluate individual configurations? To address this question, we proposed to analyze the effe...
Article
Full-text available
Buildings are the predominant objects that characterize the urban structure. For many cities, local governments establish building databases for administration as well as urban planning and monitoring. However, newly constructed buildings are often only included with a considerable time delay in the official digital cadastral maps due to processes...
Article
Full-text available
Unmanned aerial vehicles (UAVs) are widely applied for purposes of inspection, search, and rescue operations by the virtue of low-cost, large-coverage, real-time, and high-resolution data acquisition capacities. Massive volumes of aerial videos are produced in these processes, in which normal events often account for an overwhelming proportion. It...
Article
Full-text available
High-resolution remote sensing images are now available with the progress of remote sensing technology. With respect to popular remote sensing tasks, such as scene classification, image captioning provides comprehensible information about such images by summarizing the image content in human-readable text. Most existing remote sensing image caption...
Article
Full-text available
Change detection is a long-standing and challenging problem in remote sensing. Very often, features about changes are difficult to model beforehand, thus making the collection of changed samples a challenging task. In comparison, it is much easier to collect numerous no-change samples. It is possible to define a change detection approach by using o...
Preprint
Full-text available
Training semantic segmentation models with few annotated samples has great potential in various real-world applications. For the few-shot segmentation task, the main challenge is how to accurately measure the semantic correspondence between the support and query samples with limited training data. To address this problem, we propose to aggregate th...
Conference Paper
In order to estimate tree biomass, allometric equations take tree parameters such as tree height, wood density, circumference of trunk, and crown diameter as input parameters. Given that most of these quantities are challenging to be extracted from remote sensing data, we evaluate the option to approximate biomass by tree height only. We study our...
Conference Paper
With improvement in the processing of synthetic aperture radar interferometry (InSAR) data, the detection of long-term volcanic deformations becomes possible. While deep learning (DL) models are considered black-box models, challenging to debug, the advances in explainable AI (XAI) help understand the model and how it makes decisions. In this paper...
Conference Paper
Self-supervised learning (SSL) has attracted much interest in remote sensing and Earth observation due to its ability to learn task-agnostic representations without human annotation. While most of the existing SSL works in remote sensing utilize ConvNet backbones and focus on a single modality, we explore the potential of vision transformers (ViTs)...
Conference Paper
Buildings can be distinguished by their form or function and maps of building types can be used by authorities for city planning. Training models to perform this classification re- quires appropriate training data. OpenStreetMap (OSM) data is globaly available and partly provides information on build- ing types. However, this data can be incomplete...
Conference Paper
Full-text available
A significant leap forward in the performance of remote sensing models can be attributed to recent advances in machine and deep learning. Large data sets particularly benefit from deep learning models, which often comprise millions of parameters. On which part of the data a machine learner focuses on during learning, however, remains an open resear...
Conference Paper
Due to the rapid growth of earth observation (EO) data and the complexity of machine learning models, the high requirement on the computation power for EO data analysis becomes a bottleneck. Exploiting quantum computing might tackle this challenge in the future. In this paper, we present a hybrid quantum-classical convolutional neural network (QC-C...
Conference Paper
We present and evaluate a weakly-supervised methodology to quantify the spatio-temporal distribution of urban forests based on remotely sensed data with close-to-zero human interaction. Successfully training machine learning models for semantic segmentation typically depends on the availability of high-quality labels. We evaluate the benefit of hig...
Conference Paper
Existing domain adaptation (DA) approaches are usually not well suited for practical DA scenarios of remote sensing image classification, since these methods (such as unsupervised DA) rely on rich prior knowledge about the relationship between label sets of source and target domains, and source data are usually not accessible in many cases due to t...
Conference Paper
Estimating height from monocular remote sensing images is one of the most efficient ways for building large-scale 3D city models. However, existing deep learning based methods usually require a large amount of training data, which could be cost-consuming or even not possible to obtain. Towards a label-efficient deep learning model, we propose a new...
Conference Paper
In this paper, we compare trends in air quality measured by official stations on the ground with remote-sensing-based measurements in Munich, Germany. With Earth Observation data from Sentinel-5P and Sentinel-2, this study investigates the discrepancies that may arise in trends and pollutant levels of NO2. We find that the defined fixed measuring s...
Conference Paper
Deep learning has proven to be a very effective approach for Hyperspectral Image (HSI) classification. However, deep neural networks require large annotated datasets to generalize well. This limits the applicability of deep learning for HSI classification, where manually labelling thousands of pixels for every scene is impractical. In this paper, w...
Conference Paper
Out-of-distribution (OOD) detection is an emerging research topic in remote sensing where existing works focus on single sensor analysis. However, many remote sensing works use multi-modal data to benefit from different characteristics of the sensors. Data that is in-domain for one sensor may be OOD for another sensor. In this work, we address such...
Conference Paper
Distributional mismatch between training and test data may cause the remote sensing models to behave in unpredictable manner, thus reducing the trustworthiness of such models. Most existing methods for out-of-distribution (OOD) detection rely on availability of OOD samples during training. However, access to OOD data during training is counter intu...
Conference Paper
Distribution shift may pose significant challenges in Earth observation, especially when dealing with significantly differ-ent sensors like multispectral optical and Synthetic Aperture Radar (SAR). Deep learning models trained for optical image classification generally do not generalize well for SAR images. This is due to very marked differences be...
Conference Paper
We present a deep active contour model for detecting and delineating glacier calving fronts from satellite imagery. Contrary to existing deep learning-based calving front detectors, our model does not perform an intermediate segmentation or pixel-wise edge detection, but instead directly predicts the contour parametrized by a fixed number of vertic...
Conference Paper
In this work we analyse how training datasize affects the ability of a deep neural network to deal with noisy training labels in a semantic segmentation task with labels from OpenStreetMap. To this end, several versions of the training set were created by introducing varying amounts of label noise, and a model was then trained on subsets of varying...
Conference Paper
To achieve super-resolution synthetic aperture radar (SAR) tomography (TomoSAR), compressive sensing (CS)-based algorithms are usually employed, which are, however, computationally expensive, and thus is not often applied in large-scale processing. Recently, deep unfolding techniques have provided a good combination of physical model-based algorith...
Conference Paper
For global range satellite imaging mission, images captured from different areas may have large distribution biases due to different illuminations, shooting angles and atmospheric conditions. A straightforward idea to mitigate this problem is to categorize the images into different domains according the cities they belong to, and apply domain adapt...
Conference Paper
Recent developments in deep learning have pushed the capabilities of pixel-wise change detection. This work introduces the winning solution of the DynamicEarthNet WeaklySupervised Multi-Class Change Detection Challenge held at the EARTHVISION Workshop in CVPR 2021. The proposed approach is a pixel-wise change detection network coined Siamese Attent...
Conference Paper
On average, about half of all optical satellite data observing Earth is covered by haze or clouds. These atmospheric disturbances hinder the ongoing observation of our planet and prevent the seamless application of established remote sensing methods. Accordingly, to allow for an ongoing monitoring of Earth, approaches to reconstruct optical space-b...
Conference Paper
Building footprint maps are important to urban planning and monitoring. However, most existing approaches that fall back on convolutional neural networks (CNNs), require massive annotated samples for network learning. In this research, we propose a novel semi-supervised network, which can help to deal with this issue by leveraging a large amount of...
Article
Building footprints are essential for understanding urban dynamics. Planet satellite imagery with daily repetition frequency and high resolution has opened new opportunities for building mapping at large scales. However, suitable building mapping methods are scarce for less developed regions, as these regions lack massive annotated samples to provi...
Conference Paper
Full-text available
Haze and clouds in Earth's atmosphere obstruct a seamless monitoring of our planet via optical satellites. Prior work shows that models can learn to adapt and perform remote sensing downstream tasks even in the presence of such sensor noise. So what are the auxiliary benefits of incorporating an explicit cloud removal task, and what is its relation...
Article
Full-text available
Deep Learning usually requires large amounts of labeled training data. In remote sensing, deep learning is often applied for land cover and land use classification as well as street network and building segmentation. In case of the latter, a common way of obtaining training labels is to leverage crowdsourced datasets which can provide numerous type...
Conference Paper
TanDEM-X mission delivers a global digital elevation model of high quality (12 m posting) close to the HRTI-3 standard. However, when it comes to urban areas, layover effect caused by the side-looking nature of the radar satellites handicaps the use of classical multi-baseline phase unwrapping and TanDEM-X data alone for a precise 3D reconstruction...
Article
Forest biomass is a key influence for future climate, and the world urgently needs highly scalable financing schemes, such as carbon offsetting certifications, to protect and restore forests. Current manual forest carbon stock inventory methods of measuring single trees by hand are time, labour, and cost intensive and have been shown to be subjecti...
Preprint
In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities. While there has been a big success in computer vision, most of the potential of SSL in the domain of earth observation remains locked. In this paper, we provide an introduction to...
Preprint
Deep learning has proven to be a very effective approach for Hyperspectral Image (HSI) classification. However, deep neural networks require large annotated datasets to generalize well. This limits the applicability of deep learning for HSI classification, where manually labelling thousands of pixels for every scene is impractical. In this paper, w...
Conference Paper
Change detection has been a hot research topic in the field of remote sensing, and it can provide information on observing changes of Earth's surface. However, segmentation-based change results are not very friendly to end users. Thus, in order to improve user experience and offer them high-level semantic information on change detection, we introdu...
Preprint
We present and evaluate a weakly-supervised methodology to quantify the spatio-temporal distribution of urban forests based on remotely sensed data with close-to-zero human interaction. Successfully training machine learning models for semantic segmentation typically depends on the availability of high-quality labels. We evaluate the benefit of hig...
Preprint
The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover. However, the large domain gap between optical and SAR images as well as the severe speckle noise of SAR images may cause significant interference in SAR-based cloud removal, resulting in performance degener...
Conference Paper
Earth observation provides a rich source of remotely sensed information for a variety of applications such as global land cover monitoring, environmental impact due to natural disasters, and transformation of urban spaces. Although advances in deep learning provide an agile vehicle to leverage petabytes of earth observation data, it remains a chall...
Article
The fusion of two or more different data sources is a widely accepted technique in remote sensing while becoming increasingly important due to the availability of big Earth Observation satellite data. As a complementary source of geo-information to satellite data, massive text messages from social media form a temporally quasi-seamless, spatially m...
Presentation
Funded by the Helmholtz Foundation, the aim of the Artificial Intelligence for COld REgions (AI-CORE) project is to develop methods of Artificial Intelligence for solving some of the most challenging questions in cryosphere research by the example of four use cases. These use cases are of high relevance in the context of climate change but very dif...
Conference Paper
Climate change is perhaps the biggest single threat to humankind and the environment, as it severely impacts our terrestrial surface, home to most of the living species. Inspired by video prediction and exploiting the availability of Copernicus Sentinel-2 images, recent studies have attempted to forecast the land surface evolution as a function of...
Conference Paper
With the rapid rise of neural architecture search, the ability to understand its complexity from the perspective of a search algorithm is desirable. Recently, Traoré et al. have proposed the framework of Fitness Landscape Footprint to help describe and compare neural architecture search problems. It attempts at describing why a search strategy migh...
Article
Full-text available
Owing to the presence of many sensors and geographic/seasonal variations, domain adaptation is an important topic in remote sensing. However, most domain adaptation methods focus on close-set adaptation, i.e., they assume that the source and target domains share the same label space. This assumption often does not hold in practice, as there can be...