Martin Weinmann

Karlsruhe Institute of Technology | KIT · Institute of Photogrammetry and Remote Sensing

With FaSS-MVS, we present a fast, surface-aware semi-global optimization approach for multi-view stereo that allows for rapid depth and normal map estimation from monocular aerial video data captured by *UAV. The data estimated by FaSS-MVS, in turn, facilitate online 3D mapping, meaning that a 3D map of the scene is immediately and incrementally ge...

Neural Radiance Fields (NeRFs) have become a rapidly growing research field with the potential to revolutionize typical photogrammetric workflows, such as those used for 3D scene reconstruction. As input, NeRFs require multi-view images with corresponding camera poses as well as the interior orientation. In the typical NeRF workflow, the camera pos...

In this paper, we focus on investigating the potential of advanced Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting for 3D scene reconstruction from aerial imagery obtained via sensor platforms with an almost nadir-looking camera. Such a setting for image acquisition is convenient for capturing large-scale urban scenes, yet it poses particu...

3D reconstruction is a long-standing research topic in the photogrammetric and computer vision communities; although a plethora of open-source and commercial solutions for 3D reconstruction have been released in the last few years, several open challenges and limitations still exist. Undoubtedly, deep learning algorithms have demonstrated great pot...

Detecting airborne dust in common RGB images is hard. Nevertheless, monitoring airborne dust can greatly contribute to climate protection, environmentally friendly construction, research, and numerous other domains. In order to develop an efficient and robust airborne dust monitoring algorithm, various challenges have to be overcome. Airborne dust...

Forests are irreplaceable and are being studied extensively. Better forest inventory and understanding necessitate effective mapping, modeling, and automatic analysis. As a result, considerable research effort is being devoted to digitizing forest environments. Recently, digital twins have come to the attention of the geospatial community as a virt...

The classification of airborne laser scanning (ALS) point clouds is a critical task of remote sensing and photogrammetry fields. Although recent deep learning-based methods have achieved satisfactory performance, they have ignored the unicity of the receptive field, which makes the ALS point cloud classification remain challenging for the distingui...

The classification of airborne laser scanning (ALS) point clouds is a critical task of remote sensing and photogrammetry fields. Although recent deep learning-based methods have achieved satisfactory performance, they have ignored the unicity of the receptive field, which makes the ALS point cloud classification remain challenging for the distingui...

In this paper, we present an automated method for classification of binary voxel occupancy grids of discretized indoor mapping data such as point clouds or triangle meshes according to normal vector directions. Filled voxels get assigned normal class labels distinguishing between horizontal and vertical building structures. The horizontal building...

Airborne Laser Scanning (ALS) point cloud classification is a valuable and practical task in the fields of photogrammetry and remote sensing. It takes an extremely important role in many applications of surveying, monitoring, planning, production and living. Recently, driven by the wave of deep learning, the classification of ALS point clouds has a...

Remote sensing techniques are frequently applied for the surveying of remote areas, where the use of conventional surveying techniques remains difficult and impracticable. In this paper, we focus on one of the remote glacier areas, namely the Tyndall Glacier area in the Southern Patagonian Icefield in Chile. Based on optical remote sensing data in...

With the rapid development of deep learning, many deep learning-based approaches have made great achievements in object detection tasks. It is generally known that deep learning is a data-driven approach. Data directly impact the performance of object detectors to some extent. Although existing datasets include common objects in remote sensing imag...

With FaSS-MVS, we present an approach for fast multi-view stereo with surface-aware Semi-Global Matching that allows for rapid depth and normal map estimation from monocular aerial video data captured by UAVs. The data estimated by FaSS-MVS, in turn, facilitates online 3D mapping, meaning that a 3D map of the scene is immediately and incrementally...

Due to their great potential for a variety of applications, digital building models are well established in all phases of building projects. Older stock buildings however frequently lack digital representations, and creating these manually is a tedious and time-consuming endeavor. For this reason, the automated reconstruction of building models fro...

With the gaining popularity and proliferation of building information modeling (BIM) techniques, a growing demand emerges for accurate, up-to-date and semantically-enriched digital representations of built environments. In this regard, current mobile indoor mapping systems like the Microsoft HoloLens or Matterport allow to efficiently acquire trian...

The Microsoft HoloLens is a head-worn mobile augmented reality device. It allows a real-time 3D mapping of its direct environment and a self-localisation within the acquired 3D data. Both aspects are essential for robustly augmenting the local environment around the user with virtual contents and for the robust interaction of the user with virtual...

In this paper, we introduce the 2020 Gaofen Challenge and relevant scientific outcomes. The 2020 Gaofen Challenge is an international competition, which is organized by the China High-Resolution Earth Observation Conference Committee and the Aerospace Information Research Institute, Chinese Academy of Sciences and technically co-sponsored by the IE...

With the proposal of neural architecture search (NAS), automated network architecture design gradually becomes a new way in deep learning research. Due to its high capability regarding automated design, some pioneers have made an attempt to apply NAS in remote sensing and made some achievements, like 1-D/3-D Auto-convolutional neural network (CNN)...

In this paper, we present a novel pose normalization method for indoor mapping point clouds and triangle meshes that is robust against large fractions of the indoor mapping geometries deviating from an ideal Manhattan World structure. In the case of building structures that contain multiple Manhattan World systems, the dominant Manhattan World stru...

Real-time 3D reconstruction enables fast dense mapping of the environment which benefits numerous applications, such as navigation or live evaluation of an emergency. In contrast to most real-time capable approaches, our method does not need an explicit depth sensor. Instead, we only rely on a video stream from a camera and its intrinsic calibratio...

For ISPRS Technical Commission I (TC I), 76 submissions for the 2021 Congress edition of ISPRS Annals and ISPRS Archives were received. This included both full paper and abstract submissions from all over the world. Continuing the success of double blind paper reviewing in preparation of the 2016 Prague congress, the 2018 Karlsruhe symposium as wel...

For ISPRS Technical Commission I (TC I), 76 submissions for the 2021 Congress edition of ISPRS Annals and ISPRS Archives were received. This included both full paper and abstract submissions from all over the world. Continuing the success of double blind paper reviewing in preparation of the 2016 Prague congress, the 2018 Karlsruhe symposium as wel...

The field of Earth Observation (EO) and Geoinformation (GI) is gaining more and more importance due to the increasing number of data and data processing algorithms to respond even more accurately to a variety of challenges in many application areas. In order to follow recent activities and align the exponential evolution of datasets and recent proc...

With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded syste...

With the emergence of low-cost robotic systems, such as *UAV, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the re...

Fully automated interpretation and understanding of remotely sensed data by a computer has been a challenge for many decades, and many approaches have been developed over the years. Significant advances in knowledge-based image understanding, machine learning and artificial intelligence has led to this topic being the focus of much research in rece...

Classification of outdoor point clouds is an intensely studied topic, particularly with respect to the separation of vegetation from the terrain and manmade structures. In the presence of many overhanging and vertical structures, the (relative) height is no longer a reliable criterion for such a separation. An alternative would be to apply supervis...

Real-time 3D reconstruction enables fast dense mapping of the environment which benefits numerous applications, such as navigation or live evaluation of an emergency. In contrast to most real-time capable approaches, our approach does not need an explicit depth sensor. Instead, we only rely on a video stream from a camera and its intrinsic calibrat...

In this paper, we evaluate the possibility of using optical remote sensing techniques to survey remote glacier areas, where conventional surveying techniques are difficult to carry out. The extent of different spectral classes on the Tyndall glacier area in the Southern Patagonian icefield, Chile, is evaluated through classification with respect to...

With the rapid development of deep learning, many deep learning based approaches have made great achievements in object detection task. It is generally known that deep learning is a data-driven method. Data directly impact the performance of object detectors to some extent. Although existing datasets have included common objects in remote sensing i...

Supervised learning based methods for monocular depth estimation usually require large amounts of extensively annotated training data. In the case of aerial imagery, this ground truth is particularly difficult to acquire. Therefore, in this paper, we present a method for self-supervised learning for monocular depth estimation from aerial imagery th...

For ISPRS Technical Commission I (TC I), a remarkable number of 189 submissions for the 2020 Congress edition of ISPRS Annals and ISPRS Archives was received. This included both full paper and abstract submissions from all over the world. Encouraged by the success of double blind paper reviewing in preparation of the 2016 Prague congress and the 20...

Inspired by the application of state-of-the-art Fully Convolutional Networks (FCNs) for the semantic segmentation of high-resolution optical imagery, recent works transfer this methodology successfully to pixel-wise land use and land cover (LULC) classification of PolSAR data. So far, mainly single PolSAR images are included in the FCN-based classi...

Supervised learning based methods for monocular depth estimation usually require large amounts of extensively annotated training data. In the case of aerial imagery, this ground truth is particularly difficult to acquire. Therefore, in this paper, we present a method for self-supervised learning for monocular depth estimation from aerial imagery th...

Current mobile augmented reality devices are often equipped with range sensors. The Microsoft HoloLens for instance is equipped with a Time-of-Flight (ToF) range camera providing coarse triangle meshes that can be used in custom applications. We suggest to use these triangle meshes for the automatic generation of indoor models that can serve as bas...

Classification of urban materials using remote sensing data, in particular hyperspectral data, is common practice. Spectral libraries can be utilized to train a classifier since they provide spectral features about selected urban materials. However, urban materials can have similar spectral characteristic features due to high inter-class correlatio...

For ISPRS Technical Commission I (TC I), a remarkable number of 189 submissionsfor the 2020 Congress edition of ISPRS Annals and ISPRS Archives was received.This included both full paper and abstract submissions from all over the world.Encouraged by the success of double blind paper reviewing in preparation of the2016 Prague congress and the 2018 K...

3D indoor mapping and scene understanding have seen tremendous progress in recent years due to the rapid development of sensor systems, reconstruction techniques and semantic segmentation approaches. However, the quality of the acquired data strongly influences the accuracy of both reconstruction and segmentation. In this paper, we direct our atten...

Current mobile augmented reality devices are often equipped with range sensors. The Microsoft HoloLens for instance is equipped with a Time-Of-Flight (ToF) range camera providing coarse triangle meshes that can be used in custom applications. We suggest to use the triangle meshes for the automatic generation of indoor models that can serve as basis...

HyperMLPA is an interdisciplinary workshop which aims at bringing together people of different communities and disciplines involved in hyperspectral sensing, machine learning, and pattern analysis. People are invited to contribute in sensor development and calibration, to present and publish new datasets, to present innovative methodological advanc...

The Microsoft HoloLens is a head-worn mobile augmented reality device that is capable of mapping its direct environment in real-time as triangle meshes and localize itself within these three-dimensional meshes simultaneously. The device is equipped with a variety of sensors including four tracking cameras and a time-of-flight (ToF) range camera. Se...

Online augmentation of an oblique aerial image sequence with structural information is an essential aspect in the process of 3D scene interpretation and analysis. One key aspect in this is the efficient dense image matching and depth estimation. Here, the Semi-Global Matching (SGM) approach has proven to be one of the most widely used algorithms fo...

Online augmentation of an oblique aerial image sequence with structural information is an essential aspect in the process of 3D scene interpretation and analysis. One key aspect in this is the efficient dense image matching and depth estimation. Here, the Semi-Global Matching (SGM) approach has proven to be one of the most widely used algorithms fo...

Mobile Mapping is an efficient technology to acquire spatial data of the environment. The spatial data is fundamental for applications in crisis management, civil engineering or autonomous driving. The extrinsic calibration of the Mobile Mapping System is a decisive factor that affects the quality of the spatial data. Many existing extrinsic calibr...

The comparison of current image data with existing 3D model data of a scene provides an efficient method to keep models up to date. In order to transfer information between 2D and 3D data, a preliminary co-registration is necessary. In this paper, we present a concept to automatically co-register aerial imagery and untextured 3D model data. To refi...

In this paper, we address the semantic interpretation of urban environments on the basis of multi-modal data in the form of RGB color imagery, hyperspectral data and LiDAR data acquired from aerial sensor platforms. We extract radiometric features based on the given RGB color imagery and the given hyperspectral data, and we also consider different...

The comparison of current image data with existing 3D model data of a scene provides an efficient method to keep models up to date. In order to transfer information between 2D and 3D data, a preliminary co-registration is necessary. In this paper, we present a concept to automatically co-register aerial imagery and untextured 3D model data. To refi...

Mobile augmented reality devices like the Microsoft HoloLens are capable of simultaneously tracking the device location and mapping its environment in real-time. Thus, they offer potential for acquiring at least coarse point clouds and meshes of single rooms or even complete building structures that can be used in the context of building informatio...

In this paper, we investigate the potential of unsupervised feature selection techniques for classification tasks, where only sparse training data are available. This is motivated by the fact that unsupervised feature selection techniques combine the advantages of standard dimensionality reduction techniques (which only rely on the given feature ve...

The automated analysis of large areas with respect to land-cover and land-use is nowadays typically performed based on the use of hyperspectral or multispectral data acquired from airborne or spaceborne platforms. While hyperspectral data offer a more detailed description of the spectral properties of the Earth’s surface and thus a great potential...

In this paper, we address the semantic segmentation of aerial imagery based on the use of multi-modal data given in the form of true orthophotos and the corresponding Digital Surface Models (DSMs). We present the Deeply-supervised Shuffling Convolutional Neural Network (DSCNN) representing a multi-scale extension of the Shuffling Convolutional Neur...

Mobile augmented reality devices for indoor environments like the Microsoft HoloLens hold potential for the in-situ visualization of building model data. While the HoloLens has sufficient real-time inside-out tracking capacity to provide a spatially correct and stable visualization of virtual content relative to its surroundings, the placement of v...

The basic requirement for the successful deployment of a mobile augmented reality application is a reliable tracking system with high accuracy. Recently, a helmet-based inside-out tracking system which meets this demand has been proposed for self-localization in buildings. To realize an augmented reality application based on this tracking system, a...

With the technological advancements of aerial imagery and accurate 3d reconstruction of urban environments, more and more attention has been paid to the automated analyses of urban areas. In our work, we examine two important aspects that allow live analysis of building structures in city models given oblique aerial imagery, namely automatic buildi...

In this paper, we investigate the value of different modalities and their combination for the analysis of geospatial data of low spatial resolution. For this purpose, we present a framework that allows for the enrichment of geospatial data with additional semantics based on given color information, hyperspectral information, and shape information....

Since forest planning is increasingly taking an ecological, diversity-oriented perspective into account, remote sensing technologies are becoming ever more important in assessing existing resources with reduced manual effort. While the light detection and ranging (LiDAR) technology provides a good basis for predictions of tree height and biomass, t...

Classification of materials found in urban areas using remote sensing, in particular with hyperspectral data, has in recent times increased in importance. This study is conducting classification of materials found on building using hyperspectral data, by using an existing spectral library and collected data acquired with a spectrometer. Two commonl...

In this paper, we introduce a mathematical framework for obtaining spatially smooth semantic labelings of 3D point clouds from a pointwise classification. We argue that structured regularization offers a more versatile alternative to the standard graphical model approach. Indeed, our framework allows us to choose between a wide range of fidelity fu...

In this paper, we present a novel framework for the semantic labeling of airborne laser scanning data on a per-point basis. Our framework uses collections of spherical and cylindrical neighborhoods for deriving a multi-scale representation for each point of the point cloud. Additionally, spatial bins are used to approximate the topography of the co...

After scanning or reconstructing the geometry of objects, we need to inspect the result of our work. Are there any parts missing? Is every detail covered in the desired quality? We typically do this by looking at the resulting point clouds or meshes of our objects on-screen. What, if we could see the information directly visualized on the object it...

With the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In our work, we focus on algorithms that allow an online image-based dense depth estimation from video sequences, which enable...

In this paper, we focus on UAV-borne laser scanning with the objective of densely sampling object surfaces in the local surrounding of the UAV. In this regard, using a line scanner which scans along the vertical direction and perpendicular to the flight direction results in a point cloud with low point density if the UAV moves fast. Using a line sc...

In this paper, we focus on semantic point cloud classification taking into account standard failure cases reported in a variety of investigations. We present a hybrid two-step framework integrating classification, segmentation and semantic rules in a common end-to-end processing pipeline from irregularly distributed points to semantically labelled...