No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Smart Object Segmentation to Enhance the Creation of Interactive Environments
Abstract
The objective of our research is to enhance the creation of interactive environments such as in VR applications. An interactive environment can be produced from a point cloud that is acquired by a 3D scanning process of a certain scenery. The segmentation is needed to extract objects in that point cloud to, e.g., apply certain physical properties to them in a further step. It takes a lot of effort to do this manually as single objects have to be extracted and post-processed. Thus, our research aim is the real-world, cross-domain, automatic, semantic segmentation without the estimation of specific object classes.
ResearchGate has not been able to resolve any citations for this publication.
Semantic parsing of large-scale 3D point clouds is an important research topic in computer vision and remote sensing fields. Most existing approaches utilize hand-crafted features for each modality independently and combine them in a heuristic manner. They often fail to consider the consistency and complementary information among features adequately , which makes them difficult to capture high-level semantic structures. The features learned by most of the current deep learning methods can obtain high-quality image classification results. However, these methods are hard to be applied to recognize 3D point clouds due to unorganized distribution and various point density of data. In this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural network (RNN) for an efficient semantic parsing of large-scale 3D point clouds. In our method, an eye window under control of the 3D CNN and DQN can localize and segment the points of the object's class efficiently. The 3D CNN and Residual RNN further extract robust and discriminative features of the points in the eye window, and thus greatly enhance the parsing accuracy of large-scale point clouds. Our method provides an automatic process that maps the raw data to the classification results. It also integrates object localization, segmen-tation and classification into one framework. Experimental results demonstrate that the proposed method outperforms the state-of-the-art point cloud classification methods.
Today 3D models and point clouds are very popular being currently used in several fields, shared through the internet and even accessed
on mobile phones. Despite their broad availability, there is still a relevant need of methods, preferably automatic, to provide 3D data
with meaningful attributes that characterize and provide significance to the objects represented in 3D. Segmentation is the process of
grouping point clouds into multiple homogeneous regions with similar properties whereas classification is the step that labels these
regions. The main goal of this paper is to analyse the most popular methodologies and algorithms to segment and classify 3D point
clouds. Strong and weak points of the different solutions presented in literature or implemented in commercial software will be listed
and shortly explained. For some algorithms, the results of the segmentation and classification is shown using real examples at different
scale in the Cultural Heritage field. Finally, open issues and research topics will be discussed.
Unsupervised over-segmentation of an image into regions of perceptually similar pixels, known as super pixels, is a widely used preprocessing step in segmentation algorithms. Super pixel methods reduce the number of regions that must be considered later by more computationally expensive algorithms, with a minimal loss of information. Nevertheless, as some information is inevitably lost, it is vital that super pixels not cross object boundaries, as such errors will propagate through later steps. Existing methods make use of projected color or depth information, but do not consider three dimensional geometric relationships between observed data points which can be used to prevent super pixels from crossing regions of empty space. We propose a novel over-segmentation algorithm which uses voxel relationships to produce over-segmentations which are fully consistent with the spatial geometry of the scene in three dimensional, rather than projective, space. Enforcing the constraint that segmented regions must have spatial connectivity prevents label flow across semantic object boundaries which might otherwise be violated. Additionally, as the algorithm works directly in 3D space, observations from several calibrated RGB+D cameras can be segmented jointly. Experiments on a large data set of human annotated RGB+D images demonstrate a significant reduction in occurrence of clusters crossing object boundaries, while maintaining speeds comparable to state-of-the-art 2D methods.
Action Learning for 3D Point Cloud Based Organ Segmentation
- Xia Zhong
- Mario Amrehn
- Nishant Ravikumar
- Shuqing Chen
- Norbert Strobel
- Annette Birkhold
- Markus Kowarschik
- Rebecca Fahrig
- Andreas Maier
Xia Zhong, Mario Amrehn, Nishant Ravikumar, Shuqing Chen, Norbert Strobel, Annette Birkhold, Markus Kowarschik, Rebecca Fahrig,
and Andreas Maier, 'Action Learning for 3D Point Cloud Based Organ
Segmentation', Computing Research Repository (CoRR), (jun 2018).