No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Random Forest Ensembles and Extended Multi-Extinction Profiles for Hyperspectral Image Classification
... Therein, with access to label information, supervised HSI classification methods can leverage the grown community of machine learning classification algorithms to train pixel-wise classifiers. To be specific, some classical classifiers, such as nearest neighbour (NN) [17,18], support vector machine (SVM) [19,20], random forest (RF) [21][22][23][24], extreme learning machine (ELM) [25,26], sparse representation [27,28], and neural networks [29,30], have been applied to HSI classification and trained on spectral similarity features. Researchers exploit spatial information of HSI and design spatial-spectral features [31][32][33] to carry out HSI classification, which can fully utilize the structural information in HSIs and promote the accuracy of trained classifiers. ...
Due to the intricate terrain and restricted resources, hyperspectral image (HSI) datasets captured in real-world scenarios typically contain noisy labels, which may seriously affect the classification results. To address this issue, we work on a universal method that rectifies the labels first and then trains the classifier with corrected labels. In this study, we relax the common assumption that all training data are potentially corrupted and instead posit the presence of a small set of reliable data points within the training set. Under this framework, we propose a novel label-correction method named adaptive selective loss propagation algorithm (ASLPA). Firstly, the spectral–spatial information is extracted from the hyperspectral image and used to construct the inter-pixel transition probability matrix. Secondly, we construct the trusted set with the known clean data and estimate the proportion of accurate labels within the untrusted set. Then, we enlarge the trusted set according to the estimated proportion and identify an adaptive number of samples with lower loss values from the untrusted set to supplement the trusted set. Finally, we conduct label propagation based on the enlarged trusted set. This approach takes full advantage of label information from the trusted and untrusted sets, and moreover the exploitation on the untrusted set can adjust adaptively according to the estimated noise level. Experimental results on three widely used HSI datasets show that our proposed ASLPA method performs better than the state-of-the-art label-cleaning methods.
... In contrast, methods based on spectral classifiers, which are esteemed for their enhanced generalization abilities and nonlinear expression, have proven to be more effective in HSI classification. This genre includes sophisticated algorithms like k-nearest neighbour [10], random forest [11], and support vector machine (SVM) [12]. Nonetheless, owing to the prevalent high intra-class spectral variability and minimal inter-class spectral differentiation, spectral-based techniques often falter in accurately distinguishing between disparate entities and are vulnerable to unanticipated noise. ...
During the process of classifying Hyperspectral Image (HSI), every pixel sample is categorized under a land-cover type. CNN-based techniques for HSI classification have notably advanced the field by their adept feature representation capabilities. However, acquiring deep features remains a challenge for these CNN-based methods. In contrast, transformer models are adept at extracting high-level semantic features, offering a complementary strength. This paper's main contribution is the introduction of an HSI classification model that includes two convolutional blocks, a Gate-Shift-Fuse (GSF) block and a transformer block. This model leverages the strengths of CNNs in local feature extraction and transformers in long-range context modelling. The GSF block is designed to strengthen the extraction of local and global spatial-spectral features. An effective attention mechanism module is also proposed to enhance the extraction of information from HSI cubes. The proposed method is evaluated on four well-known datasets (the Indian Pines, Pavia University, WHU-WHU-Hi-LongKou and WHU-Hi-HanChuan), demonstrating that the proposed framework achieves superior results compared to other models.
... This allowed us to work with a smaller number of bands, improving the classification results. The final classification was performed using support vector machine (SVM) (Gualtieri et al., 1999;Gualtieri and Chettri., 2000) and random forest (Ham et al., 2005;Xia et al., 2018) techniques, with a majority filter applied to the results to remove erratic pixels followed by a final manual editing. ...
Human occupation of coastal strips generates pressures that can trigger severe degradation processes in marine ecosystems and threaten human livelihoods and the sustainability of the socio-ecological system. In this context, monitoring the conservation status of algal and benthic plant communities, as the basis of coastal ecosystems, is essential for the sustainability of coastal environments. The paper presents a bionomic investigation in a shallow water beach-reef coastal system called Las Canteras, in the Canary Islands (Spain). The work aims to generate a high-resolution mapping, according to standardised rules, of the marine habitats of an area with high ecological value showing signs of severe degradation due to anthropogenic pressure. For this purpose, direct observation techniques were combined with hyperspectral remote sensing techniques. The bathymetric, sedimentological and biological data collected were combined to generate detailed mapping of 13 marine habitats according to the Spanish Inventory of Marine Habitats and Species (IEHEM by its initials in Spanish) classification system. Comparison of these results with previous studies since 1960 shows total regression of the most sensitive species of marine habitats in the Canary Islands, such as Cymodocea nodosa, as well as the decline of sandy substrates which are fundamental for maintenance of the local coastal socio-ecological system.
... Although manual feature extraction has significant effects and applications, it requires domain expertise and has poor universality. Machine learning tools including Support Vector Machines (SVM) [21], K Nearest Neighbors [22], Random Forests [23], and Logistic Regression [24], have shown efficacy in hyperspectral image classification. Concurrently, deep learning-based hyperspectral classification methods [25][26][27][28] are emerging as research trends, offering new insights and techniques for spectral data processing and analysis. ...
Insect recognition, crucial for agriculture and ecology studies, benefits from advancements in RGB image-based deep learning, yet still confronts accuracy challenges. To address this gap, the HI30 dataset is introduced, comprising 2115 hyperspectral images across 30 insect categories, which offers richer information than RGB data for enhancing classification accuracy. To effectively harness this dataset, this study presents the Two-Branch Self-Correlation Network (TBSCN), a novel approach that combines spectrum correlation and random patch correlation branches to exploit both spectral and spatial information. The effectiveness of the HI30 and TBSCN is demonstrated through comprehensive testing. Notably, while ImageNet-pre-trained networks adapted to hyperspectral data achieved an 81.32% accuracy, models developed from scratch with the HI30 dataset saw a substantial 9% increase in performance. Furthermore, applying TBSCN to hyperspectral data raised the accuracy to 93.96%. Extensive testing confirms the superiority of hyperspectral data and validates TBSCN’s efficacy and robustness, significantly advancing insect classification and demonstrating these tools’ potential to enhance precision and reliability.
... HSI classification attempts to assign labels for each pixel and obtains the category of different objects [5]. In the early stages, some classical machine learning models were proposed for HSI classification, such as k-means clustering [6], multinomial logistic regression (MLR) [7], random forest (RF) [8], and support vector machine (SVM) [9], et al., which extract the representative features and assign the categories with sufficient labeled samples [10]. However, these models are difficult to capture the correlation of spectral and spatial information and to distinguish the approximate features. ...
Hyperspectral image (HSI) classification aims to recognize categories of objects based on spectral–spatial features and has been used in a wide range of real-world application areas. Attention mechanisms are widely used in HSI classification for their ability to focus on important information in images automatically. However, due to the approximate spectral–spatial features in HSI, mainstream attention mechanisms are difficult to accurately distinguish the small difference, which limits the classification accuracy. To overcome this problem, a spectral–spatial-sensorial attention network (S³AN) with controllable factors is proposed to efficiently recognize different objects. Specifically, two controllable factors, dynamic exponential pooling (DE-Pooling) and adaptive convolution (Adapt-Conv), are designed to enlarge the difference in approximate features and enhance the attention weight interaction. Then, attention mechanisms with controllable factors are utilized to build the redundancy reduction module (RRM), feature learning module (FLM), and label prediction module (LPM) to process HSI spectral–spatial features. The RRM utilizes the spectral attention mechanism to select representative band combinations, and the FLM introduces the spatial attention mechanism to highlight important objects. Furthermore, the sensorial attention mechanism extracts location and category information in a pseudo label to guide the LPM for label prediction and avoid details from being ignored. Experimental results on three public HSI datasets show that the proposed method is able to accurately recognize different objects with an overall accuracy (OA) of 98.69%, 98.89%, and 97.56%, respectively.
... Hyperspectral image classification categorizes pixels or regions within a hyperspectral image into predefined classes or land cover categories. Meanwhile, the supervised machine learning methods support vector machine (SVM) [7] and random forest (RF) [8] have been widely used in the early stages of hyperspectral image analysis using texture and color features of the land covers. These methods rely on spectral signatures to discriminate between different classes of land cover. ...
Hyperspectral image classification remains challenging despite its potential due to the high dimensionality of the data and its limited spatial resolution. To address the limited data samples and less spatial resolution issues, this research paper presents a two-scale module-based CTNet (convolutional transformer network) for the enhancement of spatial and spectral features. In the first module, a virtual RGB image is created from the HSI dataset to improve the spatial features using a pre-trained ResNeXt model trained on natural images, whereas in the second module, PCA (principal component analysis) is applied to reduce the dimensions of the HSI data. After that, spectral features are improved using an EAVT (enhanced attention-based vision transformer). The EAVT contained a multiscale enhanced attention mechanism to capture the long-range correlation of the spectral features. Furthermore, a joint module with the fusion of spatial and spectral features is designed to generate an enhanced feature vector. Through comprehensive experiments, we demonstrate the performance and superiority of the proposed approach over state-of-the-art methods. We obtained AA (average accuracy) values of 97.87%, 97.46%, 98.25%, and 84.46% on the PU, PUC, SV, and Houston13 datasets, respectively.
... Recently, the field of remote sensing classification has widely embraced ensemble classification methods. Xia et al. [22] developed five random forest ensemble models, including bagging-based, boosting-based, random subspace (RS)-based, rotation-based, and boosted-based approaches. These approaches effectively enhanced the prediction accuracy of the random forest ensemble system. ...
The Czech Republic is one of the countries along the Belt and Road Initiative, and classifying land cover in the Czech Republic helps to understand the distribution of its forest resources, laying the foundation for forestry cooperation between China and the Czech Republic. This study aims to develop a practical approach for land cover classification in the Czech Republic, with the goal of efficiently acquiring spatial distribution information regarding its forest resources. This approach is based on multi-level feature extraction and selection, integrated with advanced machine learning or deep learning models. To accomplish this goal, the study concentrated on two typical experimental regions in the Czech Republic and conducted a series of classification experiments, using Sentinel-2 and DEM data in 2018 as the main data sources. Initially, this study extracted various features, including spectral, vegetation, and terrain features, from the study area, then assessed and selected key features based on their importance. Additionally, this study also explored multi-level spatial contextual features to improve classification performance. The extracted features include texture and morphological features, as well as deep semantic information learned by utilizing a deep learning model, 3D CNN. Finally, an AdaBoost ensemble learning model with the random forest as the base classifier is designed to produce land cover classification maps, thus obtaining the spatial distribution of forest resources. The experimental results demonstrate that feature optimization significantly enhances the extraction of high-quality features of surface objects, thereby improving classification performance. Specifically, morphological and texture features can effectively enhance the discriminability between different features of surface objects, thereby improving classification accuracy. Utilizing deep learning networks enables more efficient extraction of deep feature information, further enhancing classification accuracy. Moreover, employing an ensemble learning model effectively boosts the accuracy of the original classification results from different individual classifiers. Ultimately, the classification accuracy of the two experimental areas reaches 92.84% and 93.83%, respectively. The user accuracies for forests are 92.24% and 93.14%, while the producer accuracies are 97.71% and 97.02%. This study applies the proposed approach for nationwide classification in the Czech Republic, resulting in an overall classification accuracy of 90.98%, with forest user accuracy at 91.97% and producer accuracy at 96.2%. The results in this study demonstrate the feasibility of combining feature optimization with the 3D Convolutional Neural Network (3DCNN) model for land cover classification. This study can serve as a reference for research methods in deep learning for land cover classification, utilizing optimized features.
... The model has advantages over other models for being insensitive to high-dimensional features. It ranks and tunes parameters based on their importance (Xia et al. 2018). ...
The mountainous areas are vulnerable to climate change and may have many socio-economic and environmental implications. The changing pattern of meteorological variables has deleterious effects on natural resources and livelihood. This paper makes an attempt to analyse trend and forecast metrological variables in Nainital district of India. Monthly, seasonal, and annual trends in rainfall and temperature were examined by Modified Mann–Kendall during 1989–2019. The magnitude of trend in temperature and rainfall was determined using Sen's slope estimator. Ensemble machine learning model was utilized for forecasting the variables for the next 16 years (2020–2035). The effectiveness of the model was examined through statistical performance assessors. The results revealed a significant increasing trend in the rainfall (at the rate of 9.42 mm/year) during 1989–2019. Increasing trend in the mean, minimum, and maximum temperatures on an annual basis was observed in the district. A remarkable increase in the rainfall and temperature was forecasted during various seasons. The findings of the study may help the stakeholders in devising suitable adaptation measures to climate variability. The bagging approach has shown its effectiveness in forecasting meteorological variables. The other geographical regions may find the methodology effective for analyzing climate variability and lessening its impact.
... Compared with the above methods, spectral classifier-based methods usually have a stronger generalization and nonlinear expression capability when applied to the classification of HSIs. Algorithms falling into this type include k-nearest neighbor [15], [16], random forest [17], and support vector machine (SVM) [18], [19]. Because of the high intraclass spectral variability and low interclass spectral variability, spectral-based methods, however, tend to struggle to discriminate between the different objects and further respond to unexpected noise. ...
Deep learning-based hyperspectral images (HSIs) classification methods have made significant progress recently, catching the attention of academia and industry. However, the existing studies of HSIs classification mainly focus on the closed-set environment with the assumption that ground classes are fixed and known, ignoring the complexity and diversity of ground objects in the real world. As a result, the unknown classes will be forced into known classes. To solve this problem, we propose a novel spectral-spatial evidential learning network (SSEL) that combines an improved generative adversarial network (GAN) and evidential theory for open-set HSIs classification. First, a domain adaptation strategy is embedded into GAN to generate high-quality samples by reducing the distribution discrepancy between generated and real samples. Second, the discriminator is devised to extract spectral-spatial features and output multiclass evidence for closed-set classification and uncertainty estimation. A new classification function called evidence-based loss is designed for the discriminator to guide the evidence collection process. Additionally, a novel adversarial objective function is defined, where the discriminator loss is devised to predict real samples belonging to the true class and generated samples belonging to “none of the classes”. The generator loss is developed to generate samples consistent with the label category. Finally, the class and corresponding uncertainty can be calculated based on the collected evidence, and the appropriate open-set HSIs classification. Extensive experiments on three benchmark HSIs show that our proposed method achieves competitive performance on closed-set and open-set HSIs classification compared with existing state-of-the-art methods.
... Ensemble learning (Breiman, 1996;Galar et al., 2012;Huynh et al., 2016) combines different base models through ensemble strategies to achieve superior performance compared to individual base models. This approach has been proven to be effective and widely applied in image recognition (Chen et al., 2019;Xia et al., 2018;Yang et al., 2021c). This study proposed, for the first time, an ensemble learning method based on a conservative strategy and mainstream deep learning frameworks to automatically identify wildlife images. ...
... into a low-dimensional space, such as principal component analysis, 13 support vector machines (SVM), 14 and random forests. 15 The properties of HSIs constrain these methods, and their classification results could be better. HSIs possess both spectral properties and spatial dependence, which implies a joint representation of spectral and spatial features. ...
... Thus, it is difficult to accurately extract lithological spectral features in hyperspectral images containing many high-dimensional spectral channels. Compared with traditional feature selection methods, tree-based methods, such as random forest (RF) [38,39], gradient boosting decision tree (GBDT) [40,41], and Light Gradient Boosting Machine (Light GBM) [42,43], can extract the optimal feature system that suits the needs of the model in a more targeted way, and their decision-making process is more similar to human thinking, making the model easy to understand. Its dimensional reduction speed is fast, and it can process continuous and discrete data and solve the multi-output problem. ...
Lithology classification is important in mineral resource exploration, engineering geological exploration, and disaster monitoring. Traditional laboratory methods for the qualitative analysis of rocks are limited by sampling conditions and analytical techniques, resulting in high costs, low efficiency, and the inability to quickly obtain large-scale geological information. Hyperspectral remote sensing technology can classify and identify lithology using the spectral characteristics of rock, and is characterized by fast detection, large coverage area, and environmental friendliness, which provide the application potential for lithological mapping at a large regional scale. In this study, ZY1-02D hyperspectral images were used as data sources to construct a new two-layer extreme gradient boosting (XGBoost) lithology classification model based on the XGBoost decision tree and an improved greedy search algorithm. A total of 153 spectral bands of the preprocessed hyperspectral images were input into the first layer of the XGBoost model. Based on the tree traversal structural characteristics of the leaf nodes in the XGBoost model, three built-in XGBoost importance indexes were split and combined. The improved greedy search algorithm was used to extract the spectral band variables, which were imported into the second layer of the XGBoost model, and the bat algorithm was used to optimize the modeling parameters of XGBoost. The extraction model of rock classification information was constructed, and the classification map of regional surface rock types was drawn. Field verification was performed for the two-layer XGBoost rock classification model, and its accuracy and reliability were evaluated based on four indexes, namely, accuracy, precision, recall, and F1 score. The results showed that the two-layer XGBoost model had a good lithological classification effect, robustness, and adaptability to small sample datasets. Compared with the traditional machine learning model, the two-layer XGBoost model shows superior performance. The accuracy, precision, recall, and F1 score of the verification set were 0.8343, 0.8406, 0.8350, and 0.8157, respectively. The variable extraction ability of the constructed two-layer XGBoost model was significantly improved. Compared with traditional feature selection methods, the GREED-GFC method, when applied to the two-layer XGBoost model, contributes to more stable rock classification performance and higher lithology prediction accuracy, and the smallest number of extracted features. The lithological distribution information identified by the model was in good agreement with the lithology information verified in the field.
... In the early stages, how to extract informative spatial-spectral features has been explored in traditional HSI classifiers. Various techniques involving support vector machine (SVM) [6], K -nearest neighbor (KNN) [7], random forest (RF) [8], and their variants [9], [10], [11] are used directly to predict the label for each pixel. However, their lower generalization ability often results in unsatisfactory performance when the training set becomes larger and more complex. ...
The hyperspectral image (HSI) classification aims to assign each pixel to a land-cover category. It is receiving increasing attention from both industry and academia. The main challenge lies in capturing reliable and informative spatial and spectral dependencies concealed in the HSI for each class. To address the challenge, we propose a spatial–spectral 1DSwin (SS1DSwin) Transformer with groupwise feature tokenization for HSI classification. Specifically, we reveal local and hierarchical spatial–spectral relationships from two different perspectives. It mainly consists of a groupwise feature tokenization module (GFTM) and a 1DSwin Transformer with cross-block normalized connection module (TCNCM). For GFTM, we reorganize an image patch into overlapping cubes and further generate groupwise token embeddings with multihead self-attention (MSA) to learn the local spatial–spectral relationship along the spatial dimension. For TCNCM, we adopt the shifted windowing strategy when acquiring the hierarchical spatial–spectral relationship along the spectral dimension with 1-D window-based MSA (1DW-MSA) and 1-D shifted window-based MSA (1DSW-MSA) and leverage cross-block normalized connection (CNC) to adaptively fuse the feature maps from different blocks. In SS1DSwin, we apply these two modules in order and predict the class label for each pixel. To test the effectiveness of the proposed method, extensive experiments are conducted on four HSI datasets, and the results indicate that SS1DSwin outperforms several current state-of-the-art methods. The source code of the proposed method is available at
https://github.com/Minato252/SS1DSwin
.
... In the initial exploration of hyperspectral image classification, researchers primarily focused on the spectral information contained within these images, which can effectively capture and reflect the internal mechanisms and chemical composition of ground objects. Specifically, traditional classification methods have harnessed the abundance of bands in hyperspectral images to execute machine learning algorithms for classification purposes with great efficacy, including random forest [13], decision trees [14], support vector machine [15] and K-nearest neighbor [16] algorithms. Relying solely on spectral information, these methods are capable of performing simple classification without the need for feature extraction. ...
... Similarly, machine learning technology has been successfully applied to hyperspectral image classification tasks. Many algorithms for HSI classification have been developed in recent decades, including morphology morphology [4,5,34], k-means [13], Gaussian process [3], support vector machine [42], random forest [47], extreme machine learning [14] and so on. Bandos et al. [2] developed a regularized linear discriminant analysis, which converts the data subspace into a linear molecular space. ...
Convolutional Neural Network(CNN) has been widely employed in hyperspectral image(HSI) classification. However, CNN cannot attain the relative location relation of spatial information well, hindering the further improvement of classification performance. Capsule Network(CapsNet) has been presented recently and represents features by vectors, which enhances the ability to attain feature space information and identify relative positions, and makes up for the shortcomings of CNN. To further improve the classification performance of HSI using CapsNet under limited labeled samples, this article proposes a multi-scale residual capsule network(MR-CapsNet). The proposed method adopts extended multi-scale convolution blocks to fully extract spectral-spatial features. Subsequently, the features extracted by convolution kernels of different sizes are fused by pointwise convolution. The residual structure is used for splicing with the input data, preventing the problem of vanishing gradients and overfitting. Finally, the fused feature information is classified at the capsule layer through the dynamic routing mechanism. Comparative experiments were carried out on three public datasets of hyperspectral images. The experimental results indicate that the overall classification accuracy of the proposed method has a 4.13%, 2.98%, and 1.43% improvement over the recent DC-CapsNet on three datasets, respectively.
... The RF algorithm has been widely used in remote sensing image classification. Xia et al. [51] proposed a Bagging RF (B-RF), which achieved good results in the HRS image classification. ...
This research explores a new hyperspectral remote sensing processing method that combines remote sensing and ground data, and builds a model based on a novel 3D convolutional neural network and fusion data. The method can monitor and map changes in iron ore stopes. First, we used an unmanned aerial vehicle-borne hyperspectral imager to take a hyperspectral image of the iron ore stope; second, collected iron ore samples and then used a ground-based spectrometer to measure the spectral data of these samples; third, combined the hyperspectral remote sensing data with the ground data and then proposed a data augmentation method. Fourth, based on the 3D convolutional neural network and deep residual network, an iron ore stope classification model is proposed. Finally, the model is applied to monitor and map iron ore stopes. The experimental results show that the proposed method is effective, and the overall accuracy is 99.62% for the five-class classification problem. The method provides a quick, accurate, and low-cost way to monitor iron ore stopes.
... Junshi Xia và cộng sự (2017) (Xia et al., 2017) đã sử dụng rừng ngẫu nhiên để phân loại hình ảnh. Trong công trình này, nhóm tác giả đã thực hiện một phương pháp phân loại hình ảnh bằng rừng ngẫu nhiên mở rộng nhằm nâng cao hiệu suất phân loại hình ảnh. ...
Trong bài báo này, một phương pháp phân lớp hình ảnh dựa trên cấu trúc KD-Tree Random Forest được đề xuất nhằm thực hiện phân lớp ảnh bằng nhiều cấu trúc KD-Tree độc lập. Trong đó, mỗi cấu trúc KD-Tree được sử dụng phân lớp nhiều lần cho một ảnh đầu vào theo mô hình phân lớp đa tầng. Quá trình phân lớp ảnh dựa trên cấu trúc KD-Tree Random Forest thực hiện theo phương pháp xây dựng cấu trúc KD-Tree Random Forest và huấn luyện bộ vector phân lớp. Vì vậy các thuật toán phân lớp hình ảnh dựa trên cấu trúc KD-Tree Random Forest, huấn luyện bộ véc-tơ phân lớp và mô hình phân lớp ảnh được đề xuất. Dựa trên cơ sở lý thuyết này, thực nghiệm được xây dựng trên bộ ảnh Clatech256 và so sánh với các công trình khác cùng bộ dữ liệu để minh chứng tính khả thi của phương pháp đề xuất. Theo kết quả thực nghiệm cho thấy phương pháp của chúng tôi là hiệu quả và có thể áp dụng được cho các hệ phân loại hình ảnh thuộc các lĩnh vực khác nhau.
... The First phase is feature extraction [2]; we will use different methods such as histogram gradients, HOG [3], and convolution neural network CNN [4]. The second phase is the classification mode used as support vector machine SVM [5] and Random Forest RF ensemble learning methods [6]. In the third phase, we will use clustering methods such as structural similarity index matrix SSIM [7] to put output images into similar groups to begin the final step, which is to analyze and recognize the nature of objects in the images to be as input for automated recognition system used in applications such mentioned above for CCTV system [27] and other systems that required to detect and recognize the objects on images. ...
Feature extraction is the most vital step in image classification to produce high-quality and good content images for further analysis, image detection, segmentation, and object recognition. Using machine learning algorithms, profound learning like convolutional neural network CNN became necessary to train, classify, and recognize images and objects like humans. Combined feature extraction and machine learning classification to locate and identify objects on images can then be an input of automatic recognition systems ATR such as surveillance systems CCTV, to enhance these systems and reduce time and effort for object detection and recognition in images based on digital image processing techniques especially image segmentation that differentiate from computer vision approach. This article will use machine learning and deep learning algorithms to facilitate and achieve the study's objectives.
... The dataset consisted of common rust, leaf spot, northern leaf blight disease images, and healthy images. The whole training dataset consisted of 7308 images,1634 images for leaf spot, 1907 images for common rust,1908 images for northern leaf blight, and 1859 for healthy leaf images [21]. The whole testing dataset consisted of 1826 images, 407 images for leaf spot, 477 images for common rust, 477 images for northern leaf blight, and 465 for healthy leaf images. ...
The paper presents feature extraction methods and classification algorithms used to classify maize leaf disease images. From maize disease images, features are extracted and passed to the machine learning classification algorithm to identify the possible disease based on the features detected using the feature extraction method. The maize disease images used include images of common rust, leaf spot, and northern leaf blight and healthy images. An evaluation was done for the feature extraction method to see which feature extraction method performs best with image classification algorithms. Based on the evaluation, the outcomes revealed Histogram of Oriented Gradients performed best with classifiers compared to KAZE and Oriented FAST and rotated BRIEF. The random forest classifier emerged the best in terms of image classification, based on four performance metrics which are accuracy, precision, recall, and F1-score. The experimental outcome indicated that the random forest had 0.74 accuracy, 0.77 precision, 0.77 recall, and 0.75 F1-score.
Transformer-based methods have a great ability to model non-local interactions among spectral and spatial information, while the local features are easily ignored. Graph convolutional neural networks (GCNs) tend to do well in exploiting neighborhood vertex interactions based on their unique aggregation mechanism, while the ability to extract global information is limited. In this paper, we study to comprehensively utilize the advantages of transformer and graph convolution by combing the two structures into a unified Transformer (Graphormer) to construct both local and global interactions for HSI classification, and spatial-spectral features enhanced Graphormer framework (S
2
GFormer) is proposed. Specifically, a Follow Patch mechanism is first proposed to transform the pixel in HSI to patches while preserving the local spatial features and reducing the computation cost. Moreover, a patch-wise spectral embedding block is designed to extract the spectral features of the patch, in which a neighborhood convolution is inserted for comprehensive spectral information extraction. Finally, a multi-layer Graphormer encoder module is proposed to extract the representative spatial-spectral features from the patch for HSI classification. In our network, we jointly integrate the three aforementioned parts into a unified network, and each component benefits the other. The experimental results demonstrate its suitability for HSI classification when compared with other state-of-the-art classifiers, particularly in scenarios with very limited labeled samples. The code of S
2
GFormer will be made publicly available at https://github.com/DY-HYX.
Convolutional neural networks (CNNs) have shown their potential ability in extracting discriminative features for hyperspectral image classification. However, traditional deep learning methods using CNNs tend to overlook the influence of complex environmental factors. These factors contribute to an increase in intra-class variance and a decrease in inter-class variance, making it considerably more challenging to extract meaningful features. To overcome this problem, this work develops a novel deep intrinsic decomposition with adversarial learning, namely AdverDecom, for hyperspectral image classification to mitigate the negative impact of environmental factors on classification performance. First, we develop a generative network for hyperspectral image (HyperNet) to extract the environment-related features and category-related features from the image. Then, a discriminative network is constructed to distinguish different environmental categories. Finally, an environment-category joint learning loss is developed for adversarial learning to make the deep model learn discriminative features. Experiments are conducted over four commonly used real-world datasets and the comparison results show the superiority of the proposed method. The implementation of the proposed method could be accessed at https://github.com/shendu-sw/Adversarial_Learning_Intrinsic_Decomposition for the sake of reproducibility.
The pre-trained vision-language models (VLMs) have achieved outstanding performance in various visual tasks, primarily due to the knowledge they’ve acquired from massive image-text pairs. This enables VLMs to generalize to a wide range of downstream tasks. This paper presents the first attempt to adapt VLMs for the joint classification task of hyperspectral image (HSI) and LiDAR data, aiming to leverage the well-learned VLMs to extract more generalizable features from diverse remote sensing image sources. Initially, by utilizing a patch encoder, low-dimensional patches of HSI and LiDAR data are transformed into high-dimensional latent feature representations, meeting the dimensional requirements of VLMs for visual input data. Unlike traditional classifiers that rely on discrete class labels, VLM-based classification methods depend on continuous vectors, which can be derived from textual templates with class names,
i.e
., prompts. The classification performance of VLM-based methods heavily relies on these prompts, but prompt engineering not only demands extensive expert knowledge but is also extremely time-consuming. To address this, prompt tuning methods are introduced to enhance the generalizability of VLMs by adding spectral-based prompts to the vision encoder and incorporating randomly initialized, learnable text prompts into the text encoder. Finally, through a novel class-discriminative loss function, the distance between text features of different classes is increased, thereby enhancing the model’s discriminative ability. Experimental results on the Houston 2013, Trento, and MUUFL datasets demonstrate that the proposed method can achieve competitive classification accuracy with a limited number of labeled pixels.
Hyperspectral image (HSI) classification plays an important role in the human exploration of the Earth. Recent research of deep learning-based HSI classification has been fast-growing, but still suffers from three obstacles: First, existing deep learning-based HSI works lack of extraction and utilization of multigrained multiscale information and multiscale local-to-global information. Second, most previous works have too fixed-sized receptive fields in their convolutional network parts to handle HSI classification problems, and pay no attention to the existence of asymmetries in the spectral-spatial dimension of the HSI data. Third, most networks for HSI classification are hand-craft. To this end, we propose a novel architecture in this article, which is the first to combine the advantages of nested U-Net and scale-aware Transformer, named U2ConvFormer. Specifically, the nested U-Net structure can fully extract and aggregate multiscale spectral-spatial features at both inter- and inner stage granularity. The scale-aware Transformer takes multiscale local spectral-spatial features from the encoder of nested U-Net and produces multiscale global spectral-spatial features for its decoder. After that, we design a novel plug-and-play searchable operation called asymmetric spectral-spatial convolution (A2SConv), where asymmetric spectral-spatial feature pooling and multiscale feature extraction can be concurrently searched. Furthermore, we develop a customized search strategy to automatically design U2ConvFormer, which uses advanced neural architecture search (NAS) methods to enable the customization of suitable models for different hyperspectral datasets. Experimental results on three benchmark datasets, including Indian Pines, Pavia University and Houston University 2018, validate the superiority of our proposed U2ConvFormer, which achieves new state-of-the-art performance across different benchmark datasets.
At present, hyperspectral image classification (HSIC) technology has been warmly concerned in all walks of life, especially in agriculture. However, existing classification methods operate under the closed-set assumption, which deviates from the real world with open properties. At the same time, there are more serious phenomena of different crops with similar spectrum and same crops with different spectrum in agricultural hyperspectral data, which is also a great challenge to existing methods. In this work, a fractional Fourier based frequency-spatial-spectral prototype network is proposed to address the challenges of open-set hyperspectral image classification in agricultural scenarios. Firstly, fractional Fourier transform is introduced into the network to combine the information in the frequency domain with the spatial-spectral information, so as to expand the difference between different classes on the premise of ensuring the similarity between classes. Then, the prototype learning strategy is introduced into the network to improve the feature recognition capability of the network through prototype loss. Finally, in order to break the stubbornly closed-set property of closed-set classification method, the open-set recognition module is proposed. The difference between the prototype vector and the feature vector is used to judge the unknown class. Experiments on three agricultural hyperspectral datasets show that this method can effectively identify unknown class without sacrificing the classification accuracy of closed-set, and has satisfactory classification performance.
Convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the fixed receptive field of CNN-based methods limits their capability to extract global features. In recent years, transformer has been introduced into networks to tackle this limitation, but it brings other challenges, including a significant increase in model size, the number of labeled training samples required, and the limited effectiveness of sample encoding-reconstruction pre-training methods for HSI classification. To address these issues, a center-masked transformer (CMT) approach is proposed to improve the HSI classification accuracy from two perspectives. On one hand, a local-to-global token embedding (L2GTE) framework coupled with a multiscale convolutional token embedding (MCTE) module is employed, which is well-designed to obtain local and global embedding tokens. This effectively reduces the number of model parameters. On the other hand, a regulized center-masked pre-training (RCPT) task is proposed and firstly introduced into the transformer-based network, which enables the network to learn the dependencies between central ground objects and neighboring objects without labels during the pre-training process. The experimental results conducted on five public HSI datasets demonstrate that our CMT approach outperforms other state-of-the-art methods for HSI classification when training samples are insufficient.
WaveMLP has demonstrated remarkable performance in various vision tasks; such as dense feature detection and semantic segmentation. However, WaveMLP, as a local model, imposes limitations on fully connected layers by only allowing connections between tokens within the same local window. This constraint makes the model neglect the relationship among tokens in different windows, leading to a local token fusion and a degraded modelling performance. Specially, it poses challenges when dealing with hyperspectral image (HSI) classification tasks that require capturing long-range dependencies. To address this issue, this letter proposes a new Position-Aware WaveMLP, dubbed PA-WaveMLP, which incorporates a global polar positional encoding module (PPEM) into WaveMLP. PPEM is a light-weight method to encode the spatial relationship between land objects in distance and direction by using the radius and angle. By PPEM, the proposed PA-WaveMLP enables tokens to include their own spatial position information to the fusion process, allowing for the capture of long-range dependencies while maintaining the excellent modeling capabilities of WaveMLP. The experimental results on three publicly available HSI datasets validate the effectiveness and generalizability of this newly proposed PA-WaveMLP. In particular, PA-WaveMLP model achieved an overall accuracy of 99.16%, 99.71%, and 99.47% on Indian Pines,Pavia University, and Salinas respectively.
The dissection of hyperspectral images (HSIs) into intrinsic components through hyperspectral intrinsic image decomposition (HIID) enhances the interpretability of hyperspectral data, providing a foundation for more accurate classification outcomes. However, the classification performance of HIID is constrained by the model’s representational ability. To address this limitation, this study rethinks HIID for classification tasks by introducing deep feature embedding. The proposed framework, HyperDID, incorporates the environmental feature module (EFM) and categorical feature module (CFM) to extract intrinsic features. In addition, a feature discrimination module (FDM) is introduced to separate environment-related and category-related features. Experimental results across three commonly used datasets validate the effectiveness of HyperDID in improving HSI classification performance. This novel approach holds promise for advancing the capabilities of HSI analysis by leveraging deep feature embedding principles. The implementation of the proposed method can be accessed soon at
https://github.com/shendu-sw/HyperDID
for the sake of reproducibility.
Collecting ground truth labels for hyperspectral image classification is difficult and time-consuming. Without an adequate number of training samples, hyperspectral image (HSI) classification is a challenging problem. Using generative adversarial networks (GANs) is a promising technique for solving this problem because GANs can learn features from both labeled and unlabeled samples. The cost functions widely used in current GAN methods are suitable for 2D nature images. Compared with natural images, HSIs have a simpler one-dimensional structure that facilitates image generation. Motivated by the one-dimensional spectral features of HSIs, we propose a novel semisupervised algorithm for HSI classification by introducing spectral angle distance (SAD) as a loss function and employing multilayer feature fusion. Since the differences between spectra can be quickly calculated using the spectral angle distance, the convergence speed of the GAN can be improved, and the samples generated by the generator model in the GAN are closer to the real spectrum. Once the entire GAN model has been trained, the discriminator can extract multiscale features of labeled and unlabeled samples. The classifier is then trained for HSI classification using the multilayer features extracted from a few labeled samples by the discriminator. The proposed method was validated on four hyperspectral datasets: Pavia University, Indiana Pines, Salinas, and Tianshan. The experimental results show that the proposed model provides very promising results compared with other related state-of-the-art methods.
In the past few decades, importance of the medicinal Crops is extending to a large extent due to its benefits in treating life-threatening diseases. Medicinal Crop has excellent medicinal properties on its roots, stem, and leaves to prevent human and animal health. Particularly detection and identification of the Crop classes are effectively carried out using hyperspectral images as discrimination of the target feature or objects is simple and it contains rich information containing the spatial and temporal details of underlying the land cover. However, Crop classification using machine learning architectures concerning spectral characteristics obtained on the anatomical features and morphological features. Extracted features towards classification lead to several challenges such as large spatial and temporal variability and spectral signatures similarity between different objects. A further hyperspectral image poses several difficulties with changes in illumination, environment, and atmospheric aspects. To tackle those non-trivial challenges, DenseNet-324 Densely Connected convolution neural network architecture has been designed in this work to discriminate the crop and medical Crop effectively in the interested areas. Initially, the Hyperspectral image is pre-processed against a large number of noises through the employment of the noise removal technique and bad line replacement techniques. Pre-processed image is explored to image segmentation using the global thresholding method to segment it into various regions based on spatial pieces of information on grouping the neighboring similar pixels intensity or textures. Further regions of the image are processed using principle component analysis to extract spectral features of the image. That extracted feature is employed to ant colony optimization technique to obtain the optimal features. Computed optimal features are classified using Convolution Neural Network with a hyper parameter setup. The convolution Layer of the CNN architecture process spatial, temporal, and spectral feature and generates the feature map in various context, generated feature map is max pooled in the pooling layer and classified into crops and medicinal Crop in the SoftMax layer. Experimental analysis of the proposed architecture is carried out on the Indiana Pines dataset using cross-fold validation to analyze the representation ability to discriminate the features with large variance between the different classes. From the results, it is confirmed that the proposed architecture exhibits higher performance in classification accuracy of 98.43% in classifying the Crop species compared with conventional approaches.
Recent years, in order to solve the problem of lacking accurately labeled hyperspectral image data, self-supervised learning has become an effective method for hyperspectral image classification. The core idea of self-supervised learning is to define a pretext task which helps to train the model without the labels. By exploiting both the information of the labeled and unlabeled samples, self-supervised learning shows enormous potential to handle many different tasks in the field of hyperspectral image processing. Among the vast amount of self-supervised methods, contrastive learning and masked autoencoder are well known because of their impressive performance. This article proposes a Transformer based masked autoencoder using contrastive learning (TMAC), which tries to combine these two methods and improve the performance further. TMAC has two branches, the first branch has an encoder-decoders structure, it has an encoder to capture the latent image representation of the masked hyperspectral image and two decoders where the pixel decoder aims to reconstruct the hyperspectral image at pixel-level and the feature decoder is built to extract the high-level feature of the reconstructed image. The second branch consists of a momentum encoder and a standard projection head to embed the image into the feature space. Then, by combining the output of feature decoder and the embedding vectors via contrastive learning to enhance the model’s classification performance. According to the experiments, our model shows powerful feature extraction capability and gets outstanding results on hyperspectral image datasets.
One of the major difficulties for hyperspectral imagery (HSI) classification is the hyperspectral-heterospectra, which refers to the same material presenting different spectra. Although joint spatial-spectral classification methods can relieve this problem, they may lead to falsely high accuracy because the test samples may be involved during the training process. How to address the hyperspectral-heterospectra problem remains a great challenge for pixel-wise hyperspectral imagery classification methods. Domain generalization is a promising technique that may contribute to the heterospectra problem, where the different spectra of the same material can be considered as several domains. In this paper, inspired by the theory of domain generalization, we provide a formulaic expression for hyperspectral-heterospectra. To be specific, we consider the spectra of one material as a conditional distribution and propose a domain-generalization-based method for pixel-wise HSI classification. The key of our proposed method is a new Label-indicate-Conditional-alignment (LiCa) block that focuses on aligning the spectral conditional distributions of different domains. In the LiCa block, we define two loss functions, cross-domain conditional alignment, and cross-domain entropy, to describe the heterogeneity of HSI. Moreover, we have provided the theoretical foundation for the newly-proposed loss functions, by analyzing the upper bound of classification error in any target domains. Experiments on several public data sets indicate that the LiCa block has achieved better generalization performance when compared with other pixel-wise classification methods.
The number of data points predicted correctly out of the total data points is known as accuracy in image classification models. Assessment of the accuracy is very important since it compares the correct images to the ones that have been classified by the image classification models. Image classification accuracy is a challenge since image classification models classify images to the class they don't belong to hence there is an inaccurate relationship between the predicted class and the actual class which results in a low model accuracy score. Therefore, there is a need for a model that can classify the images with the highest accuracy. The paper presents image classification models together with the feature extraction methods used to classify maize disease images.
Over the past decades, land‐cover classification has become a main topic in hyperspectral (HS) remote‐sensing applications. HS image can achieve much higher classification accuracies and is able to recognize more categories. This chapter focuses on the collaborative classification of HS‐centered multisource images, including HS image, high‐resolution panchromatic, and long‐wave infrared HS images. It reviews the current existing works of collaborative processing of multisource images and presents the main problems and challenges in multisource image collaborative processing. A comprehensive exploration of collaborative classification of multisource images is of great significance in enhancing the ability of data utilization and information acquisition, as well as the extension of the remote‐sensing application. The chapter focuses on a decision level‐based collaborative classification method for infrared HS and visible color images. High‐resolution images usually present more ground details, thereby making the extraction of finer detailed spatial information possible.
In recent years, hyperspectral image (HSI) classification methods based on generative adversarial networks (GANs) have been proposed and have made great progress, which can alleviate the dilemma of limited training samples. However, GAN-based HSI classification methods are heavily affected by the problem of imbalanced training data. The discriminator always tries to associate false labels with a few samples, which will reduce the classification accuracy. Another problem is the mode collapse based on the GAN network, which hinders the classification performance of HSI. A combined Transformer and GAN (TransGAN) model for HSI classification is proposed in this paper. First, in order to solve the problem of reduced classification accuracy caused by imbalanced training data, the discriminator is adjusted to a classifier with only one output. Second, the generator is constructed by using the Transformer, and the discriminator is added with a multi-scale pooling module (MSPM) to alleviate the problem of GAN model collapse. Experimental results on two HSI datasets show that the proposed TransGAN achieves better performance.
KeywordsMulti-scale pooling moduleTransformerGenerative adversarial networkHyperspectral image classification
Random Forest (RF) is a widely used classifier to show a good performance of hyperspectral data classification. However, such performance could be improved by increasing the diversity that characterizes the ensemble architecture. In this paper, we propose a novel ensemble approach, namely rotation random forest via kernel principal component analysis (RoRF-KPCA). In particular, the original feature space is first randomly split into several subsets, and KPCA is performed on each subset to extract high order statistics. The obtained feature sets are merged and used as input to an RF classifier. Finally, the results achieved at each step are fused by a majority vote. Experimental analysis is conducted using real hyperspectral remote sensing images to evaluate the performance of the proposed method in comparison with RF, rotation forest, support vector machines, and RoRF-PCA. The obtained results demonstrate the effectiveness of the proposed method.
In this letter, a self-improving convolutional neural network (CNN) based method is proposed for the classification of hyperspectral data. This approach solves the so-called curse of dimensionality and the lack of available training samples by iteratively selecting the most informative bands suitable for the designed network via fractional order Darwinian particle swarm optimization. The selected bands are then fed to the classification system to produce the final classification map. Experimental results have been conducted with two well-known hyperspectral data sets: Indian Pines and Pavia University. Results indicate that the proposed approach significantly improves a CNN-based classification method in terms of classification accuracy. In addition, this letter uses the concept of dither for the first time in the remote sensing community to tackle overfitting.
With different principles, support vector machines (SVMs) and multiple classifier systems (MCSs) have shown excellent performances for classifying hyperspectral remote sensing images. In order to further improve the performance, we propose a novel ensemble approach, namely, rotation-based SVM (RoSVM), which combines SVMs and MCSs together. The basic idea of RoSVM is to generate diverse SVM classification results using random feature selection and data transformation, which can enhance both individual accuracy and diversity within the ensemble simultaneously. Two simple data transformation methods, i.e., principal component analysis and random projection, are introduced into RoSVM. An empirical study on three hyperspectral data sets demonstrates that the proposed RoSVM ensemble method outperforms the single SVM and random subspace SVM. The impacts of the parameters on the overall accuracy of RoSVM (different training sets, ensemble sizes, and numbers of features in the subset) are also investigated in this paper.
Hyperspectral imaging records a detailed spectrum of the received light in each spatial position in the image. Since different substances exhibit different spectral signatures, hyperspectral imagery is a well-suited technology for accurate image classification, which is an important task in many application domains. However, the high dimensionality of the data presents challenges for image analysis. While most of the previously proposed classification techniques process each pixel independently without considering information about spatial structures, recent research in image processing has highlighted the importance of the incorporation of spatial context in a classifier. In this thesis, we propose and develop novel spectral-spatial methods and algorithms for accurate classification of hyperspectral data. First, the integration of the Support Vector Machines (SVM) technique within a Markov Random Fields (MRFs) framework for context classification is investigated. SVM and MRF models are two powerful tools for high-dimensional data classification and for contextual image analysis, respectively. In a second step, we propose classification methods using adaptive spatial neighborhoods derived from region segmentation results. Different segmentation techniques are investigated and extended to the case of hyperspectral images. Then, approaches for combining the extracted spatial regions with spectral information in a classifier are developed. In a third step, we concentrate on approaches to reduce oversegmentation in an image, which is achieved by automatically “marking” the spatial structures of interest before performing a marker-controlled segmentation. Our proposal is to analyze probabilistic classification results for selecting the most reliably classified pixels as markers of spatial regions. Several marker selection methods are proposed, using either individual classifiers, or a multiple classifier system. Then, different approaches for marker-controlled region growing are developed, using either watershed or Minimum Spanning Forest methods and resulting in both segmentation and context classification maps. Finally, we explore possibilities of high-performance parallel computing on commodity processors for reducing computational loads. The new techniques, developed in this thesis, improve classification results, when compared to previously proposed methods, and thus show great potential for various image analysis scenarios.
Classification is one of the most important techniques to the analysis of hyperspectral remote sensing images. Nonetheless, there are many challenging problems arising in this task. Two common issues are the curse of dimen-sionality and the spatial information modeling. In this work, we present a new general framework to train series of effective classifiers with spatial information for classifying hyperspectral data. The proposed framework is based on the two key observations: 1) the curse of dimensionality and the high feature-to-instance ratio can be alleviated by using Random Subspace (RS) ensembles; 2) the spatial-contextual information is modeled by the extended multi-attribute profiles (EMAPs). Two fast learning algorithms, decision tree (DT) and extreme learning machine (ELM), are selected as the base classifiers. Six RS ensemble methods, including Random subspace with DT (RSDT), Random Forest (RF), Rotation Forest (RoF), Rotation Random Forest (RoRF), RS with ELM (RSELM) and Rotation subspace with ELM (RoELM), are constructed by the multiple base learners. Experimental results on both simulated and real hyperspectral data verify the effectiveness of the RS ensemble methods for the classification of both spectral and spatial information (EMAPs). On the University of Pavia ROSIS image, our proposed approaches, both RSELM and RoELM with EMAPs, achieve the state-of-the-art performances, which demonstrates the advantage of the proposed methods. The key parameters in RS ensembles and the computational complexity are also investigated in this study.
In this paper, we propose a new spectral-spatial classification strategy to enhance the classification performances obtained on hyperspectral images by integrating rotation forests and Markov random fields (MRFs). First, rotation forests are performed to obtain the class probabilities based on spectral information. Rotation forests create diverse base learners using feature extraction and subset features. The feature set is randomly divided into several disjoint subsets; then, feature extraction is performed separately on each subset, and a new set of linear extracted features is obtained. The base learner is trained with this set. An ensemble of classifiers is constructed by repeating these steps several times. The weak classifier of hyperspectral data, classification and regression tree (CART), is selected as the base classifier because it is unstable, fast, and sensitive to rotations of the axes. In this case, small changes in the training data of CART lead to a large change in the results, generating high diversity within the ensemble. Four feature extraction methods, including principal component analysis (PCA), neighborhood preserving embedding (NPE), linear local tangent space alignment (LLTSA), and linearity preserving projection (LPP), are used in rotation forests. Second, spatial contextual information, which is modeled by MRF prior, is used to refine the classification results obtained from the rotation forests by solving a maximum a posteriori problem using the α-expansion graph cuts optimization method. Experimental results, conducted on three hyperspectral data with different resolutions and different contexts, reveal that rotation forest ensembles are competitive with other strong supervised classification methods, such as support vector machines. Rotation forests with local feature extraction methods, including NPE, LLTSA, and LPP, can lead to higher classification accuracies than that achieved by PCA. With the help of MRF, the proposed algo- ithms can improve the classification accuracies significantly, confirming the importance of spatial contextual information in hyperspectral spectral-spatial classification.
Just over a decade has passed since the concept of morphological profile was defined for the analysis of remote sensing images. Since then, the morphological profile has largely proved to be a powerful tool able to model spatial information (e.g., contextual relations) of the image. However, due to the shortcomings of using the morphological profiles, many variants, extensions, and refinements of its definition have appeared stating that the morphological profile is still under continuous development. In this case, recently introduced theoretically sound attribute profiles (APs) can be considered as a generalization of the morphological profile, which is a powerful tool to model spatial information existing in the scene. Although the concept of the AP has been introduced in remote sensing only recently, an extensive literature on its use in different applications and on different types of data has appeared. To that end, the great amount of contributions in the literature that address the application of the AP to many tasks (e.g., classification, object detection, segmentation, change detection, etc.) and to different types of images (e.g., panchromatic, multispectral, and hyperspectral) proves how the AP is an effective and modern tool. The main objective of this survey paper is to recall the concept of the APs along with all its modifications and generalizations with special emphasis on remote sensing image classification and summarize the important aspects of its efficient utilization while also listing potential future works.
Hyperspectral remote sensing technology allows one to acquire a sequence of possibly hundreds of contiguous spectral images from ultraviolet to infrared. Conventional spectral classifiers treat hyperspectral images as a list of spectral measurements and do not consider spatial dependences, which leads to a dramatic decrease in classification accuracies. In this paper, a new automatic framework for the classification of hyperspectral images is proposed. The new method is based on combining hidden Markov random field segmentation with support vector machine (SVM) classifier. In order to preserve edges in the final classification map, a gradient step is taken into account. Experiments confirm that the new spectral and spatial classification approach is able to improve results significantly in terms of classification accuracies compared to the standard SVM method and also outperforms other studied methods.
In this letter, an ensemble learning approach, Rotation Forest, has been applied to hyperspectral remote sensing image classification for the first time. The framework of Rotation Forest is to project the original data into a new feature space using transformation methods for each base classifier (decision tree), then the base classifier can train in different new spaces for the purpose of encouraging both individual accuracy and diversity within the ensemble simultaneously. Principal component analysis (PCA), maximum noise fraction, independent component analysis, and local Fisher discriminant analysis are introduced as feature transformation algorithms in the original Rotation Forest. The performance of Rotation Forest was evaluated based on several criteria: different data sets, sensitivity to the number of training samples, ensemble size and the number of features in a subset. Experimental results revealed that Rotation Forest, especially with PCA transformation, could produce more accurate results than bagging, AdaBoost, and Random Forest. They indicate that Rotation Forests are promising approaches for generating classifier ensemble of hyperspectral remote sensing.
Tree species diversity is a key parameter to describe forest ecosystems. It is, for example, important for issues such as wildlife habitat modeling and close-to-nature forest management. We examined the suitability of 8-band WorldView-2 satellite data for the identification of 10 tree species in a temperate forest in Austria. We performed a Random Forest (RF) classification (object-based and pixel-based) using spectra of manually delineated sunlit regions of tree crowns. The overall accuracy for classifying 10 tree species was around 82% (8 bands, object-based). The class-specific producer's accuracies ranged between 33% (European hornbeam) and 94% (European beech) and the user's accuracies between 57% (European hornbeam) and 92% (Lawson's cypress). The object-based approach outperformed the pixel-based approach. We could show that the 4 new WorldView-2 bands (Coastal, Yellow, Red Edge, and Near Infrared 2) have only limited impact on classification accuracy if only the 4 main tree species (Norway spruce, Scots pine, European beech, and English oak) are to be separated. However, classification accuracy increased significantly using the full spectral resolution if further tree species were included. Beside the impact on overall classification accuracy, the importance of the spectral bands was evaluated with two measures provided by RF. An in-depth analysis of the RF output was carried out to evaluate the impact of reference data quality and the resulting reliability of final class assignments. Finally, an extensive literature review on tree species classification comprising about 20 studies is presented.
We propose a novel semisupervised local discriminant analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear discriminant analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.
Opening and closing are common operators used for removing struc-tures from images according to a predeened size and shape criterion. Families of openings or closings of increasing size are at the basis of the granulometric analysis. In this paper we deene a new granulometric operator which v al-uates the extrema of greyscale images and the structures they point out with a measurement of their persistence when applying openings or closings of increasing size. The proposed mea-surement is called extinction value. The rela-tionships b e t ween extinction values and area openings or closings lead to an ecient a r e a extinction values algorithm. This transforma-tion can be seen as the spatial equivalent o f dynamics.
Connected operators are filtering tools that act by merging elementary
regions of an image. A popular strategy is based on tree-based image
representations: for example, one can compute an attribute on each node of the
tree and keep only the nodes for which the attribute is sufficiently strong.
This operation can be seen as a thresholding of the tree, seen as a graph whose
nodes are weighted by the attribute. Rather than being satisfied with a mere
thresholding, we propose to expand on this idea, and to apply connected filters
on this latest graph. Consequently, the filtering is done not in the space of
the image, but on the space of shapes build from the image. Such a processing
is a generalization of the existing tree-based connected operators. Indeed, the
framework includes classical existing connected operators by attributes. It
also allows us to propose a class of novel connected operators from the
leveling family, based on shape attributes. Finally, we also propose a novel
class of self-dual connected operators that we call morphological shapings.
Morphological attribute profiles (APs) are defined as a generalization of the recently proposed morphological profiles (MPs). APs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of the structural information. According to the type of the attributes considered in the morphological attribute transformation, different parametric features can be modeled. The generation of APs, thanks to an efficient implementation, strongly reduces the computational load required for the computation of conventional MPs. Moreover, the characterization of the image with different attributes leads to a more complete description of the scene and to a more accurate modeling of the spatial information than with the use of conventional morphological filters based on a predefined structuring element. Here, the features extracted by the proposed operators were used for the classification of two very high resolution panchromatic images acquired by Quickbird on the city of Trento, Italy. The experimental analysis proved the usefulness of APs in modeling the spatial information present in the images. The classification maps obtained by considering different APs result in a better description of the scene (both in terms of thematic and geometric accuracy) than those obtained with an MP.
Support vector machines (SVM) are attractive for the classification of remotely sensed data with some claims that the method is insensitive to the dimensionality of the data and, therefore, does not require a dimensionality-reduction analysis in preprocessing. Here, a series of classification analyses with two hyperspectral sensor data sets reveals that the accuracy of a classification by an SVM does vary as a function of the number of features used. Critically, it is shown that the accuracy of a classification may decline significantly (at 0.05 level of statistical significance) with the addition of features, particularly if a small training sample is used. This highlights a dependence of the accuracy of classification by an SVM on the dimensionality of the data and, therefore, the potential value of undertaking a feature-selection analysis prior to classification. Additionally, it is demonstrated that, even when a large training sample is available, feature selection may still be useful. For example, the accuracy derived from the use of a small number of features may be noninferior (at 0.05 level of significance) to that derived from the use of a larger feature set providing potential advantages in relation to issues such as data storage and computational processing costs. Feature selection may, therefore, be a valuable analysis to include in preprocessing operations for classification by an SVM.
Diversity among the members of a team of classifiers is deemed to be a key issue in classifier combination. However, measuring diversity is not straightforward because there is no generally accepted formal definition. We have found and studied ten statistics which can measure diversity among binary classifier outputs (correct or incorrect vote for the class label): four averaged pairwise measures (the Q statistic, the correlation, the disagreement and the double fault) and six non-pairwise measures (the entropy of the votes, the difficulty index, the Kohavi-Wolpert variance, the interrater agreement, the generalized diversity, and the coincident failure diversity). Four experiments have been designed to examine the relationship between the accuracy of the team and the measures of diversity, and among the measures themselves. Although there are proven connections between diversity and accuracy in some special cases, our results raise some doubts about the usefulness of diversity measures in building classifier ensembles in real-life pattern recognition problems.
Predictive models benefit from a compact, non-redundant subset of features that improves interpretability and generalization. Modern data sets are wide, dirty, mixed with both numerical and categorical predictors, and may contain interactive effects that require complex models. This is a challenge for filters, wrappers, and embedded feature selection methods. We describe details of an algorithm using tree-based ensembles to generate a compact subset of non-redundant features. Parallel and serial ensembles of trees are combined into a mixed method that can uncover masking and detect features of secondary effect. Simulated and actual examples illustrate the effectiveness of the approach.
This paper introduces a new supervised segmentation algorithm for remotely sensed hyperspectral image data which integrates the spectral and spatial information in a Bayesian framework. A multinomial logistic regression (MLR) algorithm is first used to learn the posterior probability distributions from the spectral information, using a subspace projection method to better characterize noise and highly mixed pixels. Then, contextual information is included using a multilevel logistic Markov–Gibbs Markov random field prior. Finally, a maximum a posteriori segmentation is efficiently computed by the -Expansion min-cut-based integer optimization algorithm. The proposed segmentation approach is experimentally evaluated using both simulated and real hyperspectral data sets, exhibiting state-of-the-art performance when compared with recently introduced hyperspectral image classification methods. The integration of subspace projection methods with the MLR algorithm, combined with the use of spatial–contextual information, represents an innovative contribution in the literature. This approach is shown to provide accurate characterization of hyperspectral imagery in both the spectral and the spatial domain.
In this letter, a technique based on independent component analysis (ICA) and extended morphological attribute profiles (EAPs) is presented for the classification of hyperspectral images. The ICA maps the data into a subspace in which the components are as independent as possible. APs, which are extracted by using several attributes, are applied to each image associated with an extracted independent component, leading to a set of extended EAPs. Two approaches are presented for including the computed profiles in the analysis. The features extracted by the morphological processing are then classified with an SVM. The experiments carried out on two hyperspectral images proved the effectiveness of the proposed technique.
We propose a method for generating classifier ensembles based on feature extraction. To create the training data for a base classifier, the feature set is randomly split into K subsets (K is a parameter of the algorithm) and Principal Component Analysis (PCA) is applied to each subset. All principal components are retained in order to preserve the variability information in the data. Thus, K axis rotations take place to form the new features for a base classifier. The idea of the rotation approach is to encourage simultaneously individual accuracy and diversity within the ensemble. Diversity is promoted through the feature extraction for each base classifier. Decision trees were chosen here because they are sensitive to rotation of the feature axes, hence the name "forest." Accuracy is sought by keeping all principal components and also using the whole data set to train each base classifier. Using WEKA, we examined the Rotation Forest ensemble on a random selection of 33 benchmark data sets from the UCI repository and compared it with Bagging, AdaBoost, and Random Forest. The results were favorable to Rotation Forest and prompted an investigation into diversity-accuracy landscape of the ensemble models. Diversity-error diagrams revealed that Rotation Forest ensembles construct individual classifiers which are more accurate than these in AdaBoost and Random Forest, and more diverse than these in Bagging, sometimes more accurate as well.
This paper sets out a new representation of an image which is contrast independent. The image is decomposed into a tree of "shapes" based on connected components of level sets, which provides a full and nonredundant representation of the image. A fast algorithm to compute the tree, the fast level lines transform (FLLT), is explained in detail. Some simple and direct applications of this representation are shown.
New remote sensing sensors will acquire High spectral, spatial and temporal Resolution Satellite Image Time Series (HR-SITS). These new data are of great interest to map land cover thanks to the combination of the three high resolutions that will allow a depiction of scene dynamics. However, their efficient exploitation involves new challenges, especially for adapting traditional classification schemes to data complexity. More specifically, it requires: (1) to determine which classifier algorithms can handle the amount and the variability of data; (2) to evaluate the stability of classifier parameters; (3) to select the best feature set used as input data in order to find the good trade-off between classification accuracy and computational time; and (4) to establish the classifier accuracy over large areas. This work aims at studying these different issues, and more especially at demonstrating the ability of state-of-the-art classifiers, such as Random Forests (RF) or Support Vector Machines (SVM), to classify HR-SITS. For this purpose, several studies are carried out by using SPOT-4 and Landsat-8 HR-SITS in the south of France. Firstly, the choice of the classifier is discussed by comparing RF and SVM algorithms on HR-SITS. Both classifiers show their ability to tackle the classification problem with an Overall Accuracy (OA) of 83.3 % for RF and 77.1 % for SVM. But RF have some advantages such as a small training time, and an easy parameterization. Secondly, the stability of RF parameters is appraised. RF parameters appear to cause little influence on the classification accuracy, about 1% OA difference between the worst and the best parameter configuration. Thirdly, different input data – composed of spectral bands with or without spectral and/or temporal features – are proposed in order to enhance the characterization of land cover. The addition of features improves the classification accuracy, but the gain in OA is weak compared with the increase in the computational cost. Eventually, the classifier accuracy is assessed on a larger area where the landscape variabilities affect the classification performances.
This letter proposes a new approach for the spectral–spatial classification of hyperspectral images, which is based on a novel extrema-oriented connected filtering technique, entitled as extended extinction profiles. The proposed approach progressively simplifies the first informative features extracted from hyperspectral data considering different attributes. Then, the classification approach is applied on two well-known hyperspectral data sets, i.e., Pavia University and Indian Pines, and compared with one of the most powerful filtering approaches in the literature, i.e., extended attribute profiles. Results indicate that the proposed approach is able to efficiently extract spatial information for the classification of hyperspectral images automatically and swiftly. In addition, an array-based node-oriented max-tree representation was carried out to efficiently implement the proposed approach.
With respect to recent advances in remote sensing technologies, the spatial resolution of airborne and spaceborne sensors is getting finer, which enables us to precisely analyze even small objects on the Earth. This fact has made the research area of developing efficient approaches to extract spatial and contextual information highly active. Among the existing approaches, morphological profile and attribute profile (AP) have gained great attention due to their ability to classify remote sensing data. This paper proposes a novel approach that makes it possible to precisely extract spatial and contextual information from remote sensing images. The proposed approach is based on extinction filters, which are used here for the first time in the remote sensing community. Then, the approach is carried out on two well-known high-resolution panchromatic data sets captured over Rome, Italy, and Reykjavik, Iceland. In order to prove the capabilities of the proposed approach, the obtained results are compared with the results from one of the strongest approaches in the literature, i.e., APs, using different points of view such as classification accuracies, simplification rate, and complexity analysis. Results indicate that the proposed approach can significantly outperform its alternative in terms of classification accuracies. In addition, based on our implementation, profiles can be generated in a very short processing time. It should be noted that the proposed approach is fully automatic.
Attribute filters and extinction filters are connected filters used to simplify greyscale images. The first kind is widely explored in the image processing literature, while the second is not much explored yet. Both kind of filters can be efficiently implemented on the max-tree. In this work, we compare these filters in terms of processing time, simplification of flat zones and reduction of max-tree nodes. We also compare their influence as a pre-processing step before extracting affine regions used in matching and pattern recognition. We perform repeatability tests using extinction filters and attribute filters, set to preserve the same number of extrema, as a pre-processing step before detecting Hessian-Affine and Maximally Stable Extremal Regions (MSER) affine regions. The results indicate that using extinction filters as pre-processing obtain a significantly higher (more than 5% on average) number of correspondences on the repeatability tests than the attribute filters. The results in processing natural images show that preserving 5% of images extrema using extinction filters achieve on average 95% of the number of correspondences compared to applying the affine region detectors directly to the unfiltered images, and the average number of max-tree nodes is reduced by a factor greater than 3. Therefore, we can conclude that extinction filters are better than attribute filters with respect to preserving the number of correspondences found by affine detectors, while simplifying the max-tree structure. The use of extinction filters as a pre-processing step is recommended to accelerate image recognition tasks.
Recent advances in spectral-spatial classification of hyperspectral images are presented in this paper. Several techniques are investigated for combining both spatial and spectral information. Spatial information is extracted at the object (set of pixels) level rather than at the conventional pixel level. Mathematical morphology is first used to derive the morphological profile of the image, which includes characteristics about the size, orientation, and contrast of the spatial structures present in the image. Then, the morphological neighborhood is defined and used to derive additional features for classification. Classification is performed with support vector machines (SVMs) using the available spectral information and the extracted spatial information. Spatial postprocessing is next investigated to build more homogeneous and spatially consistent thematic maps. To that end, three presegmentation techniques are applied to define regions that are used to regularize the preliminary pixel-wise thematic map. Finally, a multiple-classifier (MC) system is defined to produce relevant markers that are exploited to segment the hyperspectral image with the minimum spanning forest algorithm. Experimental results conducted on three real hyperspectral images with different spatial and spectral resolutions and corresponding to various contexts are presented. They highlight the importance of spectral-spatial strategies for the accurate classification of hyperspectral images and validate the proposed methods.
Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, ***, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
Extended attribute profiles and extended multi-attribute profiles are presented for the analysis of hyperspectral high-resolution images. These extended profiles are based on morphological attribute filters and, through a multi-level analysis, are capable of extracting spatial features that can better model the spatial information, with respect to conventional extended morphological profiles. The features extracted by the proposed extended profiles were considered for a classification task. Two hyperspectral high-resolution datasets acquired for the city of Pavia, Italy, were considered in the analysis. The effectiveness of the introduced operators in modelling the spatial information was proved by the higher classification accuracies obtained with respect to those achieved by a conventional extended morphological profile.
Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
This paper introduces a novel classification method termed Alternating Decision Forests (ADFs), which formulates the training of Random Forests explicitly as a global loss minimization problem. During training, the losses are minimized via keeping an adaptive weight distribution over the training samples, similar to Boosting methods. In order to keep the method as flexible and general as possible, we adopt the principle of employing gradient descent in function space, which allows to minimize arbitrary losses. Contrary to Boosted Trees, in our method the loss minimization is an inherent part of the tree growing process, thus allowing to keep the benefits of common Random Forests, such as, parallel processing. We derive the new classifier and give a discussion and evaluation on standard machine learning data sets. Furthermore, we show how ADFs can be easily integrated into an object detection application. Compared to both, standard Random Forests and Boosted Trees, ADFs give better performance in our experiments, while yielding more compact models in terms of tree depth.
Bagging predictors is a method for generating multiple versions of a predictor and using these to get an aggregated predictor. The aggregation averages over the versions when predicting a numerical outcome and does a plurality vote when predicting a class. The multiple versions are formed by making bootstrap replicates of the learning set and using these as new learning sets. Tests on real and simulated data sets using classification and regression trees and subset selection in linear regression show that bagging can give substantial gains in accuracy. The vital element is the instability of the prediction method. If perturbing the learning set can cause significant changes in the predictor constructed, then bagging can improve accuracy.
The formulation of conditional probability models for finite systems of spatially interacting random variables is examined. A simple alternative proof of the Hammersley–Clifford theorem is presented and the theorem is then used to construct specific spatial schemes on and off the lattice. Particular emphasis is placed upon practical applications of the models in plant ecology when the variates are binary or Gaussian. Some aspects of infinite lattice Gaussian processes are discussed. Methods of statistical analysis for lattice schemes are proposed, including a very flexible coding technique. The methods are illustrated by two numerical examples. It is maintained throughout that the conditional probability approach to the specification and analysis of spatial interaction is more attractive than the alternative joint probability approach.
The high number of spectral bands acquired by hyperspectral sensors increases the capability to distinguish physical materials and objects, presenting new challenges to image analysis and classification. This letter presents a novel method for accurate spectral-spatial classification of hyperspectral images. The proposed technique consists of two steps. In the first step, a probabilistic support vector machine pixelwise classification of the hyperspectral image is applied. In the second step, spatial contextual information is used for refining the classification results obtained in the first step. This is achieved by means of a Markov random field regularization. Experimental results are presented for three hyperspectral airborne images and compared with those obtained by recently proposed advanced spectral-spatial classification techniques. The proposed method improves classification accuracies when compared to other classification approaches.
Airborne lidar systems have become a source for the acquisition of elevation data. They provide georeferenced, irregularly distributed 3D point clouds of high altimetric accuracy. Moreover, these systems can provide for a single laser pulse, multiple returns or echoes, which correspond to different illuminated objects. In addition to multi-echo laser scanners, full-waveform systems are able to record 1D signals representing a train of echoes caused by reflections at different targets. These systems provide more information about the structure and the physical characteristics of the targets. Many approaches have been developed, for urban mapping, based on aerial lidar solely or combined with multispectral image data. However, they have not assessed the importance of input features. In this paper, we focus on a multi-source framework using aerial lidar (multi-echo and full waveform) and aerial multispectral image data. We aim to study the feature relevance for dense urban scenes. The Random Forests algorithm is chosen as a classifier: it runs efficiently on large datasets, and provides measures of feature importance for each class. The margin theory is used as a confidence measure of the classifier, and to confirm the relevance of input features for urban classification. The quantitative results confirm the importance of the joint use of optical multispectral and lidar data. Moreover, the relevance of full-waveform lidar features is demonstrated for building and vegetation area discrimination.
Chapter 4 discusses the fusion of label outputs. Four types of classifier outputs are listed: class labels (abstract level), ranked class labels (rank level), degree of support for the classes (measurement level) and correct/incorrect decision (oracle level). Combination methods for class label outputs are presented including majority vote, plurality vote, weighted majority vote, naive Bayes, multinomial combiners (Behavior Knowledge Space (BKS) and Werneckes methods), probabilistic combination and singular value decomposition (SVD) combination.
This paper presents a novel ensemble classifier generation technique RotBoost, which is constructed by combining Rotation Forest and AdaBoost. The experiments conducted with 36 real-world data sets available from the UCI repository, among which a classification tree is adopted as the base learning algorithm, demonstrate that RotBoost can generate ensemble classifiers with significantly lower prediction error than either Rotation Forest or AdaBoost more often than the reverse. Meanwhile, RotBoost is found to perform much better than Bagging and MultiBoost. Through employing the bias and variance decompositions of error to gain more insight of the considered classification methods, RotBoost is seen to simultaneously reduce the bias and variance terms of a single tree and the decrement achieved by it is much greater than that done by the other ensemble methods, which leads RotBoost to perform best among the considered classification procedures. Furthermore, RotBoost has a potential advantage over AdaBoost of suiting parallel execution.
Hyperspectral imaging, which records a detailed spectrum of light for each pixel, provides an invaluable source of information regarding the physical nature of the different materials, leading to the potential of a more accurate classification. However, high dimensionality of hyperspectral data, usually coupled with limited reference data available, limits the performances of supervised classification techniques. The commonly used pixel-wise classification lacks information about spatial structures of the image. In order to increase classification performances, integration of spatial information into the classification process is needed. In this paper, we propose to extend the watershed segmentation algorithm for hyperspectral images, in order to define information about spatial structures. In particular, several approaches to compute a one-band gradient function from hyperspectral images are proposed and investigated. The accuracy of the watershed algorithms is demonstrated by the further incorporation of the segmentation maps into a classifier. A new spectral–spatial classification scheme for hyperspectral images is proposed, based on the pixel-wise Support Vector Machines classification, followed by majority voting within the watershed regions. Experimental segmentation and classification results are presented on two hyperspectral images. It is shown in experiments that when the number of spectral bands increases, the feature extraction and the use of multidimensional gradients appear to be preferable to the use of vectorial gradients. The integration of the spatial information from the watershed segmentation in the hyperspectral image classifier improves the classification accuracies and provides classification maps with more homogeneous regions, compared to pixel-wise classification and previously proposed spectral–spatial classification techniques. The developed method is especially suitable for classifying images with large spatial structures.