No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Joint Hyperspectral Superresolution and Unmixing With Interactive Feedback
Abstract
This paper presents an interactive feedback scheme of spatial resolution enhancement and spectral unmixing in hyperspectral imaging. Traditionally spatial resolution enhancement and spectral unmixing operations have been carried out separately, often in series. In such sequential processing, spatially enhanced hyperspectral images (HSIs) may introduce distortion in spectral fidelity making spectral unmixing results unreliable, or vice versa. Since both high- and low-resolution HSIs have the same endmembers, the deviation in spectral unmixing between targets and estimated high-resolution HSIs can be used as feedback to control spatial resolution enhancement. The spatial difference before and after unmixing can also be used as feedback to enhance spectral unmixing. Therefore, spectral unmixing is utilized as a constraint to spatial resolution enhancement, while spatial resolution enhancement helps improve spectral unmixing results. The performance of spatial resolution enhancement and spectral unmixing can be improved since one behaves like a prior to the other. Experimental results on both simulated and real HSI data sets demonstrate that the proposed interactive feedback scheme simultaneously achieved spatial resolution enhancement and spectral unmixing fidelity. This paper is an extended version of the previous work.
... Recently, spectral unmixing-based methods have been actively investigated to fuse pairs of LR-HSI and HR-MSI [16]- [22]. Based on physics of spectral unmixing, the works [16]- [18] unmixed the input HSI to estimate a basis representing reflectance spectra, then used this representation in conjunction with the input MSI to reconstruct the desired HR-HSI. ...
... Recently, spectral unmixing-based methods have been actively investigated to fuse pairs of LR-HSI and HR-MSI [16]- [22]. Based on physics of spectral unmixing, the works [16]- [18] unmixed the input HSI to estimate a basis representing reflectance spectra, then used this representation in conjunction with the input MSI to reconstruct the desired HR-HSI. Instead of estimating spectral basis in advance and keeping them fixed, other researches [19]- [22] alternately update the spectral basis and the coefficients, while imposing nonnegative constraint on the spectral basis and coefficients. ...
... The key problem lies in how to learn accurate spatial/spectral priors from MSI/HSI inputs. Existing MSI-HSI fusion methods have supplied sparse representation [10]- [15], spectral unmixing [16]- [22], and tensor decomposition [25]- [34] views toward the regularity in latent HSI. From such perspective of computer vision, these methods consider the spatial/spectral knowledge in low-level vision. ...
% Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.
... Remote Sens. 2020, 12, 993 2 of 24 remote sensing data have become increasingly available and the corresponding applications have attracted wide interests, there are existing challenges in acquiring images with simultaneously high spatial resolution and high spectral resolution [6]. Therefore, many research focuses on recovering high quality synthetic image from low resolution (LR) inputs, including spatial resolution improvement approaches [7][8][9][10][11][12][13][14][15][16][17][18][19] and spectral resolution enhancement techniques [20-27]. ...
... Hyperspectral pan-sharpening improves spatial resolution of HSI by fusing LR HSI with a high resolution (HR) PAN image covering a spectral range from the visible to the near infrared ranges [7]. Except for the classical hyperspectral pan-sharpening approaches such as component substitution (CS) methods [8], multi-resolution analysis (MRA) methods [9], and model based optimization methods [10,11], deep learning, particularly convolutional neural network (CNN), has been widely exploited in pan-sharpening tasks. In [12], a residual CNN model is designed to describe the mapping between LR/HR MSI pairs and PAN image. ...
... The trained spatial dictionary is shared by the desired HSI and the HSpaLSpe MSI, while the relationship between their sparse coefficients is then exploited as spatial constraint to obtain high spatial resolution abundances. The sparse coefficients and abundances are solved using the feedback scheme in [11] to achieve more accurate results. In this paper, spectral-and spatial observation model are unified into a joint framework to alternately solve spectral dictionary and abundances. ...
This paper presents a joint spatial-spectral resolution enhancement technique to improve the resolution of multispectral images in the spatial and spectral domain simultaneously. Reconstructed hyperspectral images (HSIs) from an input multispectral image represent the same scene in higher spatial resolution, with more spectral bands of narrower wavelength width than the input multispectral image. Many existing improvement techniques focus on spatial- or spectral-resolution enhancement, which may cause spectral distortions and spatial inconsistency. The proposed scheme introduces virtual intermediate variables to formulate a spectral observation model and a spatial observation model. The models alternately solve spectral dictionary and abundances to reconstruct desired high-resolution HSIs. An initial spectral dictionary is trained from prior HSIs captured in different landscapes. A spatial dictionary trained from a panchromatic image and its sparse coefficients provide high spatial-resolution information. The sparse coefficients are used as constraints to obtain high spatial-resolution abundances. Experiments performed on simulated datasets from AVIRIS/Landsat 7 and a real Hyperion/ALI dataset demonstrate that the proposed method outperforms the state-of-the-art spatial- and spectral-resolution enhancement methods. The proposed method also worked well for combination of exiting spatial- and spectral-resolution enhancement methods.
... But HSI is unavoidably corrupted by various noises, e.g., Gaussian noise, mixed Poisson-Gaussian noise, dead-lines, and stripes. The noise will influence the subsequent processing, such as classification [3][4][5][6][7], segmentation [8], unmixing [9,10], object detection [11,12], background subtraction [13], and super-resolution [14]. The central limit theorem establishes that the composite effect of many independent noise sources (e.g., thermal noise, shot noise, etc.) should approach a Gaussian distribution. ...
... From a practical point of view, current imaging systems designed based on the assumption of additive Gaussian noise perform quite well. As a kind of signal independent noise, the Gaussian assumption has been broadly used in HSI denoising [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18]. From a theoretical point of view, Gaussian noise is the worst-case scenario for additive noise as the Gaussian distribution maximizes the entropy subject to a variance constraint [15]. ...
A hyperspectral image (HSI) contains abundant spatial and spectral information, but it is always corrupted by various noises, especially Gaussian noise. Global correlation (GC) across spectral domain and nonlocal self-similarity (NSS) across spatial domain are two important characteristics for an HSI. To keep the integrity of the global structure and improve the details of the restored HSI, we propose a global and nonlocal weighted tensor norm minimum denoising method which jointly utilizes GC and NSS. The weighted multilinear rank is utilized to depict the GC information. To preserve structural information with NSS, a patch-group-based low-rank-tensor-approximation (LRTA) model is designed. The LRTA makes use of Tucker decompositions of 4D patches, which are composed of a similar 3D patch group of HSI. The alternating direction method of multipliers (ADMM) is adapted to solve the proposed models. Experimental results show that the proposed algorithm can preserve the structural information and outperforms several state-of-the-art denoising methods.
... Yokoya et al. (2012) proposed a coupled nonnegative matrix factorization (CNMF) method to generate HR-HSI by estimating the HSI end-member and MSI abundance through alternate unmixing. Yi et al. (2017) introduced an interactive feedback method based on the a priori that the spatial resolution is constrained by the spectral prior information of the spectral unmixing, which in turn is controlled by the inverse of the spatial resolution. Besides matrix factorization, the tensor factorization methods are widely used for MSI and HSI fusion. ...
... Our method needs relatively more computation time on the three datasets, and the main computation burden of our method comes from solving (25) of the manuscript, where we compute TTR for each cluster. The time complexity for solving (29) is mainly related to the size of patches, the number of patches in each group, and the number of spectral ranks. (V) Scaling Analysis ...
Hyperspectral super-resolution reconstruction technique obtains a high-resolution hyperspectral image (HR-HSI) by fusing both a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI). Existing methods of hyperspectral super-resolution reconstruction are mostly concentrated on the global low-rank property, while spatial and spectral information in individual regions is not considered. To address this issue, decomposition of low-rank and sparse tensor (DLST) is proposed in this study. First, HR-HSI was decomposed into low-rank and sparse components. The former was further separated into spatial and spectral domains according to the spectral low-rank property, and the later was used to compensate for information loss caused by low-rank property. Then, a nonlocal constraint of adaptive manifold extracting structural details by the manifold structure was designed to enforce nonlocal self-similarity of the spatial domain. In order to ensure the same spatial structure of different bands and reduce the false individual regions in the sparse component, a surface-aware regularization combined with group sparsity was utilized. Finally, HR-HSI was constructed by the alternating direction method of multipliers. Experiment results on three datasets show that the proposed method outperform five common existing methods by means of both visual and quantitative evaluations. It is concluded that the new method by taking into account of the low-rank and sparse properties can improve the result of the reconstruction.
... Spectral unmixing and SRR can amplify each other in an interactive feedback framework. Hence the unmixing-based SRR not only enhances the spatial resolution but also solves the mixing pixels in HSI [58]. Linear spectral mixing (LMM) models are widely used in spectral unmixing with simplicity and efficiency [59]. ...
This paper proposes an endmember matrix constraint unmixing method for ZY-1 02D hyperspectral imagery (HSI) super-resolution reconstruction (SRR) to overcome the low resolution of ZY-1 02D HSI. The proposed method combines spectral unmixing and adds novel smoothing constraints to traditional non-negative matrix factorization to improve details and preserve the spectral information of traditional SRR methods. The full utilization of the endmember spectral matrix and endmember abundance matrix of HSI and multispectral imagery (MSI) reconstructs the high spatial resolution and high spectral fidelity HSI. Furthermore, given the ZY-1 02D HSI infrared bands are seriously corrupted by noise, the influence of denoising on the SRR accuracy is also discussed. Experiments show that the proposed method restores spatial details and spectral information and is robust for noise, preserving more spectral information. Therefore, the proposed method is a ZY-1 02D HSI SRR method with high spatial resolution and high spectral fidelity, which improves the spatial resolution while simultaneously solving spectral mixing and provides the possibility for the data further expansion.
... For instance, the spectral information of vegetation is prejudiced by a wide range of environmental situations that make it challenging to satisfactorily represent variability without the collection of site-specific field spectra. Thus, considering the aforementioned limitations, HSI analysis is categorized into the following main streams: dimensionality reduction [2,78,79], spectral unmixing [80][81][82][83][84][85], change detection [86][87][88] classification [6,[89][90][91], feature learning for classification [92][93][94], restoration and denoising [95,96], resolution enhancement [97,98]. This chapter specifically focuses on HSIC, which has achieved a phenomenal interest of the research community due to its broad applications in the areas of land use and land cover [99][100][101][102], environment monitoring and natural hazards detection [103,104], vegetation mapping [105,106] and urban planning. ...
Hyperspectral Imaging (HSI) has been extensively utilized in many real-life applications
because it benefits from the detailed spectral information contained in each pixel. Notably, the complex characteristics i.e., the nonlinear relation among the captured spectral information and the corresponding object of HSI data make accurate classification challenging for traditional methods. In the last few years, Deep Learning (DL) has been substantiated as a powerful feature extractor that effectively addresses the nonlinear problems that appeared in a number of computer vision tasks. This prompts the deployment of DL for HSI Classification (HSIC) which revealed good performance.
Keeping in mind the aforementioned issues, this thesis first enlists a systematic overview
of DL for HSIC and compared state-of-the-art strategies of the said topic. Primarily, this
thesis encapsulates the main challenges of traditional machine learning for HSIC and then acquaints the superiority of DL to address these problems. The literature is breakdown the state-of-the-art DL frameworks into spectral features, spatial features, and together spatial-spectral features to systematically analyze the achievements and future directions. This thesis also investigates the behavior and performance in terms of computational cost and classification accuracy, of the most commonly and widely used classification algorithms under different experimental setups. In a nutshell, the following specific contributions are made in this thesis:
1. A Fast and Compact 3D CNN that utilizes both spatial-spectral feature maps to improve
the performance of HSIC.
2. 3D CNNs are computationally expensive and 2D CNN alone cannot efficiently extract
discriminating spectral-spatial features. Therefore, to overcome these challenges,
this part presents a compact hybrid CNN model which overcomes the aforementioned
challenges by distributing spatial-spectral feature extraction across 3D and 2D layers.
3. CNN’s are known to be effective in exploiting joint spatial-spectral information with
the expense of lower generalization performance and learning speed due to the hard
labels and non-uniform distribution over labels. Several regularization techniques
such as dropout, L1, L2, etc., have been used to overcome the aforesaid issues. However,
sometimes models learn to predict the samples extremely confidently which is
not good from a generalization point of view. Therefore, this thesis proposed an idea
to enhance the generalization performance of a hybrid CNN for HSIC using soft labels
that are a weighted average of the hard labels and uniform distribution over ground
labels. The proposed method helps to prevent CNN from becoming over-confident.
4. DL usually required a large number of labeled training samples which is not a real
scenario. Thus, a fully automatic Spatial-Spectral approach has been proposed for the
selection of the most informative and heterogeneous samples for training using a novel
Spectral Angle Mapper (SAM) based objective function for the computation of attribute
profiles in a computationally efficient fashion.
... Considering the aforementioned limitations, HSI analysis is categorized into the following main streams: dimensionality reduction [6]- [10], spectral unmixing [11]- [17], object/change detection [18]- [22] classification [23]- [26], feature learning for classification [27]- [30], restoration and denoising [31], [32], resolution enhancement [33], [34]. Figure 2 shows an exponentially growing trend in literature published per year for HSI analysisrelated tasks and applications. ...
Hyperspectral Imaging (HSI) has been extensively utilized in many real-life applications because it benefits from the detailed spectral information contained in each pixel. Notably, the complex characteristics i.e., the nonlinear relation among the captured spectral information and the corresponding object of HSI data make accurate classification challenging for traditional methods. In the last few years, Deep Learning (DL) has been substantiated as a powerful feature extractor that effectively addresses the nonlinear problems that appeared in a number of computer vision tasks. This prompts the deployment of DL for HSI classification (HSIC) which revealed good performance. This survey enlists a systematic overview of DL for HSIC and compared state-of-the-art strategies of the said topic. Primarily, we will encapsulate the main challenges of traditional machine learning for HSIC and then we will acquaint the superiority of DL to address these problems. This survey breakdown the state-of-the-art DL frameworks into spectral-features, spatial-features, and together spatial-spectral features to systematically analyze the achievements (future research directions as well) of these frameworks for HSIC. Moreover, we will consider the fact that DL requires a large number of labeled training examples whereas acquiring such a number for HSIC is challenging in terms of time and cost. Therefore, this survey discusses some strategies to improve the generalization performance of DL strategies which can provide some future guidelines. Fig. 1: Various real-world applications of HSI. The source codes will be made publicly available at https://github.com/AnkurDeria/HSI-Traditional-to-Deep-Models
... For instance, the spectral information of vegetation is prejudiced by a wide range of environmental situations that make it challenging to satisfactorily represent variability without the collection of site-specific field spectra. Thus, considering the aforementioned limitations, HSI analysis is categorized into the following main streams: dimensionality reduction [6]- [8], spectral unmixing [9]- [14], change detection [15]- [17] classification [18]- [21], feature learning for classification [22]- [24], restoration and denoising [25], [26], resolution enhancement [27], [28]. Figure 2 shows an exponentially growing trend in literature published per year for HSI analysis related tasks and applications. ...
Hyperspectral Imaging (HSI) has been extensively utilized in many real-life applications because it benefits from the detailed spectral information contained in each pixel. Notably, the complex characteristics i.e., the nonlinear relation among the captured spectral information and the corresponding object of HSI data make accurate classification challenging for traditional methods. In the last few years, Deep Learning (DL) has been substantiated as a powerful feature extractor that effectively addresses the nonlinear problems that appeared in a number of computer vision tasks. This prompts the deployment of DL for HSI classification (HSIC) which revealed good performance. This survey enlists a systematic overview of DL for HSIC and compared state-of-the-art strategies of the said topic. Primarily, we will encapsulate the main challenges of traditional machine learning for HSIC and then we will acquaint the superiority of DL to address these problems. This survey breakdown the state-of-the-art DL frameworks into spectral-features, spatial-features, and together spatial-spectral features to systematically analyze the achievements (future research directions as well) of these frameworks for HSIC. Moreover, we will consider the fact that DL requires a large number of labeled training examples whereas acquiring such a number for HSIC is challenging in terms of time and cost. Therefore, this survey discusses some strategies to improve the generalization performance of DL strategies which can provide some future guidelines.
... The above model (14) only considers resolution enhancement, and such processing scheme tends to suffer from spectral distortions. Spectral unmixing [57], [58] has been used as a significant spectral regularization to reduce spectral distortions [59]. Once unfolding the HR-HS image X as X (3) ∈ R S×(W ×H) , then a sparse spectral unmixing model is formulated as: ...
Hyperspectral (HS) imaging has shown its superiority in many real applications. However, it is usually difficult to obtain high-resolution (HR) HS images through existing imaging techniques, due to the hardware limitations. To improve the spatial resolution of HS images, this paper proposes an effective hyperspectral-multispectral (HS-MS) image fusion method by combining the ideas of nonlocal low-rank tensor modeling and spectral unmixing. To be more precise, instead of unfolding the HS image into a matrix as done in the literature, we directly represent it as a tensor, then a designed nonlocal Tucker decomposition is used to model its underlying spatial-spectral correlation and the spatial self-similarity. The MS image serves mainly as a data constraint to maintain spatial consistency. To further reduce the spectral distortions in spatial enhancement, endmembers and abundances from the spectral are used for spectral regularization. An efficient algorithm based on the alternating direction method of multipliers (ADMM) is developed to solve the resulting model. Extensive experiments on four HS image datasets demonstrate the superiority of the proposed method over several state-of-the-art HS-MS image fusion methods.
... Hyperspectral images collected by using whisk-broom sensors and push-broom scanners can be degraded by stripes, often caused by calibration error or inconsistent responses between detectors [3]. Such stripes degrade visual quality and pose negative influence on subsequent processing, such as unmixing [4,5], super-resolution [6], classification [7,8], compressive sensing reconstruction [9,10], and recovery [11,12]. In general, three types of stripes exist: horizontal (row-by-row) [13,14], vertical (column-by-column) [15,16], and oblique stripes [17,18]. ...
This paper presents a global and local tensor sparse approximation (GLTSA) model for removing the stripes in hyperspectral images (HSIs). HSIs can easily be degraded by unwanted stripes. Two intrinsic characteristics of the stripes are (1) global sparse distribution and (2) local smoothness along the stripe direction. Stripe-free hyperspectral images are smooth in spatial domain, with strong spectral correlation. Existing destriping approaches often do not fully investigate such intrinsic characteristics of the stripes in spatial and spectral domains simultaneously. Those methods may generate new artifacts in extreme areas, causing spectral distortion. The proposed GLTSA model applies two ℓ 0 -norm regularizers to the stripe components and along-stripe gradient to improve the destriping performance. Two ℓ 1 -norm regularizers are applied to the gradients of clean image in spatial and spectral domains. The double non-convex functions in GLTSA are converted to single non-convex function by mathematical program with equilibrium constraints (MPEC). Experiment results demonstrate that GLTSA is effective and outperforms existing competitive matrix-based and tensor-based destriping methods in visual, as well as quantitative, evaluation measures.
... The specialized literature about remotely sensed HSI data covers a wide range of processing techniques that can efficiently extract the information contained in the HSI cube. The most popular ones include: (i) spectral unmixing (Bioucas-Dias et al., 2012;Heylen et al., 2014;Shi and Wang, 2014;Sánchez et al., 2015;Zhong et al., 2016a), (ii) resolution enhancement (Eismann and Hardie, 2005;Mookambiga and Gomathi, 2016;Yi et al., 2017;Yi et al., 2018), (iii) image restoration and denoising (Xu and Gong, 2008;Chen and Qian, 2011;Zhang et al., 2014;Wei et al., 2017b), (iv) anomaly detection (Stein et al., 2002;Xu et al., 2016;Kang et al., 2017), (v) dimensionality reduction (Bruce et al., 2002;Haut et al., 2018d) and (vi) data classification (Fauvel et al., 2013;Camps-Valls et al., 2014;Ghamisi et al., 2017a). In this work, we particularly focus on the topic of HSI data classification, which has received remarkable attention due its important role in land use and land cover applications (Cheng et al., 2017a), and which is currently one of the most popular techniques for HSI data exploitation (Chang, 2007). ...
Advances in computing technology have fostered the development of new and powerful deep learning (DL) techniques, which have demonstrated promising results in a wide range of applications. Particularly, DL methods have been successfully used to classify remotely sensed data collected by Earth Observation (EO) instruments. Hyperspectral imaging (HSI) is a hot topic in remote sensing data analysis due to the vast amount of information comprised by this kind of images, which allows for a better characterization and exploitation of the Earth surface by combining rich spectral and spatial information. However, HSI poses major challenges for supervised classification methods due to the high dimensionality of the data and the limited availability of training samples. These issues, together with the high intraclass variability (and interclass similarity) –often present in HSI data– may hamper the effectiveness of classifiers. In order to solve these limitations, several DL-based architectures have been recently developed, exhibiting great potential in HSI data interpretation. This paper provides a comprehensive review of the current-state-of-the-art in DL for HSI classification, analyzing the strengths and weaknesses of the most widely used classifiers in the literature. For each discussed method, we provide quantitative results using several well-known and widely used HSI scenes, thus providing an exhaustive comparison of the discussed techniques. The paper concludes with some remarks and hints about future challenges in the application of DL techniques to HSI classification. The source codes of the methods discussed in this paper are available from: https://github.com/mhaut/hyperspectral_deeplearning_review.
... Hyperspectral imaging is used in various fields of science such as remote sensing, astronomy, mineralogy and fluorescence microscopy. In these fields, a great deal of applications such as classification [1], noise removal [2], target detection [3] and super-resolution [4], [5] are studied extensively in the remote sensing community. ...
Sparse hyperspectral unmixing aims at finding the sparse fractional abundance vector of a spectral signature present in a mixed pixel. However, there are several types of noise present in the hyperspectral images. These are called mixed noise including stripes, impulse noise and Gaussian noise which deteriorate the performance of sparse unmixing algorithms. In this study, we simultaneously unmix and denoise the hyperspectral image in a unified framework in the presence of mixed noise. In the denoising step, we utilize a low-rank and sparse decomposition based on a nonconvex approach to approximate the rank of hyperspectral data and eliminate the sparse noise terms. In the unmixing part, we employ a semi-supervised sparse unmixing algorithm which uses a nonconvex heuristic similar to denoising step to promote the sparsity of the abundance matrix. We conduct several experiments on synthetic and real hyperspectral data sets to validate the effectiveness of the proposed method in denoising and unmixing processes.
... The above model (14) only considers resolution enhancement, and such processing scheme tends to suffer from spectral distortions. Spectral unmixing [45] has been used as a significant spectral regularization to reduce spectral distortions [46]. Once unfolding the HR-HS image X as X (3) ∈ R (S×(W ×H)) , then a sparse spectral unmixing model is formulated as: ...
Hyperspectral (HS) imaging has shown its superiority in many real applications. However, it is usually difficult to obtain high-resolution (HR) HS images through existing imaging techniques, due to the hardware limitations. To improve the spatial resolution of HS images, this paper proposes an effective hyperspectral-multispectral (HS-MS) image fusion method by combining the ideas of nonlocal low-rank tensor modeling and spectral unmixing. To be more precise, instead of unfolding the HS image into a matrix as done in the literature, we directly represent it as a tensor, then a designed nonlocal Tucker decomposition is used to model its underlying spatial-spectral correlation and the spatial self-similarity. The MS image serves mainly as a data constraint to maintain spatial consistency. To further reduce the spectral distortions in spatial enhancement, endmembers and abundances from the spectral are used for spectral regularization. An efficient algorithm based on the alternating direction method of multipliers (ADMM) is developed to solve the resulting model. Extensive experiments on both simulated and real data sets demonstrate the superiority of the proposed method over several state-of-the-art HS-MS image fusion methods.
... Another abundance regularization called vector-total-variation was proposed by Simões et al. [23], which controlled the spatial distribution of subspace coefficients, where the subspace can be defined either by singular value decomposition (SVD) or by endmember spectral signatures. In [24], Chen et al. proposed an interactive feedback scheme for spatial resolution enhancement, in which spectral unmixing was used as a prior for spatial resolution enhancement. Spectral unmixing based methods split the H2SI into an endmember matrix and an abundance matrix. ...
In order to reconstruct a high spatial and high spectral resolution image (H2SI), one of the most common methods is to fuse a hyperspectral image (HSI) with a corresponding multispectral image (MSI). To effectively obtain both the spectral correlation of bands in HSI and the spatial correlation of pixels in MSI, this paper proposes an adversarial selection fusion (ASF) method for the HSI–MSI fusion problem. Firstly, the unmixing based fusion (UF) method is adopted to dig out the spatial correlation in MSI. Then, to acquire the spectral correlation in HSI, a band reconstruction-based fusion (BRF) method is proposed, regarding H2SI as the product of the optimized band image dictionary and reconstruction coefficients. Finally, spectral spatial quality (SSQ) index is designed to guide the adversarial selection process of UF and BRF. Experimental results on four real-world images demonstrate that the proposed strategy achieves smaller errors and better reconstruction results than other comparison methods.
... New approaches are emerging specific to the fusion of hyperspectral imagery e.g. Yi et al. (2017) taking advantage of spectral unmixing as a constraint to spatial resolution enhancement, but these are not yet widely implemented in commercial software. ...
Hyperspectral image classification has become a hot research topic. HSI has been widely used in a wide range of real-world application areas due to the in-depth spectral information stored within each pixel. Noticeably, the detailed features - i.e., a nonlinear correlation between the obtained spectral data and the correlating HSI data object, generate efficient classification results that are complex for traditional techniques. Deep Learning (DL) has recently been validated as an influential feature extractor that efficiently identifies the nonlinear issues that have arisen in various computer vision challenges. This motivates using DL for Hyperspectral Image Classification (HSIC), which shows promising results. This survey provides a brief description of DL for HSIC and compares cutting-edge methodologies in the field. We will first summarize the key challenges for HSIC, and then we will discuss the superiority of DL and DL-ensemble in addressing these issues. In this article, we divide the state-of-the-art DL methodologies and DL with ensemble into spectral features, spatial features, and combined spatial-spectral features in order to comprehensively and critically evaluate the progress (future research directions as well) of such methodologies for HSIC. Furthermore, we will take into account that DL involves a substantial percentage of labeled training images, whereas obtaining such a number for HSI is time and cost-consuming. As a result, this survey describes some methodologies for improving the classification performance of DL techniques, which can serve as future recommendations.
Spatial resolution enhancement and its subsequent tasks are always separated in conventional hyperspectral image (HSI) processing model. The requirement of the following task, such as unmixing, cannot be referred by spatial resolution enhancement. Moreover, errors and artifacts will also be transmitted and accumulated. In this work, we propose a joint processing method of spatial resolution enhancement and spectral unmixing for HSI (J-SRE-Un), where these two tasks are treated as constraints for each other to simultaneously achieve better performance. Experiments on both simulated and real data demonstrate the effectiveness and superiority of our method.
High spectral correlations and non-local self-similarities, as two intrinsic characteristics underlying hyperspectral image (HSI), have been widely used in HSI super-resolution. However, existing methods mostly utilize the two intrinsic characteristics separately, which still inadequately exploit spatial and spectral information. To address this issue, in this study, a novel self-projected smooth prior (SPSP) is proposed for the task of HSI super-resolution. SPSP describes that two full-band patches (FBPs) are close to each other and then the corresponding subspace coefficients are also close to each other, namely smooth dependences of clustered FBPs within each group of HSI. Suppose that each group of FBPs extracted from HSI lies in smooth subspace, all FBPs within each group can be regarded as the nodes on an undirected graph, then the underlying smooth subspace structures within each group of HSI are implicitly depicted by capturing the linearly pair-wise correlation between those nodes. Utilizing each group of clustered FBPs as projection basis matrix can adaptively and effectively learn the smooth subspace structures. Besides, different from existing methods exploiting non-local self-similarities with multispectral image, to our knowledge, this work represents the first effort to exploit the non-local self-similarities on its spectral intrinsic dimension of desired HSI. In this way, spectral correlations and non-local self-similarities of HSI are incorporated into a unified paradigm to exploit spectral and spatial information simultaneously. As thus, the well learned SPSP is incorporated into the objective function solved by the alternating direction method of multipliers (ADMM). Experimental results on synthetic and real hyperspectral data demonstrate the superiority of the proposed method.
The study has been carried out on the Saranda in Jharkhand and a part of Odisha state. Geospatial data of Hyperion, a remote sensing satellite, have been used. This study has used a wide variety of patterns related to image processing to enhance and extract the mining class of Fe and Mn ores.Landsat-8, OLI sensor data have also been used to correctly explore related minerals. In this way, various processes have been applied to increase the mineralogy class and comparative evaluation with related frequency done. The Hyperion dataset for hyperspectral remote sensing has been specifically verified as an effective tool for mineral or rock information extraction within the band range of shortwave infrared used. The abundant spatial and spectral information contained in hyperspectral images enables the differentiation of different objects of any object into targeted applications for exploration such as exploration detection, mining.
We propose a novel graph Laplacian-guided coupled tensor decomposition (gLGCTD) model for fusion of hyperspectral image (HSI) and multispectral image (MSI) for spatial and spectral resolution enhancements. The coupled Tucker decomposition is employed to capture the global interdependencies across the different modes to fully exploit the intrinsic global spatial-spectral information. To preserve local characteristics, the complementary submanifold structures embedded in high-resolution (HR)-HSI are encoded by the graph Laplacian regularizations. The global spatial-spectral information captured by the coupled Tucker decomposition and the local submanifold structures are incorporated into a unified framework. The gLGCTD fusion framework is solved by a hybrid framework between the proximal alternating optimization (PAO) and the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real data sets demonstrate that the gLGCTD fusion method is superior to state-of-the-art fusion methods with a more accurate reconstruction of the HR-HSI.
While hyperspectral (HS) images play a significant role in many applications, they often suffer from issues such as low spatial resolution, low temporal resolution, and some of the acquired spectral bands are either with low signal-to-noise ratio (SNR) or invalid because of the very high-noise level. To address this issue, a spectral super-resolution method is proposed in this paper to recover a high-spectral-resolution HS image from multispectral (MS) images. The reconstructed HS image will have the same spatial resolution and coverage as the input MS image. The proposed method involves spectral improvement strategy and spatial preservation strategy. For spectral improvement strategy, auxiliary MS/HS image pairs of different landscapes are exploited to estimate spectral response relationship so that an HS image is obtained as an intermediate result. Then, spectral dictionary learning is exploited to recover a more accurate spectral reconstruction result. Spatial preservation strategy is used as a spatial constraint to ensure spatial consistency. In addition, the low-rank property of HS image is also introduced to make the use of global spectral coherence among HS bands. Experiments are conducted on both simulated and real datasets including spectral enhancement of RGB image and the MS image generated by AVIRIS data and real MS/HS data (ALI and Hyperion) captured by Earth Observing-1 (EO-1) satellite. Experiment results demonstrate the superiority of our proposed method to other state-of-the-art methods.
Super-resolution image reconstruction has been utilized to overcome the problem of spatial resolution limitation in hyperspectral (HS) imaging. To improve the spatial resolution of HS image, this paper proposes an HS-multispectral (MS) fusion method, which exploits spatial and spectral correlations and proper regularization. High spatial correlation between MS image and the desired high-resolution HS image is conserved via an over-completed dictionary, and the spectral degradation between them projected onto the space of sparsity is applied as the spectral constraint. The high spectral correlation between high-spatial- and low-spatial-resolution HS image is preserved through linear spectral unmixing. The idea of an interactive feedback proposed in our previous work is also used when dealing with spatial reconstruction and unmixing. Low-rank property is introduced in this paper to regularize the sparse coefficients of the HS patch matrix, which is utilized as the spatial constraint. Experiments on both simulated and real data sets demonstrate that the proposed fusion algorithm achieves lower spectral distortions and the super-resolution results are superior to those of other state-of-the-art methods.
Hyperspectral image (HSI) denoising is significant for correct interpretation. In this paper, a sparse representation framework that unifies denoising and spectral unmixing in a closed-loop manner is proposed. While conventional approaches treat denoising and unmixing separately, the proposed scheme utilizes spectral information from unmixing as feedback to correct spectral distortion. Both denoising and spectral unmixing act as constraints to the others and are solved iteratively. Noise is suppressed via sparse coding, and fractional abundance in spectral unmixing is estimated using the sparsity prior of endmembers from a spectral library. The abundance of endmembers is used as a spectral regularizer for denoising based on the hypothesis that spectral signatures obtained from a denoising process result are close to those of unmixing. Unmixing restrains spectral distortion and results in better denoising, which reciprocally leads to further improvements in unmixing. The strength of our proposed method is illustrated by simulated and real HSIs with performance competitive to the state-of-the-art denoising and unmixing methods.
The acquired hyperspectral data are always in low resolution in both spatial and spectral domains, which will result in lots of mixed pixels and degrade the detection and recognition performance in civil and military applications. So many super resolution techniques are applied to overcome this limit. In this paper, we propose a coupled hyperspectral spatial super-resolution and spectral unmixing method based on sparse representation. Combing spatial super-resolution and spectral unmixing can precisely conserve both spatial information and spectral correlation among different bands. Spectral unmixing is taken as a regularization term in spatial super-resolution to test spectral consistency and avoid spectral distortion, while spatial super-resolution is used to enhance the resolution of abundance map after spectral unmixing.
Pansharpening aims at fusing a panchromatic image with a multispectral one,
to generate an image with the high spatial resolution of the former and the
high spectral resolution of the latter. In the last decade, many algorithms
have been presented in the literature for pansharpening using multispectral
data. With the increasing availability of hyperspectral systems, these methods
are now being adapted to hyperspectral images. In this work, we compare new
pansharpening techniques designed for hyperspectral data with some of the state
of the art methods for multispectral pansharpening, which have been adapted for
hyperspectral data. Eleven methods from different classes (component
substitution, multiresolution analysis, hybrid, Bayesian and matrix
factorization) are analyzed. These methods are applied to three datasets and
their effectiveness and robustness are evaluated with widely used performance
indicators. In addition, all the pansharpening techniques considered in this
paper have been implemented in a MATLAB toolbox that is made available to the
community.
Hyperspectral image (HSI) denoising is an essential pre-process step to improve the performance of subsequent applications. For HSI, there is much global and local redundancy and correlation (RAC) in spatial/spectral dimensions. And denoising performance can be improved greatly if RAC is utilized efficiently in denoising process. In this paper, a HSI denoising method is proposed by jointly utilizing the global and local RAC in spatial/spectral domains. First, sparse coding is exploited to model the global RAC in spatial domain and local RAC in spectral domain. Noise can be removed by sparse approximated data with learned dictionary. At this stage, only local RAC in spectral domain is employed. It will cause spectral distortion. To compensate the shortcoming of local spectral RAC, low rank constraint is used to deal with the global RAC in spectral domain. Different hyperspectral datasets are used to test the performance of proposed method. The denoising results by the proposed method are superior to results obtained by other state-of-the-art hyperspectral denoising methods.
In this paper we apply the recently proposed J-SparseFI data fusion method to the fusion of a low-resolution hyperspectral image and a high-resolution multispectral image. The high correlation of signals in adjacent hyperspectral channels is exploited by assuming signals in different channels are jointly sparse in suitable dictionaries that are created from the multispectral image. First experimental results using airborne HySpex hyperspectral data and synthesized WorldView-2 imagery are presented.
Hyperspectral (HS) super-resolution reconstruction is an ill-posed inversion problem, for which the solution from reconstruction constraint is not unique. To address this, an HS image super-resolution method is proposed to first utilize the joint regulation of spatial and spectral nonlocal similarities. We then fused the HS and panchromatic images with sparse regulation. With these two regulation terms, edge sharpness and spectrum consistency are preserved and noises are suppressed. The proposed method is tested with Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and Hyperion images and evaluated by quantitative measures. The resulting enhanced images from the proposed method are superior to the results obtained by other well-known methods.
Data provided by most optical earth observation satellites such as IKONOS, Quick Bird and GeoEye are composed of a panchromatic channel of high spatial resolution (HR) and several multispectral channels at a lower spatial resolution (LR). The fusion of a HR panchromatic and the corresponding LR spectral channels is called “pan-sharpening”. It aims at obtaining a HR multispectral image.
In this paper, we propose a new pan-sharpening method named Sparse Fusion of Images (SparseFI, pronounced "sparsify"). SparseFI is based on the compressive sensing theory and explores the sparse representation of HR/LR multispectral image patches in the dictionary pairs co-trained from the panchromatic image and its downsampled LR version. Compared to conventional methods it “learns” from, i.e. adapts itself to, the data and has generally better performance than existing methods. Due to the fact that the SparseFI method does not assume any spectral composition model of the panchromatic image and thanks to the super-resolution capability and robustness of sparse signal reconstruction algorithms, it gives higher spatial resolution and in most cases less spectral distortion compared to the conventional methods.
The amount of noise included in a hyperspectral image limits its application and has a negative impact on hyperspectral image classification, unmixing, target detection, and so on. In hyperspectral images, because the noise intensity in different bands is different, to better suppress the noise in the high-noise-intensity bands and preserve the detailed information in the low-noise-intensity bands, the denoising strength should be adaptively adjusted with the noise intensity in the different bands. Meanwhile, in the same band, there exist different spatial property regions, such as homogeneous regions and edge or texture regions; to better reduce the noise in the homogeneous regions and preserve the edge and texture information, the denoising strength applied to pixels in different spatial property regions should also be different. Therefore, in this paper, we propose a hyperspectral image denoising algorithm employing a spectral-spatial adaptive total variation (TV) model, in which the spectral noise differences and spatial information differences are both considered in the process of noise reduction. To reduce the computational load in the denoising process, the split Bregman iteration algorithm is employed to optimize the spectral-spatial hyperspectral TV model and accelerate the speed of hyperspectral image denoising. A number of experiments illustrate that the proposed approach can satisfactorily realize the spectral-spatial adaptive mechanism in the denoising process, and superior denoising results are produced.
The K-SVD algorithm is a highly efiective method of training overcomplete dic- tionaries for sparse signal representation. In this report we discuss an e-cient im- plementation of this algorithm, which both accelerates it and reduces its memory consumption. The two basic components of our implementation are the replacement of the exact SVD computation with a much quicker approximation, and the use of the Batch-OMP method for performing the sparse-coding operations. Batch-OMP, which we also present in this report, is an implementation of the Orthogonal Matching Pursuit (OMP) algorithm which is speciflcally optimized for sparse-coding large sets of signals over the same dictionary. The Batch-OMP imple- mentation is useful for a variety of sparsity-based techniques which involve coding large numbers of signals. In the report, we discuss the Batch-OMP and K-SVD implementations and analyze their complexities. The report is accompanied by Matlabr toolboxes which implement these techniques, and can be downloaded at http://www.cs.technion.ac.il/~ronrubin/software.html.
Linear spectral unmixing is a popular tool in remotely sensed hyperspectral data interpretation. It aims at estimating the fractional abundances of pure spectral signatures (also called as endmembers) in each mixed pixel collected by an imaging spectrometer. In many situations, the identification of the end-member signatures in the original data set may be challenging due to insufficient spatial resolution, mixtures happening at different scales, and unavailability of completely pure spectral signatures in the scene. However, the unmixing problem can also be approached in semisupervised fashion, i.e., by assuming that the observed image signatures can be expressed in the form of linear combinations of a number of pure spectral signatures known in advance (e.g., spectra collected on the ground by a field spectroradiometer). Unmixing then amounts to finding the optimal subset of signatures in a (potentially very large) spectral library that can best model each mixed pixel in the scene. In practice, this is a combinatorial problem which calls for efficient linear sparse regression (SR) techniques based on sparsity-inducing regularizers, since the number of endmembers participating in a mixed pixel is usually very small compared with the (ever-growing) dimensionality (and availability) of spectral libraries. Linear SR is an area of very active research, with strong links to compressed sensing, basis pursuit (BP), BP denoising, and matching pursuit. In this paper, we study the linear spectral unmixing problem under the light of recent theoretical results published in those referred to areas. Furthermore, we provide a comparison of several available and new linear SR algorithms, with the ultimate goal of analyzing their potential in solving the spectral unmixing problem by resorting to available spectral libraries. Our experimental results, conducted using both simulated and real hyperspectral data sets collected by the NASA Jet Propulsion Laboratory's Airborne Visible I-
-
nfrared Imaging Spectrometer and spectral libraries publicly available from the U.S. Geological Survey, indicate the potential of SR techniques in the task of accurately characterizing the mixed pixels using the library spectra. This opens new perspectives for spectral unmixing, since the abundance estimation process no longer depends on the availability of pure spectral signatures in the input data nor on the capacity of a certain endmember extraction algorithm to identify such pure signatures.
This article presents a novel method for the enhancement of the spatial quality of hyperspectral (HS) images through the use of a high resolution panchromatic (PAN) image. Due to the high number of bands, the application of a pan-sharpening technique to HS images may result in an increase of the computational load and complexity. Thus a dimensionality reduction preprocess, compressing the original number of measurements into a lower dimensional space, becomes mandatory. To solve this problem, we propose a pan-sharpening technique combining both dimensionality reduction and fusion, making use of non-linear principal component analysis (NLPCA) and Indusion, respectively, to enhance the spatial resolution of a HS image. We have tested the proposed algorithm on HS images obtained from CHRIS-Proba sensor and PAN image obtained from World view 2 and demonstrated that a reduction using NLPCA does not result in any significant degradation in the pan-sharpening results.
This paper introduces the concept and principles of hyperspectral imaging (HSI) and it briefly outlines how the defence and homeland security sectors can benefit from the application of this extremely versatile technology. This paper outlines the pros and cons of the various HSI system configurations, with particular emphasis on two of the most commonly deployed spectrograph techniques, namely, the dispersive system and the narrow-band tuning filter system. It describes how HSI can be utilized for target acquisition particularly when there is no a priori knowledge of the target, and then shows how it can be used for the recognition and tracking of targets with desired or known signature characteristics. The paper also briefly mentions the possibility of remote HSI being used for recognizing a human's physiological state such as that induced by stress or anxiety. Real experimental data collected during the course of our research have been utilized throughout this paper to help understand the versatility and effectiveness of HSI technology.
Compact High Resolution Imaging Spectrometer onboard the Project for On-board Autonomy, or CHRIS/Proba, represents a new generation of satellite images that provide different acquisitions of the same scene at five different angles. Given the hyperspectral-oriented waveband configuration of the CHRIS images, the scope of its application would be much wider if the present 17m nadir resolution could be refined. This paper presents the results of three superresolution methods applied to multiangular CHRIS/Proba data. The CHRIS images were preprocessed and then calibrated into reflectance using the method described in [1][2]. Automatic registration using an intensity variation approach described in [3] was implemented for motion estimation. Three methods, namely non-uniform interpolation and de-convolution [4], iterative back-projection [5], and total variation [6] are examined. Quantitative measures including peak signal to noise ratio [7], structural similarity [8], and edge stability [9], are used for the evaluation of the image quality. To further examine the benefit of multi-frame superresolution methods, a single-frame superresolution method of bicubic resampling was also applied. Our results show that a high resolution image derived from superresolution methods enhance spatial resolution and provides substantially more image details. The spectral profiles of selected land covers before and after the application of superresolution show negligible differences, hinting the use of superresolution algorithm would not degrade the capability of the data set for classification. Among the three methods, total variation gives the best performance in all quantitative measures. Visual inspections find good results with total variation and iterative back-projection approaches. The use of superresolution algorithms, however, is complex as there are many parameters. In this paper, most of the parameter settings were tuned manually or decided empirically.
Hyperspectral imaging systems are starting to be used as a scientific tool for food quality assessment. A typical hyperspectral
image is composed of a set of a relatively wide range of monochromatic images corresponding to continuous wavelengths that
normally contain redundant information or may exhibit a high degree of correlation. In addition, computation of the classifiers
used to deal with the data obtained from the images can become excessively complex and time-consuming for such high-dimensional
datasets, and this makes it difficult to incorporate such systems into an industry that demands standard protocols or high-speed
processes. Therefore, recent works have focused on the development of new systems based on this technology that are capable
of analysing quality features that cannot be inspected using visible imaging. Many of those studies have also centred on finding
new statistical techniques to reduce the hyperspectral images to multispectral ones, which are easier to implement in automatic,
non-destructive systems. This article reviews recent works that use hyperspectral imaging for the inspection of fruit and
vegetables. It explains the different technologies available to acquire the images and their use for the non-destructive inspection
of the internal and external features of these products. Particular attention is paid to the works aimed at reducing the dimensionality
of the images, with details of the statistical techniques most commonly used for this task.
KeywordsComputer vision–Fruits–Vegetables–Quality–Non-destructive inspection–Image analysis–Hyperspectral imaging–Multispectral imaging
Spectral unmixing aims at estimating the fractional abundances of pure spectral signatures (also called endmembers) in each mixed pixel collected by a remote sensing hyperspectral imaging instrument. In recent work, the linear spectral unmixing problem has been approached in semisupervised fashion as a
sparse regression one, under the assumption that the observed image signatures can be expressed as linear combinations of pure spectra, known a priori and available in a library. It happens, however, that sparse unmixing focuses on analyzing the hyperspectral data without incorporating spatial information. In this paper, we include the total variation (TV) regularization to the classical sparse regression formulation, thus exploiting the spatial–contextual information present in the hyperspectral images and developing a new algorithm called sparse unmixing via variable splitting augmented Lagrangian and TV. Our experimental results, conducted with both simulated and real hyperspectral data sets, indicate the potential of including spatial information (through the TV term) on sparse unmixing formulations for improved
characterization of mixed pixels in hyperspectral imagery.
The spectral features in hyperspectral imagery (HSI) contain significant structure that, if properly characterized, could enable more efficient data acquisition and improved data analysis. Because most pixels contain reflectances of just a few materials, we propose that a sparse coding model is well-matched to HSI data. Sparsity models consider each pixel as a combination of just a few elements from a larger dictionary, and this approach has proven effective in a wide range of applications. Furthermore, previous work has shown that optimal sparse coding dictionaries can be learned from a dataset with no other a priori information (in contrast to many HSI “endmember” discovery algorithms that assume the presence of pure spectra or side information). We modified an existing unsupervised learning approach and applied it to HSI data (with significant ground truth labeling) to learn an optimal sparse coding dictionary. Using this learned dictionary, we demonstrate three main findings: 1) the sparse coding model learns spectral signatures of materials in the scene and locally approximates nonlinear manifolds for individual materials; 2) this learned dictionary can be used to infer HSI-resolution data with very high accuracy from simulated imagery collected at multispectral-level resolution, and 3) this learned dictionary improves the performance of a supervised classification algorithm, both in terms of the classifier complexity and generalization from very small training sets.
This paper presents a new approach to single-image superresolution, based upon sparse signal representation. Research on image statistics suggests that image patches can be well-represented as a sparse linear combination of elements from an appropriately chosen over-complete dictionary. Inspired by this observation, we seek a sparse representation for each patch of the low-resolution input, and then use the coefficients of this representation to generate the high-resolution output. Theoretical results from compressed sensing suggest that under mild conditions, the sparse representation can be correctly recovered from the downsampled signals. By jointly training two dictionaries for the low- and high-resolution image patches, we can enforce the similarity of sparse representations between the low-resolution and high-resolution image patch pair with respect to their own dictionaries. Therefore, the sparse representation of a low-resolution image patch can be applied with the high-resolution image patch dictionary to generate a high-resolution image patch. The learned dictionary pair is a more compact representation of the patch pairs, compared to previous approaches, which simply sample a large amount of image patch pairs , reducing the computational cost substantially. The effectiveness of such a sparsity prior is demonstrated for both general image super-resolution (SR) and the special case of face hallucination. In both cases, our algorithm generates high-resolution images that are competitive or even superior in quality to images produced by other similar SR methods. In addition, the local sparse modeling of our approach is naturally robust to noise, and therefore the proposed algorithm can handle SR with noisy inputs in a more unified framework.
Imaging spectrometers measure electromagnetic energy scattered in their
instantaneous field view in hundreds or thousands of spectral channels with
higher spectral resolution than multispectral cameras. Imaging spectrometers
are therefore often referred to as hyperspectral cameras (HSCs). Higher
spectral resolution enables material identification via spectroscopic analysis,
which facilitates countless applications that require identifying materials in
scenarios unsuitable for classical spectroscopic analysis. Due to low spatial
resolution of HSCs, microscopic material mixing, and multiple scattering,
spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus,
accurate estimation requires unmixing. Pixels are assumed to be mixtures of a
few materials, called endmembers. Unmixing involves estimating all or some of:
the number of endmembers, their spectral signatures, and their abundances at
each pixel. Unmixing is a challenging, ill-posed inverse problem because of
model inaccuracies, observation noise, environmental conditions, endmember
variability, and data set size. Researchers have devised and investigated many
models searching for robust, stable, tractable, and accurate unmixing
algorithms. This paper presents an overview of unmixing methods from the time
of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models
are first discussed. Signal-subspace, geometrical, statistical, sparsity-based,
and spatial-contextual unmixing algorithms are described. Mathematical problems
and potential solutions are described. Algorithm characteristics are
illustrated experimentally.
We propose a new fast algorithm for solving one of the standard approaches to ill-posed linear inverse problems (IPLIP), where a (possibly nonsmooth) regularizer is minimized under the constraint that the solution explains the observations sufficiently well. Although the regularizer and constraint are usually convex, several particular features of these problems (huge dimensionality, nonsmoothness) preclude the use of off-the-shelf optimization tools and have stimulated a considerable amount of research. In this paper, we propose a new efficient algorithm to handle one class of constrained problems (often known as basis pursuit denoising) tailored to image recovery applications. The proposed algorithm, which belongs to the family of augmented Lagrangian methods, can be used to deal with a variety of imaging IPLIP, including deconvolution and reconstruction from compressive observations (such as MRI), using either total-variation or wavelet-based (or, more generally, frame-based) regularization. The proposed algorithm is an instance of the so-called alternating direction method of multipliers, for which convergence sufficient conditions are known; we show that these conditions are satisfied by the proposed algorithm. Experiments on a set of image restoration and reconstruction benchmark problems show that the proposed algorithm is a strong contender for the state-of-the-art.
This paper presents a novel maximum a posteriori estimator for enhancing the spatial resolution of an image using co-registered high spatial-resolution imagery from an auxiliary sensor. Here, we focus on the use of high-resolution panchromatic data to enhance hyperspectral imagery. However, the estimation framework developed allows for any number of spectral bands in the primary and auxiliary image. The proposed technique is suitable for applications where some correlation, either localized or global, exists between the auxiliary image and the image being enhanced. To exploit localized correlations, a spatially varying statistical model, based on vector quantization, is used. Another important aspect of the proposed algorithm is that it allows for the use of an accurate observation model relating the "true" scene with the low-resolutions observations. Experimental results with hyperspectral data derived from the airborne visible-infrared imaging spectrometer are presented to demonstrate the efficacy of the proposed estimator.
Spectral unmixing using hyperspectral data represents a
significant step in the evolution of remote decompositional analysis
that began with multispectral sensing. It is a consequence of collecting
data in greater and greater quantities and the desire to extract more
detailed information about the material composition of surfaces. Linear
mixing is the key assumption that has permitted well-known algorithms to
be adapted to the unmixing problem. In fact, the resemblance of the
linear mixing model to system models in other areas has permitted a
significant legacy of algorithms from a wide range of applications to be
adapted to unmixing. However, it is still unclear whether the assumption
of linearity is sufficient to model the mixing process in every
application of interest. It is clear, however, that the applicability of
models and techniques is highly dependent on the variety of
circumstances and factors that give rise to mixed pixels. The outputs of
spectral unmixing, endmember, and abundance estimates are important for
identifying the material composition of mixtures
In recent years there has been a growing interest in the study of sparse representation of signals. Using an overcomplete dictionary that contains prototype signal-atoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and include compression, regularization in inverse problems, feature extraction, and more. Recent activity in this field has concentrated mainly on the study of pursuit algorithms that decompose signals with respect to a given dictionary. Designing dictionaries to better fit the above model can be done by either selecting one from a prespecified set of linear transforms or adapting the dictionary to a set of training signals. Both of these techniques have been considered, but this topic is largely still open. In this paper we propose a novel algorithm for adapting dictionaries in order to achieve sparse signal representations. Given a set of training signals, we seek the dictionary that leads to the best representation for each member in this set, under strict sparsity constraints. We present a new method-the K-SVD algorithm-generalizing the K-means clustering process. K-SVD is an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data. The update of the dictionary columns is combined with an update of the sparse representations, thereby accelerating convergence. The K-SVD algorithm is flexible and can work with any pursuit method (e.g., basis pursuit, FOCUSS, or matching pursuit). We analyze this algorithm and demonstrate its results both on synthetic tests and in applications on real image data
High-resolution image reconstruction refers to reconstructing high-resolution images from multiple low-resolution, shifted, degraded samples of a true image. In this paper, we analyze this problem from the wavelet point of view. By expressing the true image as a function in L 2 (R ), we derive iterative algorithms which recover the function completely in the L 2 sense from the given low-resolution functions. These algorithms decompose the function obtained from the previous iteration into different frequency components in the wavelet transform domain and add them into the new iterate to improve the approximation. We apply wavelet (packet) thresholding methods to denoise the function obtained in the previous step before adding it into the new iterate.
In this paper we apply the recently proposed J-SparseFI data fusion method to the fusion of a low-resolution hyperspectral image and a high-resolution multispectral image. The high correlation of signals in adjacent hyperspectral channels is exploited by assuming signals in different channels are jointly sparse in suitable dictionaries that are created from the multispectral image. First experimental results using airborne HySpex hyperspectral data and synthesized WorldView-2 imagery are presented.
In various computer vision applications, often we need to convert an image in one style into another style for better visualization, interpretation and recognition; for examples, up-convert a low resolution image to a high resolution one, and convert a face sketch into a photo for matching, etc. A semi-coupled dictionary learning (SCDL) model is proposed in this paper to solve such cross-style image synthesis problems. Under SCDL, a pair of dictionaries and a mapping function will be simultaneously learned. The dictionary pair can well characterize the structural domains of the two styles of images, while the mapping function can reveal the intrinsic relationship between the two styles' domains. In SCDL, the two dictionaries will not be fully coupled, and hence much flexibility can be given to the mapping function for an accurate conversion across styles. Moreover, clustering and image nonlocal redundancy are introduced to enhance the robustness of SCDL. The proposed SCDL model is applied to image super-resolution and photo-sketch synthesis, and the experimental results validated its generality and effectiveness in cross-style image synthesis.
An extended superresolution observation model is proposed for POCS superresolution of hyperspectral images. Multiple constraint criteria based on a priori knowledge were incorporated: data consistence, amplitude constraint, Total Variation edge smoothing constraint, outlier rejection, and PCA based denoising. The constraint criteria are applied using POCS superresolution reconstruction. The method was tested with both simulation and multi-viewing hyperspectral CHRIS images. Preliminary results of the constraint based superresolution shows potential for angular hyperspectral images.
Hyperspectral imaging is a powerful technology for remotely inferring the material properties of the objects in a scene of interest. Hyperspectral images consist of spatial maps of light intensity variation across a large number of spectral bands or wavelengths; alternatively, they can be thought of as a measurement of the spectrum of light transmitted or reflected from each spatial location in a scene. Because chemical elements have unique spectral signatures, observing the spectra at a high spatial and spectral resolution provides information about the material properties of the scene with much more accuracy than is possible with conventional three-color images. As a result, hyperspectral imaging is used in a variety of important applications, including remote sensing, astronomical imaging, and fluorescence microscopy.
The spatial resolution of a hyperspectral image is often coarse because of the limitations of the imaging hardware. Super-resolution reconstruction (SRR) is a promising signal post-processing technique for hyperspectral image resolution enhancement. This paper proposes a maximum a posteriori (MAP) based multi-frame super-resolution algorithm for hyperspectral images. Principal component analysis (PCA) is utilized in both parts of the proposed algorithm: motion estimation and image reconstruction. A simultaneous motion estimation method with the first few principal components, which contain most of the information of a hyperspectral image, is proposed to reduce computational load and improve motion field accuracy. In the image reconstruction part, different image resolution enhancement techniques are applied to different groups of components, to reduce computational load and simultaneously remove noise. The proposed algorithm is tested on both synthetic images and real image sequences. The experimental results and comparative analyses verify the effectiveness of this algorithm.
Hyperspectral remote sensing is widely used in many fields suchas agriculture, military detection, mineral exploration, and so on. Hyperspectral data has very high spectral resolution, but much lower spatial resolution than the data obtained by other types of sensors. The low spatial resolution restrains its wide applications. On the contrary, we easily obtain images with high spatial resolution but insufficient spectral resolution (like panchromatic images). Naturally, people expect to obtain images that have high spatial and spectral resolution at the same time by the hyperspectral image fusion. In this paper, a similarity measure-based variational method is proposed to achieve the fusion process. The main idea is to transform the image fusion problem to an optimization problem based on the variational model. We first establish a fusion model that constrains the spatial and spectral information of the original data at the same time, then use the split bregman iteration to obtain the final fused data. Also, we analyze the convergence of the method. The experiments on the synthetic and real data show that the fusion method preserves the information of the original images efficiently, especially on the spectral information.
Different research groups have recently studied the concept of wavelet image fusion between panchromatic and multispectral images using different approaches. In this paper, a new approach using the wavelet based method for data fusion between hyperspectral and multispectral images is presented. Using this wavelet concept of hyperspectral and multispectral data fusion, we performed image fusion between two spectral levels of a hyperspectral image and one band of multispectral image. The reconstructed image has a root mean square error of 2.8 per pixel and a signal-to- noise ratio of 36 dB. We achieved our goal of creating a composite image that has the same spectral resolution as the hyperspectral image and the same spatial resolution as the multispectral image with minimum artifacts.
We consider linear equations y = Φx where y is a given vector in ℝn and Φ is a given n × m matrix with n < m ≤ τn, and we wish to solve for x ∈ ℝm. We suppose that the columns of Φ are normalized to the unit 2-norm, and we place uniform measure on such Φ. We prove the existence of ρ = ρ(τ) > 0 so that for large n and for all Φ's except a negligible fraction, the following property holds: For every y having a representation y = Φx0by a coefficient vector x0 ∈ ℝmwith fewer than ρ · n nonzeros, the solution x1of the 1-minimization problemis unique and equal to x0. In contrast, heuristic attempts to sparsely solve such systems—greedy algorithms and thresholding—perform poorly in this challenging setting. The techniques include the use of random proportional embeddings and almost-spherical sections in Banach space theory, and deviation bounds for the eigenvalues of random Wishart matrices. © 2006 Wiley Periodicals, Inc.
As a powerful statistical image modeling technique, sparse representation has been successfully used in various image restoration applications. The success of sparse representation owes to the development of the
l
<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub>
-norm optimization techniques and the fact that natural images are intrinsically sparse in some domains. The image restoration quality largely depends on whether the employed sparse domain can represent well the underlying image. Considering that the contents can vary significantly across different images or different patches in a single image, we propose to learn various sets of bases from a precollected dataset of example image patches, and then, for a given patch to be processed, one set of bases are adaptively selected to characterize the local sparse domain. We further introduce two adaptive regularization terms into the sparse representation framework. First, a set of autoregressive (AR) models are learned from the dataset of example image patches. The best fitted AR models to a given patch are adaptively selected to regularize the image local structures. Second, the image nonlocal self-similarity is introduced as another regularization term. In addition, the sparsity regularization parameter is adaptively estimated for better image restoration performance. Extensive experiments on image deblurring and super-resolution validate that by using adaptive sparse domain selection and adaptive regularization, the proposed method achieves much better results than many state-of-the-art algorithms in terms of both PSNR and visual perception.
The goal of pan-sharpening is to fuse a low spatial resolution multispectral image with a higher resolution panchromatic image to obtain an image with high spectral and spatial resolution. The Intensity-Hue-Saturation (IHS) method is a popular pan-sharpening method used for its efficiency and high spatial resolution. However, the final image produced experiences spectral distortion. In this letter, we introduce two new modifications to improve the spectral quality of the image. First, we propose image-adaptive coefficients for IHS to obtain more accurate spectral resolution. Second, an edge-adaptive IHS method was proposed to enforce spectral fidelity away from the edges. Experimental results show that these two modifications improve spectral resolution compared to the original IHS and we propose an adaptive IHS that incorporates these two techniques. The adaptive IHS method produces images with higher spectral resolution while maintaining the high-quality spatial resolution of the original IHS.
Imaging spectroscopy, also known as hyperspectral imaging, has been transformed in less than 30 years from being a sparse research tool into a commodity product available to a broad user community. Currently, there is a need for standardized data processing techniques able to take into account the special properties of hyperspectral data. In this paper, we provide a seminal view on recent advances in techniques for hyperspectral image processing. Our main focus is on the design of techniques able to deal with the high-dimensional nature of the data, and to integrate the spatial and spectral information. Performance of the discussed techniques is evaluated in different analysis scenarios. To satisfy time-critical constraints in specific applications, we also develop efficient parallel implementations of some of the discussed algorithms. Combined, these parts provide an excellent snapshot of the state-of-the-art in those areas, and offer a thoughtful perspective on future potentials and emerging challenges in the design of robust hyperspectral imaging algorithms.
We consider the class of iterative shrinkage-thresholding algorithms (ISTA) for solving linear inverse problems arising in signal/image processing. This class of methods, which can be viewed as an ex- tension of the classical gradient algorithm, is attractive due to its simplicity and thus is adequate for solving large-scale problems even with dense matrix data. However, such methods are also known to converge quite slowly. In this paper we present a new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically. Initial promising nu- merical results for wavelet-based image deblurring demonstrate the capabilities of FISTA which is shown to be faster than ISTA by several orders of magnitude.
In this paper, we present a novel DFT- and wavelet-based estimation scheme for hyperspectral imagery. Optimal hyperspectral image estimation relies on the ability to decorrelate the signal in both space and channel at the cost of requiring second-order signal statistics. This statistical requirement is removed by the proposed estimator, which approximately decorrelates the signal in space using a 2D discrete wavelet transform and in channel using a discrete Fourier transform. In addition to allowing extremely efficient estimation, the proposed estimator vastly improves visual quality and yields typical signal-to-noise ratio gains of over 14 dB.
High correlation among the neighboring pixels both spatially and spectrally in a multispectral image makes it necessary to use an efficient data transformation approach before performing pan-sharpening. Wavelets and principal component analysis (PCA) methods have been a popular choice for spatial and spectral transformations, respectively. Current PCA-based pan-sharpening methods make an assumption that the first principal component (PC) of high variance is an ideal choice for replacing or injecting it with high spatial details from the high-resolution histogram-matched panchromatic (PAN) image. This paper presents a combined adaptive PCA-contourlet approach for pan-sharpening, where the adaptive PCA is used to reduce the spectral distortion and the use of nonsubsampled contourlets for spatial transformation in pan-sharpening is incorporated to overcome the limitation of the wavelets in representing the directional information efficiently and capturing intrinsic geometrical structures of the objects. The efficiency of the presented method is tested by performing pan-sharpening of the high-resolution (IKONOS and QuickBird) and the medium-resolution (Landsat-7 Enhanced Thematic Mapper Plus) datasets. The evaluation of the pan-sharpened images using global validation indexes reveal that the adaptive PCA approach helps reducing the spectral distortion, and its merger with contourlets provides better fusion results.
Linear spectral mixture analysis (LSMA) is a widely used technique
in remote sensing to estimate abundance fractions of materials present
in an image pixel. In order for an LSMA-based estimator to produce
accurate amounts of material abundance, it generally requires two
constraints imposed on the linear mixture model used in LSMA, which are
the abundance sum-to-one constraint and the abundance nonnegativity
constraint. The first constraint requires the sum of the abundance
fractions of materials present in an image pixel to be one and the
second imposes a constraint that these abundance fractions be
nonnegative. While the first constraint is easy to deal with, the second
constraint is difficult to implement since it results in a set of
inequalities and can only be solved by numerical methods. Consequently,
most LSMA-based methods are unconstrained and produce solutions that do
not necessarily reflect the true abundance fractions of materials. In
this case, they can only be used for the purposes of material detection,
discrimination, and classification, but not for material quantification.
The authors present a fully constrained least squares (FCLS) linear
spectral mixture analysis method for material quantification. Since no
closed form can be derived for this method, an efficient algorithm is
developed to yield optimal solutions. In order to further apply the
designed algorithm to unknown image scenes, an unsupervised least
squares error (LSE)-based method is also proposed to extend the FCLS
method in an unsupervised manner
Efficient implementation of the
- R Rubinstein
- M Zibulevsky
- M Elad