Content uploaded by Yong Ge
Author content
All content in this area was uploaded by Yong Ge on Jul 25, 2015
Content may be subject to copyright.
2040 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
Hybrid Constraints of Pure and Mixed Pixels for
Soft-Then-Hard Super-Resolution Mapping With
Multiple Shifted Images
Yuehong Chen, Yong Ge, Member, IEEE, Gerard B. M. Heuvelink, Jianlong Hu, and Yu Jiang
Abstract—Multiple shifted images (MSIs) have been widely
applied to many super-resolution mapping (SRM) approaches to
improve the accuracy of fine-scale land-cover maps. Most SRM
methods with MSIs involve two processes: subpixel sharpening
and class allocation. Complementary information from the MSIs
has been successfully adopted to produce soft attribute values of
subpixels during the subpixel sharpening process. Such informa-
tion, however, is not used in the second process of class allocation.
In this paper, a new class-allocation algorithm, named “hybrid
constraints of pure and mixed pixels” (HCPMP), is proposed to
allocate land-cover classes to subpixels using MSIs. HCPMP first
determines the classes of subpixels that overlap with the pure
pixels of auxiliary images in MSIs, after which the remaining sub-
pixels are classified using information derived from the mixed
pixels of the base image in MSIs. An artificial image and two
remote sensing images were used to evaluate the performance
of the proposed HCPMP algorithm. The experimental results
demonstrate that HCPMP successfully applied MSIs to produce
SRM maps that are visually closer to the reference images and
that have greater accuracy than five existing class-allocation algo-
rithms. Especially, it can produce more accurate SRM maps for
high-resolution land-cover classes than low-resolution cases. The
algorithm takes slightly less runtime than class allocation using
linear optimization techniques. Hence, HCPMP provides a valu-
able new solution for class allocation in SRM using auxiliary data
from MSIs.
Index Terms—Hybrid constraints, multiple shifted images
(MSIs), remotely sensed imagery, super-resolution mapping
(SRM).
Manuscript received August 07, 2014; revised March 18, 2015; accepted
March 23, 2015. Date of publication April 13, 2015; date of current version
July 20, 2015. This work was supported in part by the National Natural
Science Foundation of China under Grant 41471296 and in part by the Key
Technologies Research and Development Program of China (2012BAH33B01).
(Corresponding author: Yong Ge.)
Y. Chen and Y. Jiang are with the State Key Laboratory of Resources and
Environmental Information System, Institute of Geographical Sciences and
Natural Resources Research, University of Chinese Academy of Sciences,
Beijing 100101, China (e-mail: chenyh@lreis.ac.cn; jiangy@lreis.ac.cn).
Y. Ge is with the State Key Laboratory of Resources and Environmental
Information System, Institute of Geographical Sciences and Natural Resources
Research, University of Chinese Academy of Sciences, Beijing 100101, China
and also with Jiangsu Center for Collaborative Innovation in Geographical
Information Resource Development and Application, Nanjing 210023, China
(e-mail: gey@lreis.ac.cn).
G. B. M. Heuvelink is with the Soil Geography and Landscape Group,
Wageningen University, Wageningen 6708 PB, The Netherlands (e-mail:
gerard.heuvelink@wur.nl).
J. Hu is with the School of Computer and Information Technology, Shanxi
University, Taiyuan 030006, China (e-mail: weilong@sxu.edu.cn).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JSTARS.2015.2417191
I. INTRODUCTION
LAND COVER is a fundamental variable in many scien-
tific investigations and operational applications such as
land-cover change, ecology, and hydrology [1], [2]. Extraction
of land-cover maps from remote sensing images is typically
accomplished by classification. However, the presence of mixed
pixels in remote sensing images will often lead to an inaccurate
representation of land-cover by traditional hard and soft classi-
fication techniques [3], [4]. Super-resolution mapping (SRM),
also termed as subpixel mapping, was proposed by Atkinson [5]
to provide a solution to the mixed pixel problem in classifica-
tion. SRM transforms the output (i.e., fraction images) of soft
classification into a hard classification map with a finer spatial
resolution than the input images [6].
In past decades, various SRM approaches have been devel-
oped, including the Hopfield neural network [7], linear opti-
mization techniques [8], genetic algorithms [9], the pixel-
swapping algorithm [10], back-propagation neural networks
[11], [12], Markov random fields [13]–[16], subpixel/pixel
spatial attraction models [17]–[19], the geometric method
[20], the vectorial boundary-based method [21], geostatisti-
cal methods [22], [23], artificial intelligence-based methods
[24]–[27], and radial basis functions [28]. These approaches
have achieved relatively satisfactory performance in several
applications, such as waterline mapping [29], urban tree iden-
tification [13], enhancement of the landscape pattern index
[30], land-cover change detection [31], urban building extrac-
tion [32], lake area estimation [33], and floodplain inundation
mapping [34].
Most traditional SRM approaches are underdetermined in
which there may be multiple plausible solutions if fraction
images from only a single image are applied to predict the spa-
tial distribution of subpixels within a mixed pixel [6], [23],
[35]–[38]. Such an underdetermined process leads to ambi-
guity and uncertainty in the SRM results, which limits the
accuracy of SRM maps. One solution to this problem is to
use auxiliary data, such as prior knowledge [10], [23], [32],
[39]–[42], panchromatic images [43]–[45], land-line digital
vector data [46], light detection and ranging (LIDAR) data [47],
fused images [48], digital elevation models [29], [34], class
membership contours [49], [50], or multiple shifted images
(MSIs) [35]–[37], [51]–[53] to eliminate ambiguity and reduce
uncertainty. Compared with other auxiliary datasets, MSIs are
relatively easy to obtain by camera movements in the same area
[36]. As a result, MSIs have been widely applied to SRM to
1939-1404 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
CHEN et al.: HYBRID CONSTRAINTS OF PURE AND MIXED PIXELS FOR SOFT-THEN-HARD SUPER-RESOLUTION MAPPING 2041
improve the accuracy of land-cover maps at the subpixel scale
[6], [23], [35]–[37].
The multiobservation capability of observation satellites
enables us to readily obtain MSIs covering the same area. MSIs
include two categories: 1) multitemporal images, which are
generated when satellites observe the same area at different
times (e.g., the moderate-resolution imaging spectroradiometer
(MODIS) covers the entire Earth every 1–2 days); and 2) mul-
tiangle images, which are acquired when satellites capture the
same area from different angles using multiple sensors (e.g., the
multiangle imaging spectroradiometer (MISR) consists of nine
separate digital cameras that gather data in various directions).
These images are usually not identical but are shifted by sev-
eral subpixels owing to slight orbit-translation and the Earth’s
rotation [37]. Thus, these multiobservation satellite sensors can
provide MSIs conveniently. Ling et al. [37] first proposed to
consider MSIs as new constraints in the energy function of a
Hopfield neural network for improving the accuracy of SRM
maps. Wang and Shi [52] used MSIs in image interpolation to
enhance the accuracy of SRM maps, while Wang and Wang
[51] incorporated multiple spectral constraints from the MSIs
into a Markov random field. Xu et al. [53] improved the spatial
attraction model with MSIs (SAM_MSI), and Wang et al. [35]
enhanced the accuracy of indicator cokriging by fusing condi-
tional probability maps from MSIs (ICK_MSI). Xu et al. [36]
considered SRM as a regularization issue based on the maxi-
mum a posteriori with MSIs (MAP_MSI). All these methods
had improved performance when compared with SRM maps
generated with a single input image. They can be regarded as
soft-then-hard SRM (STHSRM), which was first summarized
by Wang et al. [54]. Note that STHSRM is not only suitable
for MSIs but also for a single input image. STHSRM contains
two processes: 1) subpixel sharpening, in which soft attribute
values of each subpixel for all land-cover classes are estimated;
and 2) class allocation, whereby hard attribute values (i.e., land-
cover labels) of subpixels within a mixed pixel are allocated
according to the soft attribute values and fraction images [54].
However, the STHSRM methods with MSIs discussed above
applied the complementary information encapsulated in the
MSIs to the first process only (i.e., subpixel sharpening). The
second process of class allocation did not use the complemen-
tary information, which may result in limited SRM accuracy
improvement.
To take full advantage of MSIs in the process of class allo-
cation, this paper proposes a new class allocation method for
STHSRM with MSIs, which allocates the classes to subpixels
using hybrid constraints of pure and mixed pixels (HCPMP).
The new HCPMP algorithm creates hybrid constraints from
both the mixed pixels of a base image and the pure pixels of
the auxiliary images in MSIs. It first determines the classes of
subpixels that overlap with the pure pixels of auxiliary images,
after which the remaining subpixels are classified using infor-
mation derived from the mixed pixels of the base image. The
base image, which can be any one image of MSIs, is considered
as the benchmark of spatial reference used to generate land-
cover maps at subpixel scale. Currently, five class-allocation
algorithms are implemented [54], including direct hardening
(DH) [7], [36], [55], units of subpixel (UOS) [22], highest
Fig. 1. Super-resolution mapping with MSIs (see main text for further explana-
tion). (a) A region partially covered by three MSIs. (b) The central mixed pixel
of image A0is overlapped by the pure pixels of other MSIs.
attribute values first (HAVF) [17], units of class (UOC) [54],
and linear optimization techniques (LOT) [8]. Compared with
the five existing class-allocation algorithms, the HCPMP algo-
rithm has several characteristics and advantages: 1) it can make
use of auxiliary images in both subpixel sharpening and class-
allocation processes to improve SRM results; 2) the accuracy of
SRM may increase, especially for high-resolution land-cover
classes, as the uncertainty in the class-allocation process can
be reduced using the constraints imposed by pure pixels from
the auxiliary images; and 3) compared with LOT, the computa-
tional efficiency may be increased as the number of subpixels
that need to assign land-cover classes decreases due to the pure
pixels in the auxiliary images.
For comparison of the proposed HCPMP algorithm with
existing algorithms, three representative subpixel sharpening
algorithms (i.e., SAM_MSI, ICK_MSI, and MAP_MSI) are
first applied to estimate soft attribute values of subpixels.
Next, five existing class-allocation algorithms and the proposed
method are employed to determine the land-cover class of each
subpixel. An artificial image and two remote sensing images are
used to evaluate the performance of HCPMP.
The remainder of this paper is organized as follows.
Section II presents the background of STHSRM using MSIs.
Section III introduces the proposed HCPMP. Results for remote
sensing images are provided in Section IV and discussed in
Section V. Finally, conclusion is drawn in Section VI.
II. BACKGROUND
A. Basic Principles of STHSRM With MSIs
Most STHSRM with MSI approaches estimate the spatial
locations of subpixels within a mixed pixel through the sub-
pixel sharpening and class-allocation processes mentioned in
Section I. Fig. 1 illustrates the basic idea of SRM with MSIs.
Fig. 1(a) shows that there are three land-cover classes—water,
forest, and buildings—within the region partially covered by
three mutually shifted images—A0,A1, and A2, each of which
comprises 3×3pixels. A0is considered as the base image,
while the other two are auxiliary (or shifted) images with slight
diagonal shift (i.e., one half-pixel) relative to A0. Given a scale
factor S=4, each pixel can be divided into 4×4smaller
subpixels, as shown in the central pixel of A0of Fig. 1(b).
STHSRM with MSIs approaches first estimates soft attribute
values of the 16 subpixels; the classes of these subpixels are
2042 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
then allocated in terms of the estimated soft attribute values and
fraction constraints from the base image A0.Fig.1(b)shows
that the central pixel in auxiliary image A1is a pure pixel, cov-
ering the four subpixels in the lower left of the central mixed
pixel in A0. The pure pixel of A1thus indicates that these four
subpixels should be labeled as buildings. Similar to the cen-
tral pixel in A1, the central pixel in A2, which is a pure pixel
with the land-cover label of forest, covers four subpixels in the
upper right of the central mixed pixel in A0. It indicates that the
four upper-right subpixels of the central pixel of A0should be
labeled as forest. In this way, MSIs provide relevant informa-
tion to improve SRM and increase accuracy of land-cover maps
at the subpixel scale [35]–[37], [51]–[53].
As the subpixel sharpening and class-allocation processes of
STHSRM with MSIs are important to produce land-cover maps,
we briefly describe these processes in the following sections.
B. Subpixel Sharpening
Subpixel sharpening disaggregates coarse fraction images
into fine soft attribute values of subpixels [54]. Pixels in coarse
images are first divided into fine subpixels for a given scale
factor, and then, the soft attribute value of each subpixel that
belongs to each land-cover class is estimated. This process
can be accomplished by many approaches under the assump-
tion of spatial dependence [5], [54]. When fraction images are
derived from a single image, the soft attribute values of sub-
pixels are estimated from the single image, whereas in case of
fraction images derived from MSIs, the soft attribute values are
obtained by fusing the subpixel sharpening results from each of
the MSIs, using approaches such as SAM_MSI [53], ICK_MSI
[35], and MAP_MSI [36]. SAM_MSI integrates spatial attrac-
tions from a base image and auxiliary images into weighted
spatial attractions, which are considered as soft attribute values
of subpixels. SAM_MSI inherits the advantages of the origi-
nal spatial attraction model, which is considered as an efficient
approach [18], [27] because it is a straightforward one-pass pro-
cess with simple rules [17]. ICK_MSI, based on geostatistics,
first computes a conditional probability map from each coarse
image of the MSIs; these probability maps are then averaged as
soft attribute values of subpixels [35]. ICK_MSI allows easy
integration of fine sample data without any iterative process
[35]. MAP_MSI, first presented by Xu et al. [36], transforms
STHSRM with MSIs into a regularization problem with the
maximum a posteriori model, and the posteriori probabilities
are considered as soft attribute values of the subpixels [36].
Recently, this method has been improved in adaptive param-
eter selection [56] and the reduction of spectral unmixing error
[55]. The three subpixel sharpening algorithms have demon-
strated their usefulness in practical situations, and thus, they
are used here to estimate the soft attribute values of subpixels.
C. Class-Allocation Algorithms
Class allocation converts the fine soft attribute values of sub-
pixels derived from subpixel sharpening to a hard classified
map. In class allocation, the number of subpixels for each class
should first be determined according to fraction images and the
given scale factor. The land-cover class of each subpixel is then
allocated according to the results of subpixel sharpening. Note
that when MSIs are applied, the number of subpixels is calcu-
lated according to fraction images from the base image. DH was
originally applied to Hopfield neural network-based SRM [7],
and subsequently, to back-propagation neural network-based
approaches [11], [12], [57]. Both UOS and HAVF assign land-
cover classes to subpixels in a sequence, while satisfying the
coherence constraint that the number of subpixels for each
class within a mixed pixel should be consistent with the frac-
tion images. UOS proceeds along a typically predefined visiting
path (S2subpixels, where Sis the scale factor) that determines
the order of visited subpixels within a pixel, while HAVF per-
forms along the descending order of all soft attribute values
(C×S2, where Cis the number of land-cover classes). UOC,
proposed by Wang et al. [54], is an advanced class-allocation
algorithm that takes intraclass spatial dependence into account.
Typically, UOC allocates labels to subpixels according to a
visiting order of land-cover classes that can be determined by
Moran’s I. Verhoeye and De Wulf [8] first applied LOT to the
class-allocation process.
DH does not guarantee the coherence constraint and may
produce overly smooth results [54]. It, however, may reduce
the spectral unmixing error of real remote sensing images [55].
UOS may produce results with a salt-and-pepper effect if the
visiting order of subpixels is not appropriate [39]. HAVF and
UOC typically generate results with almost the same accu-
racy, which is generally higher than DH and UOS. UOC is
more efficient than HAVF due to fewer comparisons of soft
attribute values in UOC [54]. Compared with DH, UOS, HAVF
and UOC, LOT usually generate the highest SRM map accu-
racy, likely due to the fact that it involves many iterations in
search for the optimal land-cover classes of subpixels. LOT,
however, needs significantly more runtime than the other four
class-allocation algorithms [54]. Note that UOC may produce
results with slightly higher accuracy than LOT if the visiting
order of land-cover classes is set appropriately, as shown in
[54]. Although all five existing class-allocation algorithms have
advantages and disadvantages, none of them use the pure pixels
in auxiliary images to increase the accuracy of SRM results.
III. HCPMP ALGORITHM
Based on the brief description of the five existing class-
allocation algorithms given in the previous section, it is clear
that they only use soft attribute values and coarse fraction
images from a base image to estimate the spatial locations of
subpixels. Complementary information in auxiliary images is
ignored in the process of class allocation. Although the comple-
mentary information may produce more accurate soft attribute
values in the subpixel sharpening process, the resulting accu-
racy of the SRM map may still be modest, especially when
using a large zoom scale [35]. It is, therefore, attractive to also
apply auxiliary images to the class-allocation process to further
reduce the uncertainty in SRM.
As shown in Fig. 1(b), the complementary information of the
two central pure pixels in the auxiliary images (i.e., images A1
and A2) is useful to decrease the uncertainty in SRM. When the
CHEN et al.: HYBRID CONSTRAINTS OF PURE AND MIXED PIXELS FOR SOFT-THEN-HARD SUPER-RESOLUTION MAPPING 2043
two pure pixels are applied to class allocation, the land-cover
classes of the eight subpixels within the central mixed pixel
of base image A0can be determined directly. The uncertainty
of the eight subpixels is completely removed, leading to a
reduction in the total SRM uncertainty. The eight remaining
subpixels can then be allocated to land-cover classes accord-
ing to the soft attribute values and constraints of the fraction
images from a base image. The HCPMP algorithm is based on
the above strategy for assigning land-cover classes to subpix-
els. HCPMP consists of two steps: 1) it first assigns land-cover
classes to those subpixels that overlap with the pure pixels of
auxiliary images in MSIs; 2) it allocates the remaining subpix-
els to land-cover classes using the mixed pixels of a base image
in MSIs. Both steps are described in detail below.
Suppose that a coarse remote sensing image has m
coarse pixels and Cland-cover classes. Let Y={Y(k)|k=
1,...,K}be the fraction images derived from the soft clas-
sification of coarse MSIs, where Kis the number of MSIs.
Let image (Y(1)) be the base image, while the other images
({Y(k)|k=2,...,K}) are the auxiliary (shifted) images in
MSIs. Let Y(k)=(y(k)
.c ),c=1,...,C be fraction images
from the kth image of MSIs, where y(k)
.c ={(y(k)
i,c )|i=
1,...,m}is the column vector composed of elements y(k)
i,c ∈
[0,1], denoting the fraction value of coarse pixel ithat belongs
to land-cover class c. Given the scale factor S, the SRM out-
put is, therefore, a fine-resolution land-cover map X, created
by dividing each coarse pixel into S×Sfine pixels (subpix-
els), where X={xj,c|j=1,...,M,c=1,...,C and M=
m×S2}and xj,c ∈{0,1}is defined in (1). This indicates
that each subpixel should be allocated to a value of one or
zero for each land-cover class, where one means that the sub-
pixel belongs to the particular class and zero that it does not.
Meanwhile, each subpixel in the SRM map should be allocated
to one and only one land-cover class, implying the condition
C
c=1 xj,c =1for all j=1,...,M
xj,c =1,subpixel jis classified as class c
0,otherwise.(1)
A. Allocating Classes Using Pure Pixels in Auxiliary Images
Pure pixels are important in class allocation because subpix-
els within pure pixels in the base image can be directly assigned
the same land-cover class before class allocation and because
HCPMP needs pure pixels in auxiliary images to improve the
accuracy of SRM maps. Therefore, pure pixels in MSIs should
be identified prior to performing the class allocation. Whether
a pixel in coarse images is pure can be derived from fraction
values of the soft classification. If a pixel is pure, there must be
a land-cover class with a fraction value greater than a chosen
threshold θ∈(0,1]. Generally, there are two ways to deter-
mine the threshold θ. First, it can be a predefined threshold
determined by experts. For example, if θ=0.95 is identified
by experts, then this means that a land-cover class must occupy
95% of the pixel area in order for this pixel to be identified as
a pure pixel. The predefined threshold can be set to different
values under different conditions. Second, the threshold may
be based on the scale factor S. It may be set to θ=1−1/S2,
where 1/S2is the subpixel fraction value. In this case, the pixel
is considered pure when the sum of fraction values of other
classes is smaller than the fraction value of a single subpixel.
The process of allocating classes to subpixels using pure
pixels in auxiliary images includes the following five steps.
Step 1) For mixed pixel iunder consideration, the number
of subpixels for each class is calculated from the
fraction images derived from the base image by
Ni,c = round y(1)
i,c ×S2(2)
where Ni,c is the number of subpixels for class cin
pixel iand round(·)is the operator that rounds its
argument toward the closest integer.
Step 2) Using (3) and the criterion of defining pure pixels,
identify pure pixels (i.e., i(2),...,i
(K)) in auxil-
iary images that may overlap with the mixed pixel
iunder consideration
Vi(k)= round (Vi(1) +Δ
k)(3)
where Vi(k)denotes the coordinates of pixel iin
the kth image and Δkis the shift between the base
image and the kth image in the MSIs.
Step 3) For each land-cover class c, remove overlapped pure
pixels with the same class using (4). If there is more
than one overlapped pure pixel with the same class
according to Step 2), the pure pixel with the high-
est overlapped area less than the fraction of class
cwithin the mixed pixel iis kept for class allo-
cation, and the other overlapped pure pixels are
removed. The reason why some overlapped pure
pixels in auxiliary images should be removed is that
the pure pixel with the highest overlapped area is
most suitable for guaranteeing the coherence con-
straint imposed by fraction images. Therefore, pure
pixels in auxiliary images that are in conflict with
the coherence constraint should be removed
y(k)
i,c =⎧
⎨
⎩
y(k)
i,c
lim
y(k)
i,c →y(1)
i,c +y(k)
i,c −y(1)
i,c =0
⎫
⎬
⎭
(4)
where y(k)
i,c denotes the kept pure pixel iin the kth
image.
Step 4) For each remaining pure pixel i(k)with land-cover
class caccording to Step 3), the number of subpix-
els Ni(k),c is calculated according to the overlapped
area between the pure pixel i(k)and mixed pixel i.
The subpixels that overlap with the pure pixel of
auxiliary images are allocated to land-cover class
c. The number of remaining subpixels within mixed
pixel iis updated by N
i,c =Ni,c −Ni(k),c.
Step 5) Repeat the above five steps until all mixed pixels
in the base image have been allocated to land-cover
classes using pure pixels in auxiliary images.
2044 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
B. Allocating Classes Using Mixed Pixels in a Base Image
The land-cover class of the remaining subpixels within a
mixed pixel of the base image can be determined by any one of
the five existing class-allocation algorithms. As LOT is a robust
approach [19], [54], it is used here to determine the optimal
land-cover classes of remaining subpixels. After the land-cover
classes of some subpixels within mixed pixel ihave been deter-
mined using the overlapped pure pixels in auxiliary images,
class allocation for the remaining subpixels is described by the
objective function in (5) and constraints given in (6). The objec-
tive function aims to maximize the soft attribute values of the
remaining subpixels, also subject to class fractions from soft
classification
max imize z=
C
c=1
N
i
j=1
xj,c ×pj,c (5)
subject to
⎧
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎩
C
c=1
xj,c =1
N
i
j=1
xj,c =N
i,c
N
i=
C
c=1
N
i,c
(6)
where N
iis the total number of remaining subpixels awaiting
class allocation within mixed pixel iand pj,c is the soft attribute
value of subpixel jfor land-cover class c.
IV. EXPERIMENTS AND ANALYSIS
A. Experimental Design
Three experiments on different images (an artificial image
and two remote sensing images) were carried out to evalu-
ate the performance of the proposed HCPMP. The artificial
imagery had a relatively simple structure of land-cover patches,
which was beneficial to visual evaluation of the performance
of the different SRM methods. The two remote sensing images
were much more complicated. Three hard-classified land-cover
images from each image were considered as reference images,
which were considered as the base images in MSIs. The land-
cover images were degraded to fraction images to simulate
the outputs of soft classification. An advantage of using frac-
tion images obtained by degrading land-cover reference images
is that errors in soft classification are avoided which facili-
tates evaluating the performance of the SRM methods. Fraction
images degraded from the fine land-cover image was also a
widely used scheme in many SRM studies [16], [35]. For eval-
uation of the performance of SRM with MSIs, MSIs were
generated by shifting reference images. In this way, errors in
coregistration and in estimating shifts between the base and
auxiliary images were avoided. In each experiment, four shifted
images were used, and the subpixel shifts given the scale
factor Swere (0,0),(0,S/2),(S/2,0), and (S/2,S/2).The
four shifted images of each land-cover reference map were
degraded into fraction images, which were used as inputs for
the subpixel sharpening and class-allocation algorithms. Note
that a shift of (0,0) refers to no shift, and hence, these were
taken as the base images. In the first experiment on the artifi-
cial imagery, three scale factors (4, 6, and 10) were considered.
This meant that each land-cover reference map was degraded
into coarse fraction images using the three scale factors, and the
fraction images were then zoomed in to create fine SRM maps.
For the other two experiments with QuickBird and Landsat TM
images, a scale factor of 4 was tested.
The three subpixel sharpening algorithms (i.e., SAM_MSI,
ICK_MSI, and MAP_MSI) described in Section II were
employed to calculate the soft attribute values of subpixels. The
five existing class-allocation algorithms described in Section II
were used for comparison with the HCPMP algorithm. All
six class-allocation algorithms were programmed in Python
2.7 version and were executed on an Intel Core2 Processor
(2.93 GHz and 4 GB memory) with the 32-bit Window 7
operating system. For SAM_MSI, the weight was set to ω=
0.4according to [58]. The parameters in ICK_MSI were the
same as those in [35]. For MAP_MSI, a Laplacian model was
selected to add prior information owing to its relatively better
performance as reported in [36]. To assess the accuracy of each
algorithm quantitatively, the adjusted overall accuracy (OA)
metric [24] was applied in all three experiments. OAis iden-
tical to the traditional measurement of OA, except that it is
calculated for mixed pixels only, which means that OAwas
calculated as the total number of correctly classified subpixels
divided by the total number of reference subpixels within mixed
pixels. The reference subpixels were the corresponding pix-
els with the same coordinates in the high-resolution reference
image. The reference subpixels within all mixed pixels were
used as testing samples for accuracy assessment in the three
experiments. OAwas used to avoid the influence of pure pix-
els and concentrate on evaluating SRM method performances
for mixed pixels [24], [35].
B. Experiment 1: Artificial Imagery
The size of the artificial image was 240 columns by 240
rows. The image included four land-cover classes (C1,C2,C3,
and C4) as shown in Fig. 2. The reference land-cover map in
Fig. 2(a) was shifted to MSIs using four shifts as mentioned in
Section IV-A. By applying the three considered scale factors,
4, 6, and 10, they were degraded into fraction images. Fig. 2(b)
shows examples of the fraction images degraded from Fig. 2(a),
in this case, using scale factor 10.
The fraction images were first used as inputs for the three
subpixel sharpening algorithms. The soft attribute values of
subpixels from the subpixel sharpening process and fraction
images were then used by the five existing class-allocation
algorithms and the proposed HCPMP algorithm to recreate
land-cover maps with the same spatial resolution as the refer-
ence image. The visiting order of the classes (i.e., C2–C4–C1–
C3) for UOC was determined by Moran’s Iin Table I.
1) SRM Results: Fig. 3 shows SRM results obtained from
the coarse fraction images in Fig. 2(b) using scale factor 10. The
first column of Fig. 3 shows that DH produced overly smooth
maps, while the structures of some land-cover patches failed
to be recreated in its results. For example, some small land-
cover patches of class C3at the image center area failed to be
CHEN et al.: HYBRID CONSTRAINTS OF PURE AND MIXED PIXELS FOR SOFT-THEN-HARD SUPER-RESOLUTION MAPPING 2045
Fig. 2. Experiment on artificial imagery. (a) Reference land-cover map.
(b) Fraction images degraded from (a) using scale factor 10.
TAB L E I
MORAN’SIOF FOUR CLASSES IN THE ARTIFICIAL IMAGE
AT THREE SCALES
recreated and were wrongly classified into class C1, especially
for the map combined with SAM_MSI. Many speckle artifacts
occur in the SRM maps created using UOS for class allocation,
as shown in the second column of Fig. 3. Focusing on the maps
generated by HAVF, UOC, and LOT, the shape of land-cover
patches (e.g., class C1 and C3 at the image center area) is more
similar to that in the reference image than for the DH results,
although there are slightly more speckled artifacts. UOS pro-
duces the most speckle artifacts. Compared with UOS, HAVF,
UOC, and LOT, fewer speckle artifacts were generated by the
proposed HCPMP algorithm as shown in the last column of
Fig. 3. Moreover, HCPMP preserved the structure and shape
of patches better than DH. Of the six class-allocation algo-
rithms, HCPMP produced results closest to the reference image
in Fig. 2(a) while also creating the most satisfactory land-cover
maps, based on visual assessment.
2) Accuracy Assessments: Table II gives the accuracy
assessment for the artificial imagery when combining
SAM_MSI, ICK_MSI, and MAP_MSI with the six class-
allocation algorithms using scale factors 4, 6, and 10. The total
number of testing samples (reference subpixels within mixed
pixels) was 7120 for the scale factor of 4, and the numbers
of testing samples for C1,C2,C3, and C4were 2416, 1680,
2807, and 217, respectively. The total number of testing sam-
ples was 13 752 for scale factor 6, and the numbers for C1,
C2,C3, and C4were 4688, 3280, 5374, and 410, respectively.
The total number of testing samples was 23 000 for scale fac-
tor 10, and the numbers for C1,C2,C3, and C4were 8439,
5830, 7901, and 830, respectively. Comparing the OAfor
the different scale factors, the accuracy in all cases gradually
decreased as the scale factor increased. The reason for this
was that the SRM process became more complicated with an
increased scale factor, while uncertainty inevitably increased as
more spatial locations of subpixels within a coarse mixed pixel
needed to be estimated [54]. The accuracies in Table II confirm
the above visual assessment. It shows that the accuracies of DH
and UOS were significantly lower than those of the other four
algorithms, although the accuracy of UOS was higher than that
of DH. The accuracy of LOT was slightly higher than those of
HAVF and UOC, while HAVF and UOC had almost identical
accuracy. The OAof HCPMP was higher than those of the
other five algorithms. Compared with LOT, which had the high-
est accuracy of the five existing algorithms, HCPMP achieved
an average increase of 2.3% using scale factor 4 and about
1.0% increase for scale factors 6 and 10. These improvements
were largely due to the fact that auxiliary images were used
in the process of class allocation and pure pixels in the auxil-
iary images provided useful information, thereby increasing the
SRM accuracy.
C. Experiment 2: QuickBird Imagery
In this experiment, a 2.44-m multispectral QuickBird image
(400 ×400 pixels) located in the Jiangsu province, China, was
used. The QuickBird image shown in Fig. 4(a) contains four
main land-cover classes: water, vegetation, buildings, and bare
ground, which were considered as endmembers for classifi-
cation and SRM. Training samples of the four classes were
manually chosen from Fig. 4(a) for classification, while the
spectral separability of the sample pair of classes was measured
by the Jeffries–Matusita distance [16]. The Jeffries–Matusita
distance is a widely used measure for evaluating sample qual-
ity, and it takes values between 0 (no separability) and 2 (total
separability) [16]. Table III shows that spectral separability
between classes is very close to the total separability value
of 2. Fig. 4(b) shows the reference image that was derived from
Fig. 4(a) by a support vector machine (SVM) hard classifier
[59]. The OA of the reference image was 94.6% and was eval-
uated using 500 ground sites in Google Earth. The reference
image was transformed to MSIs using the four shifts described
in Section IV-A. Using scale factor 4, the MSIs were degraded
into coarse fraction images, which were then used as input
for the three subpixel sharpening algorithms and the six class-
allocation algorithms to recreate land-cover maps with the same
spatial resolution as the reference image in Fig. 4(b). The visit-
ing order of classes determined by Moran’s Iin Table IV was
water–vegetation–bare ground–buildings for UOC.
1) SRM Results: Fig. 5 shows the SRM results for the
QuickBird image using scale factor 4. It suggests that DH
once again produced overly smooth maps and failed to pre-
serve some small patches, especially certain fine linear features
(e.g., vegetation inside the buildings in the top left area) when
applying SAM_MSI. As shown in the second column of Fig. 5,
UOS yielded maps with many speckle artifacts and unsmooth
land-cover boundaries. HAVF, UOC, and LOT, on the other
hand, performed better than UOS because fewer speckle arti-
facts were produced. They also recreated small features more
accurate than DH. The maps produced by HCPMP were more
similar to Fig. 4(b) than those yielded by the other algorithms,
especially compared with maps yielded by DH and UOS.
2) Accuracy Assessments: Table V shows the OAvalues
for the QuickBird image. The total number of testing samples
was 28 272, and the testing sample numbers of water, veg-
etation, buildings, and bare ground were 1746, 8526, 9738,
and 8262, respectively. The accuracies in Table II confirmed
2046 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
Fig. 3. SRM results for the artificial imagery using scale factor of 10.
TAB L E I I
ACCURACY (OA)FOR THE ARTIFICIAL IMAGERY (%)
Fig. 4. Experiment on QuickBird imagery. (a) QuickBird imagery.
(b) Reference land-cover map generated by hard classification.
the findings in visual assessment. The accuracies of DH and
UOS were smaller than those of the other four algorithms,
while the accuracies of HAVF, UOC, and LOT presented minor
differences, although LOT was slightly more accurate than
HAVF and UOC. Compared with LOT, the OAof HCPMP
had an average improvement of 1.3%. Specifically, HCPMP
was 0.89%, 1.24%, and 1.78% more accurate than LOT when
combined with SAM_MSI, ICK_MSI, and MAP_MSI, respec-
tively. Additionally, the OAof the direct hard classification
map from the fraction images of the base-degraded QuickBird
image was 67.49%, it was calculated by comparing testing
TABLE III
JEFFRIES–MATUSITA DISTANCE OF FOUR CLASSES
IN THE QUICKBIRD IMAGERY
TAB L E I V
MORAN’SIOF FOUR CLASSES IN THE QUICKBIRD IMAGERY (S=4)
samples with the corresponding sites in the hard classification
map. It suggested that SRM results were more accurate than
that of hard classification, which was most likely that SRM can
address the mixed pixel problem in classification and provide
more detailed land-cover of mixed pixels than hard classifiers.
D. Experiment 3: Landsat TM Imagery
A Landsat TM image (800 ×800 pixels) with a spatial res-
olution of 30 m was used to investigate the performance of
the proposed algorithm. The TM image shown in Fig. 6(a)
was taken over Maryland, USA, on July 6, 2000, and it cov-
ered seven main land-cover classes: buildings, forest, water,
bare ground, road, grass, and farmland. The seven classes
were considered as endmembers of classification and SRM.
Training samples were manually selected from Fig. 6(a). The
spectral separability between classes was measured by Jeffries–
Matusita distance in Table VI, which revealed that most
Jeffries–Matusita distance values were the total separability
value of 2. Fig. 6(b) shows the reference image derived from
Fig. 6(a) using a SVM hard classifier [59]. The OA of the ref-
erence image was 87.6%, evaluated using 1000 ground sites
in Google Earth. As before, the reference image in Fig. 6(b)
was shifted to MSIs and was degraded into fraction images
CHEN et al.: HYBRID CONSTRAINTS OF PURE AND MIXED PIXELS FOR SOFT-THEN-HARD SUPER-RESOLUTION MAPPING 2047
Fig. 5. SRM results for QuickBird imagery using scale factor 4.
TAB L E V
ACCURACY (OA)FOR QUICKBIRD IMAGERY(%)
Fig. 6. Experiment on Landsat TM imagery. (a) Landsat TM imagery.
(b) Reference land-cover map generated by hard classification from (a).
using scale factor 4. These fraction images were used as input
for subpixel sharpening and class allocation. The class visiting
order of UOC for the Landsat TM imagery was water–forest–
bare ground–farmland–buildings–grass–road, which was deter-
mined by Moran’s Igiven in Table VI.
1) SRM Results: SRM maps for the Landsat TM imagery
using scale factor 4 are shown in Fig. 7. As indicated in Fig. 7,
DH produces overly smooth maps, with many small land-cover
patches (e.g., the fine linear roads in the bottom right area of the
image) that are not present in Fig. 7(a). Similar to the results
in the first two experiments, some speckle artifacts shown in
Fig. 7(b) were generated by UOS. HAVF, UOC, and LOT
achieved better results than DH and UOS. Focusing on the maps
generated by HCPMP, visual assessment shows that the overall
performance of HCPMP was very close to the reference image
Fig. 6(b).
TAB L E V I
JEFFRIES–MATUSITA DISTANCE OF SEVEN CLASSES
IN LANDSAT TM IMAGERY
2) Accuracy Assessments: Table VIII presents a quantita-
tive accuracy assessment for the Landsat TM imagery. The
total number of testing samples was 267 376, and the testing
sample numbers of buildings, forest, water, bare ground, road,
grass, and farmland were 9019, 96 486, 14 567, 28 773, 19 284,
11 899, and 87 348, respectively. Similar to the findings in the
first two experiments, the performance of DH and UOS was
evidently inferior to that of the other four algorithms. There
were minor differences in the accuracies of HAVF and UOC,
while the accuracy of LOT was slightly higher than those of
HAVF and UOC. Compared with LOT, the OAof HCPMP had
an average increase of 1.1%. More precisely, the accuracy of
HCPMP was 0.95%, 0.88%, and 1.48% higher than that of LOT
when combined with SAM_MSI, ICK_MSI, and MAP_MSI,
respectively. In addition, the OAof the direct hard classifica-
tion map from the base-degraded TM image was 66.0%, it was
also calculated by comparing testing samples with the corre-
sponding sites in the hard classification map. It suggested that
SRM results were more accurate than that of hard classification
in most cases.
V. DISCUSSION
A. Analysis of the Number of MSIs
Since auxiliary images were important for providing comple-
mentary information in SRM, it is worth analyzing the impact
of the number of MSIs on the accuracy of SRM. We tested
2048 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
Fig. 7. SRM maps generated by combining ICK_MSI results with class allocation algorithms for Landsat TM imagery using scale factor 4. (a) DH. (b) UOS.
(c) HAVF. (d) UOC. (e) LOT. (f) HCPMP.
TAB L E V II
MORAN’SIOF SEVEN CLASSES IN SPOT IMAGERY (S=4)
BD, buildings; FR, forest; WT, water; BG, bare ground; RD, road;
GS, grass; FD, farmland.
four cases of the number of MSIs—1, 3, 5, and 7—on the
three experimental images using scale factor 4, with shifts
(0,0), (S/2,0),(0,S/2), (S/2,S/2), (−S/4,0), (0,−S/4), and
(−S/4,−S/4). Fig. 8 shows the accuracy change using differ-
ent numbers of MSIs for the three experimental images. Fig. 8
confirms the conclusions in [52] and [53], namely, that the accu-
racy of SRM with auxiliary images is higher than that using
only one image (i.e., the base image) and that the accuracy of
the six class-allocation algorithms increases with an increase
in the number of MSIs. However, the improvement rate of
HCPMP was slightly higher than that of the other algorithms,
especially for the artificial imagery. This was largely due to the
fact that complementary information from auxiliary images was
not applied in the existing class-allocation algorithms, whereas
HCPMP did use this information. Since HCPMP used a great
deal of complementary information to predict the spatial distri-
bution of subpixels, it was slightly more accurate than the five
existing algorithms, as shown in Fig. 8.
B. Analysis of Computational Efficiency
Table IX presents the runtime of six class-allocation algo-
rithms on the three experiments. It indicates that DH was the
TABLE VIII
ACCURACY (OA)FOR LANDSAT TM IMAGERY (%)
fastest, which is likely due to the fact that this method has the
least comparisons of soft attribute values. Similar to the conclu-
sions stated in [54], the runtime of UOS, HAVF, and UOC were
very similar, although UOC was more efficient than UOS and
HAVF. LOT was the least efficient. Although HCPMP requires
more runtime during the first process of allocating classes
using pure pixels in auxiliary images, the overall efficiency was
increased and it took only slightly less runtime compared with
LOT. The reason was that some subpixels assigned to land-
cover classes by pure pixels were already removed, and thus,
the second process of allocating classes using mixed pixels took
less runtime because fewer subpixels needed to be assigned.
Additionally, the process of allocating classes to remaining sub-
pixels using coarse mixed pixels could also be done with DH,
UOS, HAVF, and UOC. In that case, HCPMP would take less
runtime because these four algorithms are more efficient than
LOT, as shown in Table IX.
C. Analysis of High- and Low-Resolution Cases
Mixed pixels occur in two different cases: high-resolution
(H-resolution) and low-resolution (L-resolution) [6]. The
CHEN et al.: HYBRID CONSTRAINTS OF PURE AND MIXED PIXELS FOR SOFT-THEN-HARD SUPER-RESOLUTION MAPPING 2049
Fig. 8. Impact of the number of MSIs on the accuracy (OA) for the three experiments (S=4).
TAB L E I X
RUNTIME OF THE THREE EXPERIMENTS (IN SECONDS)
H-resolution case refers to pixels that are smaller than the
objects of interest (or large-size contiguous patches), while the
L-resolution case refers to pixels that are larger than the objects
of interest [6]. HCPMP was performed based on the assump-
tion that there were many pure pixels in auxiliary images. This
implies that the study area should be predominated by large-
size contiguous land-cover patches (i.e., H-resolution case). In
theory, the improvement of HCPMP was caused by pure pixels
in auxiliary images of MSIs compared with LOT, which sug-
gests that the fewer the pure pixels in the auxiliary images, the
lower the increase of accuracy. In the extreme case, the accu-
racy of HCPMP would be the same as LOT when there are no
pure pixels in the auxiliary images. It can be observed from the
reference images in Figs. 2(b), 4(b), and 6(b) that the percent-
ages of H-resolution cases in the artificial image, the QuickBird
image, and the Landsat TM image gradually decreased as the
complexity of land-cover patches was increased. Tables II, V,
and VIII show that, compared with LOT, the OAof HCPMP
(using the scale factor 4) had average increases of 2.3%, 1.3%,
and 1.1% for the artificial image, the QuickBird image, and the
Landsat image, respectively. Meanwhile, Fig. 8 shows that the
accuracy improvement rate of HCPMP gradually reduces with
a decrease of H-resolution cases, even though the number of
MSIs increases. Specifically, the accuracy improvement rate of
HCPMP changed slowly from experiment 1 to experiment 3
when the number of MSIs increased from 1 to 7. Especially,
the accuracy improvement rate of HCPMP in the Landsat TM
image was the smallest as the number of MSIs increased. This
is because the Landsat TM image contained the lowest percent-
age of H-resolution cases. Therefore, it can be concluded that
HCPMP is more suitable in the H-resolution case than in the
L-resolution case.
D. Analysis of Different Subpixel Sharpening Algorithms
As the output of subpixel sharpening was a critical input for
class allocation, the effect of subpixel sharpening on classifi-
cation accuracy was also analyzed for the HCPMP algorithm.
It can be seen in Tables II, V, and VIII that the results
obtained when combining HCPMP with SAM_MSI, ICK_MSI,
and MAP_MSI were almost the same for the artificial image.
The greatest accuracy for the QuickBird image was produced
by combining HCPMP with MAP_MSI, whereas the high-
est accuracy for the Landsat TM image was generated by
combining HCPMP with ICK_MSI. The reason for this was
that the different subpixel sharpening algorithms had differ-
ent characteristics, providing different soft attribute values of
subpixels, causing the class-allocation algorithms to achieve
different performances for SRM. Although the results from the
2050 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
three different subpixel sharpening algorithms combined with
HCPMP were slightly different, the accuracy achieved when
combining HCPMP with each of the three subpixel sharpening
algorithms was greater and the improvement in OAwas greater
than those achieved by other five existing algorithms.
E. Analysis of the Criteria of Defining Pure Pixels
Pure pixels were crucial to improve the performance of
HCPMP, because the number of pure pixels in auxiliary images
was directly associated with the accuracy improvement. Two
criteria were introduced in Section III-A. The first criterion was
determined by experts, while the second criterion was calcu-
lated according to the given scale factor. In the above three
experiments, the threshold was set to 1−1/S2, because syn-
thetic images free of errors were tested and the pure pixels
could be easy to identify by this criterion. However, the thresh-
old used here may not be optimal in case of real remote sensing
images or when the scale factor is very large, because errors
in soft classification and image registration may be propagated
into SRM [6], [23], [55]. The selection of the optimal threshold
is a valuable issue when applying real remote sensing images
in future research. Remote sensing images in different land sur-
faces may have different characteristics. Consequently, remote
sensing image could be stratified into several subareas accord-
ing to the geo-detector technique [60], and each subarea might
be assigned a different threshold.
F. Impact of Image Registration
The image registration was crucial to SRM with MSIs
[37], especially when estimating the shifts between MSIs. For
real remote sensing images, many excellent approaches were
applied to estimate the shifts between the base image and the
auxiliary images in MSIs, such as parametric models [61] and
frequency domain algorithms [62]. Although the shifts between
real MSIs could be estimated, it was difficult to evaluate the
impact of the image registration on SRM [36]. Recently, the
impact of image registration on the accuracy of SRM was ana-
lyzed with synthetic images [36], [53], [56]. All these studies
concluded that the accuracy of SRM with MSIs decreases as
the error of image registration increases. Because this study
aimed at evaluation of the performance of SRM with MSIs,
the shifts between MSIs were assumed to be known, to avoid
the error of image registration and shift estimation. Therefore,
image registration was beyond the scope of this paper, and it is
an interesting issue for analysis of the impact of image registra-
tion on the HCPMP when applying real remote sensing images.
More information about the impact of image registration on
SRM may be found in studies [36], [53], [56].
VI. CONCLUSION
In this paper, we proposed a new class-allocation algorithm,
namely HCPMP, which utilizes auxiliary images in the process
of class allocation for STHSRM with MSIs. Complementary
information from auxiliary images is commonly used in the
first process (subpixel sharpening) of STHSRM with MSIs;
however, this information is typically ignored in the second
process (class allocation). To make full use of MSIs, HCPMP
allocates land-cover classes to subpixels using both the mixed
pixels of a base image in MSIs and the pure pixels of auxiliary
images in MSIs. As HCPMP uses LOT to determine the land-
cover classes of remaining subpixels, it needs slightly longer
computing time than DH, UOS, HAVF, and UOC. Still, three
experiments showed that HCPMP successfully applies MSIs
to produce SRM maps that are visually closer to the reference
images and that have greater accuracy than five existing class-
allocation algorithms. Especially, it can produce more accurate
SRM maps for high-resolution land-cover classes than low-
resolution cases. It also takes slightly less runtime than LOT.
Hence, HCPMP is an effective solution for applying auxiliary
data to the process of class allocation in SRM for remotely
sensed imagery.
In HCPMP, complementary information of pure pixels from
auxiliary images is applied to the process of class allocation.
This paper focused on theoretically evaluating the perfor-
mances of HCPMP, and synthetic images were only used to
avoid the impact of other errors sources (e.g., soft classification
and shift estimation). In future research, testing on real MSIs
over large areas may be done to analyze whether the advan-
tages of HCPMP as shown here carry through to more realistic
real-world situations.
ACKNOWLEDGMENT
The authors would like to thank the editor and anonymous
reviewers for their valuable comments and suggestions.
REFERENCES
[1] G. M. Foody, “Status of land cover classification accuracy assessment,”
Remote Sens. Environ., vol. 80, no. 1, pp. 185–201, 2002.
[2] P. Atkinson, M. Cutler, and H. Lewis, “Mapping sub-pixel proportional
land cover with AVHRR imagery,” Int. J. Remote Sens., vol. 18, no. 4,
pp. 917–935, 1997.
[3] C. E. Woodcock and A. H. Strahler, “The factor of scale in remote
sensing,” Remote Sens. Environ., vol. 21, no. 3, pp. 311–332, 1987.
[4] A. J. Tatem, H. G. Lewis, P. M. Atkinson, and M. S. Nixon, “Multiple-
class land-cover mapping at the sub-pixel scale using a Hopfield neural
network,” Int. J. Appl. Earth Observ. Geoinf., vol. 3, no. 2, pp. 184–190,
2001.
[5] P. M. Atkinson, “Mapping sub-pixel boundaries from remotely sensed
images,” in Innovations in GIS 4. London: Taylor & Francis, 1997,
pp. 166–180.
[6] P. M. Atkinson, “Issues of uncertainty in super-resolution mapping and
their implications for the design of an inter-comparison study,” Int. J.
Remote Sens., vol. 30, no. 20, pp. 5293–5308, 2009.
[7] A. Tatem, H. Lewis, P. Atkinson, and M. Nixon, “Increasing the spa-
tial resolution of agricultural land cover maps using a Hopfield neural
network,” Int. J. Geogr. Inf. Sci., vol. 17, no. 7, pp. 647–672, 2003.
[8] J. Verhoeye and R. De Wulf, “Land cover mapping at sub-pixel scales
using linear optimization techniques,” Remote Sens. Environ., vol. 79,
no. 1, pp. 96–104, 2002.
[9] K. Mertens, L. Verbeke, E. Ducheyne, and R. De Wulf, “Using genetic
algorithms in sub-pixel mapping,” Int. J. Remote Sens., vol. 24, no. 21,
pp. 4241–4247, 2003.
[10] P. M. Atkinson, “Sub-pixel target mapping from soft-classified, remotely
sensed imagery,” Photogramm. Eng. Remote Sens., vol. 71, no. 7,
pp. 839–846, 2005.
[11] L. Wang, Y. Zhang, and J. Li, “BP neural network based subpixel mapping
method,” in Proc. Int. Conf. Intell. Comput., 2006, pp. 755–760.
[12] L. P. Zhang, K. Wu, Y. F. Zhong, and P. X. Li, “A new sub-pixel map-
ping algorithm based on a BP neural network with an observation model,”
Neurocomputing, vol. 71, no. 10–12, pp. 2046–2054, 2008.
CHEN et al.: HYBRID CONSTRAINTS OF PURE AND MIXED PIXELS FOR SOFT-THEN-HARD SUPER-RESOLUTION MAPPING 2051
[13] J. P. Ardila, V. A. Tolpekin, W. Bijker, and A. Stein, “Markov-random-
field-based super-resolution mapping for identification of urban trees in
VHR images,” J. Photogramm. Remote Sens., vol. 66, no. 6, pp. 762–775,
2011.
[14] T. Kasetkasem, M. K. Arora, and P. K. Varshney, “Super-resolution land
cover mapping using a Markov random field based approach,” Remote
Sens. Environ., vol. 96, no. 3, pp. 302–314, 2005.
[15] X. Li, Y. Du, and F. Ling, “Spatially adaptive smoothing parameter selec-
tion for Markov random field based sub-pixel mapping of remotely sensed
images,” Int. J. Remote Sens., vol. 33, no. 24, pp. 7886–7901, 2012.
[16] V. A. Tolpekin and A. Stein, “Quantification of the effects of land-cover-
class spectral separability on the accuracy of Markov-random-field-based
superresolution mapping,” IEEE Trans. Geosci. Remote Sens., vol. 47,
no. 9, pp. 3283–3297, Sep. 2009.
[17] K. C. Mertens, B. De Baets, L. P. C. Verbeke, and R. R. de Wulf, “A
sub-pixel mapping algorithm based on sub-pixel/pixel spatial attraction
models,” Int. J. Remote Sens., vol. 27, no. 15, pp. 3293–3310, 2006.
[18] Q. Wang, L. Wang, and D. Liu, “Integration of spatial attractions between
and within pixels for sub-pixel mapping,” J. Syst. Eng. Electron., vol. 23,
no. 2, pp. 293–303, 2012.
[19] Y. Chen, Y. Ge, Q. Wang, and Y. Jiang, “A subpixel mapping algorithm
combining pixel-level and subpixel-level spatial dependences with binary
integer programming,” Remote Sens. Lett., vol. 5, no. 10, pp. 902–911,
2014.
[20] Y. Ge, S. Li, and V. C. Lakhan, “Development and testing of a subpixel
mapping algorithm,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 7,
pp. 2155–2164, Jul. 2009.
[21] Y. Ge, Y. Chen, S. Li, and Y. Jiang, “Vectorial boundary-based sub-
pixel mapping method for remote-sensing imagery,” Int. J. Remote Sens.,
vol. 35, no. 5, pp. 1756–1768, 2014.
[22] A. Boucher and P. C. Kyriakidis, “Super-resolution land cover map-
ping with indicator geostatistics,” Remote Sens. Environ., vol. 104, no. 3,
pp. 264–282, 2006.
[23] Y. Ge, “Sub-pixel land-cover mapping with improved fraction images
upon multiple-point simulation,” Int. J. Appl. Earth Observ. Geoinf.,
vol. 22, no. 1, pp. 115–126, 2013.
[24] Y. F. Zhong and L. P. Zhang, “Remote sensing image subpixel mapping
based on adaptive differential evolution,” IEEE Trans. Syst. Man Cybern.
B, Cybern., vol. 42, no. 5, pp. 1306–1329, Oct. 2012.
[25] Y. F. Zhong and L. P. Zhang, “Sub-pixel mapping based on artificial
immune systems for remote sensing imagery,” Pattern Recognit., vol. 46,
no. 11, pp. 2902–2926, 2013.
[26] X. Xu, Y. Zhong, and L. Zhang, “Adaptive subpixel mapping based on
a multiagent system for remote-sensing imagery,” IEEE Trans. Geosci.
Remote Sens., vol. 52, no. 2, pp. 787–804, Feb. 2014.
[27] Q. Wang, L. Wang, and D. Liu, “Particle swarm optimization-based sub-
pixel mapping for remote-sensing imagery,” Int. J. Remote Sens., vol. 33,
no. 20, pp. 6480–6496, 2012.
[28] Q. Wang, W. Shi, and P. M. Atkinson, “Sub-pixel mapping of remote
sensing images based on radial basis function interpolation,” ISPRS J.
Photogramm. Remote Sens., vol. 92, no. 1, pp. 1–15, 2014.
[29] F. Ling, F. Xiao, Y. Du, H. Xue, and X. Ren, “Waterline mapping at the
subpixel scale from remote sensing imagery with high-resolution digital
elevation models,” Int. J. Remote Sens., vol. 29, no. 6, pp. 1809–1815,
2008.
[30] X. Li, Y. Du, F. Ling, S. Wu, and Q. Feng, “Using a sub-pixel mapping
model to improve the accuracy of landscape pattern indices,” Ecol. Indic.,
vol. 11, no. 5, pp. 1160–1170, 2011.
[31] F. Ling, W. Li, Y. Du, and X. Li, “Land cover change mapping at the
subpixel scale with different spatial-resolution remotely sensed imagery,”
IEEE Geosci. Remote Sens. Lett., vol. 8, no. 1, pp. 182–186, Jan. 2011.
[32] F. Ling, X. Li, F. Xiao, S. Fang, and Y. Du, “Object-based sub-pixel
mapping of buildings incorporating the prior shape information from
remotely sensed imagery,” Int. J. Appl. Earth Observ. Geoinf., vol. 18,
no. 1, pp. 283–292, 2012.
[33] A. M. Muad and G. M. Foody, “Super-resolution mapping of lakes from
imagery with a coarse spatial and fine temporal resolution,” Int. J. Appl.
Earth Observ. Geoinf., vol. 15, no. 1, pp. 79–91, 2012.
[34] C. Huang, Y. Chen, and J. Wu, “DEM-based modification of pixel-
swapping algorithm for enhancing floodplain inundation mapping,” Int.
J. Remote Sens., vol. 35, no. 1, pp. 365–381, 2014.
[35] Q. Wang, W. Shi, and L. Wang, “Indicator cokriging-based subpixel land
cover mapping with shifted images,” IEEE J. Sel. Topics Appl. Earth
Observ. Remote Sens., vol. 7, no. 1, pp. 327–339, Jan. 2014.
[36] X. Xu, Y. Zhong, L. Zhang, and Z. Hongyan, “Sub-pixel mapping based
on a MAP model with multiple shifted hyperspectral imagery,” IEEE J.
Sel. Topics Appl. Earth Observ. Remote Sens., vol. 6, no. 2, pp. 580–593,
Apr. 2013.
[37] F. Ling, Y. Du, F. Xiao, H. Xue, and S. Wu, “Super-resolution land-cover
mapping using multiple sub-pixel shifted remotely sensed images,” Int. J.
Remote Sens., vol. 31, no. 19, pp. 5023–5040, 2010.
[38] J. Hu, Y. Ge, Y. Chen, and D. Li, “Super-resolution land cover map-
ping based on multiscale spatial regularization,” IEEE J. Sel. Topics Appl.
Earth Observ. Remote Sens., 2015, doi: 10.1109/JSTARS.2015.2399509,
to be published.
[39] A. Boucher, “Sub-pixel mapping of coarse satellite remote sensing
images with stochastic simulations from training images,” Math. Geosci.,
vol. 41, no. 3, pp. 265–290, 2009.
[40] X. Li, Y. Du, F. Ling, Q. Feng, and B. Fu, “Superresolution mapping of
remotely sensed image based on hopfield neural network with anisotropic
spatial dependence model,” IEEE Geosci. Remote Sens. Lett., vol. 11,
no. 7, pp. 1265–1269, Jul. 2014.
[41] A. J. Tatem, H. G. Lewis, P. M. Atkinson, and M. S. Nixon, “Super-
resolution land cover pattern prediction using a Hopfield neural network,”
Remote Sens. Environ., vol. 79, no. 1, pp. 1–14, 2002.
[42] M. W. Thornton, P. M. Atkinson, and D. A. Holland, “A linearised pixel-
swapping method for mapping rural linear land cover features from fine
spatial resolution remotely sensed imagery,” Comput. Geosci., vol. 33,
no. 10, pp. 1261–1272, 2007.
[43] G. M. Foody, “Sharpening fuzzy classification output to refine the rep-
resentation of sub-pixel land cover distribution,” Int. J. Remote Sens.,
vol. 19, no. 13, pp. 2593–2599, Sep. 1998.
[44] X. Li, F. Ling, Y. Du, and Y. Zhang, “Spatially adaptive superresolu-
tion land cover mapping with multispectral and panchromatic images,”
IEEE Trans. Geosci. Remote Sens., vol. 52, no. 5, pp. 2810–2823,
May 2014.
[45] Q. M. Nguyen, P. M. Atkinson, and H. G. Lewis, “Super-resolution map-
ping using Hopfield Neural Network with panchromatic imagery,” Int. J.
Remote Sens., vol. 32, no. 21, pp. 6149–6176, 2011.
[46] P. Aplin and P. Atkinson, “Sub-pixel land cover mapping for per-field
classification,” Int. J. Remote Sens., vol. 22, no. 14, pp. 2853–2858,
2001.
[47] M. Q. Nguyen, P. M. Atkinson, and H. G. Lewis, “Superresolution map-
ping using a Hopfield neural network with lidar data,” IEEE Geosci.
Remote Sens. Lett., vol. 2, no. 3, pp. 366–370, Jul. 2005.
[48] M. Q. Nguyen, P. M. Atkinson, and H. G. Lewis, “Superresolution map-
ping using a hopfield neural network with fused images,” IEEE Trans.
Geosci. Remote Sens., vol. 44, no. 3, pp. 736–749, Mar. 2006.
[49] Y.-F. Su, G. M. Foody, A. M. Muad, and K.-S. Cheng, “Combining
Hopfield neural network and contouring methods to enhance super-
resolution mapping,” IEEE J. Sel. Topics Appl. Earth Observ. Remote
Sens., vol. 5, no. 5, pp. 1403–1417, Oct. 2012.
[50] Y.-F. Su, G. M. Foody, A. M. Muad, and K.-S. Cheng, “Combining pixel
swapping and contouring methods to enhance super-resolution mapping,”
IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 5,
pp. 1428–1437, Oct. 2012.
[51] L. G. Wang and Q. M. Wang, “Subpixel mapping using Markov random
field with multiple spectral constraints from subpixel shifted remote sens-
ing images,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 3, pp. 598–602,
May 2013.
[52] Q. Wang and W. Shi, “Utilizing multiple subpixel shifted images in sub-
pixel mapping with image interpolation,” IEEE Geosci. Remote Sens.
Lett., vol. 11, no. 4, pp. 798–802, Sep. 2013.
[53] X. Xu, Y. Zhong, and L. Zhang, “A sub-pixel mapping method based
on an attraction model for multiple shifted remotely sensed images,”
Neurocomputing, vol. 134, no. 1, pp. 79–91, 2014.
[54] Q. Wang, W. Shi, and L. Wang, “Allocating classes for soft-then-hard
subpixel mapping algorithms in units of class,” IEEE Trans. Geosci.
Remote Sens., vol. 52, no. 5, pp. 2940–2959, May 2014.
[55] Y. Zhong, Y. Wu, X. Xu, and L. Zhang, “An adaptive subpixel mapping
method based on MAP model and class determination strategy for hyper-
spectral remote sensing imagery,” IEEE Trans. Geosci. Remote Sens.,
vol. 53, no. 3, pp. 1411–1426, Mar. 2015.
[56] Y. Zhong, Y. Wu, L. Zhang, and X. Xu, “Adaptive MAP sub-pixel
mapping model based on regularization curve for multiple shifted hyper-
spectral imagery,” ISPRS J. Photogramm. Remote Sens., vol. 96, no. 1,
pp. 134–148, 2014.
[57] D. Nigussie, R. Zurita-Milla, and J. G. P. W. Clevers, “Possibilities and
limitations of artificial neural networks for subpixel mapping of land
cover,” Int. J. Remote Sens., vol. 32, no. 22, pp. 7203–7226, 2011.
[58] X. Xu, Y. Zhong, and L. Zhang, “Sub-pixel mapping with multiple shifted
remotely sensed images based on attraction model,” in Proc. Int. Conf.
Intell. Sci. Intell. Data Eng., 2012, vol. 7202, pp. 482–489.
[59] C. C. Chang and C. J. Lin. (2001). LIBSVM: A Library for Support
Vector Machines [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/
libsvm
2052 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 8, NO. 5, MAY 2015
[60] J. F. Wang et al., “Geographical detectors-based health risk assess-
ment and its application in the neural tube defects study of the Heshun
region, China,” Int. J. Geogr. Inf. Sci., vol. 24, no. 1, pp. 107–127,
2010.
[61] L. Zhang, H. Zhang, H. Shen, and P. Li, “A super-resolution reconstruc-
tion algorithm for surveillance images,” Signal Process., vol. 90, no. 3,
pp. 848–859, 2010.
[62] P. Vandewalle, S. Susstrunk, and M. Vetterli, “A frequency domain
approach to registration of aliased images with application to super-
resolution,” EURASIP J. Appl. Signal Process., vol. 2006, no. 71459,
pp. 1–14, 2006.
Yuehong Chen received the B.S. degree in geo-
graphical information system from Hohai University,
Nanjing, China, in 2010, and the M.S. degree in car-
tography and geographical information system from
the University of Chinese Academy of Sciences,
Beijing, China, in 2013. He is currently pursuing
the Ph.D. degree in cartography and geographical
information system at the State Key Laboratory of
Resources and Environmental Information System,
Institute of Geographical Sciences and Natural
Resources Research, University of Chinese Academy
of Sciences, Beijing, China.
His research interests include remote sensing image processing and super-
resolution mapping.
Yong Ge (M’14) received the Ph.D. degree in cartog-
raphy and geographical information system from the
Chinese Academy of Sciences (CAS), Beijing, China,
in 2001.
She is a Professor with the State Key Laboratory
of Resources and Environmental Information System,
Institute of Geographical Sciences and Natural
Resources Research, CAS. She has been involved in
the organization of several international conferences
and workshops. She has directed research in more
than ten national projects. She is the author or coau-
thor of over 80 scientific papers published in refereed journals, one book, and
six chapters in books; she is the editor of one book, and she holds three granted
patents in improving the accuracy of information extraction from remotely
sensed imagery. Her research interests include spatial data analysis and data
quality assessment.
Dr. Ge is a member of the Theory and Methodology Committee of
the Cartography and Geographic Information Society, the International
Association of Mathematical Geosciences, and the Editorial Board of Spatial
Statistics (Elsevier).
Gerard B. M. Heuvelink received the Ph.D. degree
in environmental sciences from Utrecht University,
Utrecht, The Netherlands, in 1993.
Since 2003, he has been employed as an Associate
Professor of Geostatistics with the Soil Geography
and Landscape Group, Wageningen University and
as a Senior Researcher Pedometrics and Digital Soil
Mapping with ISRIC World Soil Information, since
2011. He is a Visiting Professor at the Institute
of Geographical Sciences and Natural Resources
Research, Chinese Academy of Sciences, Beijing,
China, since 2011. He has authored over 220 scientific publications on geo-
statistics, spatial uncertainty analysis, and pedometrics, about 90 of which
appeared in peer-reviewed international journals.
Dr. Heuvelink is an Associate Editor of Spatial Statistics and the
European Journal of Soil Science, and Editorial Board Member of Geoderma,
Environmental, and Ecological Statistics, International Journal of Applied
Earth Observation and Geoinformation and Geographical Analysis. In 2014,
he was awarded with the Richard Webster Medal of the Pedometrics
Commission of the International Union of Soil Science.
Jianlong Hu received the B.S. degree in information
and computing science, and the M.S. degree in com-
puter application technology from Shanxi University,
Taiyuan, China, in 2003 and 2006, respectively.
Since 2014, he was a Visiting Scholar with
the State Key Laboratory of Resources and
Environmental Information System, Institute of
Geographical Sciences and Natural Resources
Research, Chinese Academy of Sciences, Beijing,
China. His research interests include remote sensing
image analysis and machine learning.
Yu Jiang received the B.S. degree in geographical
information system from Capital Normal University,
Beijing, China, in 2012. She is currently pursuing
the M.S. degree at the State Key Laboratory of
Resources and Environmental Information System,
Institute of Geographical Sciences and Natural
Resources Research, University of Chinese Academy
of Sciences, Beijing, China.
Her research interests include remote sensing
image processing and super-resolution mapping.
