666 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 3, MARCH 2015
Neural Networks and Support Vector Machine
Algorithms for Automatic Cloud Classiﬁcation
of Whole-Sky Ground-Based Images
Alireza Taravat, Fabio Del Frate, Cristina Cornaro, and Stefania Vergari
Abstract—Clouds are one of the most important meteorological
phenomena affecting the Earth radiation balance. The increasing
development of whole-sky images enables temporal and spatial
high-resolution sky observations and provides the possibility to
understand and quantify cloud effects more accurately. In this
letter, an attempt has been made to examine the machine learn-
ing [multilayer perceptron (MLP) neural networks and support
vector machine (SVM)] capabilities for automatic cloud detec-
tion in whole-sky images. The approaches have been tested on a
signiﬁcant number of whole-sky images (containing a variety of
cloud overages in different seasons and at different daytimes) from
Vigna di Valle and Tor Vergata test sites, located near Rome. The
pixel values of red, green, and blue bands of the images have been
used as inputs of the mentioned models, while the outputs provided
classiﬁed pixels in terms of cloud coverage or others (cloud-free
pixels and sun). For the test data set, the overall accuracies of
95.07%, with a standard deviation of 3.37, and 93.66%, with a
standard deviation of 4.45, have been obtained from MLP neural
networks and SVM models, respectively. Although the two ap-
proaches generally generate similar accuracies, the MLP neural
networks gave a better performance in some speciﬁc cases where
the SVM generates poor accuracy.
Index Terms—Automatic classiﬁcation, cloud classiﬁcation,
neural networks, support vector machine, whole-sky images.
CLOUD coverage or cloud fraction measurements are gen-
erally used for ﬂight planning and aviation. On the other
hand, they have a strong impact on the radiation budget and
on the climate change and variability . More recently, with
the growing interest on renewable energy sources (especially
solar energy), information about cloud coverage earned addi-
tional importance for the electricity production forecast from
photovoltaic and solar power systems .
The feedbacks of low clouds (a negative feedback) and
high thin clouds (a positive feedback) on the radiation budget
are well known. Reﬂection and absorption by cloud particles
Manuscript received June 3, 2014; revised August 14, 2014; accepted
September 6, 2014.
A. Taravat and F. Del Frate are with the Department of Civil Engineering
and Computer Science, University of Rome “Tor Vergata,” 00133 Rome, Italy
C. Cornaro is with the Department of Enterprise Engineering, University of
Rome “Tor Vergata,” 00133 Rome, Italy.
S. Vergari is with the Center of Meteorological Experimentation, Italian Air
Force, 00062 Rome, Italy.
Color versions of one or more of the ﬁgures in this paper are available online
Digital Object Identiﬁer 10.1109/LGRS.2014.2356616
depend on the volume, shape, and thickness of the clouds .
In this context, ground-based imaging devices are commonly
used to support satellite studies. There are several reasons for
using ground-based sensors for cloud recognition: 1) localized
(immediately overhead) cloud presence in a given area cannot
be determined using satellite images with high accuracy, and
2) ground-based imaging sensors are cheaper in comparison
with spaceborne platform images .
In the related literature, there are many papers which demon-
strate the increased number of ground-based instruments for
whole-sky image acquisitions . Thus, suitable and adequate
image processing procedures are necessary to fully exploit the
huge amount of data available.
Bush et al. provided a classiﬁcation method based on the
binary decision trees in order to classify the ground-based
images into ﬁve different sky conditions . Singh and Glennen
utilized cooccurrence and autocorrelation matrix for ground-
based cloud recognition . Calbo and Sabburg used Fourier
transformation to classify eight predeﬁned sky conditions .
Liu et al. extracted some cloud structure features from infrared
images . Heinle et al. proposed an approach based on textural
features such as energy and entropy as an automated classiﬁca-
tion algorithm for classifying seven different sky conditions .
Machine learning approaches such as multilayer perceptron
(MLP) neural networks and support vector machines (SVMs)
have already been demonstrated to provide excellent perfor-
mance in the classiﬁcation of remotely sensed images. Both
techniques are effective as they build input-output relationships
directly from the data without the need of aprioriassumptions
or speciﬁc preprocessing procedures. Another advantage is that,
once the training phase is over, the classiﬁcation is basically ob-
tained in real time with a strong reduction of the computational
A combination of neural networks with sky image data has
been recently proposed for direct normal irradiance forecasting
models . However, to our knowledge, a detailed analysis of
different machine learning models for automatic classiﬁcation
of whole-sky images has not been presented so far in the litera-
ture, whereas machine learning models can be very competitive
in terms of accuracy and speed for image classiﬁcation. Starting
from these motivations, the purpose of the present paper is to
demonstrate the potential of the machine learning approach
for a fast, robust, accurate, and automated whole-sky image
classiﬁcation approach. The rest of this letter is organized
in four sections. In the following section, the cloud camera
1545-598X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
TARAVAT et al.: NEURAL NETWORKS AND SVM ALGORITHMS FOR AUTOMATIC CLOUD CLASSIFICATION 667
Fig. 1. The SRF-01 Cloud Cam at the centre of meteorological experimenta-
tion (Re.S.M.A.) of Vigna di Valle.
and the associated image data are introduced. Section III
contains a description of the methodology behind the proposed
approach. The results, discussion, and conclusion follow in
Sections IV and V.
Only a few research institutions in several countries have
developed noncommercial sky cameras for their own require-
ments , , . The automatic sky imaging system which
has been used for this experiment is the SRF-01 Cloud Cam
that is a commercial sky camera produced by EKO (Fig. 1).
The SRF-01 Cloud Cams have been installed at the Centre of
Meteorological Experimentation (Re.S.M.A.) of Vigna di Valle
(42◦06N, 12◦12E; 266 m a.s.l.) and at the Solar Energy Test
and Research Laboratory (SETR Lab) of Tor Vergata University
(41◦51N, 12◦35E), South-East of Rome .
The main device consists of a Canon Power Shot A60
digital camera in a weatherproof housing, with a maximum
resolution of 1600 ×1200 pixels in 30-b color JPEG format.
Additional optics extends the ﬁeld of view to 180◦, providing
the possibility to make color pictures of the sky. The images are
rectangular, but the whole sky mapped is circular. The center
of the circle is the zenith, and the horizon is along the border
(more details in Kalisch et al.) , .
The Cloud Camera has been set to acquire two photographs
every 10 min. The ﬁrst one is well exposed (aperture =1/500
and shutter =8.0), and the second one is lightly under exposed
(aperture =1/1000 and shutter =8.0), which are the optimum
cloud camera settings achieved by . The algorithm for
cloud detection offered by EKO Co. needs both well-exposed
and under-exposed images for sun identiﬁcation, but in this
study, only the acquisitions from the ﬁrst exposure mode have
been used for the machine learning algorithms. The PC clock
has been set on UTC, and the daily time interval of image
acquisition was from 5 A.M.to9P.M. in order to consider the
period from sunrise to sunset.
The considered data set contains an overall number of 250
images, 200 from the Vigna di Valle (42◦06N, 12◦12E) test
site and 50 from the Tor Vergata University (41◦51N, 12◦35E)
TAB L E I
TAB L E I I
MIN,MAX,AND AVERAGE VALUES (COMPUTED OVER THE
NUMBER OF THE IMAGES CONSIDERED IN EACH GROUP)
OF THE ACCURACIES FOR DIFFERENT TYPES OF CLOUDS
CLASSIFIED BY MLP AND SVM (IN PERCENT)
test site. These 250 images contain all different types of clouds
under a variety of sky conditions at different selected day
times and seasons. The data set has been categorized into
four groups (Table I) according to the International Cloud
Classiﬁcation System published in WMO (1987). In order to
avoid systematic misclassiﬁcations, we have merged some of
the classes (altostratus and stratus, cirrocumulus, and altocumu-
lus). Additionally, the genera cirrus and cirrostratus have been
merged due to the difﬁculty in detecting very thin clouds (such
as some kinds of cirrostratus). Despite these generalizations, the
resulting classes represent a suitable partitioning of possible sky
conditions which are especially useful for radiation studies.
As a preprocessing task, the interesting part (which is circular
in shape) of each image in the data set has been extracted, and
then, disruptive factors like camera antenna or trees have been
eliminated from the subset images. The preprocessing phase
makes classiﬁcation and image interpretation more expedient
After the preprocessing phase, all of the images have been
classiﬁed by MLP neural networks and SVM classiﬁers. Then,
the results have been analyzed and compared with each other
and with the results of the multiband thresholding algorithm
(which is the combined result of AND operations for thresh-
olded red, green, and blue bands of the images), which is a
very popular method in this ﬁeld offered by EKO Co., and
with the results of the k-nearest neighbor classiﬁer presented
by Heinle et al. , which is the latest paper published in the
related literature about automatic cloud classiﬁcation of whole-
668 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 3, MARCH 2015
AVERAGE VALUES OF COMMISSION AND OMISSION ERRORS (IN PERCENT)ACHIEVED BY MLP AND SVM FOR DIFFERENT GROUPS OF DATA
MLPs are a very powerful neural network model for pixel-
level classiﬁcation , , which is trained by the error back-
propagation algorithm (the most commonly used model from
feedforward family). In the MLP model, the number of units in
the hidden layer and the training phase settings (the number
of training cycles and the pixel selection for training/testing
the model) represent the fundamental tasks. Normalization,
which is a preliminary phase of neural network classiﬁcations,
is performed by linear transformation from the image interval
[0–255] to the neural network interval [1,1] (the normalization
phase ensures that the distance measures respond with equal
weight for each input) .
A radial basis function kernel, a very powerful kernel for
pixel-level classiﬁcation , has been selected as the primary
SVM kernel function. In SVM (with radial basis function as the
kernel), the setting of C and γrepresents the fundamental task
in the phase of model designing.
Adjustment of the mentioned parameters (for MLPs and
SVM) affects the capability and sensitivity of the models to ﬁt
the dynamic ranges of the pixel values in the images. The IDL
programming software for the preprocessing phase and the neu-
ral network simulator (SNNS) developed at the University of
Stuttgart, Stuttgart, Germany, have been used in implementing
the classiﬁcation algorithms. Also, the SVM software (SVM-
Light package; URL: http://svmlight.joachims.org) has been
used for SVM processing.
IV. RESULTS AND DISCUSSION
In the MLP classiﬁer, 5423 pixels (extracted from 15 images)
have been used for training/testing the net. These 15 images
contain all different types of clouds under a variety of sky
conditions at different selected times during the day. The train-
ing sets contain 60% of the data, and the test sets contain the
remaining 40% which do not belong to the training sets. Pixel
selection for the training/test set has been performed randomly
and repeated four times.
Several attempts have been made to properly select the
number of units to be considered in the hidden layers of the
MLP. Architecture 3-7-2 has been ﬁnally chosen for its good
performance in terms of classiﬁcation accuracy, root-mean-
square error (rmse), and training time. Here, 10 000 training
cycles were sufﬁcient to train the network. The inputs of the net
consist of red, green, and blue bands, and the output provides
the pixel classiﬁcation in terms of cloudy pixel or others (cloud-
free pixels or sun). One MLP has been used in classifying all of
In the SVM classiﬁer, the images have been rescaled between
(0.0–1.0) for training the model. A range of values was tested
for the two SVM parameters C (1–50) and γ(0.1–10). A
grid search method has been used to examine the various
combinations of C and γ, and the best combination has been
ﬁnally chosen based on the model performance in terms of
classiﬁcation accuracy and rmse .
The training/test sets are exactly the same as those which have
been used for the MLP classiﬁer. Pixel selection for the training/
test set has been again done randomly and repeated four times.
Random pixel selection allows us to examine the robustness of
the classiﬁcation algorithms with respect to the variability
of the training data set . For the validation phase and
accuracy assessment, from each image in the data set, 250
pixels have been selected randomly and then labeled by visual
The accuracy of the whole test data set classiﬁed by MLP is
95.07%, with a standard deviation of 3.37, while the accuracy
of the SVM classiﬁer is 93.66%, with a standard deviation of
4.45. The average accuracy achieved by the EKO classiﬁcation
technique is 50.90%, with a standard deviation of 11.25%.
Visual inspection shows that the machine learning approaches
(MLP and SVM) achieve much better detection results under
a variety of conditions in comparison with the results of the
multiband thresholding algorithm. In Heinle’s work , the
average accuracy which has been obtained is about 88%. More-
over, the k-nearest neighbor classiﬁer presented by Heinle et al.
is characterized by slow runtime performance and large mem-
ory requirements , whereas the classiﬁcation by the artiﬁcial
neural network approach with a 1000 ×1000 image can be
completed in about a few seconds on a personal computer with
an Intel Pentium dual-core, a speed of 2.2 GHz, and a RAM
memory of 2.00 GB, which is much faster than some existing
methods in the literature. This might have a signiﬁcant impact
on the reduction of the computational burden when large data
sets need to be processed.
The results of the accuracy assessment applied to different
types of clouds are displayed in Tables II and III.
TARAVAT et al.: NEURAL NETWORKS AND SVM ALGORITHMS FOR AUTOMATIC CLOUD CLASSIFICATION 669
Fig. 2. Classiﬁcation results of six typical examples. (a) and (b) Clear-sky
example (no clouds and cloudiness below 5%). (c) and (d) Example of the
detection of low or midlevel layer of uniform clouds and dark thick clouds.
(e) Example of the detection of low puffy clouds with clearly deﬁned edges and
high thin clouds, wisplike. (f) Example of the detection of high patched clouds
of small cloudlets, mosaiclike. (First column) Original images. (Second and
third columns) Results of the MLP and SVM classiﬁers. (Last column) Results
of the classiﬁcation offered by EKO Co.
The MLP classiﬁer generates a satisfactory accuracy on the
clear-sky, stratus-altostratus-nimbostratus, and stratocumulus-
cumulus groups. The average accuracy of the clear-sky class is
98.65%, with a standard deviation of 0.61. The average com-
mission (percentage of extra pixels in the class) and omission
(percentage of pixels left out of the class) errors of the clear-
sky class are 5.14% and 1.33%, respectively. In the case of the
stratus-altostratus-nimbostratus group, the average accuracies
of 96.62% (standard deviation of 0.71%) with 4.35% and 3.37%
commission and omission errors are obtained by the MLP clas-
siﬁer. The accuracy of the same groups classiﬁed by SVM are
97.92%, with a standard deviation of 0.80% (5.25% and 2.08%
commission and omission error averages), and 95.74%, with
a standard deviation of 1.92% (4.78% and 4.25% commission
and omission error averages).
A signiﬁcant improvement of 3.60% in accuracy (with a
difference of 0.63% in standard deviation) has been obtained
on the cirrus-cirrostratus-altocumulus-cirrocumulus group seg-
mented by MLP with respect to the same data set segmented
by SVM (the average accuracy is 88.03%, with a standard
deviation of 3.03%).
It is necessary to identify the situations where the proposed
approaches generated poor accuracy and see why the models
failed to work correctly in those cases. The worst accuracy
generated by the MLP classiﬁer is 84.88% with 14.20% com-
mission error, which is obtained for the cirrus-cirrostratus-
altocumulus-cirrocumulus group. This accuracy is 3.96%
higher than the worst accuracy obtained by the SVM classi-
ﬁer which is for cirrus-cirrostratus-altocumulus-cirrocumulus
group (80.92% with 14.60% commission error).
This low performance is caused by the thin and transparent
parts of cirrus clouds which cannot be detected by both algo-
rithms (these parts have been classiﬁed as clear sky). Moreover,
the so-called “whitening effect” provides a misclassiﬁcation of
cloud-free pixels (which are brighter due to forward scattering
by aerosols and haze) near the solar disk, and therefore, such
pixels are classiﬁed as thin clouds by the algorithms (see also
 and ). Fig. 2(b), (c), and (f) illustrates the typical
examples where the models generate less accuracy.
V. C ONCLUSION
In the present study, the capabilities of machine learning
algorithms as automated methods for cloud segmentation in
whole-sky images have been demonstrated. Two classiﬁcation
algorithms (MLP neural networks and SVM) have been com-
pared using a data set containing 250 images from two different
test sites. The same parameters were used for all of the test
The obtained accuracies showed that the machine learning ap-
proaches (MLP and SVM) achieve better detection results under
a variety of conditions with respect to the results of the thresh-
olding algorithm, which is a popular model for whole-sky im-
age classiﬁcation. The machine learning models generate lower
accuracies for the cirrus-cirrostratus-altocumulus-cirrocumulus
group (caused by cirrus clouds), and in some other cases, the
accuracy is decreased because of the whitening effect near
the solar disk. The results show that the MLP classiﬁer as an
automated algorithm for cloud classiﬁcation works better in the
situations where SVM generates poor accuracy.
The authors would like to thank the associate editor and the
anonymous reviewers for their constructive comments, Colonel
F. Foti and Captain E. Vuerich from the Centre of Meteorolo-
gical Experimentation of the Italian Air Force for hosting the
cloud camera in their ﬁeld at Vigna di Valle, the Misure com-
pany for kindly providing the Cloud Camera, and Dr. T. Vitti
from Tecnoel Company for the installation and assistance.
670 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 3, MARCH 2015
 Z. Chen, D. Zen, Q. Zhang, and D. Ginthner, “Sky model study using
fuzzy mathematics,” J. Illuminating Eng. Soc., vol. 23, no. 1, pp. 52–58,
 M. Martinez-Chico, F. J. Bathes, and J. L. Bosch, “Cloud classiﬁcation in
a Mediterranean location using radiation data and sky images,” Energy,
vol. 36, no. 7, pp. 4055–4062, Jul. 2011.
 A. Heinle, A. Macke, and A. Srivastav, “Automatic cloud classiﬁcation
of whole sky images,” Atmos. Meas. Technol., vol. 3, no. 3, pp. 557–567,
 M. Singh and M. Glennen, “Automated ground-based cloud recognition,”
Pattern Anal. Appl., vol. 8, no. 3, pp. 258–271, Nov. 2005.
 G. Pﬁster et al., “Cloud coverage based on all-sky imaging and its impact
on surface solar irradiance,” J. Appl. Meteorol., vol. 42, no. 10, pp. 1421–
1434, Oct. 2003.
 K. A. Bush, C. H. Sun, and L. R. Thorne, “Cloud classiﬁcation using
whole-sky imager data,” in Proc. 9th Symp. Meteorol. Observ. Instrumen-
tation, 1995, pp. 353–358.
 J. Calbo and J. Sabburg, “Feature extraction from whole-sky ground-based
images for cloud-type recognition,” J. Atmos. Ocean. Technol., pp. 3–14,
 L. Liu, X. J. Sun, F. Chen, S. J. Zhao, and T. C. Gao, “Cloud classiﬁcation
based on structure features of infrared images,” J. Atmos. Ocean. Technol.,
vol. 28, no. 3, pp. 410–417, Mar. 2011.
 Y. H. Chu, H. T. C. Pedro, and C. F. M. Coimbra, “Hybrid intra-hour DNI
forecasts with sky image processing enhanced by stochastic learning,”
Solar Energy, vol. 98, pp. 592–603, Dec. 2013.
 J. Kalisch and A. Macke, “Estimation of the total cloud cover with high
temporal resolution and parametrization of short-term ﬂuctuations of sea
surface insolation,” Meteorologische Zeitschrift, vol. 17, no. 5, pp. 603–
611, Oct. 2008.
 C. Cornaro, S. Vergari, F. Foti, and F. Del Frate, “Project CL.E.A.R.
Cloudiness Experiment For Automatic Recognition,” in Proc. TECO
WMO Tech. Conf. Meteorol. Environ. Instruments Methods Observ.,
Brussels, Belgium, Oct. 16–18, 2012, pp. 1–9.
 J. F. Mas and J. J. Flores, “The application of artiﬁcial neural networks to
the analysis of remotely sensed data,” Int. J. Remote Sens., vol. 29, no. 3,
pp. 617–663, Feb. 1, 2008.
 A. Taravat and F. Del Frate, “Development of band ratioing algorithms
and neural networks to detection of oil spills using Landsat ETM
plus data,” EURASIP J. Adv. Signal Process., vol. 2012, no. 107, pp. 1–8,
 K. Topouzelis, V. Karathanassi, P. Pavlakis, and D. Rokos, “Dark forma-
tion detection using neural networks,” Int. J. Remote Sens., vol. 29, no. 16,
pp. 4705–4720, 2008.
 G. Mountrakis, J. Im, and C. Ogole, “Support vector machines in remote
sensing: A review,” ISPRS J. Photogramm. Remote Sens., vol. 66, no. 3,
pp. 247–259, May 2011.
 T. Joachims, Learning to Classify Text Using Support Vector Machines:
Methods, Theory and Algorithms. New York, USA: Springer-Verlag,
 C. N. Long, J. M. Sabburg, J. Calbo, and D. Pages, “Retrieving
cloud characteristics from ground-based daytime color all-sky images,”
J. Atmos. Ocean. Technol., vol. 23, no. 5, pp. 633–652, May. 2006.