ArticlePDF Available

Neural Networks and Support Vector Machine Algorithms for Automatic Cloud Classification of Whole-Sky Ground-Based Images


Abstract and Figures

Clouds are one of the most important meteorological phenomena affecting the Earth radiation balance. The increasing development of whole-sky images enables temporal and spatial high-resolution sky observations and provides the possibility to understand and quantify cloud effects more accurately. In this letter, an attempt has been made to examine the machine learning [multilayer perceptron (MLP) neural networks and support vector machine (SVM)] capabilities for automatic cloud detection in whole-sky images. The approaches have been tested on a significant number of whole-sky images (containing a variety of cloud overages in different seasons and at different daytimes) from Vigna di Valle and Tor Vergata test sites, located near Rome. The pixel values of red, green, and blue bands of the images have been used as inputs of the mentioned models, while the outputs provided classified pixels in terms of cloud coverage or others (cloud-free pixels and sun). For the test data set, the overall accuracies of 95.07%, with a standard deviation of 3.37, and 93.66%, with a standard deviation of 4.45, have been obtained from MLP neural networks and SVM models, respectively. Although the two approaches generally generate similar accuracies, the MLP neural networks gave a better performance in some specific cases where the SVM generates poor accuracy.
Content may be subject to copyright.
Neural Networks and Support Vector Machine
Algorithms for Automatic Cloud Classification
of Whole-Sky Ground-Based Images
Alireza Taravat, Fabio Del Frate, Cristina Cornaro, and Stefania Vergari
Abstract—Clouds are one of the most important meteorological
phenomena affecting the Earth radiation balance. The increasing
development of whole-sky images enables temporal and spatial
high-resolution sky observations and provides the possibility to
understand and quantify cloud effects more accurately. In this
letter, an attempt has been made to examine the machine learn-
ing [multilayer perceptron (MLP) neural networks and support
vector machine (SVM)] capabilities for automatic cloud detec-
tion in whole-sky images. The approaches have been tested on a
significant number of whole-sky images (containing a variety of
cloud overages in different seasons and at different daytimes) from
Vigna di Valle and Tor Vergata test sites, located near Rome. The
pixel values of red, green, and blue bands of the images have been
used as inputs of the mentioned models, while the outputs provided
classified pixels in terms of cloud coverage or others (cloud-free
pixels and sun). For the test data set, the overall accuracies of
95.07%, with a standard deviation of 3.37, and 93.66%, with a
standard deviation of 4.45, have been obtained from MLP neural
networks and SVM models, respectively. Although the two ap-
proaches generally generate similar accuracies, the MLP neural
networks gave a better performance in some specific cases where
the SVM generates poor accuracy.
Index Terms—Automatic classification, cloud classification,
neural networks, support vector machine, whole-sky images.
CLOUD coverage or cloud fraction measurements are gen-
erally used for flight planning and aviation. On the other
hand, they have a strong impact on the radiation budget and
on the climate change and variability [1]. More recently, with
the growing interest on renewable energy sources (especially
solar energy), information about cloud coverage earned addi-
tional importance for the electricity production forecast from
photovoltaic and solar power systems [2].
The feedbacks of low clouds (a negative feedback) and
high thin clouds (a positive feedback) on the radiation budget
are well known. Reflection and absorption by cloud particles
Manuscript received June 3, 2014; revised August 14, 2014; accepted
September 6, 2014.
A. Taravat and F. Del Frate are with the Department of Civil Engineering
and Computer Science, University of Rome “Tor Vergata,” 00133 Rome, Italy
C. Cornaro is with the Department of Enterprise Engineering, University of
Rome “Tor Vergata,” 00133 Rome, Italy.
S. Vergari is with the Center of Meteorological Experimentation, Italian Air
Force, 00062 Rome, Italy.
Color versions of one or more of the figures in this paper are available online
Digital Object Identifier 10.1109/LGRS.2014.2356616
depend on the volume, shape, and thickness of the clouds [3].
In this context, ground-based imaging devices are commonly
used to support satellite studies. There are several reasons for
using ground-based sensors for cloud recognition: 1) localized
(immediately overhead) cloud presence in a given area cannot
be determined using satellite images with high accuracy, and
2) ground-based imaging sensors are cheaper in comparison
with spaceborne platform images [4].
In the related literature, there are many papers which demon-
strate the increased number of ground-based instruments for
whole-sky image acquisitions [5]. Thus, suitable and adequate
image processing procedures are necessary to fully exploit the
huge amount of data available.
Bush et al. provided a classification method based on the
binary decision trees in order to classify the ground-based
images into five different sky conditions [6]. Singh and Glennen
utilized cooccurrence and autocorrelation matrix for ground-
based cloud recognition [4]. Calbo and Sabburg used Fourier
transformation to classify eight predefined sky conditions [7].
Liu et al. extracted some cloud structure features from infrared
images [8]. Heinle et al. proposed an approach based on textural
features such as energy and entropy as an automated classifica-
tion algorithm for classifying seven different sky conditions [3].
Machine learning approaches such as multilayer perceptron
(MLP) neural networks and support vector machines (SVMs)
have already been demonstrated to provide excellent perfor-
mance in the classification of remotely sensed images. Both
techniques are effective as they build input-output relationships
directly from the data without the need of aprioriassumptions
or specific preprocessing procedures. Another advantage is that,
once the training phase is over, the classification is basically ob-
tained in real time with a strong reduction of the computational
A combination of neural networks with sky image data has
been recently proposed for direct normal irradiance forecasting
models [9]. However, to our knowledge, a detailed analysis of
different machine learning models for automatic classification
of whole-sky images has not been presented so far in the litera-
ture, whereas machine learning models can be very competitive
in terms of accuracy and speed for image classification. Starting
from these motivations, the purpose of the present paper is to
demonstrate the potential of the machine learning approach
for a fast, robust, accurate, and automated whole-sky image
classification approach. The rest of this letter is organized
in four sections. In the following section, the cloud camera
1545-598X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See for more information.
Fig. 1. The SRF-01 Cloud Cam at the centre of meteorological experimenta-
tion (Re.S.M.A.) of Vigna di Valle.
and the associated image data are introduced. Section III
contains a description of the methodology behind the proposed
approach. The results, discussion, and conclusion follow in
Sections IV and V.
Only a few research institutions in several countries have
developed noncommercial sky cameras for their own require-
ments [3], [5], [10]. The automatic sky imaging system which
has been used for this experiment is the SRF-01 Cloud Cam
that is a commercial sky camera produced by EKO (Fig. 1).
The SRF-01 Cloud Cams have been installed at the Centre of
Meteorological Experimentation (Re.S.M.A.) of Vigna di Valle
(4206N, 1212E; 266 m a.s.l.) and at the Solar Energy Test
and Research Laboratory (SETR Lab) of Tor Vergata University
(4151N, 1235E), South-East of Rome [11].
The main device consists of a Canon Power Shot A60
digital camera in a weatherproof housing, with a maximum
resolution of 1600 ×1200 pixels in 30-b color JPEG format.
Additional optics extends the field of view to 180, providing
the possibility to make color pictures of the sky. The images are
rectangular, but the whole sky mapped is circular. The center
of the circle is the zenith, and the horizon is along the border
(more details in Kalisch et al.) [3], [10].
The Cloud Camera has been set to acquire two photographs
every 10 min. The first one is well exposed (aperture =1/500
and shutter =8.0), and the second one is lightly under exposed
(aperture =1/1000 and shutter =8.0), which are the optimum
cloud camera settings achieved by [11]. The algorithm for
cloud detection offered by EKO Co. needs both well-exposed
and under-exposed images for sun identification, but in this
study, only the acquisitions from the first exposure mode have
been used for the machine learning algorithms. The PC clock
has been set on UTC, and the daily time interval of image
acquisition was from 5 A.M.to9P.M. in order to consider the
period from sunrise to sunset.
The considered data set contains an overall number of 250
images, 200 from the Vigna di Valle (4206N, 1212E) test
site and 50 from the Tor Vergata University (4151N, 1235E)
test site. These 250 images contain all different types of clouds
under a variety of sky conditions at different selected day
times and seasons. The data set has been categorized into
four groups (Table I) according to the International Cloud
Classification System published in WMO (1987). In order to
avoid systematic misclassifications, we have merged some of
the classes (altostratus and stratus, cirrocumulus, and altocumu-
lus). Additionally, the genera cirrus and cirrostratus have been
merged due to the difficulty in detecting very thin clouds (such
as some kinds of cirrostratus). Despite these generalizations, the
resulting classes represent a suitable partitioning of possible sky
conditions which are especially useful for radiation studies.
As a preprocessing task, the interesting part (which is circular
in shape) of each image in the data set has been extracted, and
then, disruptive factors like camera antenna or trees have been
eliminated from the subset images. The preprocessing phase
makes classification and image interpretation more expedient
and accurate.
After the preprocessing phase, all of the images have been
classified by MLP neural networks and SVM classifiers. Then,
the results have been analyzed and compared with each other
and with the results of the multiband thresholding algorithm
(which is the combined result of AND operations for thresh-
olded red, green, and blue bands of the images), which is a
very popular method in this field offered by EKO Co., and
with the results of the k-nearest neighbor classifier presented
by Heinle et al. [3], which is the latest paper published in the
related literature about automatic cloud classification of whole-
sky images.
MLPs are a very powerful neural network model for pixel-
level classification [12], [13], which is trained by the error back-
propagation algorithm (the most commonly used model from
feedforward family). In the MLP model, the number of units in
the hidden layer and the training phase settings (the number
of training cycles and the pixel selection for training/testing
the model) represent the fundamental tasks. Normalization,
which is a preliminary phase of neural network classifications,
is performed by linear transformation from the image interval
[0–255] to the neural network interval [1,1] (the normalization
phase ensures that the distance measures respond with equal
weight for each input) [14].
A radial basis function kernel, a very powerful kernel for
pixel-level classification [15], has been selected as the primary
SVM kernel function. In SVM (with radial basis function as the
kernel), the setting of C and γrepresents the fundamental task
in the phase of model designing.
Adjustment of the mentioned parameters (for MLPs and
SVM) affects the capability and sensitivity of the models to fit
the dynamic ranges of the pixel values in the images. The IDL
programming software for the preprocessing phase and the neu-
ral network simulator (SNNS) developed at the University of
Stuttgart, Stuttgart, Germany, have been used in implementing
the classification algorithms. Also, the SVM software (SVM-
Light package; URL: has been
used for SVM processing.
In the MLP classifier, 5423 pixels (extracted from 15 images)
have been used for training/testing the net. These 15 images
contain all different types of clouds under a variety of sky
conditions at different selected times during the day. The train-
ing sets contain 60% of the data, and the test sets contain the
remaining 40% which do not belong to the training sets. Pixel
selection for the training/test set has been performed randomly
and repeated four times.
Several attempts have been made to properly select the
number of units to be considered in the hidden layers of the
MLP. Architecture 3-7-2 has been finally chosen for its good
performance in terms of classification accuracy, root-mean-
square error (rmse), and training time. Here, 10 000 training
cycles were sufficient to train the network. The inputs of the net
consist of red, green, and blue bands, and the output provides
the pixel classification in terms of cloudy pixel or others (cloud-
free pixels or sun). One MLP has been used in classifying all of
the images.
In the SVM classifier, the images have been rescaled between
(0.0–1.0) for training the model. A range of values was tested
for the two SVM parameters C (1–50) and γ(0.1–10). A
grid search method has been used to examine the various
combinations of C and γ, and the best combination has been
finally chosen based on the model performance in terms of
classification accuracy and rmse [16].
The training/test sets are exactly the same as those which have
been used for the MLP classifier. Pixel selection for the training/
test set has been again done randomly and repeated four times.
Random pixel selection allows us to examine the robustness of
the classification algorithms with respect to the variability
of the training data set [16]. For the validation phase and
accuracy assessment, from each image in the data set, 250
pixels have been selected randomly and then labeled by visual
The accuracy of the whole test data set classified by MLP is
95.07%, with a standard deviation of 3.37, while the accuracy
of the SVM classifier is 93.66%, with a standard deviation of
4.45. The average accuracy achieved by the EKO classification
technique is 50.90%, with a standard deviation of 11.25%.
Visual inspection shows that the machine learning approaches
(MLP and SVM) achieve much better detection results under
a variety of conditions in comparison with the results of the
multiband thresholding algorithm. In Heinle’s work [3], the
average accuracy which has been obtained is about 88%. More-
over, the k-nearest neighbor classifier presented by Heinle et al.
is characterized by slow runtime performance and large mem-
ory requirements [3], whereas the classification by the artificial
neural network approach with a 1000 ×1000 image can be
completed in about a few seconds on a personal computer with
an Intel Pentium dual-core, a speed of 2.2 GHz, and a RAM
memory of 2.00 GB, which is much faster than some existing
methods in the literature. This might have a significant impact
on the reduction of the computational burden when large data
sets need to be processed.
The results of the accuracy assessment applied to different
types of clouds are displayed in Tables II and III.
Fig. 2. Classification results of six typical examples. (a) and (b) Clear-sky
example (no clouds and cloudiness below 5%). (c) and (d) Example of the
detection of low or midlevel layer of uniform clouds and dark thick clouds.
(e) Example of the detection of low puffy clouds with clearly defined edges and
high thin clouds, wisplike. (f) Example of the detection of high patched clouds
of small cloudlets, mosaiclike. (First column) Original images. (Second and
third columns) Results of the MLP and SVM classifiers. (Last column) Results
of the classification offered by EKO Co.
The MLP classifier generates a satisfactory accuracy on the
clear-sky, stratus-altostratus-nimbostratus, and stratocumulus-
cumulus groups. The average accuracy of the clear-sky class is
98.65%, with a standard deviation of 0.61. The average com-
mission (percentage of extra pixels in the class) and omission
(percentage of pixels left out of the class) errors of the clear-
sky class are 5.14% and 1.33%, respectively. In the case of the
stratus-altostratus-nimbostratus group, the average accuracies
of 96.62% (standard deviation of 0.71%) with 4.35% and 3.37%
commission and omission errors are obtained by the MLP clas-
sifier. The accuracy of the same groups classified by SVM are
97.92%, with a standard deviation of 0.80% (5.25% and 2.08%
commission and omission error averages), and 95.74%, with
a standard deviation of 1.92% (4.78% and 4.25% commission
and omission error averages).
A significant improvement of 3.60% in accuracy (with a
difference of 0.63% in standard deviation) has been obtained
on the cirrus-cirrostratus-altocumulus-cirrocumulus group seg-
mented by MLP with respect to the same data set segmented
by SVM (the average accuracy is 88.03%, with a standard
deviation of 3.03%).
It is necessary to identify the situations where the proposed
approaches generated poor accuracy and see why the models
failed to work correctly in those cases. The worst accuracy
generated by the MLP classifier is 84.88% with 14.20% com-
mission error, which is obtained for the cirrus-cirrostratus-
altocumulus-cirrocumulus group. This accuracy is 3.96%
higher than the worst accuracy obtained by the SVM classi-
fier which is for cirrus-cirrostratus-altocumulus-cirrocumulus
group (80.92% with 14.60% commission error).
This low performance is caused by the thin and transparent
parts of cirrus clouds which cannot be detected by both algo-
rithms (these parts have been classified as clear sky). Moreover,
the so-called “whitening effect” provides a misclassification of
cloud-free pixels (which are brighter due to forward scattering
by aerosols and haze) near the solar disk, and therefore, such
pixels are classified as thin clouds by the algorithms (see also
[3] and [17]). Fig. 2(b), (c), and (f) illustrates the typical
examples where the models generate less accuracy.
In the present study, the capabilities of machine learning
algorithms as automated methods for cloud segmentation in
whole-sky images have been demonstrated. Two classification
algorithms (MLP neural networks and SVM) have been com-
pared using a data set containing 250 images from two different
test sites. The same parameters were used for all of the test
The obtained accuracies showed that the machine learning ap-
proaches (MLP and SVM) achieve better detection results under
a variety of conditions with respect to the results of the thresh-
olding algorithm, which is a popular model for whole-sky im-
age classification. The machine learning models generate lower
accuracies for the cirrus-cirrostratus-altocumulus-cirrocumulus
group (caused by cirrus clouds), and in some other cases, the
accuracy is decreased because of the whitening effect near
the solar disk. The results show that the MLP classifier as an
automated algorithm for cloud classification works better in the
situations where SVM generates poor accuracy.
The authors would like to thank the associate editor and the
anonymous reviewers for their constructive comments, Colonel
F. Foti and Captain E. Vuerich from the Centre of Meteorolo-
gical Experimentation of the Italian Air Force for hosting the
cloud camera in their field at Vigna di Valle, the Misure com-
pany for kindly providing the Cloud Camera, and Dr. T. Vitti
from Tecnoel Company for the installation and assistance.
[1] Z. Chen, D. Zen, Q. Zhang, and D. Ginthner, “Sky model study using
fuzzy mathematics,” J. Illuminating Eng. Soc., vol. 23, no. 1, pp. 52–58,
[2] M. Martinez-Chico, F. J. Bathes, and J. L. Bosch, “Cloud classification in
a Mediterranean location using radiation data and sky images,” Energy,
vol. 36, no. 7, pp. 4055–4062, Jul. 2011.
[3] A. Heinle, A. Macke, and A. Srivastav, “Automatic cloud classification
of whole sky images,” Atmos. Meas. Technol., vol. 3, no. 3, pp. 557–567,
Jan. 2010.
[4] M. Singh and M. Glennen, “Automated ground-based cloud recognition,”
Pattern Anal. Appl., vol. 8, no. 3, pp. 258–271, Nov. 2005.
[5] G. Pfister et al., “Cloud coverage based on all-sky imaging and its impact
on surface solar irradiance,” J. Appl. Meteorol., vol. 42, no. 10, pp. 1421–
1434, Oct. 2003.
[6] K. A. Bush, C. H. Sun, and L. R. Thorne, “Cloud classification using
whole-sky imager data,” in Proc. 9th Symp. Meteorol. Observ. Instrumen-
tation, 1995, pp. 353–358.
[7] J. Calbo and J. Sabburg, “Feature extraction from whole-sky ground-based
images for cloud-type recognition,” J. Atmos. Ocean. Technol., pp. 3–14,
[8] L. Liu, X. J. Sun, F. Chen, S. J. Zhao, and T. C. Gao, “Cloud classification
based on structure features of infrared images,” J. Atmos. Ocean. Technol.,
vol. 28, no. 3, pp. 410–417, Mar. 2011.
[9] Y. H. Chu, H. T. C. Pedro, and C. F. M. Coimbra, “Hybrid intra-hour DNI
forecasts with sky image processing enhanced by stochastic learning,”
Solar Energy, vol. 98, pp. 592–603, Dec. 2013.
[10] J. Kalisch and A. Macke, “Estimation of the total cloud cover with high
temporal resolution and parametrization of short-term fluctuations of sea
surface insolation,” Meteorologische Zeitschrift, vol. 17, no. 5, pp. 603–
611, Oct. 2008.
[11] C. Cornaro, S. Vergari, F. Foti, and F. Del Frate, “Project CL.E.A.R.
Cloudiness Experiment For Automatic Recognition,” in Proc. TECO
WMO Tech. Conf. Meteorol. Environ. Instruments Methods Observ.,
Brussels, Belgium, Oct. 16–18, 2012, pp. 1–9.
[12] J. F. Mas and J. J. Flores, “The application of artificial neural networks to
the analysis of remotely sensed data,” Int. J. Remote Sens., vol. 29, no. 3,
pp. 617–663, Feb. 1, 2008.
[13] A. Taravat and F. Del Frate, “Development of band ratioing algorithms
and neural networks to detection of oil spills using Landsat ETM
plus data,” EURASIP J. Adv. Signal Process., vol. 2012, no. 107, pp. 1–8,
[14] K. Topouzelis, V. Karathanassi, P. Pavlakis, and D. Rokos, “Dark forma-
tion detection using neural networks,” Int. J. Remote Sens., vol. 29, no. 16,
pp. 4705–4720, 2008.
[15] G. Mountrakis, J. Im, and C. Ogole, “Support vector machines in remote
sensing: A review,” ISPRS J. Photogramm. Remote Sens., vol. 66, no. 3,
pp. 247–259, May 2011.
[16] T. Joachims, Learning to Classify Text Using Support Vector Machines:
Methods, Theory and Algorithms. New York, USA: Springer-Verlag,
[17] C. N. Long, J. M. Sabburg, J. Calbo, and D. Pages, “Retrieving
cloud characteristics from ground-based daytime color all-sky images,”
J. Atmos. Ocean. Technol., vol. 23, no. 5, pp. 633–652, May. 2006.
... Clouds in a captured image can be segmented by using various algorithms that mainly rely on Red-Green-Blue (RGB) fixed threshold values for the Red to Blue (R/B) color ratio [17][18][19] or RGB differences [15,20] and normalized R/B [21]. Other methodologies focusing on cloud segmentation are the region growing algorithm [22], the 3-D [18,23] and 4-D [22] clear sky library filled by RGB values, the following graph cut algorithm [24], the Hybrid Thresholding Algorithm (HYTA) [21] or its modified version called HYTA+ [22] and machine or deep learning techniques [22,[25][26][27][28][29]. Various methodologies are available in order to retrieve the cloud type from all-sky images. ...
... Various methodologies are available in order to retrieve the cloud type from all-sky images. The majority of implemented algorithms rely on supervised learning techniques (e.g., k-Nearest-Neighbor classifier (kNN), convolutional neural networks (CNNs)) using various spectral and textural features as well as auxiliary information such as the solar zenith angle, the cloud coverage, the visible fraction of the solar disk, and the rainfall occurrence [15,20,25,28,[30][31][32]. ...
... The overall evaluation of models' nowcasts is performed using the mean bias error (MBE), relative MBE (rMBE), mean absolute error (MAE), relative MAE (rMAE), root mean square error (RMSE), relative RMSE (rRMSE), Pearson correlation coefficient (R), Interquartile Range (IQR) and Forecast Skill (FS) that are defined by, (12) with I model the nowcasted GHI by ASI-based or persistence-based algorithms and I measured the observed GHI. The p 75 and p 25 are the 75% and 25% percentiles of the difference between I model and I measured [89]. The error metrics are calculated separately at each specific lead time and for a combined data set, including the entire time window (20 min) or subsets of lead times. ...
Fluctuations of the incoming solar irradiance impact the power generation from photovoltaic and concentrating solar thermal power plants. Accurate solar nowcasting becomes necessary to detect these sudden changes of generated power and to provide the desired information for optimal exploitation of solar systems. In the framework of the International Energy Agency's Photovoltaic Power Systems Program Task 16, a benchmarking exercise has been conducted relying on a bouquet of solar nowcasting methodologies by all-sky imagers (ASI). In this paper, four ASI systems nowcast the Global Horizontal Irradiance (GHI) with a time forecast ranging from 1 to 20 min during 28 days with variable cloud conditions spanning from September to November 2019 in southern Spain. All ASIs demonstrated their ability to accurately nowcast GHI, with RMSE ranging from 6.9% to 18.1%. Under cloudy conditions, all ASIs' nowcasts outperform the persistence models. Under clear skies, three ASIs are better than persistence. Discrepancies in the observed nowcasting performance become larger at increasing forecast horizons. The findings of this study highlight the feasibility of ASIs to reliably nowcast GHI at different sky conditions, time intervals and horizons. Such nowcasts can be used either to estimate solar power at distant times or detect sudden GHI fluctuations.
... The cloud classification methods mainly contain ANN and CNN networks. Taravat A et al. inputted sky camera data into the ANN network for cloud classification tasks and achieved good classification results [19]. Afzali Gorooh et al. used an ANN network to classify satellite images and analyzed the precipitation rates of different types of clouds [20]. ...
Full-text available
The study of cloud types is critical for understanding atmospheric motions and climate predictions; for example, accurately classified cloud products help improve meteorological predicting accuracies. However, the current satellite cloud classification methods generally analyze the threshold change in a single pixel and do not consider the relationship between the surrounding pixels. The classification development relies heavily on human recourses and does not fully utilize the data-driven advantages of computer models. Here, a new intelligent cloud classification method based on the U-Net network (CLP-CNN) is developed to obtain more accurate, higher frequency, and larger coverage cloud classification products. The experimental results show that the CLP-CNN network can complete a cloud classification task of 800 × 800 pixels in 0.9 s. The classification area covers most of China, and the classification task only needs to use the original L1-level data, which can meet the requirements of a real-time operation. With the Himawari-8 CLTYPE product and the CloudSat 2B-CLDCLASS product as the test comparison target, the CLP-CNN network results match the Himawari-8 product highly, by 84.4%. The probability of detection (POD) is greater than 0.83 for clear skies, deep-convection, and Cirrus–Stratus type clouds. The probability of detection (POD) and accuracy are improved compared with other deep learning methods.
... To identify the most relevant color components, clustering and dimensionality reduction techniques were applied (Dev et al., 2014(Dev et al., , 2016. There are also studies on supervised learning techniques for classifying pixels in ASIs such as neural networks, support vector machines (SVMs), random forests and Bayesian classifiers (Taravat et al., 2014;Cheng and Lin, 2017;Ye et al., 2019). Lately, also deep learning approaches using convolutional neural networks (CNNs) were presented (Dev et al., 2019;Xie et al., 2020;Song et al., 2020). ...
Full-text available
Semantic segmentation of ground-based all-sky images (ASIs) can provide high-resolution cloud coverage information of distinct cloud types, applicable for meteorology-, climatology- and solar-energy-related applications. Since the shape and appearance of clouds is variable, and there is high similarity between cloud types, a clear classification is difficult. Therefore, most state-of-the-art methods focus on the distinction between cloudy and cloud-free pixels without taking into account the cloud type. On the other hand, cloud classification is typically determined separately at the image level, neglecting the cloud's position and only considering the prevailing cloud type. Deep neural networks have proven to be very effective and robust for segmentation tasks; however they require large training datasets to learn complex visual features. In this work, we present a self-supervised learning approach to exploit many more data than in purely supervised training and thus increase the model's performance. In the first step, we use about 300 000 ASIs in two different pretext tasks for pretraining. One of them pursues an image reconstruction approach. The other one is based on the DeepCluster model, an iterative procedure of clustering and classifying the neural network output. In the second step, our model is fine-tuned on a small labeled dataset of 770 ASIs, of which 616 are used for training and 154 for validation. For each of them, a ground truth mask was created that classifies each pixel into clear sky or a low-layer, mid-layer or high-layer cloud. To analyze the effectiveness of self-supervised pretraining, we compare our approach to randomly initialized and pretrained ImageNet weights using the same training and validation sets. Achieving 85.8 % pixel accuracy on average, our best self-supervised model outperforms the conventional approaches of random (78.3 %) and pretrained ImageNet initialization (82.1 %). The benefits become even more evident when regarding precision, recall and intersection over union (IoU) of the respective cloud classes, where the improvement is between 5 and 20 percentage points. Furthermore, we compare the performance of our best model with regards to binary segmentation with a clear-sky library (CSL) from the literature. Our model outperforms the CSL by over 7 percentage points, reaching a pixel accuracy of 95 %.
... Many studies show that deep learning methods can adaptively learn the deep features of clouds and have higher detection accuracy than traditional machine learning methods [26][27][28][29][30][31]. Liu et al. introduced a neural network for satellite cloud detection tasks, and conducted experiments on the FY-2C satellite cloud image dataset. ...
Full-text available
Cloud recognition is a basic task in ground meteorological observation. It is of great significance to accurately identify cloud types from long-time-series satellite cloud images for improving the reliability and accuracy of weather forecasting. However, different from ground-based cloud images with a small observation range and easy operation, satellite cloud images have a wider cloud coverage area and contain more surface features. Hence, it is difficult to effectively extract the structural shape, area size, contour shape, hue, shadow and texture of clouds through traditional deep learning methods. In order to analyze the regional cloud type characteristics effectively, we construct a China region meteorological satellite cloud image dataset named CRMSCD, which consists of nine cloud types and the clear sky (cloudless). In this paper, we propose a novel neural network model, UATNet, which can realize the pixel-level classification of meteorological satellite cloud images. Our model efficiently integrates the spatial and multi-channel information of clouds. Specifically, several transformer blocks with modified self-attention computation (swin transformer blocks) and patch merging operations are used to build a hierarchical transformer, and spatial displacement is introduced to construct long-distance cross-window connections. In addition, we introduce a Channel Cross fusion with Transformer (CCT) to guide the multi-scale channel fusion, and design an Attention-based Squeeze and Excitation (ASE) to effectively connect the fused multi-scale channel information to the decoder features. The experimental results demonstrate that the proposed model achieved 82.33% PA, 67.79% MPA, 54.51% MIoU and 70.96% FWIoU on CRMSCD. Compared with the existing models, our method produces more precise segmentation performance, which demonstrates its superiority on meteorological satellite cloud recognition tasks.
... However, various alternative procedures have been developed for CIE standard sky classification [9], due to the scarcity of sky scanners available to gather sky luminance data at ground meteorological stations. In this task, Supervised Machine Learning (SML) procedures are proposed as effective tools for sky classification, based on accessible meteorological indices [10] such as decision trees (DTs) [11], Support Vector Machines (SVMs) [12], and Artificial Neural Networks (ANNs) [13][14][15]. ...
Full-text available
Digital sky images are studied for the definition of sky conditions in accordance with the CIE Standard General Sky Guide. Likewise, adequate image-processing methods are analyzed that highlight key image information, prior to the application of Artificial Neural Network classification algorithms. Twenty-two image-processing methods are reviewed and applied to a broad and unbiased dataset of 1500 sky images recorded in Burgos, Spain, over an extensive experimental campaign. The dataset comprises one hundred images of each CIE standard sky type, previously classified from simultaneous sky scanner data. Color spaces, spectral features, and texture filters image-processing methods are applied. While the use of the traditional RGB color space for image-processing yielded good results (ANN accuracy equal to 86.6%), other color spaces, such as Hue Saturation Value (HSV), which may be more appropriate, increased the accuracy of their global classifications. The use of either the green or the blue monochromatic channels improved sky classification, both for the fifteen CIE standard sky types and for simpler classification into clear, partial, and overcast conditions. The main conclusion was that specific image-processing methods could improve ANN-algorithm accuracy, depending on the image information required for the classification problem.
Cloud image segmentation plays an important role in ground-based cloud observation. Recently, most existing methods for ground-based cloud image segmentation learn feature representations using the convolutional neural network (CNN), which results in the loss of global information because of the limited receptive field size of the filters in the CNN. In this article, we propose a novel deep model named TransCloudSeg, which makes full use of the advantages of the CNN and transformer to extract detailed information and global contextual information for ground-based cloud image segmentation. Specifically, TransCloudSeg hybridizes the CNN and transformer as the encoders to obtain different features. To recover and fuse the feature maps from the encoders, we design the CNN decoder and the transformer decoder for TransCloudSeg. After obtaining two sets of feature maps from two different decoders, we propose the heterogeneous fusion module to effectively fuse the heterogeneous feature maps by applying the self-attention mechanism. We conduct a series of experiments on Tianjin Normal University large-scale cloud detection database and Tianjin Normal University cloud detection database, and the results show that our method achieves a better performance than other state-of-the-art methods, thus proving the effectiveness of the proposed TransCloudSeg.
Full-text available
One of the largest challenges in Numerical Weather Prediction (NWP) is cloud forecasting. The reason is twofold: first, clouds are constantly changing size and shape over short periods of time, and second, cloud formation processes occur at wide scales, from sub-micrometer to few kilometers. Typical NWP models operate at scales between 1 to 20 km resolution and run between 2 to 8 times per day, thus spatial and temporal limited for accurate cloud forecasting. Data extrapolation techniques have emerged to forecast cloud cover using deep learning and satellite imagery. However, satellite resolution remains impractical for hyperlocal cloudiness forecasting, which is relevant for many industries such as solar power, astronomy, and aviation. Over the last years, higher resolution models have been developed (<1km). One example is ClimaCell Bespoke Atmospheric Model (CBAM), which claims to be the world’s highest resolution weather forecasting model. CBAM initial conditions are fed with data from power grids, cellular networks, road cameras, connected vehicles, and smartphones. It is well established that the accuracy of NWP models improves significantly with the assimilation of high-resolution meteorological observations. However, most data-assimilation techniques omit cloud observations, thus the benefits of highly localized cloud-assimilation are still largely unknown. Since clouds directly influence solar radiation, surface temperature, precipitation and are useful signs to predict upcoming weather, I have hypothesized that the next generation of NWP models will require assimilation of high-resolution cloud data. The overall goal of this thesis is to retrieve cloud cover, cloud type, cloud motion vectors, cloud base height, and cloud transmittance from the combined use of a skycam, a pyranometer, and a ceilometer, aiming to ultimately improve hyperlocal short-term weather forecasting. This study comprised summer daytime data from July, August, and September 2020. The datasets generated by this dissertation are publicly available on
A conceptual runoff model mathematically expresses hydrological phenomena originating from spatial and temporal changes based on physical laws; thus, it can describe causal relationships for runoff changes. However, these models are difficult to run, owing to their complex structures and the need for a large amount of data. The development of runoff models using deep neural networks (DNN), which belong to the category of empirical models, is continuously increasing owing to the advantages of DNN, such as the modeling convenience and high prediction performance. However, DNN–based runoff models have difficulty reflecting topographical and spatial characteristics, such as land cover and physical soil characteristics, because they mainly depend on meteorological data. The objective of the present study was to overcome the limitation of data use that depends on meteorological data and to reflect topographical characteristics in DNN–based runoff models by suggesting a data creation methodology that reflects spatial characteristics. This paper proposes a methodology for producing two types of two–dimensional features—surface flow features (SFF) and base flow features (BFF)—using the Soil Conservation Service curve number and groundwater level. To determine the applicability of the features, the daily runoff was simulated using the generated features as input data of the convolutional neural network (CNN), which is a type of DNN. The CNN architecture was improved to have a multi–input structure and derive continuous variables, because the ordinary CNN was difficult to apply in this study. After the CNN was trained, the daily runoff was simulated, and the applicability of the features as input data of the DNN was evaluated. The daily runoff predictions of the CNN using the SFF and BFF exhibited moderate levels. This indicates that the SFF and BFF have sufficient value as input data of the CNN.
Photovoltaic systems are sensitive to cloud shadow projection, which needs to be forecasted to reduce the noise impacting the intra-hour forecast of global solar irradiance. We present a comparison between different kernel discriminative models for cloud detection. The models are solved in the primal formulation to make them feasible in real-time applications. The performances are compared using the j-statistic. The infrared cloud images have been preprocessed to remove debris, which increases the performance of the analyzed methods. The use of the pixels’ neighboring features also leads to a performance improvement. Discriminative models solved in the primal yield a dramatically lower computing time along with high performance in the segmentation.
Full-text available
The recently increasing development of whole sky imagers enables temporal and spatial high-resolution sky observations. One application already performed in most cases is the estimation of fractional sky cover. A distinction between different cloud types, however, is still in progress. Here, an automatic cloud classification algorithm is presented, based on a set of mainly statistical features describing the color as well as the texture of an image. The k-nearest-neighbour classifier is used due to its high performance in solving complex issues, simplicity of implementation and low computational complexity. Seven different sky conditions are distinguished: high thin clouds (cirrus and cirrostratus), high patched cumuliform clouds (cirrocumulus and altocumulus), stratocumulus clouds, low cumuliform clouds, thick clouds (cumulonimbus and nimbostratus), stratiform clouds and clear sky. Based on the Leave-One-Out Cross-Validation the algorithm achieves an accuracy of about 97%. In addition, a test run of random images is presented, still outperforming previous algorithms by yielding a success rate of about 75%, or up to 88% if only "serious" errors with respect to radiation impact are considered. Reasons for the decrement in accuracy are discussed, and ideas to further improve the classification results, especially in problematic cases, are investigated.
Full-text available
In Lauder, Central Otago, New Zealand, two all-sky imaging systems have been in operation for more than 1 yr, measuring the total, opaque, and thin cloud fraction, as well as indicating whether the sun is obscured by clouds. The data provide a basis for investigating the impact of clouds on the surface radiation field. The all-sky cloud parameters were combined with measurements of global, direct, and diffuse surface solar irradiance over the spectral interval from 0.3 to 3 μm. Here, the results of ongoing analysis of this dataset are described. As a reference for the magnitude of the cloud influence, clear-sky irradiance values are estimated as a simple function of solar zenith angle and the earth-sun distance. The function is derived from a least squares fit to measurements taken when available cloud images show clear-sky situations. Averaged over a longer time period, such as 1 month, cloud fraction and surface irradiance are clearly negatively correlated. Monthly means in the ratio of the measured surface irradiance to the clear-sky value had a correlation coefficient of about -0.9 with means of cloud fraction for the months from July 2000 to June 2001. In the present work reductions in the surface irradiance and situations in which clouds cause radiation values to exceed the expected clear-sky amount are analyzed. Over 1 yr of observations, 1-min-averaged radiation measurements exceeding the expected clear-sky value by more than 10% were observed with a frequency of 5%. In contrast, a reduction of more than 10% below estimated clear-sky values occurred in 66% of the cases, while clear-sky irradiances (measured irradiance within ±10% of estimated clear-sky value) were observed 29% of the time. Low cloud fractions frequently lead to moderate enhancement, because the sun is often unobscured and the clouds are brighter than the sky that they hide. As cloud fraction increases the sun is likely to be obscured, causing irradiance values to fall well below clear-sky values. However, in the case of unobscured sun, there is a tendency for strongest enhancements when cloud fractions are highest. Enhancements, especially at high solar zenith angle, are also often observed in association with thin clouds.
Full-text available
Accurate knowledge of the spatial extents and distributions of an oil spill is very impor-tant for efficient response. This is because most petroleum products spread rapidly on the water surface when released into the ocean, with the majority of the affected area becoming covered by very thin sheets. This article presents a study for examining the feasibility of Landsat ETM+ images in order to detect oil spills pollutions. The Landsat ETM+ images for 1st, 10th, 17th May 2010 were used to study the oil spill in Gulf of Mexico. In this article, an attempt has been made to perform ratio operations to enhance the feature. The study concluded that the bands difference between 660 and 560 nm, division at 660 and 560 and division at 825 and 560 nm, normalized by 480 nm provide the best result. Multilayer perceptron neural network classifier is used in order to perform a pixel-based supervised classification. The result indicates the potential of Landsat ETM+ data in oil spill detection. The promising results achieved encourage a further analysis of the potential of the optical oil spill detection approach.
Full-text available
Some cloud structure features that can be extracted from infrared images of the sky are suggested for cloud classification. Both the features and the classifier are developed over zenithal images taken by the whole-sky infrared cloud-measuring system (WSIRCMS), which is placed in Nanjing, China. Before feature extraction, the original infrared image was smoothed to suppress noise. Then, the image was enhanced using top-hat transformation and a high-pass filtering. Edges are detected from the enhanced image after adaptive optimization threshold segmentation and morphological edge detection. Several structural features are extracted from the segment image and edge image, such as cloud gray mean value (ME), cloud fraction (ECF), edge sharpness (ES), and cloud mass and gap distribution parameters, including very small-sized cloud mass and gaps (SMG), middle-sized cloud gaps (MG), medium-small-sized cloud gaps (MSG), and main cloud mass (MM). It is found that these features are useful for distinguishing cirriform, cumuliform, and waveform clouds. A simple but efficient supervised classifier called the rectangle method is used to do cloud classification. The performance of the classifier is assessed with an a priori classification carried out by visual inspection of 277 images. The index of agreement is 90.97%.
We propose novel smart forecasting models for Direct Normal Irradiance (DNI) that combine sky image processing with Artificial Neural Network (ANN) optimization schemes. The forecasting models, which were developed for over 6 months of intra-minute imaging and irradiance measurements, are used to predict 1 min average DNI for specific time horizons of 5 and 10 min. We discuss optimal models for low and high DNI variability seasons. The different methods used to develop these season-specific models consist of sky image processing, deterministic and ANN forecasting models, a genetic algorithm (GA) overseeing model optimization and two alternative methods for training and validation. The validation process is carried over by the Cross Validation Method (CVM) and by a randomized training and validation set method (RTM). The forecast performance for each solar variability season is evaluated, and the models with the best forecasting skill for each season are selected to build a hybrid model that exhibits optimal performance for all seasons. An independent testing set is used to assess the performance of all forecasting models. Performance is assessed in terms of common error statistics (mean bias and root mean square error), but also in terms of forecasting skill over persistence. The hybrid forecast models proposed in this work achieve statistically robust forecasting skills in excess of 20% over persistence for both 5 and 10 min ahead forecasts, respectively.
Text Classification, or the task of automatically assigning semantic categories to natural language text, has become one of the key methods for organizing online information. Since hand-coding classification rules is costly or even impractical, most modern approaches employ machine learning techniques to automatically learn text classifiers from examples. However, none of these conventional approaches combines good prediction performance, theoretical understanding, and efficient training algorithms. \ \ Based on ideas from Support Vector Machines (SVMs), Learning To Classify Text Using Support Vector Machines presents a new approach to generating text classifiers from examples. The approach combines high performance and efficiency with theoretical understanding and improved robustness. In particular, it is highly effective without greedy heuristic components. The SVM approach is computationally efficient in training and classification, and it comes with a learning theory that can guide real-world applications. \ \ Learning To Classify Text Using Support Vector Machines gives a complete and detailed description of the SVM approach to learning text classifiers, including training algorithms, transductive text classification, efficient performance estimation, and a statistical learning model of text classification. In addition, it includes an overview of the field of text classification, making it self-contained even for newcomers to the field. This book gives a concise introduction to SVMs for pattern recognition, and it includes a detailed description of how to formulate text-classification tasks for machine learning. \ \ Learning To Classify Text Using Support Vector Machines is designed as a reference for researchers and practitioners, and is suitable as a secondary text for graduate-level students in Computer Science within Machine Learning and Language Technology.
Clouds are one of the most important moderators of the earth radiation budget and one of the least understood. The effect that clouds have on the reflection and absorption of solar and terrestrial radiation is strongly influenced by their shape, size, and composition. Physically accurate parameterization of clouds is necessary for any general circulation model (GCM) to yield meaningful results. The work presented here is part of a larger project that is aimed at producing realistic three-dimensional (3D) volume renderings of cloud scenes, thereby providing the important shape information for parameterizing GCMs. The specific goal of the current study is to develop an algorithm that automatically classifies (by cloud type) the clouds observed in the scene. This information will assist the volume rendering program in determining the shape of the cloud. Much work has been done on cloud classification using multispectral satellite images. Most of these references use some kind of texture measure to distinguish the different cloud types and some also use topological features (such as cloud/sky connectivity or total number of clouds). A wide variety of classification methods has been used, including neural networks, various types of clustering, and thresholding. The work presented here utilizes binary decision trees to distinguish the different cloud types based on cloud feature vectors.