Conference PaperPDF Available

Solar Event Classification Using Deep Convolutional Neural Networks

Authors:

Abstract and Figures

The recent advances in the field of neural networks, more specifically deep convolutional neural networks (CNN), have considerably improved the performance of computer vision and image recognition systems in domains such as medical imaging, object recognition, and scene characterization. In this work, we present the first attempt into bringing CNNs to the field of Solar Astronomy, with the application of solar event recognition. With the objective of advancing the state-of-the-art in the field, we compare the performance of multiple well established CNN architectures against the current methods of multiple solar event classification. To evaluate the effectiveness of deep learning in the solar image domain, we experimented with well-known architectures such as LeNet-5, CifarNet, AlexNet, and GoogLenet. We investigated the recognition of four solar event types using image regions extracted from the high-resolution full disk images of the Sun from the NASA’s Solar Dynamics Observatory (SDO) mission. This work demonstrates the feasibility of using CNNs by obtaining improved results over the conventional pattern recognition methods used in the field.
Content may be subject to copyright.
Solar Event Classification using Deep Convolutional
Neural Networks
Ahmet Kucuk, Juan M. Banda, and Rafal A. Angryk
Georgia State University,
Department of Computer Science,
Atlanta, GA, USA
{akucuk1,jbanda, rangryk}@cs.gsu.edu
Abstract. The recent advances in the field of neural networks, more specifically
deep convolutional neural networks (CNN), have considerably improved the per-
formance of computer vision and image recognition systems in domains such
as medical imaging, object recognition, and scene characterization. In this work,
we present the first attempt into bringing CNNs to the field of Solar Astronomy,
with the application of solar event recognition. With the objective of advancing
the state-of-the-art in the field, we compare the performance of multiple well es-
tablished CNN architectures against the current methods of multiple solar event
classification. To evaluate the effectiveness of deep learning in the solar image
domain, we experimented with well-known architectures such as LeNet-5, Ci-
farNet, AlexNet, and GoogLenet. We investigated the recognition of four solar
event types using image regions extracted from the high-resolution full disk im-
ages of the Sun from the NASAs Solar Dynamics Observatory (SDO) mission.
This work demonstrates the feasibility of using CNNs by obtaining improved
results over the conventional pattern recognition methods used in the field.
Keywords: Image Classification, Solar Event Classification, Deep Learning, Con-
volutional Neural Networks
1 INTRODUCTION
The effectiveness of deep neural networks in computer vision has been demonstrated
over the years in several different works [21, 25, 38, 39]. One of the first major successes
of deep learning was handwritten character recognition in 1990s [25]. More recently,
Krizhevsky et al. won the 2012 ImageNet Large Scale Visual Recognition Competition
(ILSRVC) with his graphics processing unit (GPU) supported method. Nowadays, the
usage of deep convolutional neural networks (CNN) in computer vision tasks has ex-
ploded [21]. Several variations and techniques for CNNs are developed to be used in
wide range of applications including object detection [38], image recognition [19], and
autonomous driving systems [11]. Today, we are witnessing the effectiveness of deep
learning in industry-level applications of major companies like Google’s PlaNet for im-
age geolocation [42] or Facebook’s DeepFace for face recognition [39], among other
applications.
2 Lecture Notes in Computer Science: Authors’ Instructions
…....
…....
…....
….
Region Extraction Resizing
Convolution Pooling Fully Connected Layers
AR
CH
FL
SG
QS
Output
Fig. 1. Example of solar event classification using deep convolutional neural networks.
To eliminate the filtering effect of the atmosphere during solar observations, NASA
launched a satellite named Solar Dynamic Observatory (SDO) into orbit [43]. Since
2010, SDO has been successfully taking high-resolution full-disk images of the Sun
in 10 different wavelengths [26]. SDO generates 1.5 TB image data in a day, which
makes it one of the largest solar dataset available. The data is used by solar physicists
for a better understanding of the Sun, which is crucial because coronal mass ejection
and solar flares on the Sun might endanger astronauts by changing the radiation levels
or interrupt Global Positioning System (GPS) and intercontinental communications or
damage power grids [23]. For this purpose, several computer vision tasks including ob-
ject recognition [14], image classification [6], and image retrieval [8] have been utilized
on this data for solar physics research. However, we believe that emerging deep learning
techniques can further improve the effectiveness of previous studies.
Deep learning has several advantages compared to conventional techniques in com-
puter vision. One of these advantages is that deep learning does not require extensive
domain knowledge or expertise to select the features on the images because a general
purpose learning algorithm can configure the parameters on its own; thus, the network
can automatically extract the most useful features for its decision process [24]. This
phenomenon is extremely compelling for the solar domain because of the fuzzy bound-
aries of solar events in the images. Additionally, as it is illustrated in Figure 1, deep
learning models can be fed with raw image data, which avoids daily feature extraction
process required by conventional methods [8]. Although the training process can be
time-consuming for large CNNs, response time in production is reasonable for real-
time applications [18].
We believe that deep learning can be an alternative to conventional methods, which
extract the image parameters and use well-known machine learning algorithms, to solve
solar related computer vision tasks [4]. Our approach has a promising potential bene-
Solar Event Classification using Deep CNNs 3
fit for several applications such as Content-Based Image Retrieval (CBIR) systems for
solar images [6], event detection, validation of existing event detection modules [40],
and solar event prediction [31]. To introduce the deep learning in the solar domain, we
selected the task of classifying solar events in image regions. To the best of our knowl-
edge, this is the first work that demonstrates the effectiveness of deep convolutional
neural networks on solar event classification that forms a foundation for other computer
vision tasks related to solar image domain.
The rest of this paper is organized as follows: Section 2 gives background informa-
tion about both deep learning and solar image classification. We then present the solar
image data preparation steps and configuration details of used CNN architectures in
Section 3. In Section 4, we present results of the experiments on different CNN archi-
tectures with a detailed discussion. Lastly, in Section 5, we present our conclusions and
provide insights into our future work.
2 BACKGROUND
2.1 Convolutional Neural Networks
LeCun et al. successfully applied CNN on a handwritten digit recognition task for the
first time in the 1990s [25]. At the time, lack of large datasets and computational power
needed for training the deeper models to solve more complicated tasks were an obstacle
to the popularity of the technique. The last decade brought very large datasets and made
higher computational power available at a very affordable prices. Along with increasing
central processing unit (CPU) computing power, GPU computing has been adapted for
deep learning for faster computational power in certain tasks. Thanks to these advance-
ments and novel improvements in CNN architecture, deep CNNs gained major success
in computer vision tasks such as ILSVRC competitions [19, 21, 38]. The success gave
community an alternative to conventional methods like feature engineering.
A simple CNN differs from other deep learning architectures in its convolutional
layers that are specifically designed for image data. Convolutional layers use a kernel
to filter the image to extract local features from an image and it is usually followed by a
pooling layer that reduces the number of features extracted while increasing the robust-
ness of output to shifts and distortions [25]. In fully connected layers, extracted features
are collected to detect higher level features [25]. To eliminate non-significant neurons
and emphasize significant ones, an activation function is used. CNNs can be trained
using back propagation that allows the network to adjust weights of the network to re-
duce the output of loss function. In other words, this process simulates hand-picking of
features that is usually performed by domain experts in most of the traditional computer
vision approaches.
Several techniques based on simple CNN architecture are proposed to address dif-
ferent problems of the learning process. The most relevant techniques include Rectified
Linear Unit (ReLU), data augmentation, and dropout. Using ReLU activation increases
the training speed significantly when it is compared to tanh activation function [21,29].
Data augmentation is artificially populating the existing data to reduce overfitting [21].
Dropout is randomly picking some neurons to be silenced temporarily. Theoretically,
4 Lecture Notes in Computer Science: Authors’ Instructions
a network with nneurons behaves like an assembly of up to 2nmodel while mak-
ing decision [37]. This technique is also very effective against overfitting. Besides the
architectural advancements, the efficiency of deep learning has significantly improved
thanks to hardware support. GPU usage during training process increase the training
speed significantly so that larger models can be trained to solve more complex models
in reasonable times.
Network in Network (NIN) structure is another technique introduced recently that
has proved its effectiveness [27]. NIN structure consists of the multilayer perceptron
(MLP) and global averaging pooling layer. Multilayer perceptrons, which consist of
several layers to increase the effectiveness of local feature extraction, are used instead
of convolution filters. Fully connected layers, followed by convolutional layers, were
replaced with global average pooling that introduce feature map for each class. Un-
like fully connected layer, global average pooling is not prone to overfitting. Inception
model introduced in [38] is inspired by NIN structure. The main idea in their work is to
increase the depth of a network by adding 1x1 convolutional layers and enhancing the
feature re-use. While increasing the network depth, this strategy decreases the number
of parameters used in the network.
Increasing the depth and width of a deep learning model is the simplest way to
increase the performance of the network. Although increment in the depth size brings
problems with it such as overfitting, computational complexity, and vanishing gradient,
deep models (with 22 layers) have been trained in these works [36, 38]. Even deeper
models with 152 layers have been trained using residual learning technique [19].
While advancements on deep learning architectures are continuing at a fast pace,
CNN applications produced significant results in real life problems such as face ver-
ification by Facebook [39], vehicle type classification [16], species recognition for
wild animal monitoring [12], land use classification in remote sensing images [10].
CNNs also started to give good results on medical imaging, which is interesting for our
work because of the similarity between medical images and solar images [7]. Usages
of CNNs on medical image includes, but are not limited to, glaucoma detection [13],
segmentation of neuronal membranes [15], and identification of breast cancer [41].
2.2 Solar Event Classification
With the start of the SDO mission in 2010 the Feature Finding Team (FFT) has been in
charge of identifying solar events present in SDO images via two different approaches:
a) traditional computer vision via specialized solar event detection modules, and b) the
trainable module, as described in [28]. The specialized solar event detection modules
are in charge of detecting individual events like Flares [14], Coronal Holes and Active
Regions [9] and Sigmoids [32]. The trainable module utilizes statistical classification
algorithms to identify solar phenomena and has been detailed in [1].
The SDO images are processed by extracting a set of ten image parameters: entropy,
fractal dimension, the mean intensity, the third and fourth moments, relative smooth-
ness, the standard deviation of the intensity, Tamura contrast, Tamura directionality and
uniformity. These parameters have been tested to perform the best based on computa-
tional expense and classification accuracy for SDO images. More details as well as their
formulas are beyond the scope of this work and can be found in [2, 3, 8].
Solar Event Classification using Deep CNNs 5
a) Active Region (AR)
171
b) Sigmoid (SG)
131
c) Coronal Hole (CH)
193
d) Flare (FL)
131
e) Quiet Sun (QS)
131
Fig. 2. Example image regions for each event type with corresponding wavelengths
Once the image parameters have been extracted, multiple classification algorithms
have been tested in [5] allowing researchers to identify which ones perform the best
depending on individual combinations of image parameters and types of solar events
[1]. With the advantage of building a general purpose classifier that is able to identify
more than one different type of solar event at a time, the current state of the art in
terms of solar event classification has been achieved in [6]. The average classification
accurracy for Active Regions (AR), Coronal Holes (CH), Flares (FL), Sigmoids (SG)
has reach 70% while the individual accurracies are 79%, 84%, 72%, and 68%. These
values will be considered our baseline results to compare against in this work.
In order to provide researchers with training and testing data for classification and
retrieval analyses, the FFT team has released multiple datasets which include solar im-
age parameters corresponding to labeled events and can be found in [34, 35]. It was
not until [22] where researchers had also have the access to the corresponding high-
resolution image files for the solar events in datasets [34, 35]. This allows for analyses
like ones presented in this work.
3 METHODS
3.1 Data
We used the Large-Scale Solar Dynamic Observatory Dataset (LSDO) that can be found
in [22]. The data used in this work consists of solar image regions that are labeled as a
particular type of solar event. We conducted several preprocessing steps to prepare an
appropriate dataset for our task. The first preprocessing step was to ensure that we feed
our models with unique records that does not contain overlaps of multiple event types.
Event records in LSDO are reported from several different detector modules, which
means that the same solar activity can be reported multiple times, if different types of
events co-occur or different module can independently report the same events multi-
ple times. To discard the possible duplicate reports, we only used reports from Spatial
Possibilistic Clustering Algorithm (SPoCA) for AR and CH [40], Flare Detective for
FL [17], and Sigmoid Sniffer for SG [32]. In addition, we found out that some events
occur spatiotemporally close to each other which may lead to ambiguity in labeling. To
remove this ambiguity, we eliminated the events that are spatiotemporally overlapping
(i.e. co-occurring).
Before we extract the regions from 4K full disk images of the Sun using bounding
box information, we conducted an additional step. The network models that we use for
6 Lecture Notes in Computer Science: Authors’ Instructions
experiments are designed for square regions. However, bounding boxes of solar events
have different sizes and aspect ratios. We shortened the list of regions by selecting the
ones with the aspect ratio min(w, h)/max(w, h), ratio between width (w) and height
(h) greater than ratio threshold 0.90, which we picked according to the distribution of
ratio for each event type to get desired number of regions. We introduced ratio threshold
because event regions with low aspect ratio may change the characteristic of the event
pattern on the region significantly when it is rescaled to be fit into the models. Finally,
we extracted regions and labeled them accordingly. Samples of each class is shown in
Figure 2.
Moreover, to evaluate our models under more realistic conditions, we created a fifth
class, Quiet Sun (QS), which consists of unlabeled regions of the Sun. QS represents
regions of the Sun without any specific activity. To create QS regions, in each image
we randomly picked a region in size of 32x32 to 256x256 pixels and checked if the
selected region is spatiotemporally overlapping with any event records in all dataset.
We avoided to pick regions close to the edge of the image in order not to include the
background region behind the Sun. We repeated this process until we found a non-
overlapping region or gave up on that image. In Figure 2, region eis an example QS
region.
Instead of having a single dataset, we created three datasets to use in different CNN
models and test different aspects of the models. For each event type, we used the optimal
reported wavelength, which is listed as follows: 171 ˚
A for AR, 193 ˚
A for CH, 131 ˚
A for
FL and SG. We chose 1,500 regions with the highest resolution for each event type on
the specified wavelength. For the fifth class (i.e. QS), we used 500 regions from each
wavelength (131 ˚
A, 171 ˚
A, 193 ˚
A) to get 1,500 regions so that this class contains the
wavelength of all other event types. We used 32x32, 128x128, 256x256 pixels region
sizes to evaluate and understand the behavior of models against different image sizes.
To resize images, we used Lanczos algorithm to get better quality pacthes [30]. For
convenience, we named them as follows: ”R32” for 32x32, ”R128” for 128x128, and
”R256” for 256x256 regions.
3.2 CNN Architectures
Several different CNN architectures are presented to solve computer vision problems
in different domains and in Section 2.1, we mention the most significant techniques
employed by these models. However, for solar image domain, there is no previously
proposed CNN architecture and selection of a CNN model for a particular dataset is not
trivial. To see the performance of CNN architectures in solar image classification task,
we used several different models that have success in other domains. These models
consist of different number of layers and employ different techniques, which allow
us to measure the effectiveness of deep learning on the solar domain under different
configurations.
We used four different CNN architectures: LeNet-5, CifarNet, AlexNet, and GoogLenet.
LeNet-5 is trained by using stochastic gradient descent (SGD) with learning rate of 0.01,
momentum of 0.9, batch size of 100, and weight decay of 0.0005. CifarNet model is
trained by using SGD with learning rate of 0.001, momentum of 0.9, batch size of 100,
and decay of 0.004. In AlexNet model, we used data augmentation. Random subregions
Solar Event Classification using Deep CNNs 7
Model Dataset Epochs Accuracy in %
AR CH FL SG QS Average
LeNet-5 R32 65 97.7 97.0 87.0 92.0 94.0 93.5
CifarNet R32 80 99.7 97.3 91.3 85.0 96.3 93.9
CifarNet R128 80 78.3 88.7 86.3 92.0 82.3 85.5
CifarNet R256 75 99.0 99.3 75.0 82.0 85.0 88.1
AlexNet R128 35 97.3 95.0 85.0 88.3 92.7 91.7
AlexNet R256 40 98.0 95.7 85.0 84.0 91.0 90.7
GoogLenet R256 40 98.0 99.3 84.0 91.0 98.3 94.1
Table 1. Highest average testing accuracy of models with class-based classification accuracy
of the input image are given to the network as an input in order to reduce overfitting.
For 256x256 regions, 227x227 sub-images are used as it is used in [21]. For 128x128
regions, we selected 96x96 sub-images for data augmentation. We trained this mode
using SGD with learning rate of 0.01, momentum of 0.9, batch size of 100, and decay
of 0.0005. GoogLenet is trained by using SGD with learning rate of 0.01, momentum
of 0.9, batch size of 50, and decay of 0.0002.
4 EXPERIMENTS
4.1 Experimental Setup
Experiments are conducted on a dedicated server with 128 cores (Intel Xeon CPU E7-
8860 with Ubuntu 14.04). We used the CPU-only version of popular deep learning
framework, Caffe [20]. OpenBLAS is a dependency of Caffe and we configured Open-
BLAS library to compile for 128 cores. All the results are derived from random split of
80% training data and 20% test data for each dataset.
4.2 Evaluation of the Models
Our goal is not only to accurate classification of the regions, but also to evaluate the
general behavior of deep learning on solar imagery. Table 1 depicts best average ac-
curacy of each model, class-based accuracy, and number of epochs. We experimented
on different models and different datasets (different image sizes). Some model-dataset
combination were not listed in Table 1 because some of the models can not be feed
with all image sizes and some of the experiments never give interpretable results after
several parameter search.
In Figure 3, we present 10 epochs runtime of each model in log scale. There is
a considerable performance gap between fastest model, LeNet-5, and slowest model,
GoogLeNet. CifarNet is close to AlexNet when it is trained on same dataset. Another
implication of Figure 3 is runtime by image size. As size of images increases, runtime
changes significantly which can be clearly observed in CifarNet experiments.
Region size is important aspect of our experiments that needed to be investigated,
because solar events are reported on a wide range of sizes (e.g. 16x16 to 512x512) and
8 Lecture Notes in Computer Science: Authors’ Instructions
110 100 1000 10000
Goog Lenet -R256
AlexNet-R256
AlexNet-R128
CifarNet-R32
LeNet-5-R32
Runtime;(m in)
Fig. 3. Runtime of models for 10 epochs in log scale
most of the time, regions are small part in a high resolution full disk image (4096x4096).
Aspect ratio of region is also important, but it is beyond the scope of the paper because
datasets are prepared from square like regions as it is described in Section 3. We exper-
imented with three different region sizes: 32x32, 128x128, 256x256. Smaller regions
tend to give better results, which may imply that for classification of 5 classes, CNN
models do not need high-resolution detail.
20
30
40
50
60
70
80
90
100
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80
ACCURACY
EPOCHS
LeNet-5-R32 CifarNet-R32
CifarNet-R128 CifarNet-R256
AlexNet-R128 AlexNet-R256
GoogLenet-R256
Fig. 4. Classification accuracy of models in epochs
For a better comparison among experiments, we provide accuracy of models cal-
culated every epochs in Figure 4. We had to stop training of GoogLenet model at 40
epochs because of the very long runtime but it does not influence our analysis since the
model already converged at 40 epochs. We can share following observations consider-
ing the overall picture. LeNet-5 is the fastest model with reasonable accuracy but as the
complexity of task increases, performance of LeNet-5 might not sustain because of its
simplicity. CifarNet classify accurately in R32 dataset but it converges later than LeNet-
5. GoogLenet gives the best result. However, when we consider the size of GoogLenet
and its training time, GoogLenet is a complex model for our classification task. In R128
and R256 experiments, CifarNet does not give close distribution of accuracies among
Solar Event Classification using Deep CNNs 9
Fig. 5. Confusion matrix for LeNet-5 and GoogLenet
classes. On the other hand, AlexNet is performing well on those datasets. In Figure 4,
we observe that AlexNet converges after 15 epochs, while it takes more than 30 epochs
for CifarNet and GoogLenet. Accuracy of AlexNet is less than GoogLenet but the model
is significantly faster. As a result, LeNet-5 performs well within limited time in our
classification task while AlexNet might be a good candidate model for more complex
classification tasks.
Figure 5 illustrates the confusion matrix for LeNet-5 and GoogLenet. Both mod-
els have similar distribution of error. One of the important observations is very minor
transitivity between FL and SG classes, which may stems from the wavelength used
for FL and SG is same, 131 ˚
A. We believe that having QS regions in three different
wavelengths significantly reduces this kind of transitivity. CH events are generally not
occurring close to other events like AR. As expected, there is less confusion between
CH and other events. However, having confusion between QS and FL is unexpected
and may show that the model is having difficulty to differentiation FL and QS regions
that are generated from 131 ˚
A images.
4.3 Comparison with Conventional Methods
Experiment/Method Epochs Accuracy in %
AR CH FL SG QS Average
Image Parameters N/A 79.0 85.0 72.0 68.0 N/A 76.0
LeNet-5-R32 65 97.7 97.0 87.0 92.0 94.0 93.5
CifarNet-R32 80 99.7 97.3 91.3 85.0 96.3 93.9
AlexNet-R128 35 97.3 95.0 85.0 88.3 92.7 91.7
GoogLenet-R256 40 98.0 99.3 84.0 91.0 98.3 94.1
Table 2. Accuracy comparison between traditional and deep learning approaches
Comparison between traditional methods and a deep learning approach is important
to understand the effectiveness of our efforts. For comparison, we found the best results
10 Lecture Notes in Computer Science: Authors’ Instructions
previously introduced for solar image classification tasks in [6] and the baseline results
are shown in Table 2 at ”Image Parameters” row. These results are produced using
the dataset in [33]. Banda et al. are using ten different image parameters that are well
performing on solar data [6]. Then, they present the best performing combinations of
multiple metrics and classifiers, from which we picked the best results for each class.
Our deep learning models are giving better accuracy on each class and overall accuracy.
Without creating a domain specific model, achieving better accuracy for each class
shows that deep learning approach is valuable and encouraging for using deep learning
on solar image domain.
5 CONCLUSIONS
We presented results for four well-known deep convolutional neural networks models
on a solar image classification task. We tested these models with three datasets with
different region sizes. Our experiments show that for basic solar image classification
task, well-known algorithms performs well while LeNet-5 and AlexNet are slightly
better option in terms of time and complexity. Also, accuracy of our models is better
than traditional approaches, based on extracting statistical image features and using
machine learning classification algorithms. For future work, we believe that spending
more time on a personalized CNN architecture for solar domain can lead to an optimal
model. In addition, ensemble models have shown to produce better results in other
image classification/object detection tasks [15, 19,38]. Based on the fact that a model is
better on classifying particular event types while its overall accuracy is low, ensemble
of multiple models can increase the performance of classification. Finally, the scope of
this work was limited to four event types, and adding more types to the classification
task (e.g. Filaments, Sunspots) may help us to built better CNN models and lead to new
and valuable observations.
References
1. Banda, J., Angryk, R., Martens, P.: Steps toward a large-scale solar image data analysis to
differentiate solar phenomena. Solar Physics (2013)
2. Banda, J.M., Angryk, R.A.: On the effectiveness of fuzzy clustering as a data discretization
technique for large-scale classification of solar images. In: Fuzzy Systems, 2009. FUZZ-
IEEE 2009. IEEE Int. Conf. on. pp. 2019–2024. IEEE (2009)
3. Banda, J.M., Angryk, R.A.: An experimental evaluation of popular image parameters for
monochromatic solar image categorization. In: FLAIRS Conf. pp. 380–385 (2010)
4. Banda, J.M., Angryk, R.A.: Selection of image parameters as the first step towards creating a
CBIR system for the solar dynamics observatory. In: Int. Conf. on Digital Image Computing:
Techniques and Applications, DICTA 2010, Sydney, Australia, 1-3 December, 2010. pp.
528–534 (2010)
5. Banda, J.M., Angryk, R.A.: Selection of image parameters as the first step towards creating
a cbir system for the solar dynamics observatory. In: Digital Image Computing: Techniques
and Applications (DICTA), 2010 Int. Conf. on. pp. 528–534. IEEE (2010)
6. Banda, J.M., Angryk, R.A.: Large-scale region-based multimedia retrieval for solar images.
In: Artificial Intelligence and Soft Computing - 13th Int. Conf., ICAISC 2014, Zakopane,
Poland, June 1-5, 2014, Proc., Part I. pp. 649–661 (2014)
Solar Event Classification using Deep CNNs 11
7. Banda, J.M., Angryk, R.A., Martens, P.C.: On the surprisingly accurate transfer of image
parameters between medical and solar images. In: 18th IEEE Int. Conf. on Image Processing,
ICIP 2011, Brussels, Belgium, September 11-14, 2011. pp. 3669–3672 (2011)
8. Banda, J.M., Angryk, R.A., Martens, P.C.: Imagefarmer: Introducing a data mining frame-
work for the creation of large-scale content-based image retrieval systems. Int. Journal of
Computer Applications 79(13) (2013)
9. Barra, V., Delouille, V., Kretzschmar, M., Hochedez, J.F.: Fast and robust segmentation of
solar euv images: algorithm and results for solar cycle 23. Astronomy and Astrophysics
505(1), 361–371 (2009)
10. Castelluccio, M., Poggi, G., Sansone, C., Verdoliva, L.: Land use classification in remote
sensing images by convolutional neural networks. CoRR abs/1508.00092 (2015)
11. Chen, C., Seff, A., Kornhauser, A.L., Xiao, J.: Deepdriving: Learning affordance for direct
perception in autonomous driving. In: 2015 IEEE Int. Conf. on Computer Vision, ICCV
2015, Santiago, Chile, December 7-13, 2015. pp. 2722–2730 (2015)
12. Chen, G., Han, T.X., He, Z., Kays, R., Forrester, T.: Deep convolutional neural network
based species recognition for wild animal monitoring. In: 2014 IEEE Int. Conf. on Image
Processing, ICIP 2014, Paris, France, October 27-30, 2014. pp. 858–862 (2014)
13. Chen, X., Xu, Y., Wong, D.W.K., Wong, T.Y., Liu, J.: Glaucoma detection based on deep con-
volutional neural network. In: 37th Annual Int. Conf. of the IEEE Engineering in Medicine
and Biology Society, EMBC 2015, Milan, Italy, August 25-29, 2015. pp. 715–718 (2015)
14. Christe, S., Hannah, I.G., Krucker, S., McTiernan, J., Lin, R.P.: Rhessi microflare statistics.
i. flare-finding and frequency distributions. The Astrophysical Journal 677(2), 1385 (2008)
15. Ciresan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks seg-
ment neuronal membranes in electron microscopy images. In: Advances in Neural Informa-
tion Processing Systems 25: 26th Annual Conf. on Neural Information Processing Systems
2012. pp. 2852–2860 (2012)
16. Dong, Z., Pei, M., He, Y., Liu, T., Dong, Y., Jia, Y.: Vehicle type classification using unsuper-
vised convolutional neural network. In: 22nd Int. Conf. on Pattern Recognition, ICPR 2014,
Stockholm, Sweden, August 24-28, 2014. pp. 172–177 (2014)
17. Grigis, P., Davey, A., Martens, P., Testa, P., Timmons, R., Su, Y., SDO Feature Finding Team:
The SDO flare detective 41, 402.08 (May 2010)
18. Gurghian, A., Koduri, T., Bailur, S.V., Carey, K.J., Murali, V.N.: Deeplanes: End-to-end lane
position estimation using deep neural networks. In: Proc. of the IEEE Conf. on Computer
Vision and Pattern Recognition Workshops. pp. 38–45 (2016)
19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016
IEEE Conf. on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA,
June 27-30, 2016. pp. 770–778 (2016)
20. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R.B., Guadarrama, S.,
Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proc. of the
ACM Int. Conf. on Multimedia, MM ’14, Orlando, FL, USA, November 03 - 07, 2014. pp.
675–678 (2014)
21. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional
neural networks. In: Advances in Neural Information Processing Systems 25: 26th Annual
Conf. on Neural Information Processing Systems 2012. pp. 1106–1114 (2012)
22. Kucuk, A., Banda, J.M., Angryk, R.A.: A large scale solar dynamic observatory image
dataset for computer vision applications. Submitted to Nature Scientific Data (2017), http:
//lsdo.dmlab.cs.gsu.edu/
23. Langhoff, S.R., Straume, T.: Highlights of the space weather risks and society workshop.
Space Weather 10(6) (2012)
24. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (May 2015)
12 Lecture Notes in Computer Science: Authors’ Instructions
25. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document
recognition. Proc. of the IEEE 86(11), 2278–2324 (1998)
26. Lemen, J.R., Akin, D.J., Boerner, P.F., Chou, C., Drake, J.F., Duncan, D.W., Edwards, C.G.,
Friedlaender, F.M., Heyman, G.F., Hurlburt, N.E., et al.: The atmospheric imaging assembly
(aia) on the solar dynamics observatory (sdo). In: The Solar Dynamics Observatory, pp. 17–
40. Springer (2011)
27. Lin, M., Chen, Q., Yan, S.: Network in network. CoRR abs/1312.4400 (2013)
28. Martens, P., Attrill, G., Davey, A., Engell, A., Farid, S., Grigis, P., Kasper, J., Korreck, K.,
Saar, S., Savcheva, A., et al.: Computer vision for the solar dynamics observatory (sdo). Solar
Physics 275(1-2), 79–113 (2012)
29. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In:
Proc. of the 27th Int. Conf. on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel.
pp. 807–814 (2010)
30. Parlett, B.N., Scott, D.S.: The lanczos algorithm with selective orthogonalization. Mathe-
matics of computation 33(145), 217–238 (1979)
31. Qahwaji, R., Colak, T.: Automatic short-term solar flare prediction using machine learning
and sunspot associations. Solar Physics 241(1), 195–211 (2007)
32. Raouafi, N., Bernasconi, P.N., Georgoulis, M.K.: The sigmoid sniffer and the advanced au-
tomated solar filament detection and characterization code modules 216, 402.32 (May 2010)
33. Schuh, M., Angryk, R., Ganesan Pillai, K., Banda, J., Martens, P.: A large scale solar image
dataset with labeled event regions. Proc. Int. Conf. on Image Processing (ICIP) pp. 4349–
4353 (2013)
34. Schuh, M.A., Angryk, R.A., Martens, P.C.: A large-scale dataset of solar event reports from
automated feature recognition modules. Journal of Space Weather and Space Climate 6, A22
(2016)
35. Schuh, M.A., Angryk, R.A., Pillai, K.G., Banda, J.M., Martens, P.C., et al.: A large-scale
solar image dataset with labeled event regions. In: ICIP. pp. 4349–4353 (2013)
36. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recog-
nition. CoRR abs/1409.1556 (2014)
37. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a
simple way to prevent neural networks from overfitting. Journal of Machine Learning Re-
search 15(1), 1929–1958 (2014)
38. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.E., Anguelov, D., Erhan, D., Vanhoucke,
V., Rabinovich, A.: Going deeper with convolutions. In: IEEE Conf. on Computer Vision
and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015. pp. 1–9 (2015)
39. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level per-
formance in face verification. In: 2014 IEEE Conf. on Computer Vision and Pattern Recog-
nition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014. pp. 1701–1708 (2014)
40. Verbeeck, C., Delouille, V., Mampaey, B., De Visscher, R.: The spoca-suite: Software for
extraction, characterization, and tracking of active regions and coronal holes on euv images.
Astronomy & Astrophysics 561, A29 (2014)
41. Wang, D., Khosla, A., Gargeya, R., Irshad, H., Beck, A.H.: Deep learning for identifying
metastatic breast cancer. CoRR abs/1606.05718 (2016)
42. Weyand, T., Kostrikov, I., Philbin, J.: Planet - photo geolocation with convolutional neural
networks. CoRR abs/1602.05314 (2016)
43. Withbroe, G.: Living with a star. In: Bulletin of the American Astronomical Society. vol. 32,
p. 839 (2000)
... Solar phenomena detection has been attempted in the field of solar physics. Kucuk, Banda, and Angryk (2017) studied the classification of solar events, such as active regions, coronal holes, flares, sigmoids, and quiet-Sun regions using deep CNN models, and they showed that the accuracy of deep learning CNNs was significantly higher than that of conventional classification methods. Kucuk, Aydin, and Angryk (2017) detected active regions and coronal holes in multi-wavelength solar images from the Solar Dynamics Observatory (SDO) using Faster R-CNN, and they showed that supervised deep-learningbased object detection can be applied to the solar physics domain. ...
... • Unlike previous studies (Kucuk, Aydin, and Angryk, 2017;Kucuk, Banda, and Angryk, 2017), we created our own datasets for training and testing using SDO data. SDO is one of the largest real solar image repositories. ...
... • Solar event detection using deep learning methods has been attempted. However, previous studies trained classification models using cropped images (Kucuk, Banda, and Angryk, 2017;Armstrong and Fletcher, 2019), which output only the image class, but not the location of the event. Kucuk, Aydin, and Angryk (2017) employed object detection methods; however, the accuracy of these methods was relatively low. ...
Article
Full-text available
Research on the detection of solar events has been conducted over many years. Recently, deep learning and data-driven approaches have been applied to solar event recognition. In this study, we present solar event detection using deep-learning-based object detection methods for real-time space weather monitoring. First, we construct a new object detection dataset using imaging data obtained by the Solar Dynamics Observatory with bounding boxes as labels for three representative features: coronal holes, sunspots, and prominences. Second, we train two representative object detection models: the Single Shot MultiBox Detector (SSD) and the Faster Region-based Convolutional Neural Network (R-CNN) using the new dataset. The results show that both models perform similarly well for coronal hole and sunspot detection. For prominence detection, the SSD and Faster R-CNN exhibited relatively low performance. This study demonstrates that deep-learning-based object detection can successfully detect multiple types of solar events, and it may be extended to detect other solar events. In addition, we provide the dataset for further achievements of object detection studies in solar physics.
... In this paper we describe some initial results we obtain using a convolutional neural network (CNN) to analyse SDO data. Basic applications of CNNs to solar physics data classification is shown in e.g., Kucuk et al., 2017;Armstrong and Fletcher, 2019, however CNNs have also started being applied to the prediction of solar events, in particular flares and CMEs, that can affect space weather as considered, for example, by Bobra and Couvidat (2015), Nagem et al. (2018), and Fang et al. (2019). ...
... The network created is a shallow CNN with only two convolutional layers, unlike deeper networks used on solar image data (Kucuk et al., 2017;Armstrong and Fletcher, 2019). Both of these papers tried to classify solar events such as flares, coronal holes and sunspots, with varying instruments used. ...
Article
Full-text available
In order to efficiently analyse the vast amount of data generated by solar space missions and ground-based instruments, modern machine learning techniques such as decision trees, support vector machines (SVMs) and neural networks can be very useful. In this paper we present initial results from using a convolutional neural network (CNN) to analyse observations from the Atmospheric Imaging Assembly (AIA) in the 1,600Å wavelength. The data is pre-processed to locate flaring regions where flare ribbons are visible in the observations. The CNN is created and trained to automatically analyse the shape and position of the flare ribbons, by identifying whether each image belongs into one of four classes: two-ribbon flare, compact/circular ribbon flare, limb flare, or quiet Sun, with the final class acting as a control for any data included in the training or test sets where flaring regions are not present. The network created can classify flare ribbon observations into any of the four classes with a final accuracy of 94%. Initial results show that most of the images are correctly classified with the compact flare class being the only class where accuracy drops below 90% and some observations are wrongly classified as belonging to the limb class.
... That is to say, many coronal features, such as coronal holes, persist over long periods of time, and thus a pixel that was previously classified as a coronal hole is likely to remain classified as a coronal hole in future images. Non-probabilistic approaches, such as neural networks have also been applied to multi-theme solar feature classification (Kucuk et al., 2017). ...
Preprint
Full-text available
In order to utilize solar imagery for real-time feature identification and large-scale data science investigations of solar structures, we need maps of the Sun where phenomena, or themes, are labeled. Since solar imagers produce observations every few minutes, it is not feasible to label all images by hand. Here, we compare three machine learning algorithms performing solar image classification using extreme ultraviolet and Hα images: a maximum likelihood model assuming a single normal probability distribution for each theme from Rigler et al. (2012), a maximum-likelihood model with an underlying Gaussian mixtures distribution, and a random forest model. We create a small database of expert-labeled maps to train and test these algorithms. Due to the ambiguity between the labels created by different experts, a collaborative labeling is used to include all inputs. We find the random forest algorithm performs the best amongst the three algorithms. The advantages of this algorithm are best highlighted in: comparison of outputs to hand-drawn maps; response to short-term variability; and tracking long-term changes on the Sun. Our work indicates that the next generation of solar image classification algorithms would benefit significantly from using spatial structure recognition, compared to only using spectral, pixel-by-pixel brightness distributions.
... That is to say, many coronal features, such as coronal holes, persist over long periods of time, and thus a pixel that was previously classified as a coronal hole is likely to remain classified as a coronal hole in future images. Non-probabilistic approaches, such as neural networks have also been applied to multi-theme solar feature classification (Kucuk et al., 2017). ...
Preprint
Full-text available
In order to utilize solar imagery for real-time feature identification and large-scale data science investigations of solar structures, we need maps of the Sun where phenomena, or themes, are labeled. Since solar imagers produce observations every few minutes, it is not feasible to label all images by hand. Here, we compare three machine learning algorithms performing solar image classification using extreme ultraviolet and Hydrogen-alpha images: a maximum likelihood model assuming a single normal probability distribution for each theme from Rigler et al. (2012), a maximum-likelihood model with an underlying Gaussian mixtures distribution, and a random forest model. We create a small database of expert-labeled maps to train and test these algorithms. Due to the ambiguity between the labels created by different experts, a collaborative labeling is used to include all inputs. We find the random forest algorithm performs the best amongst the three algorithms. The advantages of this algorithm are best highlighted in: comparison of outputs to hand-drawn maps; response to short-term variability; and tracking long-term changes on the Sun. Our work indicates that the next generation of solar image classification algorithms would benefit significantly from using spatial structure recognition, compared to only using spectral, pixel-by-pixel brightness distributions.
... In the case of large data, an automatic and rapid detection algorithm of CD and CW is very research-oriented. In 2017, Kucuk et al. used deep convolutional neural networks to classify solar events [14], including Sigmoid (SG) and Flare (FL). At the same time, they also used a deep learning model to detect solar events, such as Active Region (AR) [15]. ...
Article
Full-text available
Automatic detection of solar events, especially uncommon events such as coronal dimming (CD) and coronal wave (CW), is very important in solar physics research. The CD and CW are not only related to the detection of coronal mass ejections (CMEs) but also affect space weather. In this paper, we have studied methods for automatically detecting them. In addition, we have collected and processed a dataset that includes the solar images and event records, where the solar images come from the Atmospheric Imaging Assembly (AIA) of Solar Dynamics Observatory (SDO) and the event records come from Heliophysics Event Knowledgebase (HEK). Different from the methods used before, we introduce the idea of deep learning. We train single-wavelength and multiwavelength models based on Faster R-CNN. In terms of accuracy, the single-wavelength model performs better. The multiwavelength model has a better detection performance on multiple solar events than the single-wavelength model.
... Although, solar event detection with deep learning is not studied earlier, we investigated solar event classification using deep convolutional neural networks (DCNNs) in our previous study [10]. The study shows that DCNNs can classify solar regions cropped from full-disk images with high accuracy. ...
... W is the HL weights which are represented with 4D tensor in which features for every mixture of map of feature destination, map of feature source, vertical position source, and horizontal position source are contained. B is the biases which are represented as vectors in which one single element for each map of feature destination is contained [30][31][32][33][34][35][36]. In our case, the best accuracy results can be chosen from different structures of CNN classification technique, as it is shown in Fig. 1 there is a loop where a Result is considered as an initialised variable and the result accuracy of CNN is saved in X variable. ...
Article
In recent times, there has been increasing awareness of employee behaviour prediction in healthcare, trade and industry systems worldwide and its value on returns and profits of these systems. Nevertheless, determining the top employees with capacities and endorsing them for promotion is depending more or less on features which are dynamic and serving these systems’ interest. The current structure in organising and academic firms in Kurdistan-Iraq is non-systematic and manually performed, thus, the evaluation of employees’ behaviours is carried out by the directors at different branches, sections, and subsections, as a result, in some cases the outcomes of employees’ performance cause a low level of acceptance among staffs who believe that most of these cases are falsely assessed. This paper suggests an intelligent and vigorous structure to examine the performance of employees. It aims at presenting a solution to employee behaviour prediction through a joint effective feature selection method, then, Fuzzy Rough Set theory (FRST) is used to selected relevant features, next the classification task is conducted via Fuzzy-Rough Nearest Neighbours (FRNN), Decision Tree (DT), Naïve Bayes (NB) and Convolution Neural Network (CNN). FRNN and CNN classifiers have the best classification accuracy rate.
Article
Full-text available
The National Aeronautics Space Agency (NASA) Solar Dynamics Observatory (SDO) mission has given us unprecedented insight into the Sun’s activity. By capturing approximately 70,000 images a day, this mission has created one of the richest and biggest repositories of solar image data available to mankind. With such massive amounts of information, researchers have been able to produce great advances in detecting solar events. In this resource, we compile SDO solar data into a single repository in order to provide the computer vision community with a standardized and curated large-scale dataset of several hundred thousand solar events found on high resolution solar images. This publicly available resource, along with the generation source code, will accelerate computer vision research on NASA’s solar image data by reducing the amount of time spent performing data acquisition and curation from the multiple sources we have compiled. By improving the quality of the data with thorough curation, we anticipate a wider adoption and interest from the computer vision to the solar physics community.
Article
Is it possible to build a system to determine the location where a photo was taken using just its pixels? In general, the problem seems exceptionally difficult: it is trivial to construct situations where no location can be inferred. Yet images often contain informative cues such as landmarks, weather patterns, vegetation, road markings, and architectural details, which in combination may allow one to determine an approximate location and occasionally an exact location. Websites such as GeoGuessr and View from your Window suggest that humans are relatively good at integrating these cues to geolocate images, especially en-masse. In computer vision, the photo geolocation problem is usually approached using image retrieval methods. In contrast, we pose the problem as one of classification by subdividing the surface of the earth into thousands of multi-scale geographic cells, and train a deep network using millions of geotagged images. While previous approaches only recognize landmarks or perform approximate matching using global image descriptors, our model is able to use and integrate multiple visible cues. We show that the resulting model, called PlaNet, outperforms previous approaches and even attains superhuman levels of accuracy in some cases. Moreover, we extend our model to photo albums by combining it with a long short-term memory (LSTM) architecture. By learning to exploit temporal coherence to geolocate uncertain photos, we demonstrate that this model achieves a 50% performance improvement over the single-image model.
Technical Report
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Conference Paper
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif- ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implemen- tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called dropout that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry
Conference Paper
Glaucoma is a chronic and irreversible eye disease, which leads to deterioration in vision and quality of life. In this paper, we develop a deep learning (DL) architecture with convolutional neural network for automated glaucoma diagnosis. Deep learning systems, such as convolutional neural networks (CNNs), can infer a hierarchical representation of images to discriminate between glaucoma and non-glaucoma patterns for diagnostic decisions. The proposed DL architecture contains six learned layers: four convolutional layers and two fully-connected layers. Dropout and data augmentation strategies are adopted to further boost the performance of glaucoma diagnosis. Extensive experiments are performed on the ORIGA and SCES datasets. The results show area under curve (AUC) of the receiver operating characteristic curve in glaucoma detection at 0.831 and 0.887 in the two databases, much better than state-of-the-art algorithms. The method could be used for glaucoma detection.
Conference Paper
Is it possible to determine the location of a photo from just its pixels? While the general problem seems exceptionally difficult, photos often contain cues such as landmarks, weather patterns, vegetation, road markings, or architectural details, which in combination allow to infer where the photo was taken. Previously, this problem has been approached using image retrieval methods. In contrast, we pose the problem as one of classification by subdividing the surface of the earth into thousands of multi-scale geographic cells, and train a deep network using millions of geotagged images. We show that the resulting model, called PlaNet, outperforms previous approaches and even attains superhuman accuracy in some cases. Moreover, we extend our model to photo albums by combining it with a long short-term memory (LSTM) architecture. By learning to exploit temporal coherence to geolocate uncertain photos, this model achieves a 50 % performance improvement over the single-image model.