ArticlePDF Available

Abstract and Figures

The interconnected and heterogeneous nature of the next-generation Electrical Grid (EG), widely known as Smart Grid (SG), bring severe cybersecurity and privacy risks that can also raise domino effects against other Critical Infrastructures (CIs). In this paper, we present an Intrusion Detection System (IDS) specially designed for the SG environments that use Modbus/Transmission Control Protocol (TCP) and Distributed Network Protocol 3 (DNP3) protocols. The proposed IDS called MENSA (anoMaly dEtection aNd claSsificAtion) adopts a novel Autoencoder-Generative Adversarial Network (GAN) architecture for (a) detecting operational anomalies and (b) classifying Modbus/TCP and DNP3 cyberattacks. In particular, MENSA combines the aforementioned Deep Neural Networks (DNNs) in a common architecture, taking into account the adversarial loss and the reconstruction difference. The proposed IDS is validated in four real SG evaluation environments, namely (a) SG lab, (b) substation, (c) hydropower plant and (d) power plant, solving successfully an outlier detection (i.e., anomaly detection) problem as well as a challenging multiclass classification problem consisting of 14 classes (13 Modbus/TCP cyberattacks and normal instances). Furthermore, MENSA can discriminate five cyberattacks against DNP3. The evaluation results demonstrate the efficiency of MENSA compared to other Machine Learning (ML) and Deep Learning (DL) methods in terms of Accuracy, False Positive Rate (FPR), True Positive Rate (TPR) and the F1 score.
Content may be subject to copyright.
A Unified Deep Learning Anomaly Detection and
Classification Approach for Smart Grid
Environments
Ilias Siniosoglou, Panagiotis Radoglou-Grammatikis, Georgios Efstathopoulos,
Panagiotis Fouliras§and Panagiotis Sarigiannidis
Abstract—The interconnected and heterogeneous nature of the
next-generation Electrical Grid (EG), widely known as Smart
Grid (SG), bring severe cybersecurity and privacy risks that can
also raise domino effects against other Critical Infrastructures
(CIs). In this paper, we present an Intrusion Detection System
(IDS) specially designed for the SG environments that use
Modbus/Transmission Control Protocol (TCP) and Distributed
Network Protocol 3 (DNP3) protocols. The proposed IDS called
M EN S A (anoMaly dEtection aNd claSsificAtion) adopts a novel
Autoencoder-Generative Adversarial Network (GAN) architec-
ture for (a) detecting operational anomalies and (b) classifying
Modbus/TCP and DNP3 cyberattacks. In particular, M EN S A
combines the aforementioned Deep Neural Networks (DNNs)
in a common architecture, taking into account the adversarial
loss and the reconstruction difference. The proposed IDS is
validated in four real SG evaluation environments, namely (a)
SG lab, (b) substation, (c) hydropower plant and (d) power plant,
solving successfully an outlier detection (i.e., anomaly detection)
problem as well as a challenging multiclass classification problem
consisting of 14 classes (13 Modbus/TCP cyberattacks and
normal instances). Furthermore, M EN S A can discriminate five
cyberattacks against DNP3. The evaluation results demonstrate
the efficiency of M EN S A compared to other Machine Learning
(ML) and Deep Learning (DL) methods in terms of Accuracy,
False Positive Rate (FPR), True Positive Rate (TPR) and the F1
score.
Index Terms—Anomaly Detection, Auto-encoder, Cybersecu-
rity, Generative Adversarial Network, Deep Learning, Machine
Learning, Modbus, Smart Grid,
I. INTRODUCTION
The rapid advance of the Industrial Internet of Things
(IIoT) leads the conventional Electrical Grid (EG) into
a new digital paradigm called Smart Grid (SG), providing
significant benefits, such as better utilisation of the existing
resources, pervasive control and self-healing. According to
[1], the SG will compose the biggest Internet of Things
This project has received funding from the European Unions Horizon
2020 research and innovation programme under grant agreement No. 787011
(SPEAR).
I. Siniosoglou, P. Radoglou-Grammatikis and P. Sarigiannidis are
with the Department of Electrical and Computer Engineering, University
of Western Macedonia, Kozani, Greece - E-Mail: {isiniosoglou,
pradoglou, psarigiannidis}@uowm.gr
G. Efstathopoulos is with the 0infinity Limited, Imperial Offices, London,
UK, E6 2JG - E-Mail: george@0infinity.net
§P. Fouliras is with the Department of Applied Informatics, University of
Macedonia, Thessaloniki, Greece - E-Mail: pfoul@uom.edu.gr
(IoT) application. However, the evolution of the smart tech-
nologies introduces severe cybersecurity issues due to (a)
the necessary presence of insecure, legacy systems, such as
Industrial Control Systems (ICS) and Supervisory Control and
Data Acquisition (SCADA) [2], (b) the vulnerability nature
of Transmission Control Protocol/Internet Protocol (TCP/IP)
[3] and (c) the new attack surface introduced by the smart
technologies [4].
Denial of Service (DoS), unauthorised access and False Data
Injection (FDI) compose expected attack vectors targeting the
SG with disastrous consequences. The first one target the
availability of the involved systems, while the other ones
exploit the vulnerabilities of the industrial protocols in order
to compromise the confidentiality, integrity and authenticity
of the exchanged information. A characteristic Advanced
Persistent Threat (APT) [5] was the BlackEnergy3 [6] in 2015
against a Ukrainian substation, resulting in the power outage
for more than 225,000 people. Moreover, the Crashoverride
APT in 2016 caused another blackout in Ukraine [6]. Other
devastating APTs against Critical Infrastructures (CIs) are
Stuxnet, Flame, Duqu [7] and TRITON [8]. Also, in 2014 and
2017, the Dragonfly and Dragonfly 2.0 APTs targeted multiple
energy companies [2].
Both industry and academia have provided valuable coun-
termeasures [9]–[13]. In particular, IEC 62351 [14], [15]
specifies a set of guidelines in order to enhance the security
of ICS/SCADA. Furthermore, based on the aforementioned
remarks, the timely, accurate and consistent intrusion detection
is necessary. In particular, signature-based Intrusion Detection
Systems (IDS), such as Snort and Suricata can recognise
a plethora of known intrusions. Moreover, anomaly-based
IDS adopting statistical analysis, Machine Learning (ML) and
Deep Learning (DL) methods can detect zero-day attacks
and unknown anomalies. However, despite the benefits of the
aforementioned solutions, they are characterised by essential
limitations [16]. First, in many CIs, such as the SG, the
adoption of the IEC 62351 is challenging, especially for
the adjustments that need to be taken place in real-time.
On the other side, the signature-based IDS can detect only
known cyberattack patterns and include only a limited set of
signature rules related to industrial communication protocols
like Modbus, Distributed Network Protocol 3 (DNP3) and IEC
61850 [2]. Finally, the anomaly-based IDS suffer from a high
The published version is available by IEEE Xplore at https://ieeexplore.ieee.org/document/9425573
number of False Positives (FP).
In this paper, we provide an anomaly detection model
capable of: (a) detecting anomalies and (b) classifying anoma-
lies into particular cyberattack types. The anomaly detection
refers to the process of identifying whether an action is
malicious or not. On the other side, the anomaly classification
categorises the malicious activities into particular cyberattack
types. The proposed model called MEN SA (anoMaly dEtec-
tion aNd claSsificAtion) combines simultaneously two Deep
Neural Networks (DNNs): (a) autoencoder and (b) Generative
Adversarial Network (GAN). We validated the efficiency of
MEN SA with three types of datasets: (a) Modbus/TCP
network flows, (b) DNP3 network flows and (c) operational
data (i.e., time-series electricity measurements). The datasets
related to Modbus/TCP and the operational data are originating
from four SG environments: (a) SG lab, (b) substation, (c)
hydropower plant and (d) power plant. The DNP3 cyberattacks
are related only to the substation environment. Consequently,
the contributions of this paper are summarised in the following
sentences.
Providing a DL-based anomaly detection and classi-
fication model called MENSA.MEN S A can detect in
parallel both anomalies and particular cyberattacks with
high performance in terms of Accuracy, True Positive
Rate (TPR), False Positive Rate (FPR) and the F1 score.
In particular, the average Accuracy, TPR, FPR and F1
are calculated at 0.947,0.812,0.036 and 0.7942, respec-
tively. Compared to the existing anomaly-based IDS [16],
MEN SA addresses efficiently the FP.
Detecting a plethora of Modbus/TCP and DNP3 cy-
berattacks:MEN SA is able to solve a difficult classifi-
cation problem by detecting and discriminating efficiently
14 Modbus/TCP-related cyberattacks. Moreover, it can
recognise five DNP3 cyberattacks. The MENSA detec-
tion capability relies on TCP/IP network flow statistics.
Therefore, the MEN SA detection efficiency demon-
strates also its scalability since similar statistics can be
used for detecting cyberattacks against any protocol at
the application layer.
Detecting anomalies upon operational data:MEN SA
can detect anomalies upon various operational data (i.e.,
electricity measurements) coming from different SG en-
vironments.
Validating MENSA with real data originating from
four use cases: The efficiency of MENSA was validated
using network traffic data and operational data originating
from four SG evaluation environments: (a) SG lab, (b)
substation, (c) hydropower plant and (d) power plant.
Evaluating a plethora of ML/DL methods: Various
ML/DL models were evaluated and compared with each
other in terms of Accuracy, TPR, FPR and the F1 score.
MEN SA DL models provide the best performance.
The rest of this paper is organised as follows. Section II dis-
cusses previous relevant works. Section III provides the nec-
essary background. Finally, section IV analyses the MENSA
architecture, while section V describes how MEN SA is im-
plemented in a SG environment. Finally, section VII concludes
this paper.
II. RE LATE D WORK
Several papers have investigated the IoT and SG security
issues. Some remarkable cases are listed in [16]–[36]. In
particular, in our previous work in [16], we present a compre-
hensive study related to the SG intrusion detection solutions.
After introducing the necessary background related to the
architectural ingredients of the SG, 37 cases are analysed,
taking into account the architecture schema, the detection
method and their efficiency. Accordingly, in [18] R. Mitchel
and I. Chen provide a survey related to the intrusion detec-
tion techniques for Cyber-Physical Systems (CPS). Similarly,
after giving the necessary information regarding the CPS and
intrusion detection methods, R. Mitchel and I. Chen study a
plethora of specially designed IDS for the CPS. In [22], S.
Rakas et al. examine 26 IDS cases related to SCADA systems.
The authors define first an evaluation methodology, which
considers the IDS performance, test environment, implemen-
tation tools, detection techniques and protocols. Next, after
explaining the factors affecting the design and development
of the SCADA IDS, they briefly discuss 26 SCADA IDS
cases, thereby identifying research gaps and directions for
future research work. In parallel, multiple survey papers have
studied DL techniques for detecting and classifying anomalies.
Characteristic examples are provided in [37]–[39]. Therefore,
taking into account the aforementioned points, subsequently,
we discuss some specific IDS cases that use DL techniques for
detecting intrusions against the SG and SCADA systems. Each
paragraph focuses on a dedicated case. Finally, we highlight
how our work is differentiated.
In [40], R. Shire et al. provide a malware intrusion detection
system for IoT environments, utilising a Convolutional Neural
Network (CNN). The proposed IDS consists of three main
steps. Fist, a network sniffer undertakes to capture the overall
network traffic. For this purpose, a socket Python library
is adopted. Next, the Binvis tool [41] is used to convert
the stored network traffic (i.e., pcap file) into an image. In
particular, the Hilbert space-filling curve clustering algorithm
[42] is used to extract the images. The specific algorithm
overcomes other solutions in maintaining the locality among
the objects in multi-dimensional spaces, thus generating a
more suitable image imprint. Finally, the image is inserted
into a CNN, which undertakes to identify the corresponding
malware. The CNN is constructed, utilising Tensorflow
and more precisely the MobileNet module. The performance
analysis demonstrates the effectiveness of the proposed IDS.
In [43], Y. He et al. present a DL-based detection method,
which is capable of recognising FDI attacks against SCADA
systems for stealing energy. In particular, the proposed method
is composed of two main detection schemes: (a) State Vector
Estimator (SVE) and (b) Deep-Learning Based Identification
(DLBI). SVE assesses the real-time measurements by com-
puting the l2norm and comparing it with a particular
threshold value t, which is defined experimentally. If the
calculation result is higher than t, then an alarm is reported.
Otherwise, the DLBI is activated for evaluating further the
real-time measurements. DLBI constitutes a Deep Belief Net-
work (DBN) called Conditional DBN (CDBN), which utilises
a Conditional Gaussian Bernoulli Restrictive Boltzmann Ma-
chine (CGBRBM) in order to identify the appropriate features.
The resiliency of the proposed method is demonstrated based
on four simulated cases, utilising an IEEE 118-bus power test
system and an IEEE 300-bus system. Moreover, the efficiency
of the proposed method is validated by comparing its detection
results with the outcomes of two ML solutions: (a) Artificial
Neural Network (ANN) and (b) Support Vector Machine
(SVM).
In [44], M. Saharkhizan et al. provide an intrusion de-
tection mechanism for the Modbus IoT environments, which
aggregates an ensemble of multiple Long-Short-Term-Memory
(LSTM) networks. LSTM is a fundamental type of Recurrent
Neural networks (RNNs) that can learn the log-term pattern
of the training data. The proposed mechanism utilises the
dataset of I. Fazao et al. [45] that consists of four cyberattacks-
categories, namely (a) Man In The Middle (MITM) attacks,
(b) Ping Distributed DoS (DDoS) attacks, (c) TCP SYN DoS
attacks and (d) Modbus query flood attacks. Moreover, the au-
thors use the CICFlowMeter to generate the corresponding
bidirectional network flows. Finally, the output of six LSTM
networks is aggregated with the help of a decision tree in
order to classify the exported network flows into the categories
mentioned above. Based on the evaluation results, the accuracy
of the proposed mechanism reaches 99%.
In [46], H. Yang et al. present a network IDS for the DNP3
SCADA systems. The proposed IDS relies on a CNN, which
consists of five convolutional layers that are followed by the
Rectified Linear Unit (ReLu) to increase the non-linearity of
the feature maps. Next, the max-pooling function is applied in
order to increase the spatial invariance. The input of CNN is
a two-dimensional matrix with an rxD size where rdenotes
a time window and Dthe total size of the DNP3 packets’
attributes. The time window ris equal to the number of the
DNP3 packets transmitted within a second. On the other side,
Dis equal to 25, i.e., there are 25 DNP3 network packets’
attributes originating from the (a) link layer, (b) network layer,
(c) transport layer and (d) the application layer. The proposed
IDS solves a difficult classification problem consisting of
multiple attacks-categories, namely (a) Address Resolution
Protocol (ARP) poisoning attacks, (b) TCP SYN Flood attacks,
(c) TCP RST attacks, (d) User Datagram Protocol (UDP) flood
attacks, (e) DNP3 application transmission attacks, (f) outsta-
tion DFC flag attacks, (g) function reset attacks, (h) pseudo-
transport layer sequence modification attacks, (i) fragmented
message interruption attacks, (j) data-link layer length over-
flow attacks, (k) configuration capture attacks, (l) outstation
data reset attacks, (m) clear object attacks, (n) outstation write
without reading, (o) address alteration attacks, (p) unavailable
function attacks, (q) dual single-packet attacks and (r) dual
multiple-packet attacks. Based on the evaluation results, the
overall accuracy of the proposed CNN reaches 99.38%.
In [47], the authors present an Intrusion Prevention System
(IPS) focused on the DNP3 cyberattacks. The architecture
of the proposed IPS is composed of three modules (a) Data
Monitoring Module, (b) DIDEROT Analysis Engine and (c)
Response Module. The Data Monitoring Module undertakes
to monitor and capture the DNP3 network traffic, extracting
the respective network. Then, the DIDEROT Analysis Engine
applies a decision tree and an autoencoder in order to recognise
potential DNP3 cyberattacks and anomalies, respectively. The
decision tree focuses on a classification problem, which is
composed of five cyberattacks, namely (a) injection, (b) flood-
ing, (c) DNP3 reconnaissance, (d) replay and (e) masquerad-
ing. On the other side, the autoencoder solves an anomaly
detection problem, which tries to identify DNP3 anomalies.
Finally, the Response Module informs the Software-Defined-
Networking (SDN) controller to disrupt the malicious DNP3
flows by transmitting the necessary OpenFlow commands
to the SDN Switches. Based on the evaluation results, the
F1 score of the proposed decision tree and the DIDEROT
autoencoder reach 0.991 and 0.953, respectively.
In our previous work in [48], we provide an anomaly-based
IDS called ARIES (smArt gRid Intrusion dEtection System),
which secures the SG communications. The architecture of
the proposed IDS consists of three modules, namely (a) Data
Collection Module, (b) ARIES Analysis Engine and (c)
Response Module. The Data Collection module sniffs the
overall network traffic, producing the analogous bidirectional
network flow statistics. These statistics are analysed by the
ARIES Analysis Engine, thus detecting successfully relevant
cyberattacks and anomalies. Finally, the Response Module
informs the system operator about potential cyberattacks. The
ARIES Analysis Engine is composed of three detection
layers, namely (a) Network-flow Based detection, (b) Packet-
based detection and (c) Operational Data based detection.
The first layer is responsible for recognising specific cyber-
attacks and anomalies by processing network flow statistics.
In particular, it can detect (a) DoS cyberattacks, (b) Secure
Shell (SSH) brute-force attacks, (c) File Transfer Protocol
(FTP) brute-force attacks, (d) port-scanning cyberattacks and
(e) bots. To this end, a decision tree classifier is applied.
The second layer focuses on Modbus/TCP anomalies by
processing Modbus/TCP packets’ attributes via the Isolated
Forest algorithm. Finally, the third layer analyses operational
data (i.e., time-series electricity measurements) via a GAN
called ARIESGAN. The evaluation analysis demonstrates
the efficiency of all ARIES detection layers. In particular,
the F1 score of the first detection layer reaches 0.982, while
the F1 score of the second and third layer reaches 0.751 and
0.853, respectively.
Undoubtedly, the works analysed earlier provide valuable
insights and methodologies concerning the intrusion detection
in CIs. DL is an emerging technology, which can contribute
significantly to the defence against the rapid evolution of the
cyberthreats and malware. In particular, the lack of labelled
data renders DL techniques an ideal solution for constructing
effective security applications since they can identify the ap-
propriate features autonomously. Nevertheless, it is noteworthy
that most of the previous works have not been validated with
real SG environments and data. Furthermore, apart from [46],
[47], most of them either do not consider the SCADA proto-
cols that constitute the root of the most anomalies/intrusions in
CIs or cover them partially (i.e., they recognise only a few rele-
vant attacks). Therefore, based on the aforementioned remarks,
this paper extends our previous work in [48] by enhancing
ARIESGAN and introducing an Autoencoder-GAN archi-
tecture with novel minimisation functions, taking into account
both the adversarial error and the reconstruction difference. In
particular, the proposed Autoencoder-GAN architecture was
validated in four real SG evaluation environments that use
the Modbus/TCP and DNP3 protocols. Our previous work in
[48] could detect only Modbus/TCP anomalies. In contrast,
this paper examines and detects a plethora of Modbus/TCP
cyberattacks that can be performed by Smod [49], a widely
known penetration-testing tool related to Modbus.
III. BACKGRO UN D
This section provides the necessary background regarding
(a) Modbus, (b) DNP3, (c) Autoencoders and (d) GANs. In
particular, after describing the core architecture of the Modbus
and DNP3 protocols, we specify which Modbus/TCP and
DNP3 attacks can be successfully recognised by MEN SA.
Next, the functionality of the Autoencoder and GAN DNNs is
provided so that the reader can normally proceed to the unified
Autoencoder-GAN architecture described in the following
sections. More detailed information about Autoencoders and
GAN is provided in [37]–[39].
A. Modbus/TCP and DNP3 Threat Identification
Modbus is an industrial communication protocol adopted
widely by SCADA systems in the energy sector due to its
simplicity, easy deployment and open specifications. In partic-
ular, the general Modbus frame is called Application Data Unit
(ADU), which in turn consists of (a) the Protocol Data Unit
(PDU), (b) Addressing and (c) Error Checking. PDU encloses
the primary information of the Modbus packets, including
the function code and the respective data [2]. Each function
code defines a different functionality. The addressing and error
checking functionalities rely on the Modbus version (i.e., (a)
Modbus/Remote Terminal Unit (RTU) or (b) Modbus/TCP).
In the Modbus/RTU version, the master and each slave are
characterised by unique IDs, while the error checking is
achieved through Cyclic Redundancy Check (CRC). On the
other hand, in the Modbus/TCP version, the Slave ID field is
replaced by the Modbus Application Protocol (MBAP) header,
which in turn includes (a) the Transaction Identifier, (b) the
Protocol Identifier, (c) Length and (d) Unit Identifier. The
protocol identifier is always equal to zero for the current
Modbus services, while other values are reserved for potential
extensions. Length indicates the size of the remaining field,
including Unit ID, Function Code and Data. The Unit ID is
utilised for serial connecting to a Modbus device, which does
not use the Modbus/TCP version. Finally, the error checking
functionality was replaced by the corresponding mechanisms
of TCP/IP.
DNP3 is a reliable protocol applied largely in the CIs in the
US. In the SG, DNP3 is used to transfer messages between
master devices and outstations. It supports several topologies,
comprising (a) point-to-point, where an outstation and one
master communicate with each other, (b) multiple-drop, where
several masters and outstations interact with each other and (c)
hierarchical interface, where an entity can operate with both
roles. DNP3 includes three layers: (a) link layer, (b) transport
layer and (c) application layer. The link-layer offers addressing
services, multiplexing, data fragmentation, error checking and
link control. On the other side, the transport layer is used as
in the case of the Open Systems Interconnection (OSI) model,
and it is represented with one byte utilised for fragmenting the
DNP3 packets. Finally, the application layer defines a set of
functional commands utilised for managing and controlling the
SG entities, such as RTUs, Programmable Logic Controllers
(PLCs), Intelligent Electronic Devices (IEDs) and smart me-
ters. Apart from the DNP3 serial line communication, DNP3
can be used over TCP/IP, wherein the aforementioned DNP3
layers are incorporated into the application layer of TCP/IP.
Both Modbus and DNP3 are characterised by severe security
issues since they were not constructed having cybersecurity in
mind [2]. In our previous work in [49] we have identified
the Modbus/TCP cyberattacks based on Smod. Similarly, in
[50], N. Rodofile et al. discuss possible cyberattacks against
DNP3. Based on these works, Table I and Table II enumerate
the Modbus/TCP and DNP3 cyberattacks that MEN SA can
classify, respectively.
B. Autoencoder and GAN
A GAN [51], [52] relies on two sub-neural networks, the
Generator Gand the Discriminator D. The Generator Gtakes
random noise data and generates data similar to the real data.
On the other hand, the Discriminator Dinputs a data sample
and tries to classify it as real or fake. The GAN aims to push
and train both sub-networks that rival each other so that the
Generator Gcan produce data that the Discriminator Dcannot
distinguish from the real ones. Equation (1) shows the relation
between Gand D.
min
Gmax
DV(G, D) = min
Gmax
D
Expdata [log(D(x))]+
Ezpz[log(1 D(G(z)))] (1)
Gaccumulates noise zfrom space Zmapping it to the space
Xfrom which Dinputs x.(pdata(x)and (pz(z)) denote the
probability distribution of spaces Xand Z, respectively.
The autoencoders are DNNs that learn to imitate the in-
put data by compressing and inflating it into a multilayer
pipeline. In particular, an autoencoder consists of two sub-
networks, the Encoder and the Decoder. The Encoder sub-
network compresses the input data of space Xto a manifold
F. In contrast, the decoder sub-network inflates the data of
manifold Fto a sample P, where PX. The goal of the
TABLE I: Modbus/TCP Cyberattacks
Modbus/TCP Cyberattack Description
modbus/dos/writeSingleCoils This DoS Modbus/TCP cyberattack uses Modbus/TCP packets with the
function code 05 to change the value of a single coil to ON or OFF.
modbus/dos/writeSingleRegister
This DoS Modbus/TCP attack transmits continuously Modbus/TCP
packets with the function code 06 to the target system. The goal of the
cyberattacker is to affect the availability of the target.
modbus/function/readCoils
This Modbus/TCP unauthorised access cyberattack accesses the content
of a single coil. To this end, a Modbus/TCP packet with the function
code 01 is utilised.
modbus/function/readCoils (DoS)
This Modbus/TCP cyberattack is another DoS attack, which exploits
the function code 01. The attacker sends continuously to the target
system a plethora of Modbus/TCP packets with the function code 01
that read the status of a single coil.
modbus/function/readDiscreteInput
This Modbus/TCP unauthorised access cyberattack violates the
confidentiality of a Modbus/TCP device by reading the content of
multiple discrete inputs. It uses Modbus/TCP packets with the
function code 02.
modbus/function/readDiscreteInputs (DoS) This DoS Modbus/TCP cyberattacks sends a plethora of Modbus/TCP
packets with the function code 02.
modbus/function/readHoldingRegister
It constitutes the most usual unauthorised access attack against
Modbus/TCP targeting the content of a holding register via a
Modbus/TCP packet with the function code 03.
modbus/function/readHoldingRegister (DoS)
This Modbus/TCP cyberattack also targets the availability of a
Modbus/TCP device by sending multiple Modbus/TCP packets with the
function code 03. This function code is used to read the content of a
holding register.
modbus/function/readInputRegister This unauthorised access Modbus/TCP cyberattack aims to violate the
confidentiality of a Modbus/TCP input register by reading its content.
modbus/function/readInputRegister (DoS)
This Modbus/TCP attack sends continuously a plethora of Modbus/TCP
packets with the function code 04 (Modbus Read Input Register packet)
to the target system, thus aiming to corrupt its availability.
modbus/function/writeSingleCoils
This unauthorised access Modbus/TCP attack takes full advantage of the
lack of authentication and authorisation mechanisms by changing the
status of single coil either to ON or OFF through a Modbus/TCP packet
with the function code 05.
modbus/function/writeSingleRegister
This unauthorised access Modbus/TCP cyberattack targets both the
confidentiality and integrity of a Modbus/TCP single register by sending
a Modbus/TCP packet with the function code 06, thus changing its
content.
modbus/scanner/getfunc This reconnaissance Modbus/TCP attack enumerates all Modbus/TCP
function codes supported by the target system.
modbus/scanner/uid This Modbus/TCP reconnaissance cyberattack enumerates the slave IDs
supported by the target system.
TABLE II: DNP3 Cyberattacks
DNP3 Cyberattack Description
DNP3 Injection
This cyberattack takes full advantage that DNP3 does not include any authentication and
authorisation mechanism. It injects malicious DNP3 packets between the communication of a
DNP3 master and DNP3 outstation.
DNP3 Flooding This cyberattack floods continually the target system with DNP3 messages.
DNP3 Reconnaissance This cyberattack diagnoses whether the DNP3 protocol is used by the target system.
Replay This cyberattack replays the DNP3 messages originating from a legitimated entity.
Masquerading It imitates the behaviour of a legitimate DNP3 entity.
autoencoder architecture is to help the network through the
training process, thus producing samples pthat are similar to
the given real data r. After the training process, the network
inputs new data similar to the training data. Equation (2) shows
the data pipeline of the autoencoder architecture.
r, p :argmin
r,p
X(pr)X
2
r:XF, p :FP
(2)
IV. MENSA ARCHITECTURE
MEN SA combines the DNNs mentioned above to com-
pose a unified DNN architecture for (a) anomaly detection
and (b) anomaly classification purposes. This union is accom-
plished by encapsulating the autoencoder architecture into the
structure of the GAN network. The Generator takes the form
of the Decoder, while the Discriminator takes the structure of
the Encoder. In this schema, the Generator-Decoder takes an
input of a noise sample N×M, where Nis the number
of the noise points in a sample and Mis the number of
the input samples. Next, the Generator-Decoder inflates those
samples to produce samples that imitate the desired data. The
Discriminator-Encoder compresses the Generator-Decoder’s
output into a single point, which is the validity label of the
sample. This function is used to discriminate the real and
fake samples. An intermediate model is exported after the
training process from the Discriminator-Encoder sub-network.
This model is part of the Discriminator-Encoder and it is
utilised for the anomaly detection procedure. It comprises the
input layer up to the latent layer before the output of the
network. In particular, it is used to reduce the input dimension
into a specified latent space. Two samples pass through the
intermediate model: (a) the real data sample and (b) the
generated sample. At this point, the Generator-Decoder has
learned to generate close to real data that imitates the normal
samples. To calculate the anomaly score for the real sample,
the Adversarial Loss function is utilised. The Adversarial Loss
is the difference between the generated and the real sample.
Since the Generator-Decoder has learned to produce normal
samples, the greater the Adversarial Loss, the higher the
probability of the real sample being abnormal. The equation
below describes the Adversarial Loss.
AdvL(dr, dp) = kdrdpk(3)
where AdvL(x)is the adversarial loss score, drand dpare
the prediction of the latent model in the real and the generated
sample, respectively. On the other side, regarding the anomaly
classification purpose, a second, lightweight implementation
of the combined Autoencoder-GAN architecture is adopted.
Both Autoencoder-GAN architectures for anomaly detection
and anomaly classification are analysed in the following sub-
sections.
A. MENSA Autoencoder-GAN for Anomaly Detection
In this case, the combined MEN S A Autoencoder-GAN
works as an anomaly detector. It is trained only with a
set of normal samples and can discriminate outliers in a
dataset containing both normal and anomalous samples. The
structure of the entire network can be separated into three
components, (a) the input layer, (b) the Generator-Decoder and
(c) the Discriminator-Encoder. Fig. 1 depicts the M ENSA
Autoencoder-GAN network for anomaly detection.
Fig. 1: M EN S A Autoencoder-GAN for anomaly detection
Input Layer for Anomaly Detection: The input layer repre-
sents the input of the proposed DNN. It takes a noise vector
of size Ngenerated based on the uniform distribution with
mean µand standard deviation σ.
Generator-Decoder for Anomaly Detection: The Generator-
Decoder is in charge of inflating a random noise input vector
of size z= 10 to a size M, where Mis the number of features,
while the generated data imitates the real one. It is trained to
produce normal samples. The Generator-Decoder’s structure
consists of thirteen layers, an input layer, an output T anh
layer and a sequence of Dense,ReLU ,LeakyReLU ,Batch,
Normalization and Dropout layers.
tanh(x)=2s(2x)1, tanh [1,1] (4)
where equation (4) describes the T anh function. tanh(x)is
the output of the tanh function, s(x)is the sigmoid function
(6) and xis the input vector.
An explanatory illustration of the Generator-Decoder’s
structure is shown in Fig. 2. This network is compiled with the
Binary Cross-Entropy function (equation 5) and the RMSprop
optimizer with a learning rate parameter of lr = 0.0002. The
Binary Cross-Entropy function is defined as follows. Nis the
number of samples given, while yis the label. p(yi)is the
probability of the sample being a match to the label sample
when 1p(yi)presents the inverse of that probability. Finally,
Hrepresents the result of the Binary Cross-Entropy loss in a
given point.
Hp(q) = 1
N
N
X
i=1
yi·log(p(yi)) +(1yi)·log(1p(yi)) (5)
Fig. 2: Generator-decoder structure for anomaly detection
Discriminator-Encoder for Anomaly Detection: The role
of the Discriminator-Encoder is to distinguish the real and
the generated data samples (i.e., the samples generated by
the Generator-Decoder). It takes a vector of Mfeatures
representing a data instance sample. Next, it compresses
the data through a multi-layer pipeline into a single point
representing the validity layer (i.e., the binary classification of
the sample being real or fake). The Discriminator-Encoder is
trained alongside the Generator-Decoder, receiving both real
and generated samples, each with a ground truth label. The
ground truth labels given as input to the Discriminator-Encoder
are represented by tl 1for the Generator-Decoder’s
output, while fl 0represents the real sample. In the
training process, the Discriminator-Encoder’s training ability is
deactivated when the Generator-Decoder is trained. From the
Discriminator-Encoder, the intermediate model is extracted.
This network is also compiled with the Binary Cross-Entropy
function (equation 5) and the RMSprop optimizer with a
learning rate parameter of 0.0002. Thirteen layers compose the
Discriminator-Encoder: an input layer, an output Sigmoid layer
(equation 6) and a sequence of Dense, ReLU, Leaky ReLU,
Batch Normalization and Dropout layers. Fig. 3 illustrates
the Discriminator-Encoder’s structure. It is noteworthy that
the Discriminator-Encoder operates also as an encoder. This
means that it reduces the dimension of the input sample from
its original dimension to a manifold of size 1, indicating
the validity of the sample. The extracted latent model is an
intermediate model, describing the fist nlayers of Dbefore
the output sequence. This nth layer outputs a reduced manifold
of size k, which makes the detection easier and faster than
comparing the original samples. There is no standard way
to determine the nth latent layer. Usually, nth is defined
experimentally, identifying the best accuracy/information-loss
trade-off.
s(x) = 1
1 + ex, s [0,1] (6)
Fig. 3: Discriminator-encoder structure for anomaly detection
Based on the aforementioned remarks, the MEN SA
anomaly detection process uses the following steps. Given a
real sample gr, first, the Generator-Decoder generates a sample
gpusing random noise data. Subsequently, both grand gpare
given to the latent model, which in turn outputs the reduced
samples drand dpof size k. Next, drand dpare given
to formula (3). In order to detect the anomaly, a threshold
t[0,1], is leveraged. Finally, if the AdvL outcome is
greater than t, then an anomaly is detected.
B. MENSA Autoencoder-GAN for Anomaly Classification
The MEN SA Autoencoder-GAN for anomaly classifica-
tion is derived by the previous MENSA Autoencoder-GAN
for anomaly detection. This implementation combines both the
process of anomaly detection and anomaly classification into
a single DNN. In particular, it produces three ground-truth
label points, (a) one for the validity of the sample, (b) one
for the anomaly approximation and (c) one describing the
anomaly class of the sample. This architecture can also be
separated into three parts, (a) the Input layer, (b) the Generator-
Decoder and (c) the Discriminator-Encoder. The structure of
this DNN is depicted in Fig. 4. The main difference with the
previous MEN SA Autoencoder-GAN for anomaly detection
is that this network is designed to handle multiclass data with
fewer features. In contrast, the MEN SA Autoencoder-GAN
for anomaly detection is designed to handle one class and data
with a large number of features.
Input Layer for Anomaly Classification: The input layer
takes a noise vector input of size Nand a vector containing
the classes of the sample. The elements of the random noise
vector follow a normal distribution with µ= 0 and σ= 1.
The label vector with a dimension of [1 xC], is a zero vector
with 1in the position of the class. Cdenotes the number of
classes that exist in the given dataset. The class of the sample is
represented by cp, which is derived by the following formula.
cp=argmax(Vlabel )(7)
where Vlabel is the label vector.
Generator-Decoder for Anomaly Classification: The
Generator-Decoder is a modified version of the Generator-
Decoder used in Autoencoder-GAN for anomaly detection.
In this case, the Generator-Decoder inputs the two vectors
Fig. 4: M EN S A Autoencoder-GAN for anomaly classification
explained in the input layer and concatenates them in order
to pass through the Generator-Decoder’s structure. The
Generator-Decoder’s structure is illustrated in Fig. 5. It
consists of nine layers, an input layer, an output Relu layer
and a sequence of Dense and ReLU layers. This network
is compiled with the Categorical Cross-Entropy function
(equation 8) and the Adadelta optimizer [53]. During the
training process, the Generator-Decoder learns to reproduce
the data representing each class in the dataset using a label
vector. This means that it produces a sample of a certain
class, which is introduced as a label vector. The output of
this module is a vector of size M, where Mis the number
of features of the sample.
Lcc(r, p) =
M
X
j=0
N
X
i=0
(rij log(pij )) (8)
The above equation denotes the Categorical Cross-Entropy
loss function used to compile the Generator-Decoder. Lcc(y, p)
is the Categorical Cross-Entropy output, ris the real sample,
and pis the generated sample.
Fig. 5: Generator-Decoder structure for anomaly classification
Discriminator-Encoder for Anomaly Classification: The
Discriminator-Encoder takes an input vector of Mfeatures,
representing a data sample. Since the proposed architecture
produces not only the validity approximation but also the
anomaly classification of the introduced sample, the output of
the Discriminator-Encoder includes two parts. The first part
is the validity label of the given sample, distinguishing the
sample as real or fake. The second part is a label vector that
denotes the multiclass classification of the sample based on
the classes given in the dataset. This vector of size Ccontains
the numbers predicted by the Discriminator-Encoder in the
range between [0,1], using the Softmax activation function
(equation 9),
softmax(z)i=ezi
Pj,n ezj(9)
where softmax(z)iis the output of the layer, nis the
dimension of the encoded input vector, zidenotes the input
score and zjdescribes each individual score of the encoded
input vector.
The class of the sample is the position of the highest
value in that vector, as described by equation 7. As in the
case of MEN SA Autoencoder-GAN for anomaly detection,
the Discriminator-Encoder is trained alongside the Generator-
Decoder, receiving both real and generated samples, each
with a ground truth label and a label vector. The ground
truth labels given as input to the Discriminator-Encoder are
tl 1for the Generator-Decoder’s output and fl 0for
the real sample. In the case of the label vectors, for the real
sample, the corresponding label vector is given as input to the
Discriminator-Encoder, while for the fake or predicted sample,
a vector with a random label is given. As previously, the
Discriminator-Encoder’s training ability is deactivated when
the Generator-Decoder is trained. The Discriminator-Encoder
is compiled with the Binary Cross-Entropy (equation 5) for
the validity. For the classification procedure, the Categorical
Cross-Entropy (equation 8) and the Adadelta optimizer [53]
are used.
Fig. 6: Discriminator-Encoder structure for anomaly classification
Therefore, to solve the anomaly classification problem the
MEN SAAutoencoder GAN f orAnomalyDetection is
extended a step further. Since classification is a multi-class
problem, the comparison between the input sample with a
randomly generated sample is not adequate. To overcome this
issue, a new conditional architecture is defined. A conditional
GAN can generate samples for each class. By asking the
Generator-Decoder to generate samples from all the avail-
able classes and applying the above methodology used for
the anomaly detection, MEN SA produces Coutputs. By
applying the AdvL for each combination of samples dc
rand
dc
p, where cC, and choosing the cwith the lowest loss,
we result in the best approximation of the class of the given
sample. Fortifying the effort to optimize the classification
process, an additional utility has been added to the MEN SA
Autoencoder-GAN for anomaly classification. In particular,
apart from predicting the validity of the input samples, the
Discriminator-Encoder is also designed to predict the classes.
Thus, it overtakes the role of a classifier. During the training,
the Discriminator-Encoder optimises both the validation and
classification processes.
V. M EN SA I MPLEMENTATION CAPABILITIES
The SG comprises multiple environments and infrastruc-
tures related to the energy generation, transmission and distri-
bution. Therefore, a reliable IDS for the entire SG ecosystem
should be able to be adapted appropriately based on the
corresponding conditions. These conditions can be expressed
sufficiently by the communication protocols and the opera-
tional data (i.e., time series electricity measurements) used
and exchanged respectively by the components of each SG
infrastructure. Furthermore, an essential safety requirement for
an IDS in an SG environment is to consider the computing
resources of the SG components. In general, the cybersecurity
and privacy solutions should not affect and burden the func-
tionality of the SG components [16]. Finally, an IDS solution
should act timely and reliably, detecting the possible anomalies
and intrusions [16].
Based on the aforementioned remarks, Fig. 7 depicts how
MEN SA is implemented in an SG environment. MENSA
is running on a dedicated computing system without deploying
software sensors or services in the SG environment. Thus, it
does not affect the computing resources and the normal oper-
ation of the SG equipment. In particular, the implementation
of MEN SA follows five steps: (a) network traffic sniffing,
(b) operational data collection, (c) network flow extraction
statistics, (d) MENSA anomaly detection and classification
and (e) notification. The first step is responsible for capturing
the entire network traffic through a Switched Port Analyser
(SPAN). To this end, Tshark is adopted. Tshark can be
configured to monitor and sniff the overall network traffic per a
specific time threshold, which is defined based on the network
characteristics of each SG environment. Next, the appropriate
operational data (i.e., time series electricity measurements) is
received. This kind of data is received per a specific thresh-
old time through a REpresentational State Transfer (REST)
Application Programming Interface (API) with a centralised
server called Master Terminal Unit (MTU). MTU is a common
ingredient of the SCADA systems, collecting measurements
and statistics from the SG equipment, such as PLCs, RTUs
and IEDs. Next, the network flow statistics are produced from
the network traffic data received from the first step. For this
purpose, CICFlowMeter is utilised. CICFlowMeter is
a TCP/IP network flow generator that extracts bidirectional
network flow statistics on a predefined flow timeout [54].
Subsequently, the MENSA anomaly detection and classifi-
cation is applied, as analysed in section IV. The anomaly
detection is applied to the operational data, while the anomaly
classification is used to discriminate particular cyberattacks
based on the network flow statistics. Finally, the last step
includes the user notification based on the outcome of the
previous step.
Fig. 7: MENSA Implementation
VI. EVALUATIO N ANALYSI S
This section is devoted to the MEN SA evaluation analysis.
First, the SG evaluation environments and the operation of
MEN SA in the prediction phase are described. Next, the
datasets and the comparative methods follow. Finally, after in-
troducing the necessary definitions and the evaluation metrics,
the MEN SA evaluation results are presented in a comparative
study with other ML and DL methods.
A. Evaluation Environments
MEN SA was evaluated and validated in four real SG
evaluation environments coming from the SPEAR project [55],
namely (a) SG lab, (b) distribution substation, (c) hydropower
plant and (d) power plant. Table III summarises them, showing
what communication protocols are supported for each case.
Moreover, each of the aforementioned SG environments gener-
ates different operational data. Each SG environment possesses
its own SCADA system, which monitors and controls the
automation procedures. In particular, they are characterised
by the presence of appropriate RTUs that manage the op-
eration of industrial elements, such as generators, turbines
and transformers. The RTUs communicate with the MTU,
utilising the Modbus/TCP protocol. For each SG environment,
TABLE III: MENSA Evaluation: Smart Grid Environment, Protocols and Operational Data
SG Environment Modbus/TCP DNP3 Operational Data
SG Lab X X(SG Lab operational data)
Distribution Substation X X X(Distribution Substation operational data)
Hydropower Plant X X(Hydropower Plant operational data)
Power Plant X X(Power Plant operational data)
the Modbus/TCP protocol is utilised in a different way (i.e.,
different Modbus/TCP function codes). In the substation en-
vironment, there are also some IEDs that communicate with
the MTU via DNP3. Through a Human Machine Interface
(HMI) installed on MTU, the system operator can transmit the
necessary commands to the RTUs. Moreover, based on SPAN,
MEN SA receives the entire Modbus/TCP network traffic
and performs the MEN SA Autoencoder-GAN for anomaly
classification in order to discriminate the Modbus/TCP and
DNP3 cyberattacks. In addition, each SG environment gen-
erates and stores in MTU respective operational data (i.e.,
time-series electricity measurements) that is inserted in the
MEN SA Autoencoder-GAN for anomaly detection. This
operational data is received by ME N SA through a REST
API. MEN SA is running in a separate computing system
with an Intel(R) Core(TM) i7-8550U CPU - 1.80GHz, 16GB
Random Access Memory (RAM) and Ubuntu 20.04.2.0 LTS
(Focal Fossa). This machine is also used to extract the evalu-
ation results. Consequently, MENSA is evaluated in several
different ways. First, MEN SA is evaluated against four SG
environments (SG Lab, Distribution Substation, Hydropower
Plant, and Power Plant) using the Modbus/TCP protocol in
a different way (i.e., different Modbus/TCP function codes).
Second, MESA is evaluated in a Distribution Substation
under both protocols, i.e., Modbus/TCP and DNP3. Finally,
MEN SA is evaluated with respect to different operational
data per SG environment.
B. Datasets and Comparative Methods
Appropriate datasets were constructed in order to evaluate
both MEN SA Autoencoder-GAN for anomaly detection and
MEN SA Autoencoder-GAN for anomaly classification. In
the first case, statistically created anomalous samples were
injected manually in the database of MTU, thus creating
a dataset composed of normal and anomalous time-series
electricity measurements for each SG environment mentioned
earlier. This data is different for each SG environment. During
the pre-processing step, the data is formatted utilising a sliding
window of 30 instances and is normalized in the range of
[0,1]. On the other side, regarding the validation of MEN SA
for anomaly classification, the Modbus/TCP cyberattacks of
Table I were emulated in a safe manner, utilising Smod
[49]. Regarding the DNP3 cyberattacks, the intrusion detection
dataset of N.Rodofile et al. [50] was combined with normal
DNP3 network flows of the substation environment. Thus,
datasets consisting of normal and malicious Modbus/TCP and
DNP3 network flows were generated. CICFlowMeter was
used to extract the Modbus/TCP and DNP3 network flows
from the network packet capturing files (i.e., pcap files). Both
datasets were labelled since in the first case, the anomalous
instance were known, while in the second, the malicious IPs
were known. Furthermore, multiple ML and DL methods
were adopted in each case in order to compare and evaluate
the performance of MEN SA. In particular, for the anomaly
detection, the following ML and DL methods were used: (a)
Angle-Based Outlier Detection (ABOD) [56] [57], (b) Isola-
tion Forest (Iforest) [58], (c) Principal Component Analysis
(PCA) [59], (d) Minimum Covariance Determinant (MCD)
[60], (e) Local Outlier Factor (LOF) [61], (f) DIDEROT
Autoencoder [47], (g) ARIES GAN [48] and BlackBox IDS
[62]. Similarly, for the anomaly classification, the subsequent
methods were utilised: (a) Logistic Regression [63], (b) Linear
Discriminant Analysis (LDA) [64], (c) Decision Tree Classifier
[65], (d) Gaussian Naive Bayes (Gaussian NB) [66], (e)
Support Vector Machine (SVM), (f) Random Forest [67],
(g) Multilayer Perceptron (MLP) [68], (h) Adaptive Boosting
(AdaBoost) [69], (i) Quadratic Discriminant Analysis [70], (j)
Dense DNN ReLU [48] and (k) Dense DNN Tanh [48]. The
DIDEROT Autoencoder and the ARIES GAN, Dense DNN
Relu and Dense DNN Tanh originate from our previous works
in [47] and [48], respectively. It is worth mentioning that
the ARIES GAN [48] and the BlackBox IDS [62] constitute
advanced, custom DNNs for anomaly detection and anomaly
classification, respectively. Finally, for the anomaly classifica-
tion, Suricata was also used with the Quickdraw ICS IDS
signatures [71]. Suricata is a widely known network IDS,
which can detect malicious packets [71]. In order to compare
the efficacy of Suricata with ME N SA, we correlated
the packets-related alerts extracted by Suricata with the
corresponding malicious flows.
C. Evaluation Results
Before explaining the evaluation results of MENSA, we
have to introduce the necessary terms and determine the
appropriate evaluation metrics. T P denotes the number of
the classifications that recognise correctly an anomaly or a
cyberattack as an intrusion. Accordingly, T N implies the
amount of the correct classifications that recognise the nor-
mal instances as normal. On the other side, F P denotes
the number of the incorrect classifications that categorise
the normal instances as intrusions. Finally, FN signifies the
wrong classifications that classify the anomalous or malicious
instances as normal. Therefore, based on these values, the
following metrics (equations 10- 13) are defined.
Accuracy (ACC) (equation (10)) expresses the proportion of
the correct classifications and the overall instances. It is a fair
evaluation metric when the training dataset consists of an equal
number of instances for all categories.
Accuracy(AC C) = T P +T N
T P +T N +F P +F N (10)
The False Positive Rate (FPR) (equation (11)) represents
the symmetry of the normal instances that were detected as
anomalous/malicious. FPR is determined by dividing F P
with the sum of T N and F P .
FPR =F P
F P +T N (11)
The True Positive Rate (TPR) (equation (12) ) defines what
ratio of the original anomalous or intrusion instances were
recognised as anomalous/intrusions. TPR is computed by
dividing T P with the sum of T P and F N.
TPR =T P
T P +F N (12)
Finally, the F1 score (equation (13)) denotes the golden ratio
of P recision and TPR, considering both F P and F N.
F1 = 2×P recision ×TPR
P recision +TPR where P r ecision =T P
T P +F P
(13)
Table IV presents the M ENSA evaluation results for
detecting operational anomalies in the first evaluation en-
vironment (i.e., the SG lab). MEN S A achieves the best
performance where ACC = 0.9647,TPR = 0.9418,FPR =
0.0282 and F1=0.9257. On the other side, MCD presents
the worst evaluation results where ACC = 0.7151,TPR =
0.2994,FPR = 0.1584 and F1 = 0.329. Accordingly,
Table V shows the evaluation results for detecting anomalies
in the substation environment. In this case, LOF achieves the
best performance, where ACC = 0.8732,TPR = 0.9938,
FPR = 0.15716 and F1=0.7591. In contrast, the ACC,
TPR,FPR and the F1Score of M EN SA reach 0.8810,
0.7163,0.0775 and 0.7076, respectively. Table VI reflects the
evaluation results for recognising anomalies in the hydropower
plant environment. In this case, the best performance is carried
out by MEN SA where AC C = 0.8835,TPR = 0.8715,
FPR = 0.1134 and F1=0.7498. On the contrary, the lowest
performance is accomplished by MCD where ACC = 0.7337,
TPR = 0.2103,FPR = 0.1351 and F1 = 0.2403.
In a similar manner, Table VII reflects the evaluation results
of MEN SA for distinguishing the Modbus/TCP cyberattacks
emulated in the SG lab. Based on the comparative results,
MEN SA overcomes the other ML and DL solutions since
its ACC ,TPR,FPR and the F1Score reach 0.964,
0.7307,0.0192 and 0.7307. On the other side, the lowest
performance is accomplished by AdaBoost, where ACC =
0.9111,TPR = 0.3333,FPR = 0.0476 and F1=0.3333.
Accordingly, Table VIII includes the evaluation results for
discriminating the Modbus/TCP cyberattacks in the substation
environment. Again, MEN SA achieves the best performance,
where ACC = 0.9655,TPR = 0.7591,FPR = 0.0185 and
TABLE IV: MENSA evaluation results for detecting operational anomalies in
the first SG environment - SG Lab.
Model ACC TPR FPR F1
ABOD 0.692 0.989 0.397 0.600
Iforest 0.813 0.960 0.231 0.705
PCA 0.851 0.982 0.187 0.755
LOF 0.829 0.992 0.220 0.730
MCD 0.715 0.299 0.158 0.329
DIDEROT Autoencoder 0.851 0.982 0.188 0.755
ARIES GAN 0.930 0.875 0.053 0.853
MENSA 0.964 0.941 0.028 0.925
TABLE V: MENSA evaluation results for detecting operational anomalies in
the second SG environment - substation.
Model ACC TPR FPR F1
ABOD 0.839 0.995 0.200 0.713
Iforest 0.850 0.951 0.175 0.718
PCA 0.847 0.961 0.181 0.716
LOF 0.873 0.993 0.1571 0.759
MCD 0.822 0.991 0.220 0.691
DIDEROT Autoencoder 0.840 0.961 0.189 0.708
ARIES GAN 0.834 0.653 0.120 0.613
MENSA 0.881 0.716 0.077 0.707
TABLE VI: MENSA evaluation results for detecting operational anomalies in
the third SG environment - hydropower plant.
Model ACC TPR FPR F1
ABOD 0.581 0.993 0.522 0.487
Iforest 0.716 0.948 0.341 0.572
PCA 0.745 0.978 0.312 0.606
LOF 0.579 0.996 0.525 0.486
MCD 0.733 0.210 0.135 0.240
DIDEROT Autoencoder 0.746 0.978 0.311 0.607
ARIES GAN 0.817 0.966 0.219 0.680
MENSA 0.883 0.871 0.113 0.749
F1 = 0.7591. Moreover, as previously, AdaBoost shows the
worst performance, where ACC = 0.9183,TPR = 0.4281,
FPR = 0.0439 and F1 = 0.4281. In the same SG envi-
ronment, Table XI shows the efficiency of MEN SA against
the DNP3 cyberattacks. MEN SA exceeds the performance
of the other solutions, while the lowest efficiency is accom-
plished by Quadratic Discriminant Analysis. Table IX presents
the evaluation results related to classifying the Modbus/TCP
cyberattacks of Table I in the hydropower plant environ-
ment. Similarly, MENSA achieves the best outcome, where
ACC = 0.9668,TPR = 0.7679,FPR = 0.0178 and
F1=0.7679. In this case, Adaboost achieves even lower
evaluation results compared to the previous environments,
where ACC = 0.8877,TPR = 0.2142,FPR = 0.0604 and
F1=0.2142. Finally, Table X illustrates the evaluation results
of MEN SA for discriminating the Modbus/TCP cyberattacks
in the power plant environment. Also, in this case, MENSA
accomplishes the best outcome where ACC ,TPRm, FPR
and the F1Score reach 0.9646,0.7349,0.0189 and 0.7349.
On the other hand, AdaBoost shows again the worst results,
TABLE VII: MENSA evaluation results for classifying Modbus/TCP cyber-
attacks in the first SG environment - SG Lab.
Model ACC TPR FPR F1
Logistic Regression 0.945 0.588 0.029 0.588
LDA 0.937 0.529 0.033 0.529
Decision Tree Classifier 0.960 0.706 0.020 0.706
Gaussian NB 0.941 0.563 0.031 0.563
SVM RBF 0.931 0.485 0.036 0.485
SVM Linear 0.932 0.494 0.036 0.494
Random Forest 0.945 0.588 0.029 0.588
MLP 0.941 0.559 0.031 0.559
AdaBoost 0.911 0.333 0.047 0.333
Quadratic Discriminant Analysis 0.937 0.528 0.033 0.528
Dense DNN ReLU 0.943 0.578 0.030 0.578
Dense DNN Tanh 0.940 0.552 0.031 0.552
BlackBox IDS 0.948 0.612 0.027 0.601
Suricata 0.839 0.664 0.000 0.798
MENSA 0.964 0.730 0.019 0.730
TABLE VIII: MENSA evaluation results for classifying Modbus/TCP cyber-
attacks in the second SG environment - substation.
Model ACC TPR FPR F1
Logistic Regression 0.944 0.614 0.029 0.614
LDA 0.944 0.608 0.030 0.608
Decision Tree Classifier 0.964 0.749 0.019 0.749
Gaussian NB 0.938 0.566 0.033 0.566
SVM RBF 0.931 0.518 0.037 0.518
SVM Linear 0.930 0.511 0.037 0.511
Random Forest 0.947 0.631 0.028 0.631
MLP 0.940 0.584 0.031 0.584
AdaBoost 0.918 0.428 0.043 0.428
Quadratic Discriminant Analysis 0.944 0.613 0.029 0.613
Dense DNN ReLU 0.945 0.619 0.029 0.619
Dense DNN Tanh 0.944 0.611 0.029 0.611
BlackBox IDS 0.948 0.948 0.027 0.633
Suricata 0.839 0.664 0.000 0.798
MENSA 0.965 0.759 0.018 0.759
TABLE IX: MENSA evaluation results for classifying Modbus/TCP cyberat-
tacks in the third SG environment - hydropower plant
Model ACC TPR FPR F1
Logistic Regression 0.943 0.603 0.030 0.603
LDA 0.943 0.604 0.030 0.604
Decision Tree Classifier 0.964 0.749 0.019 0.749
Gaussian NB 0.928 0.497 0.038 0.497
SVM RBF 0.918 0.426 0.044 0.426
SVM Linear 0.921 0.453 0.042 0.453
Random Forest 0.947 0.633 0.028 0.633
MLP 0.938 0.570 0.033 0.570
AdaBoost 0.887 0.214 0.060 0.214
Quadratic Discriminant Analysis 0.941 0.593 0.031 0.593
Dense DNN ReLU 0.945 0.619 0.029 0.619
Dense DNN Tanh 0.945 0.619 0.029 0.619
BlackBox IDS 0.948 0.641 0.029 0.630
Suricata 0.839 0.664 0.000 0.798
MENSA 0.966 0.767 0.017 0.767
where ACC = 0.9111,TPR = 0.333,FPR = 0.0476 and
F1=0.3333.
Even though the data samples per case are morphologically
similar, they differ in various ways, such as the features, the
values magnitude and the sparsity. Thus, it is impracticable
TABLE X: MENSA evaluation results for classifying Modbus/TCP cyberat-
tacks in the fourth SG environment - power plant
Model ACC TPR FPR F1
Logistic Regression 0.946 0.597 0.028 0.597
LDA 0.939 0.548 0.032 0.548
Decision Tree Classifier 0.960 0.703 0.021 0.703
Gaussian NB 0.942 0.565 0.031 0.565
SVM RBF 0.929 0.468 0.037 0.468
SVM Linear 0.933 0.502 0.035 0.502
Random Forest 0.949 0.623 0.026 0.623
MLP 0.941 0.562 0.031 0.562
AdaBoost 0.911 0.333 0.047 0.333
Quadratic Discriminant Analysis 0.937 0.529 0.033 0.529
Dense DNN ReLU 0.948 0.616 0.027 0.616
Dense DNN Tanh 0.940 0.551 0.032 0.551
BlackBox IDS 0.940 0.611 0.027 0.604
Suricata 0.839 0.664 0.000 0.798
MENSA 0.964 0.734 0.018 0.734
TABLE XI: MENSA evaluation results for classifying DNP3 cyberattacks in
the second SG environment - substation
Model ACC TPR FPR F1
Logistic Regression 0.907 0.722 0.055 0.722
LDA 0.896 0.688 0.062 0.688
Decision Tree Classifier 0.977 0.991 0.001 0.991
Gaussian NB 0.910 0.731 0.053 0.731
SVM RBF 0.864 0.592 0.081 0.592
SVM Linear 0.893 0.680 0.063 0.680
Random Forest 0.931 0.733 0.053 0.733
MLP 0.911 0.733 0.053 0.733
AdaBoost 0.798 0.396 0.120 0.396
Quadratic Discriminant Analysis 0.722 0.166 0.166 0.166
Dense DNN ReLU 0.941 0.823 0.035 0.823
Dense DNN Tanh 0.932 0.797 0.040 0.797
BlackBox IDS 0.965 0.896 0.021 0.895
Suricata 0.795 0.636 0.000 0.777
MENSA 0.994 0.983 0.003 0.983
to formulate a model using standard hyperparameters per
case. In contrast, each case is optimised experimentally. To
evaluate the MEN SA performance in terms of the various
hyperparameters, two evaluation metrics are utilised: (a) the
F1 score variation per threshold tand (b) the F1 score
saturation curve per iteration. Both cases aim to maximise
the F1 score. In Fig. 8, the behaviour of the F1 score is
depicted for four different experiments. As illustrated, the F1
score changes exponentially after a value of t. In particular,
this value describes the optimal threshold leading to the most
efficient discrimination between the normal and the abnormal
instances. After this value, the F1 score saturates completely.
Subsequently, Fig. 9 shows how the F1 score is improved
based on the number of epochs. For the first iterations, the
F1 score increases exponentially. Next, it saturates slowly for
the rest of the training process. When the curve starts to flatten,
the training is stopped to avoid overfitting and memorisation.
Thus, the best checkpoint is selected. Finally, regarding the
batch size, the larger the number of features, the higher the
batch scaling in 2a, a N, while the learning rate of each
optimizer is kept in the scale 1/1000.
Therefore, according to Tables IV- X, almost in all SG
Fig. 8: F1-Score variation through the change of t
Fig. 9: F1-Score Saturation Curve
environments ME N SA achieves the best performance either
for detecting operational anomalies or discriminating the Mod-
bus/TCP and DNP3 cyberattacks. In general, a high TPR and
low FPR can be observed. This is due to ME N SA dynamic
deep threshold discovery. Usually, ML and DL classifiers use
a threshold to provide the optimal outcome. Since MEN SA
is designed to be adaptable for each SG environment and
type of data, MEN SA dynamically calculates the appropriate
threshold during the training process, thus achieving the best
detection results. To this end, a brute force approach is utilised.
It is noteworthy that ME N SA overcomes other advanced DL
solutions, such as ARIES GAN [48] and BlackBox IDS [62]
for anomaly detection and anomaly classification, respectively.
Moreover, MENSA exceeds the efficiency of Suricata
since the existing Quickdraw ICS IDS signatures [71] do
not cover all possible intrusions related to the Modbus/TCP
and DNP3 payloads. In addition, the TCP/IP network flow
statistics generated by CICFlowMeter render MEN SA
a scalable solution for detecting and classifying anomalies
for other application-layer protocols, such as IEC 60870-5-
104, Message Queuing Telemetry Transport (MQTT) and IEC
61850 Manufacturing Message Specification (MMS). Finally,
the successful anomaly detection against different kinds of
operational data demonstrates the MEN SA scalability.
VII. CONCLUSION
The next generation EG, commonly called SG, creates
significant advantages and challenges in society. On the one
side, valuable services are already provided, such as the two-
way power flow and self-monitoring, but on the other side, new
cybersecurity concerns are generated. It is worth mentioning
that the interconnected nature of the SG ecosystem also affects
the safety status of other CIs. Therefore, the presence of novel
intrusion and anomaly detection mechanisms and eliminating
FP and FN are necessary. The ML and DL solutions compose
valuable mechanisms capable of detecting zero-day attacks.
In this paper, we implemented an anomaly detection and
classification model capable of detecting 13 Modbus/TCP
cyberattacks, 5DNP3 cyberattacks and potential anomalies
related to operational data (i.e., time-series electricity mea-
surements). The proposed model called MEN SA combines
two DNNs: (a) Autoencoder and (b) GAN in a prototype
architecture, which applies a novel minimisation function,
taking into account (a) the adversarial error and (b) the recon-
struction difference. The efficiency of MENSA was validated
and evaluated in four SG evaluation environments: (a) SG lab,
(b) substation, (c) hydropower plant and (d) power plant. To
this end, other ML and DL methods were also adopted.
Our future plans in this field include the implementation
of other DL models in order to detect cyberattacks against
other ICS/SCADA protocols, such as Profinet and EtherCat.
Moreover, sufficient association rules will be examined to
correlate the outcome of these DL models with each other.
Finally, optimisation solutions mitigating sufficiently such
cyberattacks in CIs will be investigated.
VIII. ACKNOWLEDGMENT
This project has received funding from the European Unions
Horizon 2020 research and innovation programme under grant
agreement No. 787011 (SPEAR).
REFERENCES
[1] S. Tan, D. De, W.-Z. Song, J. Yang, and S. K. Das, “Survey of security
advances in smart grid: A data driven approach,IEEE Communications
Surveys & Tutorials, vol. 19, no. 1, pp. 397–422, 2017.
[2] D. Pliatsios, P. Sarigiannidis, T. Lagkas, and A. G. Sarigiannidis,
“A survey on scada systems: Secure protocols, incidents, threats and
tactics,” IEEE Communications Surveys & Tutorials, 2020.
[3] M. De Vivo, G. O. de Vivo, R. Koeneke, and G. Isern, “Internet
vulnerabilities related to tcp/ip and t/tcp,” ACM SIGCOMM Computer
Communication Review, vol. 29, no. 1, pp. 81–85, 1999.
[4] R.-G. Panagiotis, P. G. Sarigiannidis, and I. D. Moscholios, “Securing
the internet of things: Challenges, threats and solutions,” Internet of
Things, vol. 5, pp. 41–70, 2019.
[5] A. Alshamrani, S. Myneni, A. Chowdhary, and D. Huang, “A survey on
advanced persistent threats: Techniques, solutions, challenges, and re-
search opportunities,” IEEE Communications Surveys Tutorials, vol. 21,
no. 2, pp. 1851–1877, 2019.
[6] S. Kwon, H. Yoo, and T. Shon, “Ieee 1815.1-based power system
security with bidirectional rnn-based network anomalous attack detection
for cyber-physical system,IEEE Access, vol. 8, pp. 77 572–77 586,
2020.
[7] B. Bencs´
ath, G. P´
ek, L. Butty´
an, and M. Felegyhazi, “The cousins of
stuxnet: Duqu, flame, and gauss,” Future Internet, vol. 4, no. 4, pp.
971–1003, 2012.
[8] A. Di Pinto, Y. Dragoni, and A. Carcano, “Triton: The first ics cyber
attack on safety instrument systems,” in Proc. Black Hat USA, 2018, pp.
1–26.
[9] J. Sakhnini, H. Karimipour, A. Dehghantanha, R. M. Parizi, and G. Sri-
vastava, “Security aspects of internet of things aided smart grids: A
bibliometric survey,” Internet of things, p. 100111, 2019.
[10] P. Kumar, Y. Lin, G. Bai, A. Paverd, J. S. Dong, and A. Martin, “Smart
grid metering networks: A survey on security, privacy and open research
issues,” IEEE Communications Surveys Tutorials, vol. 21, no. 3, pp.
2886–2927, 2019.
[11] A. Ghosal and M. Conti, “Key management systems for smart grid
advanced metering infrastructure: A survey,” IEEE Communications
Surveys Tutorials, vol. 21, no. 3, pp. 2831–2848, 2019.
[12] A. S. Musleh, G. Chen, and Z. Y. Dong, “A survey on the detection algo-
rithms for false data injection attacks in smart grids,” IEEE Transactions
on Smart Grid, vol. 11, no. 3, pp. 2218–2234, 2020.
[13] R. Leszczyna, “Cybersecurity and privacy in standards for smart grids–a
comprehensive survey,” Computer Standards & Interfaces, vol. 56, pp.
62–73, 2018.
[14] S. M. S. Hussain, T. S. Ustun, and A. Kalam, “A review of iec
62351 security mechanisms for iec 61850 message exchanges,” IEEE
Transactions on Industrial Informatics, vol. 16, no. 9, pp. 5643–5654,
2020.
[15] T. S. Ustun and S. S. Hussain, “Iec 62351-4 security implementations
for iec 61850 mms messages,” IEEE Access, 2020.
[16] P. I. Radoglou-Grammatikis and P. G. Sarigiannidis, “Securing the smart
grid: A comprehensive compilation of intrusion detection and prevention
systems,” IEEE Access, vol. 7, pp. 46595–46 620, 2019.
[17] S. V. B. Rakas, M. D. Stojanovi, and J. D. Markovi-Petrovi, “A review
of research work on network-based scada intrusion detection systems,”
IEEE Access, vol. 8, pp. 93 083–93 108, 2020.
[18] R. Mitchell and I.-R. Chen, “A survey of intrusion detection techniques
for cyber-physical systems,ACM Computing Surveys (CSUR), vol. 46,
no. 4, pp. 1–29, 2014.
[19] S. Ghosh and S. Sampalli, “A survey of security in scada networks:
Current issues and future challenges,” IEEE Access, vol. 7, pp. 135812–
135 831, 2019.
[20] P. Radoglou-Grammatikis, P. Sarigiannidis, T. Liatifis, T. Apostolakos,
and S. Oikonomou, “An overview of the firewall systems in the smart
grid paradigm,” in 2018 Global Information Infrastructure and Network-
ing Symposium (GIIS), 2018, pp. 1–4.
[21] D. Pliatsios, P. Sarigiannidis, T. Lagkas, and A. G. Sarigiannidis,
“A survey on scada systems: Secure protocols, incidents, threats and
tactics,” IEEE Communications Surveys Tutorials, vol. 22, no. 3, pp.
1942–1976, 2020.
[22] S. V. B. Rakas, M. D. Stojanovi´
c, and J. D. Markovi´
c-Petrovi´
c, “A
review of research work on network-based scada intrusion detection
systems,” IEEE Access, vol. 8, pp. 93083–93 108, 2020.
[23] G. Efstathopoulos, P. Radoglou-Grammatikis, P. Sarigiannidis, V. Ar-
gyriou, A. Sarigiannidis, K. Stamatakis, M. K. Angelopoulos, and S. K.
Athanasopoulos, “Operational data based intrusion detection system for
smart grid,” in 2019 IEEE 24th International Workshop on Computer
Aided Modeling and Design of Communication Links and Networks
(CAMAD), 2019, pp. 1–6.
[24] S. Yoon, J.-H. Cho, D. S. Kim, T. J. Moore, F. Free-Nelson, and
H. Lim, “Attack graph-based moving target defense in software-defined
networks,” IEEE Transactions on Network and Service Management,
vol. 17, no. 3, pp. 1653–1668, 2020.
[25] I. Hafeez, M. Antikainen, A. Y. Ding, and S. Tarkoma, “Iot-keeper:
Detecting malicious iot network activity using online traffic analysis at
the edge,” IEEE Transactions on Network and Service Management,
vol. 17, no. 1, pp. 45–59, 2020.
[26] S. Garg, K. Kaur, N. Kumar, G. Kaddoum, A. Y. Zomaya, and R. Ranjan,
“A hybrid deep learning-based model for anomaly detection in cloud
datacenter networks,” IEEE Transactions on Network and Service Man-
agement, vol. 16, no. 3, pp. 924–935, 2019.
[27] T. V. Phan, T. G. Nguyen, N.-N. Dao, T. T. Huong, N. H. Thanh, and
T. Bauschert, “Deepguard: Efficient anomaly detection in sdn with fine-
grained traffic flow monitoring,IEEE Transactions on Network and
Service Management, vol. 17, no. 3, pp. 1349–1362, 2020.
[28] R. Doriguzzi-Corin, S. Millar, S. Scott-Hayward, J. Martinez-del Rincon,
and D. Siracusa, “Lucid: A practical, lightweight deep learning solution
for ddos attack detection,” IEEE Transactions on Network and Service
Management, vol. 17, no. 2, pp. 876–889, 2020.
[29] C. Yinka-Banjo and O.-A. Ugot, “A review of generative adversarial
networks and its application in cybersecurity,Artificial Intelligence
Review, pp. 1–16, 2019.
[30] J.-Y. Kim, S.-J. Bu, and S.-B. Cho, “Malware detection using deep
transferred generative adversarial networks,” in International Conference
on Neural Information Processing. Springer, 2017, pp. 556–564.
[31] A. Arora and Shantanu, “A review on application of gans in cybersecu-
rity domain,” IETE Technical Review, pp. 1–9, 2020.
[32] P. Dixit and S. Silakari, “Deep learning algorithms for cybersecurity
applications: A technological and status review,” Computer Science
Review, vol. 39, p. 100317, 2021.
[33] S.-C. Li, Y. Huang, B.-C. Tai, and C.-T. Lin, “Using data mining
methods to detect simulated intrusions on a modbus network,” in 2017
IEEE 7th International Symposium on Cloud and Service Computing
(SC2). IEEE, 2017, pp. 143–148.
[34] W. Yusheng, F. Kefeng, L. Yingxu, L. Zenghui, Z. Ruikang, Y. Xi-
angzhen, and L. Lin, “Intrusion detection of industrial control system
based on modbus tcp protocol,” in 2017 IEEE 13th International
Symposium on Autonomous Decentralized System (ISADS). IEEE, 2017,
pp. 156–162.
[35] I. N. Fovino, A. Carcano, T. D. L. Murel, A. Trombetta, and M. Masera,
“Modbus/dnp3 state-based intrusion detection system,” in 2010 24th
IEEE International Conference on Advanced Information Networking
and Applications. IEEE, 2010, pp. 729–736.
[36] T. Morris, R. Vaughn, and Y. Dandass, “A retrofit network intrusion
detection system for modbus rtu and ascii industrial control systems,” in
2012 45th Hawaii International Conference on System Sciences. IEEE,
2012, pp. 2338–2345.
[37] D. S. Berman, A. L. Buczak, J. S. Chavis, and C. L. Corbett, “A survey
of deep learning methods for cyber security,Information, vol. 10, no. 4,
p. 122, 2019.
[38] S. Mahdavifar and A. A. Ghorbani, “Application of deep learning to
cybersecurity: A survey,” Neurocomputing, vol. 347, pp. 149–176, 2019.
[39] M. A. Ferrag, L. Maglaras, S. Moschoyiannis, and H. Janicke, “Deep
learning for cyber security intrusion detection: Approaches, datasets, and
comparative study,” Journal of Information Security and Applications,
vol. 50, p. 102419, 2020.
[40] R. Shire, S. Shiaeles, K. Bendiab, B. Ghita, and N. Kolokotronis,
“Malware squid: a novel iot malware traffic analysis framework using
convolutional neural network and binary visualisation,” in Internet of
Things, Smart Spaces, and Next Generation Networks and Systems.
Springer, 2019, pp. 65–76.
[41] binvis.io, “binvis.io,” 2018. [Online]. Available: https://binvis.io/
[42] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang,
T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convo-
lutional neural networks for mobile vision applications,” arXiv preprint
arXiv:1704.04861, 2017.
[43] Y. He, G. J. Mendis, and J. Wei, “Real-time detection of false data
injection attacks in smart grid: A deep learning-based intelligent mech-
anism,” IEEE Transactions on Smart Grid, vol. 8, no. 5, pp. 2505–2516,
2017.
[44] M. Saharkhizan, A. Azmoodeh, A. Dehghantanha, K. K. R. Choo,
and R. M. Parizi, “An ensemble of deep recurrent neural networks
for detecting iot cyber attacks using network traffic,IEEE Internet of
Things Journal, vol. 7, no. 9, pp. 8852–8859, 2020.
[45] P. Simoes, “Denial of service attacks: Detecting the frailties of machine
learning algorithms in the classification process,” in Critical Information
Infrastructures Security: 13th International Conference, CRITIS 2018,
Kaunas, Lithuania, September 24-26, 2018, Revised Selected Papers,
vol. 11260. Springer, 2019, p. 230.
[46] H. Yang, L. Cheng, and M. C. Chuah, “Deep-learning-based network
intrusion detection for scada systems,” in 2019 IEEE Conference on
Communications and Network Security (CNS), 2019, pp. 1–7.
[47] P. Radoglou-Grammatikis, P. Sarigiannidis, G. Efstathopoulos, P.-A.
Karypidis, and A. Sarigiannidis, “Diderot: an intrusion detection and
prevention system for dnp3-based scada systems,” in Proceedings of the
15th International Conference on Availability, Reliability and Security,
2020, pp. 1–8.
[48] P. Radoglou Grammatikis, P. Sarigiannidis, G. Efstathopoulos, and
E. Panaousis, “Aries: A novel multivariate intrusion detection system
for smart grid,” Sensors, vol. 20, no. 18, p. 5305, 2020.
[49] P. Radoglou-Grammatikis, I. Siniosoglou, T. Liatifis, A. Kourouniadis,
K. Rompolos, and P. Sarigiannidis, “Implementation and detection of
modbus cyberattacks,” in 2020 9th International Conference on Modern
Circuits and Systems Technologies (MOCAST). IEEE, 2020, pp. 1–4.
[50] N. R. Rodofile, K. Radke, and E. Foo, “Framework for scada cyber-
attack dataset creation,” in Proceedings of the Australasian Computer
Science Week Multiconference, 2017, pp. 1–10.
[51] A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta,
and A. Bharath, “Generative adversarial networks: An overview,IEEE
Signal Processing Magazine, vol. 35, 10 2017.
[52] Y. Hong, U. Hwang, J. Yoo, and S. Yoon, “How generative adversarial
nets and its variants work: An overview of gan,ACM Computing
Surveys, vol. 52, 11 2017.
[53] M. D. Zeiler, “Adadelta: An adaptive learning rate method,ArXiv, vol.
abs/1212.5701, 2012.
[54] P. Radoglou-Grammatikis, P. Sarigiannidis, A. Sarigiannidis, D. Mar-
gounakis, A. Tsiakalos, and G. Efstathopoulos, “An anomaly detection
mechanism for iec 60870-5-104,” in 2020 9th International Conference
on Modern Circuits and Systems Technologies (MOCAST). IEEE, 2020,
pp. 1–4.
[55] P. Radoglou-Grammatikis, P. Sarigiannidis, E. Iturbe, E. Rios, A. Sari-
giannidis, O. Nikolis, D. Ioannidis, V. Machamint, M. Tzifas, A. Gian-
nakoulias, M. Angelopoulos, A. Papadopoulos, and F. Ramos, “Secure
and private smart grid: The spear architecture,” in 2020 6th IEEE
Conference on Network Softwarization (NetSoft), 2020, pp. 450–456.
[56] N. Pham and R. Pagh, “A near-linear time approximation algorithm for
angle-based outlier detection in high-dimensional data,” Proceedings of
the ACM SIGKDD International Conference on Knowledge Discovery
and Data Mining, 08 2012.
[57] X. Li, J. C. Lv, and D. Cheng, “Angle-based outlier detection algo-
rithm with more stable relationships,” in Proceedings of the 18th Asia
Pacific Symposium on Intelligent and Evolutionary Systems, Volume 1,
H. Handa, H. Ishibuchi, Y.-S. Ong, and K. C. Tan, Eds. Cham: Springer
International Publishing, 2015, pp. 433–446.
[58] F. T. Liu, K. M. Ting, and Z. Zhou, “Isolation forest,” in 2008 Eighth
IEEE International Conference on Data Mining, 2008, pp. 413–422.
[59] M. ling Shyu, S. ching Chen, K. Sarinnapakorn, and L. Chang, “A novel
anomaly detection scheme based on principal component classifier,” in
in Proceedings of the IEEE Foundations and New Directions of Data
Mining Workshop, in conjunction with the Third IEEE International
Conference on Data Mining (ICDM03, 2003, pp. 172–179.
[60] P. Rousseeuw and K. Driessen, “A fast algorithm for the minimum
covariance determinant estimator,Technometrics, vol. 41, pp. 212–223,
08 1999.
[61] D. Pokrajac, A. Lazarevic, and L. J. Latecki, “Incremental local outlier
detection for data streams,” in 2007 IEEE Symposium on Computational
Intelligence and Data Mining, 2007, pp. 504–515.
[62] Z. Lin, Y. Shi, and Z. Xue, “Idsgan: Generative adversarial net-
works for attack generation against intrusion detection,” arXiv preprint
arXiv:1809.02077, 2018.
[63] Y. Wang, “A multinomial logistic regression modeling approach for
anomaly intrusion detection,” Computers and Security, vol. 24, pp. 662–
674, 11 2005.
[64] K. Kim, H. Choi, C. Moon, and C.-W. Mun, “Comparison of k-
nearest neighbor, quadratic discriminant and linear discriminant analysis
in classification of electromyogram signals based on the wrist-motion
directions,” Current Applied Physics - CURR APPL PHYS, vol. 11, pp.
740–745, 05 2011.
[65] M. Shafiq, X. Yu, A. A. Laghari, L. Yao, N. K. Karn, and F. Abdessamia,
“Network traffic classification techniques and comparative analysis
using machine learning algorithms,” in 2016 2nd IEEE International
Conference on Computer and Communications (ICCC), 2016, pp. 2451–
2455.
[66] N. Williams, S. Zander, and G. Armitage, “A preliminary performance
comparison of five machine learning algorithms for practical ip
traffic flow classification,SIGCOMM Comput. Commun. Rev.,
vol. 36, no. 5, p. 516, Oct. 2006. [Online]. Available: https:
//doi.org/10.1145/1163593.1163596
[67] Y. Chang, W. Li, and Z. Yang, “Network intrusion detection based on
random forest and support vector machine,” in 2017 IEEE International
Conference on Computational Science and Engineering (CSE) and IEEE
International Conference on Embedded and Ubiquitous Computing
(EUC), vol. 1, 2017, pp. 635–638.
[68] P. I. Radoglou-Grammatikis and P. G. Sarigiannidis, “Flow anomaly
based intrusion detection system for android mobile devices,” in 2017
6th International Conference on Modern Circuits and Systems Technolo-
gies (MOCAST), 2017, pp. 1–4.
[69] W. Hu, W. Hu, and S. Maybank, “Adaboost-based algorithm for network
intrusion detection.” IEEE Transactions on Systems, Man, and Cyber-
netics, Part B, vol. 38, pp. 577–583, 04 2008.
[70] S. Naseer, D. Y. Saleem, S. Khalid, M. Khawar, J. Han, M. Iqbal, and
K. Han, “Enhanced network anomaly detection based on deep neural
networks,” IEEE Access, pp. 1–1, 08 2018.
[71] K. Wong, C. Dillabaugh, N. Seddigh, and B. Nandy, “Enhancing suricata
intrusion detection system for cyber security in scada networks,” in
2017 IEEE 30th Canadian Conference on Electrical and Computer
Engineering (CCECE), 2017, pp. 1–5.
Ilias Siniosoglou received his Diploma degree (5
years) from the Dept. of Electrical and Computer
Eng., University of Western Macedonia, Greece,
in 2020. He is now a Ph.D.student in the same
department. His main area of research is Deep and
Federated Learning on Next Generation IoT plat-
forms, primarily focusing on optimization, deploy-
ment and scalability methodologies. Currently, he is
working as a research associate at the University
of Western Macedonia in national and European
funded research projects, including (a) H2020-DS-
SC7-2017 (DS-07-2017), SPEAR: Secure and PrivatE smArt gRid, (b) H2020-
SU-DS-2018 (SU-DS04-2018), SDN-microSENSE: SDN-microgrid reSilient
Electrical eNergy SystEm, (c) MARS: sMart fArming with dRoneS (Compet-
itiveness, Entrepreneurship, and Innovation) and (d) H2020-ICT-2020-1 (ICT-
56-2020) TERMINET: nexT gEneRation sMart INterconnectEd ioT.
Panagiotis Radoglou-Grammatikis received the
Diploma degree (MEng, 5 years) from the Dept.
of Informatics and Telecommunications Eng. (now
Dept. of Electrical and Computer Eng.), Faculty
of Eng., University of Western Macedonia, Greece,
in 2016. He is now a PhD candidate in the same
department. His main research interests are in the
area of cybersecurity and mainly focus on intru-
sion detection, vulnerability research and applied
cryptography. He has published 18 research papers
in international scientific journals, conferences and
book chapters, including IEEE Access, Computer Networks (ELSEVIER),
Internet of Things (ELSEVIER) and Sensors (MDPI). Moreover, he received
the Best Paper award in 2019 IEEE International Workshop on Computer
Aided Modeling and Design of Communication Links and Networks (IEEE
CAMAD). He has served as a reviewer for several scientific journals and
possesses working experience as a security engineer and software developer.
Currently, he is working as a research associate at the University of Western
Macedonia in national and European funded research projects, including
(a) H2020-DS-SC7-2017 (DS-07-2017), SPEAR: Secure and PrivatE smArt
gRid, (b) H2020-SU-DS-2018 (SU-DS04-2018), SDN-microSENSE: SDN-
microgrid reSilient Electrical eNergy SystEm, (c) MARS: sMart fArming with
dRoneS (Competitiveness, Entrepreneurship, and Innovation), (d) H2020-ICT-
2020-1 (ICT-56-2020) TERMINET: nexT gEneRation sMart INterconnectEd
ioT and (e) H2020-LC-SC3-EE-2020-1 (LC-SC3-EC-4-2020) EVIDENT:
bEhaVioral Insgihts anD Effective eNergy policy acTions. Finally, he is a
member of the IEEE and the Technical Chamber of Greece.
Georgios Efstathopoulos studied in National Tech-
nical University of Athens, where he received the
Diploma of Electrical and Computer Engineer with
distinction. He received his PhD degree under the
supervision of Professor A. Manikas in the Commu-
nications and Signal Processing Group, Department
of Electrical and Electronic Engineering, Imperial
College London. Also, he worked as software de-
veloper and quantitative analyst at the Investment
Bank sector. He has been working as a quantitative
analyst in the nancial sector for the last 9 years.
Over the last 3 years, Georgios has been actively involved in a number of
data analytics, machine learning and AI projects in various industries, which
includes autonomous vehicles, nance, smart grid, insurance and healthcare
sectors.
Dr. Panayotis Fouliras is Assistant Professor at the
Department of Applied Informatics at the University
of Macedonia, Thessaloniki, Greece. He obtained
his B.Sc. in Physics (Aristotle University of Thes-
saloniki, Greece), M.Sc. and Ph.D in Computer
Science from University of London, UK (QMW).
His research interests span computer networks, QoS,
multimedia and system evaluation methods.
Prof. Panagiotis Sarigiannidis is an Associate Pro-
fessor in the Department of Electrical and Com-
puter Engineering in the University of Western
Macedonia, Kozani, Greece since 2016. He received
the B.Sc. and Ph.D. degrees in computer science
from the Aristotle University of Thessaloniki, Thes-
saloniki, Greece, in 2001 and 2007, respectively.
He has published over 180 papers in international
journals, conferences and book chapters, including
IEEE Communications Surveys and Tutorials, IEEE
Transactions on Communications, IEEE Internet of
Things, IEEE Transactions on Broadcasting, IEEE Systems Journal, IEEE
Wireless Communications Magazine, IEEE/OSA Journal of Lightwave Tech-
nology, IEEE Access, and Computer Networks. He has been involved in
several national, European and international projects. He is currently the
project coordinator of three H2020 projects, namely a) H2020-DS-SC7-
2017 (DS-07-2017), SPEAR: Secure and PrivatE smArt gRid, b) H2020-
LC-SC3-EE-2020-1 (LC-SC3-EC-4-2020), EVIDENT: bEhaVioral Insgihts
anD Effective eNergy policy acTions, and c) H2020-ICT-2020-1 (ICT-56-
2020), TERMINET: nexT gEneRation sMart INterconnectEd ioT, while he
coordinates the Operational Program MARS: sMart fArming with dRoneS
(Competitiveness, Entrepreneurship, and Innovation) and the Erasmus+ KA2
ARRANGE-ICT: SmartROOT: Smart faRming innOvatiOn Training. He also
serves as a principal investigator in the H2020-SU-DS-2018 (SU-DS04-2018),
SDN-microSENSE: SDN-microgrid reSilient Electrical eNergy SystEm and
in three Erasmus+ KA2: a) ARRANGE-ICT: pArtneRship foR AddressiNG
mEgatrends in ICT, b) JAUNTY: Joint undergAduate coUrses for smart
eNergy managemenT sYstems, and c) STRONG: advanced firST RespONders
traininG (Cooperation for Innovation and the Exchange of Good Practices).
His research interests include telecommunication networks, internet of things
and network security. He is an IEEE member and participates in the Editorial
Boards of various journals, including International Journal of Communication
Systems and EURASIP Journal on Wireless Communications and Networking.
... In the literature, there are so many approaches presented to describe the price determination and make the consumers informed of it [5,6]. Traditional methods, such as local electricity auctions, may lead to cyber security gaps and privacy issues [7]. Overcoming the mentioned issues, the decentralized smart grid control (DSGC) has been recently announced to the academic audience by the studies [8,9]. ...
Article
Full-text available
As the backbone of modern society and industry, the need for a more efficient and sustainable electrical grid is crucial for proper energy management. Governments have recognized this need and have included energy management as a key component of their plans. Decentralized Smart Grid Control (DSGC) is a new approach that aims to improve demand response without the need for major infrastructure upgrades. This is achieved by linking the price of electricity to the frequency of the grid. While DSGC solutions offer benefits, they also involve several simplifying assumptions. In this proposed study, an enhanced analysis will be conducted to investigate how data analytics can be used to remove these simplifications and provide a more detailed understanding of the system. The proposed data-mining strategy will use detailed feature engineering and explainable artificial intelligence-based models using a public dataset. The dataset will be analyzed using both classification and regression techniques. The results of the study will differ from previous literature in the ways in which the problem is handled and the performance of the proposed models. The findings of the study are expected to provide valuable insights for energy management-based organizations, as it will maintain a high level of symmetry between smart grid stability and demand-side management. The proposed model will have the potential to enhance the overall performance and efficiency of the energy management system.
... The first cyberattack targets the communication between a smart meter and the Active Distribution Management System (ADMS), while the second attack focuses on the communication between a smart inverter and ADMS. In both cases, the confidentiality is violated through a Man In The Middle (MITM) attack [3] against the Modbus/Transmission Control Protocol (TCP) [4], [5]. Finally, a relevant Intrusion Detection System (IDS) is presented. ...
Conference Paper
Full-text available
The transformation of the conventional electrical grid into a digital ecosystem brings significant benefits, such as two-way communication between energy consumers and utilities, self-monitoring and pervasive controls. However, the advent of the smart electrical grid raises severe cybersecurity and privacy concerns, given the presence of legacy systems and communications protocols. This paper focuses on False Data Injection (FDI) cyberattacks against a low-voltage distribution system, taking full advantage of Man In The Middle (MITM) actions. The first cyberattack targets the communication between a smart meter and an Active Distribution Management System (ADMS), while the second FDI cyberattack targets the communication between a smart inverter and ADMS. In both cases, the cyberattacks affect the operation of the distribution transformer, thus resulting in devastating consequences. Moreover, this paper provides an Artificial Intelligence (AI)-based Intrusion Detection System (IDS), detecting and mitigating the above cyberattacks in a timely manner. The evaluation results demonstrate the efficiency of the proposed IDS.
... An algorithm's performance measurements are its accuracy, precision, recall, and f1-score. These scenarios are mathematically represented as in Albulayhi and Sheldon (2021), Khoei, Aissou, Hu, and Kaabouch (2021), Peng et al. (2019), Radoglou-Grammatikis and Sarigiannidis (2018), Sharafaldin et al. (2018) and Siniosoglou, Radoglou-Grammatikis, Efstathopoulos, Fouliras, and Sarigiannidis (2021) and written in subsequent equations as ...
Article
The Smart Grid’s objective is to increase the electric grid’s dependability, security, and efficiency through extensive digital information and control technology deployment. As a result, it is necessary to apply real-time analysis and state estimation-based techniques to ensure efficient controls are implemented correctly. These systems are vulnerable to cyber-attacks, posing significant risks to the Smart Grid’s overall availability due to their reliance on communication technology. Therefore, effective intrusion detection algorithms are required to mitigate such attacks. In dealing with these uncertainties, we propose a hybrid deep learning algorithm that focuses on Distributed Denial of Service attacks on the communication infrastructure of the Smart Grid. The proposed algorithm is hybridized by the Convolutional Neural Network and the Gated Recurrent Unit algorithms. Simulations are done using a benchmark cyber security dataset of the Canadian Institute of Cybersecurity Intrusion Detection System. According to the simulation results, the proposed algorithm outperforms the current intrusion detection algorithms, with an overall accuracy rate of 99.7%.
... In recent years, cyberattacks have caused significant damage to industrial production and national infrastructure [1]; the Stuxnet virus swept the global industry in 2010 and was able to carry out targeted attacks on infrastructure, with Iran suffering the most severe effects [2]. In 2015, a malicious program called BlackEnergy affected multiple substations in the Ukrainian power sector [3]. ...
Article
Full-text available
In this paper, we propose an unsupervised anomaly detection method based on the Autoencoder with Long Short-Term Memory (LSTM-Autoencoder) network and Generative Adversarial Network (GAN) to detect anomalies in industrial control system (ICS) using cyber–physical fusion features. This method improves the recall of anomaly detection and overcomes the challenges of unbalanced datasets and insufficient labeled samples in ICS. As a first step, additional network features are extracted and fused with physical features to create a cyber–physical dataset. Following this, the model is trained using normal data to ensure that it can properly reconstruct the normal data. In the testing phase, samples with unknown labels are used as inputs to the model. The model will output an anomaly score for each sample, and whether a sample is anomalous depends on whether the anomaly score exceeds the threshold. Whether using supervised or unsupervised algorithms, experimentation has shown that (1) cyber–physical fusion features can significantly improve the performance of anomaly detection algorithms; (2) the proposed method outperforms several other unsupervised anomaly detection methods in terms of accuracy, recall, and F1 score; (3) the proposed method can detect the majority of anomalous events with a low false negative rate.
Article
Machine learning (ML)-based intrusion detection system (IDS) approaches have been significantly applied and advanced the state-of-the-art system security and defense mechanisms. In smart grid computing environments, security threats have been significantly increased as shared networks are commonly used, along with the associated vulnerabilities. However, compared to other network environments, ML-based IDS research in a smart grid is relatively unexplored although the smart grid environment is facing serious security threats due to its unique environmental vulnerabilities. In this paper, we conducted an extensive survey on ML-based IDS in smart grids based on the following key aspects: (1) The applications of the ML-based IDS in transmission and distribution side power components of a smart power grid by addressing its security vulnerabilities; (2) dataset generation process and its usage in applying ML-based IDSs in the smart grid; (3) a wide range of ML-based IDSs used by the surveyed papers in the smart grid environment; (4) metrics, complexity analysis, and evaluation testbeds of the IDSs applied in the smart grid; and (5) lessons learned, insights, and future research directions.
Article
In the rough signal processing of AC intelligent sensor, the effective value and initial phase of voltage/current determine the test accuracy. To improve the harmonic detection and compensation performance of the existing APF and promote the improvement of power grid power quality. The direct positioning method is used as the comparison method, and the error LMS method is proposed to obtain and test the voltage and current signals of intelligent sensors. The simulation results of error LMS method show that the accuracy of voltage RMS and initial phase value calculated by method 1 increases with the increase of the number of sampling points, while the accuracy of voltage RMS of method 2 and method 3 does not change significantly. The results of correlation analysis method show that the test accuracy of the proposed method is 1/2–1/3 of the direct definition method when the amplitude of interference noise signal is 5%, 10% and 15%. Compared with the direct definition method, the rough signal processing technology has lower sampling amount and higher test accuracy, which helps to simplify the system and save the overhead cost.
Article
Detecting a large number of attack classes accurately applying machine learning (ML) and deep learning (DL) techniques depends on the number of representative samples available for each attack class. In most cases, the data samples are highly imbalanced that results in a biased intrusion detection model towards the majority classes. Under-sampling, over-sampling and SMOTE are some techniques among the solutions that turn the imbalanced dataset to balanced one. These techniques have not had much impact on the improvement of detection accuracy. To deal with this problem, this paper proposes a Wasserstein Conditional Generative Adversarial Network (WCGAN) combined with an XGBoost Classifier. Gradient penalty along with the WCGAN is used for stable learning of the model. The proposed model is evaluated with some other GAN models (i.e., standard/vanilla GAN, Conditional GAN) which shows the significance of applying WCGAN in this paper. The loss on generated and real data shows a similar pattern and is lower for the Wasserstein variants of GAN compared to the other variants of the GAN model. The performance is benchmarked on three datasets NSL-KDD, UNSW-NB15 and BoT-IoT. The comparison of performance metrics before and after using the proposed framework with XGBoost classifier shows improvement in terms of higher precision, recall and F-1 score. However, comparatively less improvement is observed in FAR compared to other classifiers such as Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM). The proposed work is also compared with a recent similar technique called DGM, which uses conditional GAN along with different ML classification models. The performance of the proposed model outperforms DGM. The proposed model creates a significant footprint (or, attack signatures) to tackle with the problem of data-imbalance during the design of the Intrusion Detection System (IDS).
Article
We propose a unified machine learning (ML) framework for simultaneously performing electrical load forecasting and unsupervised anomaly detection in real time. For load forecasting, we introduce a training data generator (TDG) and a look-back optimizer (LBO) to enhance the performance of a baseline ML-driven prediction approach. The proposed TDG selects a subset of the entire historical dataset of power consumption to derive a relatively small training dataset that is used for forecasting using ML. It is designed to operate on raw meter data, without requiring any input data conditioning or additional information (e.g., weather, building occupancy), thus providing a data-efficient training approach. The proposed LBO optimizes prediction performance by tracking the accuracy of prior forecasts and automatically adjusting the framework to improve prediction outcomes. For the problem of anomaly detection, the proposed framework is unsupervised and determines abnormality by comparing fluctuations in the current load with power consumption changes in the training dataset. Therefore, in addition to not requiring any data-labeling, it circumvents the problem of imbalanced classes associated with ML-based anomaly detection methods. The proposed framework is evaluated using a real-world power consumption dataset from a large US-based university campus with considerable power usage complexity. Twelve different evaluation metrics are used to comprehensively assess the effectiveness of our approach. Our evaluation results show that compared with alternative methods, the proposed framework consistently achieves superior outcomes with regard to both load forecasting and anomaly detection, representing a promising approach to electrical load forecasting and anomaly detection in practice.
Conference Paper
Full-text available
Supervisory Control and Data Acquisition (SCADA) systems play a significant role in Critical Infrastructures (CIs) since they monitor and control the automation processes of the industrial equipment. However, SCADA relies on vulnerable communication protocols without any cybersecurity mechanism, thereby making it possible to endanger the overall operation of the CI. In this paper, we focus on the Modbus/TCP protocol, which is commonly utilised in many CIs and especially in the electrical grid. In particular, our contribution is twofold. First, we study and enhance the cyberattacks provided by the Smod pen-testing tool. Second, we introduce an anomaly-based Intrusion Detection System (IDS) capable of detecting Denial of Service (DoS) cyberattacks related to Modbus/TCP. The efficacy of the proposed IDS is demonstrated by utilising real data stemming from a hydropower plant. The accuracy and the F1 score of the proposed IDS reach 81% and 77% respectively.
Conference Paper
Full-text available
The transformation of the conventional electricity grid into a new paradigm called smart grid demands the appropriate cybersecurity solutions. In this paper, we focus on the security of the IEC 60870-5-104 (IEC-104) protocol which is commonly used by Supervisory Control and Data Acquisition (SCADA) systems in the energy domain. In particular, after investigating its security issues, we provide a multivariate Intrusion Detection System (IDS) which adopts both access control and outlier detection mechanisms in order to detect timely possible anomalies against IEC-104. The efficiency of the proposed IDS is reflected by the Accuracy and F1 metrics that reach 98% and 87%, respectively.
Article
Full-text available
The advent of the Smart Grid (SG) raises severe cybersecurity risks that can lead to devastating consequences. In this paper, we present a novel anomaly-based Intrusion Detection System (IDS), called ARIES (smArt gRid Intrusion dEtection System), which is capable of protecting efficiently SG communications. ARIES combines three detection layers that are devoted to recognising possible cyberattacks and anomalies against a) network flows, b) Modbus/Transmission Control Protocol (TCP) packets and c) operational data. Each detection layer relies on a Machine Learning (ML) model trained using data originating from a power plant. In particular, the first layer (network flow-based detection) performs a supervised multiclass classification, recognising Denial of Service (DoS), brute force attacks, port scanning attacks and bots. The second layer (packet-based detection) detects possible anomalies related to the Modbus packets, while the third layer (operational data based detection) monitors and identifies anomalies upon operational data (i.e., time series electricity measurements). By emphasising on the third layer, the ARIES Generative Adversarial Network (ARIES GAN) with novel error minimisation functions was developed, considering mainly the reconstruction difference. Moreover, a novel reformed conditional input was suggested, consisting of random noise and the signal features at any given time instance. Based on the evaluation analysis, the proposed GAN network overcomes the efficacy of conventional ML methods in terms of Accuracy and the F1 score.
Conference Paper
Full-text available
In this paper, an Intrusion Detection and Prevention System (IDPS) for the Distributed Network Protocol 3 (DNP3) Supervisory Control and Data Acquisition (SCADA) systems is presented. The proposed IDPS is called DIDEROT (Dnp3 Intrusion DetEction pReventiOn sysTem) and relies on both supervised Machine Learning (ML) and unsupervised/outlier ML detection models capable of discriminating whether a DNP3 network flow is related to a particular DNP3 cyberattack or anomaly. First, the supervised ML detection model is applied, trying to identify whether a DNP3 network flow is related to a specific DNP3 cyberattack. If the corresponding network flow is detected as normal, then the unsupervised/outlier ML anomaly detection model is activated, seeking to recognise the presence of a possible anomaly. Based on the DIDEROT detection results, the Software Defined Networking (SDN) technology is adopted in order to mitigate timely the corresponding DNP3 cyberattacks and anomalies. The performance of DIDEROT is demonstrated using real data originating from a substation environment.
Article
Full-text available
Software-Defined Networking (SDN) leverages the implementation of reliable, flexible and efficient network security mechanisms which make use of novel techniques such as artificial intelligence (AI) and machine learning (ML). In particular, these techniques -together with SDN -are the key enablers for the design of anomaly detection methods which are based on efficient traffic flow monitoring. In this paper, we tackle this problem by proposing an efficient anomaly detection framework, denoted as DeepGuard, which improves the detection performance of cyberattacks in SDN based networks by adopting a fine-grained traffic flow monitoring mechanism. Specifically, the proposed framework utilizes a deep reinforcement learning technique, i.e., Double Deep Q-Network (DDQN), to learn traffic flow matching strategies maximizing the traffic flow granularity while proactively protecting the SDN data plane from being overloaded. Afterwards, by implementing the learned optimal traffic flow matching control policy, the most beneficial traffic information for anomaly detection is acquired at runtime — thereby improving the cyberattack detection performance. The performance of the proposed framework is validated by extensive experiments, and the results show that DeepGuard yields significant performance improvements compared to existing traffic flow matching mechanisms regarding the level of traffic flow granularity. In the case of distributed denial-of-service (DDoS) attacks, DeepGuard achieves a remarkable attack detection performance while effectively preventing forwarding performance degradation in the SDN data plane.
Article
Full-text available
With the deployment of advanced information and communication technologies (ICT) the legacy power grid is being transformed as smart grid. However, the extensive use of ICT makes it vulnerable to cyberattacks. Standardization of power system communication with interoperable protocols has many benefits and at the same time the standardized semantics makes it much more vulnerable to cyberattacks. IEC has published a new standard IEC 62351 which provides the security guidelines for securing power system communication against cyber-attacks. In this paper, the cybersecurity considerations for IEC 61850 Manufacturing Message Specification (MMS) messages as per the IEC 62351-4 standard are discussed in detail. Furthermore, the implementation of IEC 62351-4 security specifications for MMS messages are demonstrated through experiments in lab.
Chapter
As an essential tool in security, the intrusion detection system bears the responsibility of the defense to network attacks performed by malicious traffic. Nowadays, with the help of machine learning algorithms, intrusion detection systems develop rapidly. However, the robustness of this system is questionable when it faces adversarial attacks. For the robustness of detection systems, more potential attack approaches are under research. In this paper, a framework of the generative adversarial networks, called IDSGAN, is proposed to generate the adversarial malicious traffic records aiming to attack intrusion detection systems by deceiving and evading the detection. Given that the internal structure and parameters of the detection system are unknown to attackers, the adversarial attack examples perform the black-box attacks against the detection system. IDSGAN leverages a generator to transform original malicious traffic records into adversarial malicious ones. A discriminator classifies traffic examples and dynamically learns the real-time black-box detection system. More significantly, the restricted modification mechanism is designed for the adversarial generation to preserve original attack functionalities of adversarial traffic records. The effectiveness of the model is indicated by attacking multiple algorithm-based detection models with different attack categories. The robustness is verified by changing the number of the modified features. A comparative experiment with adversarial attack baselines demonstrates the superiority of our model.
Article
Cybersecurity is essential to protect the tremendous increase in data stored on servers and its transmission on networks. The techniques used to detect threats and preserve data need to be updated regularly to prevent advanced attacks in the future. The information transferred over a network is converted into unintelligible data using codes and cyphers to achieve data integrity and confidentiality. At present, a firewall is used to block unknown traffic, antimalware software to detect viruses, trojan, worms and intrusion detection systems to detect attacks. The security professionals are employing Generative Adversarial Networks (GANs) to produce amazing results in fields such as Intrusion Detection, Steganography, Password Cracking, and Anomaly Generation. This paper presents a systematic literature review of GANs applications in the cybersecurity domain, including analysis of specific extended GAN frameworks and currently used stable cybersecurity datasets.
Article
Cybersecurity mainly prevents the hardware, software, and data present in the system that has an active internet connection from external attacks. Organizations mainly deploy cybersecurity for their databases and systems to prevent it from unauthorized access. Different forms of attacks like phishing, spear-phishing, a drive-by attack, a password attack, denial of service, etc. are responsible for these security problems In this survey, we analyzed and reviewed the usage of deep learning algorithms for Cybersecurity applications. Deep learning which is also known as Deep Neural Networks includes machine learning techniques that enable the network to learn from unsupervised data and solve complex problems. Here, 80 papers from 2014 to 2019 have been used and successfully analyzed. Deep learning approaches such as Convolutional Neural Network (CNN), Auto Encoder (AE), Deep Belief Network (DBN), Recurrent Neural Network (RNN), Generative Adversal Network (GAN) and Deep Reinforcement Learning (DIL) are used to categorize the papers referred. Each specific technique is effectively discussed with its algorithms, platforms, dataset, and potential benefits. The paper related to deep learning with cybersecurity is mainly published in the year 2018 in a large number and 18% of published articles originate from the UK. In addition, the papers are selected from a variety of journals, and 30% of papers used are from the Elsevier journal. From the experimental analysis, it is clear that the deep learning model improved the accuracy, scalability, reliability, and performance of the cybersecurity applications when applied in realtime.