ChapterPDF Available

Towards the Identification of Context in 5G Infrastructures

Authors:

Abstract and Figures

The evolution of communication networks, bringing the fifth generation (5G) of mobile communications in the foreground, gives the vertical industries opportunities that were not possible until now. Flexible network and computing infrastructure management can be achieved, hence bringing more freedom to the service providers, to maximize the performance of their computing resources. Still, challenges regarding the orchestration of these resources may arise. For this reason, an engine that can recognize possible factors that might affect the use of these resources and come up with solutions when needed in real-time, is required. In this paper, we present a novel Complex Event Processing engine that is enriched with Machine Learning capabilities in order to be fully adaptive to its environment, as a solution for monitoring application components deployed in 5G infrastructures. The proposed engine utilizes Incremental DBSCAN to identify the normal behavior of the deployed services and adjust the rules accordingly.
Content may be subject to copyright.
Towards the Identification of Context
in 5G Infrastructures
Chrysostomos Symvoulidis(B
), Ilias Tsoumas, and Dimosthenis Kyriazis
University of Piraeus, 18532 Piraeus, Attica, Greece
{simvoul,itsoum,dimos}@unipi.gr
Abstract. The evolution of communication networks, bringing the fifth
generation (5G) of mobile communications in the foreground, gives the
vertical industries opportunities that were not possible until now. Flexi-
ble network and computing infrastructure management can be achieved,
hence bringing more freedom to the service providers, to maximize the
performance of their computing resources. Still, challenges regarding the
orchestration of these resources may arise. For this reason, an engine that
can recognize possible factors that might affect the use of these resources
and come up with solutions when needed in real-time, is required. In
this paper, we present a novel Complex Event Processing engine that is
enriched with Machine Learning capabilities in order to be fully adaptive
to its environment, as a solution for monitoring application components
deployed in 5G infrastructures. The proposed engine utilizes Incremental
DBSCAN to identify the normal behavior of the deployed services and
adjust the rules accordingly.
Keywords: 5G ·Complex Event Processing ·
Dynamic complex event processing ·Anomaly detection ·
Event identification ·Context awareness
1 Introduction
Unlike previous generations, 5G will be more than just a mobile network. Accord-
ing to Soldani et al. [2]andMoyse[3] 5G will enable a reliable and efficient
way of deploying several services in multiple distant facilities. The utilization of
such diverse infrastructures should lead us to considerable advantages. Numer-
ous paradigms and methodologies such as Software-as-a-Service (SaaS) and
Infrastructure-as-a-Service (IaaS) can be now coupled [3], thus leading to greater
performance of the services per se. Services will be substantially more efficient,
since the service providers will have the ability to orchestrate them, even in
the cases where they are deployed in heterogeneous infrastructures. Monitoring
would present a more complete image to the service providers, showing aggre-
gated results on all their resources on all infrastructures [4,5,7].
As already mentioned, 5G is an emerging technology that will bring many
advantages both for the end-users, and the service providers, due to its increased
c
Springer Nature Switzerland AG 2019
K. Arai et al. (Eds.): CompCom 2019, AISC 998, pp. 406–418, 2019.
https://doi.org/10.1007/978-3-030-22868-2_31
Towards the Identification of Context in 5G Infrastructures 407
bandwidth, high throughput and particularly high reliability [1]. But in order to
take full advantage of a 5G infrastructure’s capabilities and its functionalities,
an intelligent way to orchestrate both the network and the infrastructure is
necessary. Anomaly detection in such diverse environments consisting of several
factors is a demanding process, hence an intelligent and interactive way to bypass
such issues is required.
Complex Event Processing (CEP) is a technology, mainly used for the analy-
sis and the process series of correlated events, using a set of techniques and tools
[10]. CEP is a perfect fit for the aforementioned issues, since it can be used as a
reactive alternative to process incoming information in order to identify complex
states and propose solutions to problems instantaneously.
Given the fact that in 5G infrastructures anomaly detection is a complex
process, common anomaly detection solutions might not work effectively. Hence,
we present a novel CEP engine for anomaly detection for such environments. This
engine focuses mainly on the identification of events that affect the deployment
and the performance of both the applications and the network per se. The engine
has been developed using the Drools Fusion framework [12], a tool created by
RedHat JBoss with respect to operating CEP jobs. The proposed approach also
includes an intelligent mechanism that can make the CEP engine more adaptive
to the current environment, via the exploitation of Machine Learning techniques,
in order to provide more customized results based on its environment every time,
thus making the CEP engine context aware [6].
The rest of the paper is organized as follows: Sect. 2focuses on the delivery
of the state of the art in the area of network anomaly focusing mainly in the new
era networks, and the CEP architectures that are currently being used. Section 3
depicts the overall architecture of the proposed engine. Section 4regards the
experimental evaluation of the engine and depicts the results. Finally, Sect. 5
concludes the paper and suggests future work that can be done based on the
current engine.
2 Related Work
This section provides an overview of the current State of the Art in the CEP
and Anomaly Detection areas.
2.1 Complex Event Processing
As already declared, CEP is a technique that is primarily used for the identi-
fication of certain situations (events), through the process of streams of infor-
mation from various sources. Its purpose is to identify such meaningful events
and respond quickly. CEP is part of the Rule Engines area [13]. CEP can be
used along other technologies, such as Business Process Management (BPM)
or Business Activity monitoring, but its distinguished due to its agility [14]in
comparison with the rest. CEP is widely used in a great amount of information
408 C. Symvoulidis et al.
systems, including network monitoring and anomaly detection, business pro-
cess automation (BPA) and business intelligence (BI), among others. Complex
Event Processors can identify events immediately, thus speeding up the decision-
making process significantly. CEP engines can identify events from numerous
data streams that are often correlated, or possible event patterns and situations,
as well as alert when necessary and trigger actions that have to be done [11]. An
integral element of the CEP engines is the exploitation of potential incidents,
namely the events, that are in general, things that happen in a certain environ-
ment, or changes of state in a current situation. Yet, the creation of events is
not a simple process, on the ground that complex events can exist through the
combination of several individual events.
Another highly important term of the CEP engines regards the Sliding Win-
dows. The Sliding Windows are a way of grouping (correlating) complex events
together based on some factors. A widely used technique to combine such events
is done through the notion of time (e.g. a number of events are occurred in a
specified time window), or by the number of times they are occurred (e.g. an
event X has occurred Y times).
One of the first known systems in the CEP area is Aurora [15]. Aurora
supported a number of basic operations including Filtering, Resampling and
Tumbling that could be applied into streams of information in a window time,
or filters such as joins, maps and GroupBys that could be applied into tuples
of data, once a time. Borealis [15] came as the second generation of Aurora
that could operate as distributed stream processing engine. Through Borealis
dynamical revision of query results and modification of the query specifications
was achieved.
Gyllstrom et al. [16] proposed SASE+ witch was eventually developed in
the University of Massachusets, a new approach which described a temporal
sequence of events that are correlated based on their attribute values. Endler
et al. [22] presented a semantic model for streaming data as an extension of a
CEP engine in order to provide Semantic reasoning. Brenna et al. [18] in their
work presented CAYUGA, a CEP engine that uses non-finite automation to
match event patterns to queries, through an SQL-like language. Bhargavi et al.
[17], proposed an adaptive CEP engine that can update its Knowledge Base at
run-time in order to make it more dynamic and interactive. Petersen et al. [21]
in their work have presented a CEP rule alteration engine using Online Learning
techniques, thus making the engine more efficient over time, since the rules would
adapt better to the current state of the environment.
In terms of production CEP tools, WSO2 created a CEP engine called WSO2
Complex Event Processor [8,9]. In its core, the WSO2 CEP uses the Siddhi query
engine [19] which describes each event as a tuple. It provides a lot of features
including time windows, filters, joins, event patterns, event partitions and it
focuses mostly on the real-time analyzation of the identified events. A robust
and versatile known CEP engine is Esper [20]. Esper describes an attribute of
an event as a composition of other events, divides each event in fragments and
offers domain-specific information per event. It uses an extension of SQL, the
Towards the Identification of Context in 5G Infrastructures 409
Esper Query Language (EQL), which offers filters such as sliding windows and
patterns over the streams of information. Other systems that could operate CEP
processes include Cordies [38], LogCEP [37], and SPA [32], along with Apache
Flink [34,45], Apache Storm [33,44] and Apache Spark [35,43]; which regard
general purpose frameworks, that could also work as CEP engines.
2.2 Anomaly Detection
Anomaly detection is used broadly in communication networks. Depending on
the situation there are numerous ways to analyze and detect anomalies [24]; clas-
sification based, clustering-based, or using statistical methods and information
theory. Deljac et al. in [23,24] used a Bayesian network for the outage detection of
telecommunication networks. Clustering-based techniques regard unsupervised
learning engines, mainly for unlabeled data. Ahmed et al. in [25] used regular
clustering techniques to cluster the data and detect outliers to improve the accu-
racy of anomaly detection engines. Another interesting approach regards the one
presented in [26] where Ester et al. used the DBSCAN algorithm to discover noise
in databases. Statistical-oriented methods are widely used in anomaly detection,
including [27], that suggests using signal processing techniques in order to iden-
tify anomalies in IP networks. Shyu et al. [28] proposed a novel paradigm to
handle and analyze multidimensional network traffic datasets that could iden-
tify whether an instance in the dataset regarded an anomaly or not.
Yang et al. [29] presented a rule-based Intrusion Detection System (IDS)
for Supervisory Control and Data Acquisition (SCADA) networks, using an in-
depth analysis and deep packet inspection techniques. Balabine and Velednitsky
in [31] published a patent that uses Support Vector Machines (SVM) for network
traffic anomaly detection. Other than SVMs, Neural networks are also used for
such purposes, like the one presented by Poojitha et al. in [30] where they tried
to detect anomalies in a given dataset with labeled information.
Taking under consideration all the aforementioned approaches, an issue that
has not been answered yet, regards the ability of a given CEP engine to be fully
adaptive to the environment it concerns and provide customized insights. For this
reason, our approach regards a CEP engine in order to combine these two areas
and provide better results in diverse ecosystems such as a 5G infrastructure.
Our approach aims at providing an engine that can be adaptive and support the
deployment of applications and services in such environments trouble-free with
customized events, taking under consideration the current conditions.
3 Overall Architecture
This section describes the architecture of the developed engine. More specifically,
it describes the mechanisms responsible for the collection of the data from the
various components, along with the way the data are fed to the CEP engine,
how this engine matches the real-time data with the rules in predefined the
Knowledge Base (KB), and finally what is our proposed solution to the creation
of an adaptive CEP engine.
410 C. Symvoulidis et al.
Fig. 1. CEP Engine overall architecture
The proposed engine is divided in six major steps which are also depicted in
Fig. 1: (i) the creation of the rules that are matched to the events by the domain
expert, (ii) the collection of the data by the various monitoring engines, (iii) the
data cleaning part along with the transformation of the data into CEP-utilizable
information, (vi) the event processing part where the streams of information are
matched to some rules stored in the KB, (v) the intelligent rules generator;
meaning the engine responsible for adapting to the environment and update the
rules accordingly, and finally (vi) the propagation of the decisions made by the
CEP engine to the interested mechanisms. These steps are described in more
detail in the sections below.
3.1 Creation of the Events
The whole process starts with the creation of the events by a domain expert,
in 5G networks and infrastructures. Events related to the performance of the
network and the resource infrastructures. Most of these events have to do with
the resource usage of the computing resources and the network usage. These
events are then “translated” into rules and are used for the identification of the
events by the engine. This translation regards the creation of rules based on the
decisions of the domain expert.
Towards the Identification of Context in 5G Infrastructures 411
3.2 Data Collection
The process continues with the collection of the data by the various monitoring
engines. The architecture suggests using Netdata [39] probes in each application
component, that collect information related to the usage of the machine, includ-
ing CPU, Memory and Disk usage percentage, currently running applications,
system up-time, IPv4/IPv6 received, sent and aborted packets and Kbps, etc.
These application components regard Virtual Machines, Containers, or Hard-
ware resources.
As depicted in Fig. 1, the collected data are scraped by the Prometheus mon-
itoring engine [40]. Its job is to provide aggregated results with respect to the
whole infrastructure and specific results per deployed application component, in
order for both the CEP engine and the Intelligent rules generator to have an
overall image of the current state of the facility.
Additionally, data are also collected by the Consul [41] service discovery
mechanism. Its purpose is to collect valuable information related to the services,
including health checking, service communication, check whether the APIs of
the services are functional, etc.
All the data that are collected by both the Prometheus and Consul services
are then sent to a Kafka Bus [42], as the proposed Publish/Subscribe (Pub-
Sub) platform. From there anyone that is interested in retrieving that data, can
subscribe to each topic and collect them.
3.3 Data Cleaning
As soon as the data are collected from the Prometheus engine and the Con-
sul mechanism from the input adapters of the CEP engine, the data cleaning
process is instantiated. This process regards mainly the data collected from the
Prometheus system. Netdata and Prometheus collect a great number of infor-
mation that only a just a small amount of data are used by the CEP engine. In
addition, the data are received in a form that the CEP engine cannot handle, so
they are transformed into CEP engine - utilizable information, and are finally
fed to the engine.
3.4 Complex Event Processing
After the collection of the data collection and data cleaning processes are com-
plete, the CEP engine is now able to initiate the event identification process.
Drools Fusion [12] is the preferred CEP engine for the reason that it has sev-
eral advantages on the way the events are handled, its ease of creation rules and
has numerous useful features, including Sliding Time and Length Windows, sup-
port of both Stream and Batch Processing and several ways of grouping events
together apart from Sliding Windows.
Before the collection of the data, the engine collects the rules from the KB.
These rules are the representation of the events and are in the form of WHEN
... THEN ..., as also presented in Fig. 2.
412 C. Symvoulidis et al.
After the streaming data are collected and cleaned they are forwarded to the
CEP engine. Here starts the process of the identification of complex events. The
engine collects the rules from the KB and when the incoming data match a rule,
that specific rule gets triggered.
Fig. 2. Rules file example
3.5 Personalized Events on the Fly
This is the most important part of the engine. The proposed engine includes a
dual mode of operations regarding the update of the KB.
The first mode regards the Rules variation engine. Its purpose is to check
the numerical difference between the streaming data and the expressions in the
rules (i.e the expression in the WHEN part, Fig. 2, that triggers the rule). If
that difference is greater that a given amount, the rules are updated in order to
reduce that void, in cases where it is considered necessary. Also, through this
engine, the domain expert can update the KB personally, by modifying, adding,
or deleting rules without the engine having to shut-down or restart. Then a
listener notifies the inference engine, in order to act based on the updated KB.
The second mode, regards a more sophisticated way of updating the KB in
a more personalized manner, with respect to the deployed application graphs,
using Machine Learning techniques. The intention of this module, is to identify
the normal behavior of the deployed applications and services, based on the
data streams provided by the monitoring and the service discovery mechanisms,
in order to create customized rules. In such diverse environments like a 5G
infrastructure, even the domain experts cannot be fully aware of the current state
Towards the Identification of Context in 5G Infrastructures 413
of the infrastructure, so the events are more generalized. Hence, the existence of
a mechanism that can adapt on the current state of the environment is necessary.
Therefore, we developed an engine that uses Incremental Density Based
Spatial Clustering of Applications with Noise (Incremental DBSCAN) [36], an
enhancement of the known DBSCAN algorithm that can work with continuously
incoming data.
Using DBSCAN the engine can identify the normal behavior of the applica-
tion graph and the corresponding application components and adjust the rules
accordingly, and group it into clusters. In more detail, the Intelligent rules gen-
erator component, as depicted in Fig. 1, is comprised of two major components;
the Cluster creator and the Rule generator.
The Cluster creator collects the data from the Kafka bus and with the use
of Incremental DBSCAN, it creates clusters that present the behavior of the
networking and computing resources of the application components. It then dis-
covers the limits of the major cluster that regards the normal behavior of the
component. The cluster that is considered to be the one that depicts the normal
behavior is the one with the most core elements in it.
The limits of the clusters are then forwarded to the Rule Generator. Its job
is to update the rules in the KB the same way the Rules Variation engine does,
but it is now based on the outcomes of the Cluster creator. After the update of
the KB is complete, the listener notifies the CEP engine so that it can now act
based on the updated KB.
3.6 Propagation of the Decisions Made by the CEP Engine
When an event is identified, a rule is triggered. After such a rule is triggered,
the process of the communication of the outcomes begins. The outcome of every
rule which is triggered is forwarded to the Kafka Bus that is described above and
the corresponding services and engines that are subscribed to the topic related
to the CEP, in order to get notified.
4 Experimental Evaluation
This section presents the experiments that the developed CEP is put under.
Due to the absence of a real 5G infrastructure, an experiment was executed in
a comparable environment.
4.1 Working Environments
For the testing of the developed CEP engine we used two Virtual Machines
(VM) that are comprised of 2 vCPUs, 8 GB of memory and 20 GB of Hard Disk.
They both ran the Ubuntu 14.04 operating system. During the test we put these
VMs under pressure using benchmarks. Another VM with the same computing
414 C. Symvoulidis et al.
Fig. 3. NumberofIdentiedeventsvs.Timetakentofirerules(ms)ina8-hperiod
resources had the responsibility to collect the monitoring data and the service
discovery information from Prometheus and Consul and forward them to the
Kafka bus.
The CEP engine, as well as the rule generator engine ran on a desktop PC
with Windows 10 operating system, with Intel dual-core i3 CPU, 8 GB of RAM
and 500 GB of Hard Disk Drive.
Fig. 4. CPU usage vs. Identified events in a 8-h period
4.2 Results
For the evaluation part, we are based on metrics that can describe the overall
performance of the engine. More specifically, these metrics regard the number of
Towards the Identification of Context in 5G Infrastructures 415
the rules that are fired, the total time taken to fire the corresponding rules from
the time the data arrived in the engine, and if that changed when the amount
of the rules triggered was increased.
The CEP engine presented indeed the current situation of the resources with
minimum delay, since the events were being identified in less than 300 ms from
the moment the data were received in the engine, regardless of the number of the
number of the rules that were being triggered at the same time, as also depicted
in Fig. 3.
Fig. 5. Clusters created by the Intelligent rules generator engine
In Fig. 4the number of the rules that were being triggered is depicted, in
comparison to the CPU usage. At first, the number of the identified events was
quite large. This was due to the fact that the rules that were used at first were the
predefined ones. After the Rule generator engine was activated, the number of
the identified events dropped significantly. Still though, events that were essential
and described a change of state in the environment were identified.
Figure 5presents the clusters that were created by the DBSCAN algorithm.
The cluster with the orange color was identified as the normal behavior of the
deployed VMs. The limits of the cluster (i.e minimum CPU, Memory, Disk usage,
etc.) were forwarded to the Rule generator, in order to update the KB. When
the process was complete the CEP engine identified as an event anything outside
these boundaries. This led to the creation of events that were not general, but
were adapted to the specific situation.
416 C. Symvoulidis et al.
5 Conclusion and Future Work
The identification of factors that affect the use of such diverse infrastructures and
networks, such as 5G is indeed a difficult and complex process. In this paper, we
have presented a novel CEP engine that can overcome issues, when dealing with
multidimensional data and be considered a trustworthy alternative for a more
interactive network anomaly detection. The use of the Incremental DBSCAN
algorithm makes the current engine more adaptive and the events identified are
customized to the current situation.
It is within our future plans to extend the present approach, by making
the Incremental DBSCAN more precise and continue our research in Machine
Learning and CEP engines in order to make the engine more generic and flexible
to changes in its environment, thus being capable of providing insights to other
areas of interest including Cloud computing.
Acknowledgment. This work has received funding from the European Unions Hori-
zon 2020 research and innovation program under grant agreement No 761898 project
MATILDA.
References
1. Suriano, D.: The Future of Networking Is 5G: Businesses Must Prepare Now, 23
September 2018. https://www.forbes.com/sites/oracle/2018/09/24/the-future-of-
networking-is-5g-businesses-must-prepare-now
2. Soldani, D., Manzalini, A.: A 5G infrastructure for “anything-as-a-service”. J.
Telecommun. Syst. Manag. 3(2), 1 (2014)
3. Moyse, I.: 5 Reasons for a Multi-Cloud Infrastructure—Dyn Blog (n.d.). https://
dyn.com/blog/5-reasons-for-a- multi-cloud- infrastructure/
4. Uittenbogaard, T.: What are the advantages of a multi-site/multi-domain solu-
tion and who’s to benefit from it? (2015). https://oneshoe.com/news/what-are-
advantages-multi-sitemulti-domain-solution-and-whos- benefit- it
5. Hume, A.C., Al-Hazmi, Y., Belter, B., Campowsky, K., Carril, L. M., Carrozzo, G.,
Engen, V., Garcia-Perez, D., Ponsati, J.J., Kubert, R., Rohr, C., van Seghbroeck,
G., Liang, Y.: Bonfire: a multi-cloud test facility for internet of services experimen-
tation. In: International Conference on Testbeds and Research Infrastructures, pp.
81–96. Springer, Heidelberg (2012)
6. Schilit, B., Adams N., Want R.: Context-aware computing applications. In: Work-
shop on Mobile Computing Systems and Applications, pp. 85–90. IEEE, Santa
Cruz (1994)
7. Baldin, I., Chase, J., Xin, Y., Mandal, A., Ruth, P., Castillo, C., Orlikowski,
V., Heermann, C., Mills, J.: ExoGENI: a multi-domain infrastructure-as-a-service
testbed. In: The GENI Book, pp. 279–315. Springer, Cham (2016)
8. Perera, S., Sriskandarajah, S., Vivekanandalingam, M., Fremantle, P.,
Weerawarana, S.: Solving the grand challenge using an opensource CEP
engine. In: Proceedings of the 8th ACM International Conference on Distributed
Event-Based Systems, pp. 288–293. ACM, May 2014
9. WSO2 Complex Event Processor. https://wso2.com/products/complex-event-
processor
Towards the Identification of Context in 5G Infrastructures 417
10. Luckham, D.: The Power of Events, vol. 204. Addison-Wesley, Reading (2002)
11. Garcia, J.: A complex event processing system for monitoring manufacturing sys-
tems. Tampere University of Technology (2012). https://dspace.cc.tut.fi/dpub/
bitstream/handle/123456789/20958/garcia izaguirre.pdf
12. Drools Fusion 6.2.0 documentation. https://docs.jboss.org/drools/release/6.2.0.
CR3/drools-docs/html/
13. Fulop, L.J., Toth, G., Rcz, R., Panczel, J., Gergely, T., Beszedes, A., Farkas, L.:
Survey on complex event processing and predictive analytics. In: Proceedings of
the Fifth Balkan Conference in Informatics, pp. 26–31, July 2010
14. Saboor, M., Rengasamy, R.: Designing and developing complex event processing
applications. Sapient Global Markets (2013)
15. Carney, D., Cetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G.,
Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring streams: a new class of data
management applications. In: Proceedings of the 28th International Conference on
Very Large Data Bases, pp. 215–226. VLDB Endowment, August 2002
16. Diao, Y., Immerman, N., Gyllstrom, D.: SASE+: an agile language for kleene
closure over event streams. UMass Technical Report (2007)
17. Bhargavi, R., Pathak, R., Vaidehi, V.: Dynamic complex event processing—
adaptive rule engine. In: 2013 International Conference on Recent Trends in Infor-
mation Technology (ICRTIT), pp. 189–194. IEEE, July 2013
18. Brenna, L., Gehrke, J., Hong, M., Johansen, D.: Distributed event stream pro-
cessing with non-deterministic finite automata. In: Proceedings of the Third ACM
International Conference on Distributed Event-Based Systems, p. 3. ACM, July
2009
19. Suhothayan, S., Gajasinghe, K., Loku Narangoda, I., Chaturanga, S., Perera, S.,
Nanayakkara, V.: Siddhi: a second look at complex event processing architectures.
In: Proceedings of the 2011 ACM Workshop on Gateway Computing Environments,
pp. 43–50. ACM, November 2011
20. ETES Intelligence - Esper and NEsper: Where Complex Event Processing meets
Open Source. Esper & NEsper, 2, 2006-2013 (2006)
21. Petersen, E., To, M.A., Maag, S.: An online learning based approach for CEP rule
generation. In: 2016 8th IEEE Latin-American Conference on Communications
(LATINCOM), pp. 1–6. IEEE, November 2016
22. Endler, M., Briot, J.P., e Silva, F.S., de Almeida, V.P., Haeusler, E.H.: Towards
stream-based reasoning and machine learning for IoT applications. In: Intelligent
Systems Conference (IntelliSys), pp. 202–209. IEEE, September 2017
23. Deljac, Z., Randic, M., Krcelic, G.: Early detection of network element outages
based on customer trouble calls. Decis. Support Syst. 73, 57–73 (2015)
24. Ahmed, M., Mahmood, A.N., Hu, J.: A survey of network anomaly detection tech-
niques. J. Netw. Comput. Appl. 60, 19–31 (2016)
25. Ahmed, M., Mahmood, A.N.: A novel approach for outlier detection and clus-
tering improvement. In: 2013 8th IEEE conference on Industrial electronics and
applications (ICIEA), pp. 577–582. IEEE, June 2013
26. Ester, M., Kriegel, H. P., Sander, J., Xu, X.: A density-based algorithm for discov-
ering clusters in large spatial databases with noise. In: KDD, vol. 96, no. 34, pp.
226–231, August 1996
27. Thottan, M., Ji, C.: Anomaly detection in IP networks. IEEE Trans. Signal Pro-
cess. 51(8), 2191–2204 (2003)
28. Shyu, M.L., Chen, S.C., Sarinnapakorn, K., Chang, L.: A novel anomaly detec-
tion scheme based on principal component classifier. Department of Electrical and
Computer Engineering, Miami University Coral Gables, FL (2003)
418 C. Symvoulidis et al.
29. Yang, Y., McLaughlin, K., Littler, T., Sezer, S., Wang, H.F.: Rule-based intrusion
detection system for SCADA networks (2013)
30. Poojitha, G., Kumar, K.N., Reddy, P.J.: Intrusion detection using artificial neural
network. In: 2010 International Conference on Computing Communication and
Networking Technologies (ICCCNT), pp. 1–7. IEEE, July 2010
31. Balabine, I., Velednitsky, A.: U.S. Patent No. 9,843,488. U.S. Patent and Trade-
mark Office Washington, DC (2017)
32. Dijkman, R., Peters, S., ter Hofstede, A.: A toolkit for streaming process data anal-
ysis. In: 2016 IEEE 20th International Enterprise Distributed Object Computing
Workshop (EDOCW), pp. 1–9. IEEE, September 2016
33. Gaunitz, B., Roth, M., Franczyk, B.: Dynamic and scalable real-time analytics in
logistics combining Apache Storm with complex event processing for enabling new
business models in logistics. In: 2015 International Conference on Evaluation of
Novel Approaches to Software Engineering (ENASE), pp. 289–294. IEEE, April
2015
34. Bansod, R., Kadarkar, S., Virk, R., Raval, M., Rashinkar, R., Nambiar, M.: High
performance distributed in-memory architectures for trade surveillance system.
In: 2018 17th International Symposium on Parallel and Distributed Computing
(ISPDC), pp. 101–108. IEEE, June 2018
35. Liu, G., Zhu, W., Saunders, C., Gao, F., Yu, Y.: Real-time complex event process-
ing and analytics for smart grid. Proc. Comput. Sci. 61, 113–119 (2015)
36. Chakraborty, S., Nagwani, N.K.: Analysis and study of Incremental DBSCAN
clustering algorithm. arXiv preprint arXiv:1406.4754 (2014)
37. Cao, J., Wei, X., Liu, Y.Q., Mao, D., Cai, Q.: LogCEP-Complex event processing
based on pushdown automaton. Int. J. Hybrid Inform. Technol. 7(6), 71–82 (2014)
38. Koch, G. G., Koldehofe, B., Rothermel, K.: Cordies: expressive event correlation in
distributed systems. In: Proceedings of the Fourth ACM International Conference
on Distributed Event-Based Systems, pp. 26–37. ACM, July 2010
39. NetData. https://my-netdata.io/
40. Prometheus monitoring system. https://prometheus.io/
41. Consul by HashiCorp. https://consul.io/
42. Apache Kafka. https://apache.kafka.org/
43. Apache Spark: Lightning-fast cluster computing. http://spark.apache.org
44. Marz, N.: Storm: distributed and fault-tolerant realtime computation (2013).
https://www.infoq.com/presentations/Storm-Introduction. Accessed 21 Oct 2011
45. Flink, A.: Scalable batch and stream data processing (2016). https://flink.apache.
org
... There are innumerable non-monolithic applications that run in a distributed manner. For the proper deployment of these distributed applications in the virtual environments of 5G networks, it is crucial to capture their computing and networking requirements, as well as to monitor their components at runtime and adapt to any environment changes may appear [17] in order to make possible their consistent instantiation inside of 5G. Thus, the applications will be ready to be deployed in 5G environments, following and extending the cloud-native rules of the containerization, the dynamic orchestration, and the micro-services orientation [12]. ...
Chapter
Full-text available
The new-coming 5G network is considered to be one of the most significant innovations today. This is due to the opportunities that is going to provide to the vertical industries. 5G infrastructures will introduce a new way for low-delay, reliable deployment of services. In fact, such infrastructures can be used for the placement of application services in the form of application graphs. An application graph consists of several application components (i.e. micro-services) that may be hosted in the same infrastructure or in different ones. Conflicting requirements that arise when deploying in such infrastructures are now handled through network slicing, which regards a way for partitioning conventional network and computing resources into virtual elements. In this paper, we define a universal application metamodel of a 5G compatible application in order to guarantee the annotation of each application descriptor with its proper requirements for their fulfillment at the instantiation time. In terms of application architecture, we consider each application graph as a service mesh topology in order to adopt this novel service architecture as a dominant methodology that is well fitting in the promising 5G capabilities.
Article
Full-text available
Electric power networks are among the world's most complex human-made systems. The developing smart grid is an inherently complex system which is rapidly evolving in both definition and implementation. Deployment of advanced technologies within the electric utility sector and usage of state-of-the-art computing systems provides companies with innovative capabilities to forecast electricity demand, influence customer usage patterns, create demand response program, optimize unit commitment, and prevent power outages. At the same time, these advances also lead to the generation of unprecedented data volumes, high data communications requirements, and increased system complexity. Utility companies must be capable of high-volume, high-speed data management and advanced analytics which are designed to transform data into actionable insights, if they strive to successfully implement a modern smart grid. As smart grid operations will leverage Advanced Metering Infrastructure (AMI) to drive more real time decision making and operational activities, complex event processing and stream computing are needed for the modern smart grid. This paper explores the challenges and benefits of transitioning to a smart grid, and explores new architectural approaches built on Lambda Architecture and other emerging software standards which may more effectively leverage established forms of complex event processing.
Chapter
This chapter describes ExoGENI, a multi-domain testbed infrastructure built using the ORCA control framework. ExoGENI links GENI to two advances in virtual infrastructure (IaaS) services outside of GENI: open cloud computing (OpenStack) and dynamic circuit fabrics. It orchestrates a federation of independent cloud sites and circuit providers through their native IaaS interfaces, and links them to other GENI tools and resources. ExoGENI slivers are instances of basic IaaS resources: variously sized virtual machines, bare-metal nodes, iSCSI block storage volumes, and Layer 2 network links with optional OpenFlow control. ExoGENI offers a powerful unified hosting platform for deeply networked, multi-domain, multi-site cloud applications. ExoGENI operates its own stitching engine and Layer 2 (L2) network exchanges that work in concert to interconnect the sites with dynamic point-to-point and multi-point L2 links via multiple circuit providers. It also supports stitchports—named attachment points enabling direct L2 connections to resources outside the system’s control.ExoGENI is seeding a larger, evolving platform linking third-party cloud sites, transport networks, new resource types, and other infrastructure services. It facilitates real-world deployment of innovative distributed services, leading to a new vision of a future federated, more resilient, and deeply networked cyber-infrastructure. This chapter explores the unique features of ExoGENI and, in particular, how it differs from other GENI testbeds.
Conference Paper
In this paper we present an approach for an information system which is capable of processing and analysing vast amounts of data. In addition to Big Data solutions we do not focus on ex post batch processing but on online stream processing. We use Apache Storm in combination with Complex Event Processing to provide a scalable and dynamic event-driven information system, providing logistics businesses with relevant information in real-time to increase their data and process transparency
Article
This paper deals with the issue of early detection of network element outages. Timeliness of outage detection as well as accuracy in finding outages on equipment in a telecommunication network depend on the monitoring system used and its performance. The intent of this paper is to investigate and propose a complementary solution to improve performance of the existing systems in detecting faults earlier than it was able to do before. In developing our approach two constraints are given. The existing operational environment cannot be changed, threshold tuning and parameter changing cannot be done, furthermore no additional infrastructure investment has been planned. Hence, our approach relies on an alternative method based on a two-stage hybrid statistical and diagnostic detector which we designed in a way that exploits additional available data and avoids alarm monitoring system imperfections. The role of this detector is twofold: early detection of network element outages based on customer trouble calls and rule-based decision making for faulty-element isolation based on knowledge derived from fault and network management data. In this paper we present results of statistical analysis of trouble-reporting data. The analysis showed that timing of customers' trouble reports and their content have information potential that can be utilized for early detection of outages. The detector is explained in detail and its accuracy and reduction delay is evaluated. The method presented can reduce the outage detection delay time by 2.33 hours on average observed in relation to the performance of an existing fault management process which was designed to detect outages solely on the basis of an alarm monitoring system, for the "difficulties in work" type of malfunction. We attained an overall probability of correct detection of 95.3%. Out of the total number of outages that hypothetically could be detected, by using this method we were able to detect 77.5% of cases one hour before the alarm was raised in the existing alarm system, while 23% of cases were detected four hours before the actual alarm. The approach has been tested on real telecommunication network data over the period of one year.