Conference PaperPDF Available

TRAFFIC ANOMALY DETECTION WITH SNORT

Authors:

Abstract and Figures

Snort is open source intrusion detection system based on signature detection. In the paper we present information about the second version of anomalydetection – preprocessor designed to log and analyse network traffic information. We also collect network traffic information from a few local area networks and made a few simply traffic statistical analysis which could be usefull to anomalies detection.
Content may be subject to copyright.
Maciej Szmit, Radosław Wężyk, Maciej Skowroński, Anna Szmit: "Traffic Anomaly Detection
with Snort", [in:] Information Systems Architecture and Technology. Information Systems
and Computer Communication Networks, Wydawnictwo Politechniki Wrocławskiej,
Wrocław 2007 ISBN 978-83-7493-348-3
Maciej SZMIT*
Radosław WĘŻYK**
Maciej SKOWROŃSKI**
Anna SZMIT***
TRAFFIC ANOMALY DETECTION WITH SNORT
Snort is open source intrusion detection system based on signature detection. In the paper we pre-
sent information about the second version of anomalydetection preprocessor designed to log and
analyze network traffic information. We also collect network traffic information from a few local area
networks and made a few simply traffic statistical analysis which could be usefull to anomalies detec-
tion.
1. INTRODUCTION
Snort is open source network intrusion detection and prevention system (IDS and
IPS) utilizing a rule driven language, which possibility of use signature, protocol and
anomaly based inspection methods. Snort authors said that it is the most widely de-
ployed intrusion detection and prevention technology worldwide and has become the de
facto standard for the industry [1]. The most popular form of using Snort is to build an
Network based Intrusion Detection System (NIDS) based on standard signatures data-
base or build an IPS by adding active response program, like Guardian [2], which works
in conjunction with Snort to automatically update firewall (usually iptables [3]) rules
based on alerts generated by Snort, but there are a lot of other possibilities: using Snort
in inline mode [4], using Snort in bridge mode computer (which can be good idea from
security point of view because of prevention from attacks against IDS machine) or using
Snort like a sniffer etc. (see: [1], [5], [6]).
Because of Snort popularity there are a lot of accessories – beginning from front ends
– visualization and analysis tools (like ACID - Analysis Console for Intrusion Databases
[7] or BASE – Basic Analysis and Security Engine [8]), through experimental preproc-
__________
* Computer Engineering Department, Technical University of Lodz, Maciej@szmit.info
** Computer Engineering Department, Technical University of Lodz
** Computer Engineering Department, Technical University of Lodz
*** Faculty of Organization and Management, Technical University of Lodz
essors (like Snort+AI [10] or SPADE Statistical Packet Anomaly Detection Engine
[11]), finishing on dedicated hacker tools like Snot – a Snort alert generator and general
NIDS decoy utility [12].
2. ANOMALYDETECTION PREPROCESSOR
Our project Anomalydetection [13] is a simply preprocessor designed to log and
analyze information about network traffic and detect possibly network traffic anomalies.
You can find more information about idea and program data structures in our articles
[14] and [15], in thesis [6] and in project page [13]. Basically: Anomalydetection logs
25 parameters of network traffic:
1) number of TCP packets,
2) number of outgoing TCP packets,
3) number of incoming TCP packets,
4) number of TCP packets in its own subnet (LAN),
5) number of UDP datagrams,
6) number of outgoing UDP datagrams,
7) number of incoming UDP datagrams,
8) number of UDP datagrams in its own subnet (LAN),
9) number of ICMP messages,
10) number of outgoing ICMP messages,
11) number of incoming ICMP messages,
12) number of ICMP messages in its own subnet (LAN),
13) number of TCP packets with set SYN and ACK flags,
14) number of outgoing TCP packet send on port 80 (WWW),
15) number of incoming TCP packet from port 80 (WWW),
16) number of outgoing UDP datagrams send on port 53 (DNS),
17) number of incoming UDP datagrams from port 53 (DNS),
18) outgoing IP bitrate [kBps],
19) incoming IP bitrate [kBps],
20) outgoing TCP port 80 bitrate [kBps],
21) incoming TCP port 80 bitrate [kBps],
22) outgoing UDP bitrate [kBps],
23) incoming UDP bitrate [kBps],
24) outgoing UDP port 53 bitrate [kBps],
25) incoming UDP port 53 bitrate [kBps].
After a period of working in collecting data mode, anomalydetection builds “network
profile” which contains information about mean and variation of each parameter. In
anomaly detection mode the current version of our preprocessor generates alert when
one of the parameter exceed value meaning plus (or minus) X standard deviation, where
X is parameter setting by administrator.
The second version of anomalydetection, which will be released soon (a beta release
is available on project pages) will also detect ARP-spoofing attack. This attack (and a
few similar like DNS-spoofing) can be easy detected by comparation numbers of request
with number of answers. When you receive more than one ARP-reply with different
content you can expect that you are under attack.
In our research we collected data from two campus networks (Net A and Net C on
figures below). The first was rather small (about 20 computers), the second was larger
(more than 400 computers). We tried to make simply statistical analysis of these time
series to recognize their characteristics and in the future – implement additional alert
generating mechanism in our preprocessor.
3. NETWORK TRAFFIC – A STATISTICAL OVERVIEW
Because the both of networks were similar, we expected day and eventually week
seasonality (see [6], [14]). The first parameter which we investigate was number of TCP
packets which usually should be about 80% of network traffic (see. figure 1). On figure
2 and figure 3 you can see average overall TCP traffic in each of the networks in each
day of the week. As we could expect, each of the two networks has different characters
and the Network C with smallest variation and more regular traffic seems to be better for
statistical analysis.
Overall TCP traffic [packets per hour]
0
500000
1000000
1500000
2000000
2500000
1460919137818372296275532143673
Net C
Net A
Fig. 1. Overall TCP traffic in the Net A and the Net C
Network A average overall TCP traffic
[packet per hour]
0
50000
100000
150000
200000
250000
300000
350000
00:00
02:00
04:00
06:00
08:00
10:00
12:00
14:00
16:00
18:00
20:00
22:00
Std. Dev. Mo
Tu We
Th Fr
Sa
Su
Fig.2. The Net A average overall TCP traffic
Network C average overall TCP traffic
[packet per hour]
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
00:00
02:00
04:00
06:00
08:00
10:00
12:00
14:00
16:00
18:00
20:00
22:00
Std. Dev. Mo
Tu We
Th Fr
Sa
Su
Fig.3. The Net C average overall TCP traffic
Coefficient of determination 2
R
(proportion of variation of each kind of traffic ex-
plained by hour is shown on table 1).
TCP
overall
TCP
outgoing
TCP
incoming
ICMP
incoming
TCP with
SYN/ACK
UDP port 53
incoming
47% 61% 62% 22% 65% 36%
Tab. 1. Coefficient of determination values
On figure 4 you can see average values of TCP and other protocols traffic in the
Network C in 24-hours period.
Avr. Packet Rate in 24-hour (NetworkC)
00:00
02:00
04:00
06:00
08:00
10:00
12:00
14:00
16:00
18:00
20:00
22:00
TCP ov er all
TCP ou tgoin g
TCP inc oming
ICMP in c omi ng
TCP w it SY N and ACK
UDP p or t 53 inco ming
Fig.4. Average traffic in the Net C (selected protocols)
The most interesting protocol for detecting anomalies is of course ICMP, because we
can expect high ICMP traffic when any problem or error occurs. In current version of
anomalydetection the alerts are generated based on average and standard deviation val-
ues, but it seemed to be interesting to analyze histogram of incoming ICMP distribution.
In figure 5 we presented histograms of average number of TCP packets and ICMP mes-
sages for the Network C at 6 am. and in figure 6 – the same histogram for 9 pm.
Histogram of TCP and ICMP (inc oming) in
the Network C at 6 am.
-5 -3 -2 -1 0 1 2 3 5
[number of s igma]
TCP
ICMP i nc omi ng
Fig.5. Histogram of TCP and ICMP (incoming) in the Net C at 6 am.
Histogram of TCP and ICMP (inc oming) in
the Network C at 9 pm.
-5 -3 -2 -1 0 1 2 3 5
[number of s igma]
TCP
ICMP i nc omi ng
Fig.6. Histogram of TCP and ICMP (incoming) in the Net C at 9 pm.
As you can see ICMP traffic has asymmetric distribution (or even not unimodal dis-
tribution) so it should be investigated if average and standard deviation can be used.
We are going to investigate this problem in details in future, after getting more time
series from a few anomalydetection which we have installed in a few networks.
3. CONCLUSIONS
Statistical analysis (like self-similarity analysis [20], time series analysis [21] etc.)
and Artificial Intelligence based methods (like Neural Networks [10], Genetic Algo-
rithms [16], Immunity based algorithms [17], Data Mining methods [18], Simulated
Annealing algorithms [19] etc.) of network anomalies is perceived as very interesting
and promising methods for Intruder Detection and Prevention Systems. It possibly can
detect new or unknown methods of attack, like the zero day exploits. But one should
remember that anomaly based detection (neither nor signature based detection) is not
The Silver Bullet. For example: a lot of attacks can be not recognized by network traffic
anomaly detection systems because of small amount of data which they use (even only
one packet) and a lot of false positives and false negatives could be generated by them
because of using bandwidth management methods which can falsify traffic profile. On
the other hand, traffic anomalies detection can be useful for even non technical attacks
(for example: spy, who transfer a big amount of data from secure network to external
server). So we think that this method can be used as subsidiary method in “classic”,
signature-based IDS.
REFERENCES
[1] Snort homepage http://www.snort.org (01.03.2007)
[2] Guardian homepage http://www.chaotic.org/guardian (01.03.2007)
[3] Iptables and Netfilter project pages http://www.netfilter.org (01.03.2007)
[4] Snort inline project page http://snort-inline.sourceforge.net (01.03.2007)
[5] Szmit M., Gusta M., Tomaszewski M.: 101 zabezpieczeń przed atakami w sieci komputerowej,
Helion, Gliwice 2005
[6] Skowroński M., Wężyk R.: Systemy detekcji intruzów i aktywnej odpowiedzi, praca magisterska
napisana w Katedrze Informatyki Stosowanej Politechniki Łódzkiej pod kierunkiem Macieja
Szmita, maszynopis, Łódź 2006
[7] Analysis Console for Intrusion Databases project page http://acidlab.sourceforge.net
(01.03.2007)
[8] Basic Analysis and Security Engine homepage http://base.secureideas.net (01.03.2007)
[9] Snort Setup for Statistics howto
http://www.faqs.org/docs/Linux-HOWTO/Snort-Statistics-HOWTO.html (01.03.2007)
[10] Snort+AI project page http://afrodita.unicauca.edu.co/%7Eaarboleda/snort_ai.htm (01.03.2007)
[11] SPADE CVS repository http://www.bleedingsnort.com/cgi bin/viewcvs.cgi/?cvsroot=SPADE
(01.03.2007)
[12] Copy of old (2001) Snot page
http://web.archive.org/web/20010424080846/http://www.geocities.com/sniph00/ (01.03.2007)
[13] Anomalydetection project page http://www.anomalydetection.info (01.03.2007)
[14] Skowroński M., Wężyk R., Szmit M., Detekcja anomalii ruchu sieciowego w programie Snort,
Hakin9 nr 3/2007
[15] Maciej Skowroński, Radosław Wężyk, Maciej Szmit, "Preprocesory detekcji anomalii dla
programu Snort" w: "Sieci komputerowe Tom 2. Aplikacje i zastosowania", WKŁ 2007, pp.
333-338
[16] Li W.: Using Genetic Algorithm for Network Intrusion Detection, United States Department of
Energy Cyber Security Group 2004 Training Conference, Kansas City, 2004
[17] Dasgupta D.: Immunity Based Intrusion Detection System: A General Framework, 22nd Na-
tional Information Systems Security Conference (NISSC), 1999
[18] Cichocki R.: Algorytmy indukcji reguł decyzyjnych w Systemach Wykrywania Intruzów, XII
konferencja Sieci Komputerowe , Zakopane 2005
[19] Kruk T. J., Wrzesień J.: Korelacja w wykrywaniu anomalii, Materiały konferencji CERT
Secure 2003, Warszawa 2003
[20] Kolbusz J., Lewicki A., Majdański A., Karmelita S.: Badanie samopodobieństwa ruchu w
sieciach LAN – metody i narzędzia”, Informatyka Stosowana ISSN 83-914678-6-4, VII
Lubelskie Akademickie Forum Informatyczne, Kazimierz Dolny 2003, s. 97-103.
[21] Hao Y., Chuang L. Berton S. Bo L. Geyong M., Network traffic prediction based on a new time
series model: Research Articles, International Journal of Communication Systems, Volume 18,
Issue 8, 2005
... Moreover, sFlow-RT based on traffic measurements can be modified by other tools (e.g., NetFlow [52]) to classify the specific flows. Our current system does not solve all problems in SDN, but RAD can expand with other tools or methods [53][54][55] to support new features (e.g., detection against ...
... Moreover, sFlow-RT based on traffic measurements can be modified by other tools (e.g., NetFlow [52]) to classify the specific flows. Our current system does not solve all problems in SDN, but RAD can expand with other tools or methods [53][54][55] to support new features (e.g., detection against unknown anomalies). Provisioning interfaces to other tools for IDS and flow classification in RAD is a point for future study. ...
Article
Full-text available
The main advantage of software defined networking (SDN) is that it allows intelligent control and management of networking though programmability in real time. It enables efficient utilization of network resources through traffic engineering, and offers potential attack defense methods when abnormalities arise. However, previous studies have only identified individual solutions for respective problems, instead of finding a more global solution in real time that is capable of addressing multiple situations in network status. To cover diverse network conditions, this paper presents a comprehensive reactive system for simultaneously monitoring failures, anomalies, and attacks for high availability and reliability. We design three main modules in the SDN controller for a robust and agile defense (RAD) system against network anomalies: a traffic analyzer, a traffic engineer, and a rule manager. RAD provides reactive flow rule generation to control traffic while detecting network failures, anomalies, high traffic volume (elephant flows), and attacks. The traffic analyzer identifies elephant flows, traffic anomalies, and attacks based on attack signatures and network monitoring. The traffic engineer module measures network utilization and delay in order to determine the best path for multi-dimensional routing and load balancing under any circumstances. Finally, the rule manager generates and installs a flow rule for the selected best path to control traffic. We implement the proposed RAD system based on Floodlight, an open source project for the SDN controller. We evaluate our system using simulation with and without the aforementioned RAD modules. Experimental results show that our approach is both practical and feasible, and can successfully augment an existing SDN controller in terms of agility, robustness, and efficiency, even in the face of link failures, attacks, and elephant flows.
... Standard intrusion detection systems made by network equipment developers such as Sophos (Sophos Labs), Fortinet (Fortiguard Labs), and others, store intelligent system parameters in private databases that are accessible only for prepaid users of products. Some open-source IPS providers like "Snort" store well-known anomalies detection preprocessing algorithm structures in public repositories (Szmit et al., 2007). ...
Article
Full-text available
Traffic analysis is a common question for most of the production systems in various segments of computer networks. Attacks, configuration mistakes, and other factors can cause network increased accessibility and as a result danger for data privacy. Analyzing network flow and their single packets can be helpful for anomalies detection. Well-known network equipment has predeveloped network flow monitoring software. “NetFlow” data collector software “Nfsen” is an open-source way to collect information from agents. Also “Nfsen” is designed for data sorting and dataset for instruction detection system preparation. Prepared data can be split into fragments for artificial intelligent learning and testing. As AI unit can be used multilayer perceptron developed in a python programming language. This paper focused on real-world traffic dataset collection and multilayer perceptron deployment for TCP flood traffic detection.
... Tools included in the Anomaly Detection 3.0 allows analysis of movement, its forecasting with help of its advanced statistical algorithms, evaluation of created forecasts, real-time monitoring and verifying that the individual volumes of network traffic parameters do not exceed the forecasted value and in case of exceeding the norms to generate the appropriate messages for the administrator who should check each alarm for potential threats. Current (3.0) version (see e.g. [5] [6] ) of Anomaly Detection provides monitoring of following network traffic parameters: total number of TCP, UDP, and ICMP packets,number of outgoing TCP, UDP, and ICMP packets, number of incoming TCP, UDP, and ICMP packets, number of TCP, UDP, and ICMP packets from current subnet, number of TCP packets with SYN/ACK flags, number of outgoing and incoming WWW packets – TCP on port 80, number of outgoing and incoming DNS packets – UDP outgoing on port 53, number of ARP-request and ARP-reply packets, number of non TCP/IP stacks packets, total number of packets, TCP, WWW, UDP, and DNS upload and download speed [kBps]. Whole Anomaly Detection application consists of three parts: Snorts preprocessor, Profile Generator and Profile Evaluator. ...
Conference Paper
Full-text available
This paper presents information about Anomaly-Detection - a Snort-based network traffic monitoring tool. The article concerns use of based on Holt-Winters forecasting method in real-time behavioral analysis of network traffic.
Chapter
Structured Query Language (SQL) is a common database language. SQL Slammer is so named because it exploits a vulnerability in the database and then reproduces automatically through scanning for other databases to exploit.
Conference Paper
Cyber espionage campaigns and cyber attacks make use of data exfiltration on a regular basis causing damages for billions of dollars. Nowadays, they represent one of the primary threats, and they are performed by criminals, companies and states. Normally, data exfiltration uses classic application-layer protocols (e.g. FTP or HTTP) in combination with very basic obfuscation mechanisms. Even though in most cases these techniques are effective enough, this paper describes how instead they can be detected using properly configured IDSs. Moreover, we introduce a novel approach named polymorphic blending exfiltration that serves to avoid detection from signature-based as well as anomaly-based IDSs. This technique permits to blend the exfiltrated data in the normal and legitimate traffic. We show how IDSs can be easily improved in order to be able to detect such exfiltration. Finally, we conclude presenting different evasion techniques that can be included in the polymorphic blending exfiltration to keep providing a safe undetectable exfiltration.
Article
Full-text available
Serangan atau intrusi yang masuk ke dalam sebuah sistem adalah sesuatu yang hampir pasti terjadi dalam dunia teknologi informasi saat ini. Untuk mengatasi hal tersebut ada beberapa teknologi yang dapat digunakan, seperti firewall atau sistem deteksi intrusi (intrusion detection system/IDS). Tidak seperti firewall yang hanya menyeleksi paket yang masuk berdasarkan alamat IP dan port, IDS bekerja dengan cara memantau isi paket yang masuk ke dalam sebuah komputer untuk kemudian memutuskan apakah rangkaian paket yang masuk tersebut merupakan sebuah serangan atau tidak. Salah satu aplikasi open sources dari IDS adalah Snort yang menggunakan pencocokan string untuk mengambil keputusan. Kelemahan dari IDS yang berbasis pencocokan string adalah kemunculan string dalam sebuah paket harus sama persis, sehingga sulit untuk mendeteksi serangan yang mirip tetapi memiliki pola string yang berbeda. Oleh karena itu paper ini mengusulkan sebuah metode deteksi intrusi menggunakan n-gram dan cosine similarity untuk mencari kemiripan dari serangkaian paket, sehingga yang dicari bukan yang sama persis, tetapi seberapa mirip dengan signature yang ada. Berbeda dengan Snort, paket tidak dicocokkan dengan pola serangan, tetapi dengan pola pengaksesan sebuah halaman web oleh pengguna yang sesungguhnya, sehingga yang memiliki kemiripan tinggi akan dianggap sebagai paket yang sah, sedangkan yang rendah akan dianggap sebagai serangan. Dari hasil ujicoba dengan berbagai variasi nilai ambang batas, maka didapatkan nilai 0.8 dengan n = 3 memberikan akurasi yang terbaik. Sistem deteksi intrusi ini juga mampu mendeteksi berbagai jenis serangan tanpa harus mendefinisikan serangan yang ada sebelumnya, sehingga lebih tahan terhadap zero-day attack.
Article
Full-text available
This paper considers the problem of a choice of algorithms and data structures to achieve the effective processing of events generated by intrusion detection systems. The proposed approach is based on balanced binary trees and speeds up the operations of adding and searching records in the structure. The paper provides the theoretical and experimental confirmation of the efficiency of the developed approach
Conference Paper
The article presents the results of analysis of a few kinds of network traffic using Holt-Winters method in the analysis of network traffic. The data were obtained from five real computer networks using Snort intruder detection system and preprocessor AnomalyDetection.
Article
Full-text available
This paper describes a technique of applying Genetic Algorithm (GA) to network Intrusion Detection Systems (IDSs). A brief overview of the Intrusion Detection System, genetic algorithm, and related detection techniques is presented. Parameters and evolution process for GA are discussed in detail. Unlike other implementations of the same problem, this implementation considers both temporal and spatial information of network connections in encoding the network connection information into rules in IDS. This is helpful for identification of complex anomalous behaviors. This work is focused on the TCP/IP network protocols.
Article
This paper focuses on investigating immunological principles in designing a multi-agent system for intrusion/anomaly detection and response in networked computers. In this approach, the immunity-based agents roam around the machines (nodes or routers), and monitor the situation in the network (i.e. look for changes such as malfunctions, faults, abnormalities, misuse, deviations, intrusions, etc.). These agents can mutually recognize each other's activities and can take appropriate actions according to the underlying security policies. Specifically, their activities are coordinated in a hierarchical fashion while sensing, communicating and generating responses. Such an agent can learn and adapt to its environment dynamically and can detect both known and unknown intrusions. This research is the part of an effort to develop a multi-agent detection system that can simultaneously monitor networked computer's activities at different levels (such as user level, system level, process level and packet level) in order to determine intrusions and anomalies. The proposed intrusion detection system is designed to be flexible, extendible, and adaptable that can perform real-time monitoring in accordance with the needs and preferences of network administrators. This paper provides the conceptual view and a general framework of the proposed system.
Article
Fast and accurate methods for predicting traffic properties and trend are essential for dynamic network resource management and congestion control. With the aim of performing online and feasible prediction of network traffic, this paper proposes a novel time series model, named adaptive autoregressive (AAR). This model is built upon an adaptive memory-shortening technique and an adaptive-order selection method originally developed by this study. Compared to the conventional one-step ahead prediction using traditional Box–Jenkins time series models (e.g. AR, MA, ARMA, ARIMA and ARFIMA), performance results obtained from actual Internet traffic traces have demonstrated that the proposed AAR model is able to support online prediction of dynamic network traffic with reasonable accuracy and relatively low computation complexity. Copyright © 2005 John Wiley & Sons, Ltd.
Systemy detekcji intruzów i aktywnej odpowiedzi, praca magisterska napisana w Katedrze Informatyki Stosowanej Politechniki Łódzkiej pod kierunkiem Macieja Szmita, maszynopis
  • M Skowroński
  • R Wężyk
Skowroński M., Wężyk R.: Systemy detekcji intruzów i aktywnej odpowiedzi, praca magisterska napisana w Katedrze Informatyki Stosowanej Politechniki Łódzkiej pod kierunkiem Macieja Szmita, maszynopis, Łódź 2006
Algorytmy indukcji reguł decyzyjnych w Systemach Wykrywania Intruzów, XII konferencja Sieci Komputerowe
  • R Cichocki
Cichocki R.: Algorytmy indukcji reguł decyzyjnych w Systemach Wykrywania Intruzów, XII konferencja Sieci Komputerowe, Zakopane 2005
Preprocesory detekcji anomalii dla programu Snort" w: "Sieci komputerowe Tom 2. Aplikacje i zastosowania
  • Maciej Skowroński
  • Radosław Wężyk
  • Maciej Szmit
Maciej Skowroński, Radosław Wężyk, Maciej Szmit, "Preprocesory detekcji anomalii dla programu Snort" w: "Sieci komputerowe Tom 2. Aplikacje i zastosowania", WKŁ 2007, pp. 333-338
  • Spade Cvs
  • Http
SPADE CVS repository http://www.bleedingsnort.com/cgi bin/viewcvs.cgi/?cvsroot=SPADE (01.03.2007) [12] Copy of old (2001) Snot page http://web.archive.org/web/20010424080846/http://www.geocities.com/sniph00/ (01.03.2007)
Badanie samopodobieństwa ruchu w sieciach LAN -metody i narzędzia
  • J Kolbusz
  • A Lewicki
  • A Majdański
  • S Karmelita
Kolbusz J., Lewicki A., Majdański A., Karmelita S.: Badanie samopodobieństwa ruchu w sieciach LAN -metody i narzędzia", Informatyka Stosowana ISSN 83-914678-6-4, VII Lubelskie Akademickie Forum Informatyczne, Kazimierz Dolny 2003, s. 97-103.