A visualization paradigm for network intrusion detection

Abstract

We present a novel paradigm for visual correlation of network alerts from disparate logs. This paradigm facilitates and promotes situational awareness in complex network environments. Our approach is based on the notion that, by definition, an alert must possess three attributes, namely: what, when, and where. This fundamental premise, which we term w³, provides a vehicle for comparing between seemingly disparate events. We propose a concise and scalable representation of these three attributes, that leads to a flexible visualization tool that is also clear and intuitive to use. Within our system, alerts can be grouped and viewed hierarchically with respect to both their type, i.e., the what, and to their where attributes. Further understanding is gained by displaying the temporal distribution of alerts to reveal complex attack trends. Finally, we propose a set of visual metaphor extensions that augment the proposed paradigm and enhance users' situational awareness. These metaphors direct the attention of users to many-to-one correlations within the current display helping them detect abnormal network activity.

Full text

Page 1

Proceedings of the 2005 IEEE
Workshop on Information Assurance and Security
United States Military Academy, West Point, NY, 17–19 June 2005
A Visualization Paradigm for Network Intrusion Detection
Yarden Livnat, Jim Agutter, Shaun Moon, Robert F. Erbacher, Stefano Foresti
Abstract—
We present a novel paradigm for visual correlation of net-
work alerts from disparate logs. This paradigm facilitates
and promotes situational awareness in complex network en-
vironments. Our approach is based on the notion that, by
definition, an alert must possess three attributes, namely:
What, When, and Where. This fundamental premise, which
we term w3, provides a vehicle for comparing between seem-
ingly disparate events. We propose a concise and scalable
representation of these three attributes that leads to a flex-
ible visualization tool that is also clear and intuitive to use.
Within our system, alerts can be grouped and viewed hi-
erarchically with respect to both their type, i.e., the What,
and to their Where attributes.
gained by displaying the temporal distribution of alerts to
reveal complex attack trends.
of visual metaphor extensions that augment the proposed
paradigm and enhance users’ situational awareness. These
metaphors direct the attention of users to many-to-one cor-
relations within the current display helping them detect ab-
normal network activity.
Further understanding is
Finally, we propose a set
I. Introduction
The spread of malicious network activities poses great
risks to the operational integrity of many corporations,
institutions, and organizations, in addition to imposing
heavy economic burdens.
(IDS) analyze network traffic and host-based processes in
an attempt to detect such malicious activities. The prolif-
eration of different IDSs and the sophistication of attacks
lead to a large number of alert types. This complexity is
compounded by the sheer number of alerts these systems
generate and a high rate of false positives.
There is a growing body of research that validates the
role of visualization as a means for solving complex data
problems. Visualization elevates the comprehension of in-
formation by fostering rapid correlation and perceived as-
sociations. To that end, the design of the display must
support the decision making process: identifying prob-
lems, characterizing them, and determining appropriate re-
sponses. It is imperative that information be presented in
a manner that facilitates the user’s ability to process the
Intrusion detection systems
Y. Livnat: Scientific Computing and Imaging Institute, University
of Utah
J. Agutter: College of Architecture+Planning, University of Utah
S. Moon: College of Architecture+Planning, University of Utah
R. Erbacher:Department of Computer Science, Utah State Univer-
sity
S. Foresti: Center for High Performance Computing, University of
Utah
information and minimize any mental transformations that
must be applied to the data. Our goal is to enable users to
quickly decide how pervasive and how severe problems are.
In this work, we focus on developing visualizations that
take advantage of human perceptive and cognitive fa-
cilities in order to enhance users’ situational awareness
and support decision-making. We propose a visualization
paradigm, Figure 1, for the correlation of various network-
and host-based alerts from disparate IDS logs. The display
shows the local network topology map in the center, with
the various alert logs (divided into specific or group of alert
types) on the surrounding ring. The ring width represents
time and is divided into several history periods. Finally,
the alerts themselves are shown as lines from the most re-
cent history period to the local node being attacked. This
visual system provides a holistic view of the local networks,
promotes situational awareness, and supports the decision
making process. This design thereby increases the likeli-
hood that complex malicious attacks are detected early.
Fig. 1
VisAlert: A novel visualization paradigm for network
intrusion detection.
We wish to emphasize that our tool does not perform
any analysis on the alerts, nor does it process raw data, i.e.,
ISBN 0-7803-9814-9/$10.00 c ?2005 IEEE30

Page 2

the network traffic and alerts. Our aim is to enhance sit-
uational awareness via visual correlation of existing alerts.
In this respect, our approach can accommodate any past
or future IDS, or any other system for that matter, which
generates network alerts of any kind. Section III describes
in detail the rationale behind this statement and the con-
text within which it is valid. It is precisely this lack of
additional analysis that frees us from any constraints and
dependencies on the kind of alerts we visualize, and pro-
vides the flexibility and power of this paradigm.
This paper is structured as follows. Section II reviews
earlier visualization work. Section III presents the basic
premise of this work, i.e., the notion of network alerts as
space-time events of a given type. We then present our
visualization paradigm in section IV, followed by a set of
visual extensions in section IV-C. Section V presents initial
results and we offer some conclusions and future work in
section VI.
II. Previous Work
Historically, visualization has been applied extensively to
network monitoring and analysis, primarily for monitoring
network health and performance [1], [2]. Initial visualiza-
tion techniques for IDS environments focused on simple
scales and color representations to indicate state or level
of threat [3], [4], [5], [6]. Other environments provide ba-
sic graphical user interfaces but fall short of providing the
needed visual capabilities for analysis [7], [8], [9], [10].The
few visualization techniques that have been developed for
intrusion detection have been limited as to their applica-
bility and effectiveness.
With the need for better analysis mechanisms for se-
curity and IDS-related data, more advanced visualization
techniques have been increasingly explored over the last
few years.Many of these techniques have been proven
to be effective at allowing users to see malicious activities
such as worm [11], [12] or DoS attacks [13], [11]. However,
these visualization techniques tend to be focused on specific
problems and not on general alert correlation for an entire
enterprise. Other techniques, such as those by Secure De-
cisions [14], have focused on visual pattern matching, i.e.,
the representation of already-known attacks. Of the more
advanced techniques, additional techniques of note include:
The work by Teoh et al. [12] focuses on Internet routing
data which allows for the analysis of worms and other large-
scale attacks within its limited focus. Similarly, McPherson
et al. [13] developed a technique for the visualization of
port activity that is geared toward monitoring of large scale
networks for naive port scans and DoS attacks.
The work by Yin et al. [15] and Lakkaraju et al. [16]
focuses on the representation of netflows and associated
link relationships. Such techniques are critical for analyz-
ing attacks and IDS data but these two techniques quickly
suffer scalability issues and are limited as to the number of
representable parameters.
Wood [17] describes basic graph-based visualization tech-
niques, such as pie charts and bar graphs, and how these
techniques can be applied to typical network data avail-
able to all system administrators. This work provides a
fundamental description of how visualization can be im-
plemented and its application to such data, as well as the
meaning behind the identified results. The technique is
limited only in the simplicity of the visualization tech-
niques, which currently cannot analyze the high-volume,
high-dimensional data currently being generated by today’s
environments. This remains a major challenge for IDS data
analysis in general.
Traditional representations and network alert report-
ing techniques tend to use a single sensor - single indi-
cator(SSSI) display paradigm. Each sensor has a unique
way of representing its information (indicator) and does
not depend on information gathered by other sensors. The
benefit of such an approach lies in the separation of the
various sensors. Each sensor’s indicator can thus be opti-
mized for the particular data produced by its sensor, and
the user can pick and choose which sensors to use in an
analysis. Furthermore, the failure of one sensor does not
impact the capability of the rest of the system.
Consequently, the separation between the various sen-
sors is also the weakness of this representation technique.
Because each indicator is isolated, the user must observe,
condense, and integrate information generated by the inde-
pendent sensors across the entire enterprise. This process
of sequential, piecewise data gathering makes it difficult to
develop a coherent, real-time understanding of the inter-
relationship between the information being displayed, and
particularly the identification of malicious attacks.
Our work is designed to expand the visualization capabil-
ities to provide more robust general capabilities and tools
to aid administrators and analysts in the analysis of IDS-
related data.
III. Alerts, Resources and Instances
The various IDSs employ numerous complex approaches
and techniques; yet fundamentally their only purpose is to
generate alerts when such activity is detected. In addition,
some IDSs generate alerts based on alerts logged by other
IDSs. Automated response systems do exist; however, most
decision-making is done by humans.
One approach to resolving these issues is to correlate var-
ious alerts by common attributes. This approach is based
on the premise that while a false positive alert should not
exhibit correlation to other alerts, a sustained attack will
likely raise several alerts. Furthermore, real attack activ-
ities will most likely generate multiple alerts of different
types. There exists a large body of work aimed at cor-
relating these disparate alert logs based upon clustering,
probability and similarity to predefined attacks [18], [19].
ISBN 0-7803-9814-9/$10.00 c ?2005 IEEE31

Page 3

Alert correlation aims at establishing the validity or gen-
erating a confidence measure for the participating alerts.
The main problem in correlating alerts from disparate logs
is the seeming lack of mutual grounds on which to base any
kind of comparison between the alerts. By its very nature,
any alert, or any event for that matter, must possess what
we term the w3premise, namely, the When, Where, and
What attributes. Whenrefers to the point in time when the
alert happened. Whererefers to the local network node,
e.g., an IP address, to which the alert pertains. Finally,
the Whatrefers to some global indication of the type of the
alert, (e.g., log = snort,gid = 1,sid = 103). Put another
way, a network alert is a point in space-time, that has a
generic type associated with it and may contain additional
information specific to that alert type.
Typically, alerts are correlated based on either their
When or What attributes. In the latter, the alerts are first
grouped based on the What and then correlated, within
each group, based on the additional attributes associated
with that particular What attribute. However, the real
value of an alert is with respect to the local resource(s) it
pertains to. It is, in fact, the preservation of the resources’
status and integrity that is the main focus of IDS to begin
with. The alerts’ (What, When) attributes by themselves
have little if any inherent value. As a consequence, it is
the correlation of alerts with respect to resources that is
the focus of this work.
We also distinguish between an alert’s definition and an
alert’s instances. An alert definition is static and describes
the preconditions and meaning of an alert. Though there
are collections of predefined and widely-used general defi-
nitions, each installation usually extends them with its own
private sets of definitions. An alert instance, on the other
hand, refers to a particular point in time when the alert
preconditions were met. An alert instance may also in-
clude detailed information about how and what triggered
a particular alert. Correlating alerts and resources, there-
fore, means correlating alert instances with respect to a
particular resource.
IV. The Visualization Paradigm
The proposed visualization paradigm is based on the w3
premise (Section III), and correlates disparate alerts based
on their What, When, and Where attributes. What we
need is a way to visually organize the alert instances in
a clear and comprehensive fashion with respect to these
attributes.
One possible approach is to use a three-dimension Carte-
sian coordinates system and map the What, When, and
Where to the three axises. In this configuration, a network
event is represented by a three-dimensional point, as seen
in Figure 2. There are, however, several problems with
this approach. First, the points do not exhibit any obvi-
ous correlation, for example, two nearby points may not
Type
Time
Node
Network Alerts
A
B
C
Fig. 2
Cartesian representation of the What, When, and Where
attributes. Note that while alert A has the same Where as
B, and the same What as C, it is not evident in this
representation.
share any attribute with each other. Second, the three-
dimensional view introduce visual obstacles such as occlu-
sion and depth perception. Finally, the visual perception
will depend on the user’s specific view point, which only
adds another unnecessary degree of freedom to the already
complicated situation.
Alternatively, we base our approach on representing the
network alerts as connections between two domains. These
two domains are a one dimensional domain representing the
Where attribute, and a two-dimensional domain represent-
ing the When and What attributes. Note that the Where
and What spaces are finite while the When space is semi-
infinite. A network alert instance, in this scheme, is thus a
straight line from a point in the What-When domain to a
point in the Where domain, as shown in Figure 3.
Where
When
What
Network
Alerts
Fig. 3
Domains: Representing network alerts as lines between two
domains. The design facilitates the correlation of alerts
with respect to the same resources.
We choose to separate the Where attribute from the
What and When as resources provide a more or less static
ISBN 0-7803-9814-9/$10.00 c ?2005 IEEE32

Page 4

set of objects that we can use as visualization anchors for
the transient alert instances. In addition, we can now ex-
pand the Where domain into a two-dimensional domain,
which enables us to layout the resources in a more flexi-
ble and meaningful way for the user, such as the network
topology map.
Our design layout, as shown in Figure 4, maps the Where
domain onto a finite circle, while the What-When domain is
wrapped around it in the shape of a ring. We maintain the
orthogonality of the What and When spaces by mapping
the What attribute to the angle around the circle and the
When attribute to the radial direction from the center of
the circle.
network
topology
Type
Time
Fig. 4
Mapping: The finite one-dimensional Where domain is mapped
into a central circle containing the network topology map,
while the When-Where domain is mapped to a ring around it.
Alert instances can now be visualized, Figure 5, as lines
from ρ(what,when) → (angle,radius) on the outer ring,
to ψ(where) → (x,y) in the inner circle, where ρ and ψ are
general projections of the alerts into our two domains. In
our system we enable the user to dynamically control and
configure these two projections as necessary. We discuss
these projections in more detail in the rest of this section.
Finally, to reduce the possible cluttering when showing
all the alerts simultaneously, we divide the When space
into varying intervals and show the alerts instances for the
most recent history period only. The rest of the history pe-
riods show only the number of alert instances that occurred
during that period as described in Section IV-B.4.
A. Model and Presentation
The issues of organization and scalability with respect
to the number of resources, alert definitions and alert in-
1
Fig. 5
The visualization paradigm of VisAlert.
stances can be addressed by providing hierarchical group-
ings, also known as levels of detail (LOD). However, in or-
der to provide changing views of the logs and alerts, we em-
ploy the model, view, control (MVC) methodology. MVC
is based on separating the model, namely the data and its
organization, from the presentation, and using a controller
to generate the visualization or presentation based on the
model. This separation of the data and the presentation
components provides a mechanism by which different vi-
sualizations can be applied to the data based on outside
input.
B. Level of Detail
Level of detail refers to the notion of a dynamic repre-
sentation on a model based on a set of constraints. For
example, as an object moves farther away, the finer details
fade away, and thus the object representation can be simpli-
fied without effecting how it is perceived. LOD frameworks
are based on either discrete or continuous representation.
In the discrete case, the framework contains an ordered set
of different representation of the object, e.g., coarser views
with less details. At runtime, the framework chooses one
of these representation based on external input. The dis-
crete approach can therefore lead to abrupt changes in the
display of the object. To alleviate this effect, care must
be taken to ensure that these transitions occur only when
the differences between two adjacent representations are
not visible. The continuous framework, on the other hand,
provides a representation that changes smoothly based on
a continuous parameter. To this end the framework may
have a continuous representation of the object or it may
interpolate from a fixed collection of representations.
ISBN 0-7803-9814-9/$10.00 c ?2005 IEEE 33

Page 5

In our visual paradigm we employ LOD for each of the
three components of the w3premise, namely the type, lo-
cation and time. This provides us with the flexibility to
restrict our view of the data to only subsections as in a
drill-down or zoom-in operations. In addition, we gain the
capability to get global views of the data from different
perspectives.
B.1 Logs and Views
Conceptually, we tend to think of alerts as being grouped
in two levels: the log they belong to and the kind of alert
they represent, e.g., FTP, access-attempt, policy-violation.
Using this approach, alert instances are collected into bins
based on their actual type and these bins are then orga-
nized into the various groups which comprise the log. This
approach, however, is too restricted and rigid.
In this work we chose to separate between the model and
its presentation. The model comprises a fixed two-level hi-
erarchy, in which a log represents an unsorted collection of
alert instances, or in our terminology, events. Our rationale
for this fixed hierarchy is that the various logs are usually
collected separately from each other using different IDSs
and sensors (thus the log grouping). Furthermore, even
for the same kind of alert log, e.g., Snort, the actual alert
types it contains can vary considerably between different
organization (thus the notion of a log as an unordered set
of alerts).
It is the responsibility of the presentation layer to orga-
nize the logs and the alerts in each log in a fashion that is
meaningful to the local operator. This, in turn, allows us to
use different presentation schemes for different logs. Fur-
thermore, a single log can be presented in multiple ways-
based on the current operator tasks, capabilities, clearance,
or even as a filtering operation.
The different presentation schemes can differ in hierar-
chical structure as well. Each node in the hierarchy can
have a different number of children, and the height of each
branch can vary as well. The structure can be static or dy-
namic and can adapt to user needs and tasks. Finally, the
graphic presentation can also differ and employ different
visual motifs.
B.2 Transition
The visualization paradigm allows the user to zoom in
or out of any group within any view, as well as open or
close new views. Yet by their very nature, the log views
are discrete, leading to a discrete LOD representation. In
order not to confuse the user, we must take care of smooth-
ing the transition between the various levels. Consider the
transition from a single view to two views of a Snort log,
as depicted in Figure 6. The first view organizes the alerts
based on the typical Snort groups (Attach, FTP, P2P, X11,
etc.)while the second view organizes the alerts based
on their Snort classification (attempted-recon, shellcode-
detect, policy-violation, etc). Note that while these two
snapshots of the display look fine one next to each other,
the user may get confused and disoriented if the left im-
age is instantaneously replaced by the right. Not only will
a whole new view popup, but the locations of the bins,
groups and alerts instances of the first view will suddenly
shift.
We can, however, achieve a smooth transition between
the two levels of details via animation. In particular, when-
ever a group or view is opened or closed we animate it by
smoothly varying the length of the arc occupied by the ob-
ject. During the animation, we multiply the length of an
object’s arc by a parameter that varies between zero (the
object is not seen) to one (the object is fully visible).
(a)Single View (b)Multiple
view occupies part of the
ring
Views,Each
Fig. 6
A single and multiple views of the same log. Note that the
same alert may be displayed twice, once per view.
B.3 The resources map
In the scope of this work, the term resources refers
to individual network nodes such as hosts, switches, and
routers. Each such node may include additional informa-
tion such as its name, IP address, mission(s), how critical
it is to the organization, its operating system and even the
OS patch level. However, an unordered collection of nodes
can make it hard on the operator to comprehend the rela-
tionship between the various nodes. We, thus, use the local
network topology map as the base for the resources presen-
tation inside the inner circle. The use of the topology map
has both pros and cons. On one hand, the topology map
of an organization may not be available, or may not be up
to date. On the other hand, the topology map can help to
correlate attacks based on sub-group, such as a particular
class C, or the physical location of the nodes (assuming the
topology map is laid out in that way), or even a combina-
tion of the two.
The scalability issue arises when the topology map be-
comes too big to fit into the inner circle. To this end, we let
the user shift the view (a pan operation) of the underlying
topology map as well as zoom in and out. However, as the
ISBN 0-7803-9814-9/$10.00 c ?2005 IEEE 34