Conference PaperPDF Available

Operator Impressions of 3D Visualizations for Cybersecurity Analysts

Authors:

Abstract and Figures

Cybersecurity analysts ingest and process significant amounts of data from diverse sources in order to acquire network situation awareness. Visualizations can enhance the efficiency of analysts' workflow by providing contextual information, various sets of cybersecurity related data, information regarding alerts, among others. However, textual displays and 2D visualizations have limited capabilities in displaying complex, dynamic and multidimensional information. There have been many attempts to visualize data in 3D, while being displayed on 2D displays, but success has been limited. We propose that customized, stereoscopically perceivable 3D visualizations aligned with analysts' internal representations of network topology, may enhance their capability to understand their networks' state in ways that 2D displays cannot afford. These 3D visualizations may also provide a path for users who are trained and comfortable with textual and 2D representations of data to assess visualization methods that may be suitably aligned to implicit knowledge of their networks. Thus, the premise of custom data-visualizations forms the foundation for this study. Herein, we report on findings from a comparative, qualitative, within-subjects usability analysis between 2D and 3D representations of the same network traffic dataset. Study participants (analysts) provided information on: 1.) ability to create an initial understanding of the network, 2.) ease of finding task-relevant information in the representation, and 3.) overall usability. Results indicated that interviewees indicated a preference for 3D visualizations over the 2D alternatives and we discuss possible explanations for this preference.
Content may be subject to copyright.
Operator Impressions of 3D Visualizations for Cybersecurity Analysts
Kaur Kullman1, Noam Ben Asher2, Char Sample3
TalTech University, Tallinn, Estonia, EU1
ORAU, Oak Ridge TN, USA2
ICF Inc. Columbia, MD, USA3
Abstract: Cybersecurity analysts ingest and process significant amounts of data from diverse sources in order
to acquire network situation awareness. Visualizations can enhance the efficiency of analysts workflow by
providing contextual information, various sets of cybersecurity related data, information regarding alerts, among
others. However, textual displays and 2D visualizations have limited capabilities in displaying complex, dynamic
and multidimensional information. There have been many attempts to visualize data in 3D, while being displayed
on 2D displays, but success has been limited. We propose that customized, stereoscopically perceivable 3D
visualizations aligned with analysts’ internal representations of network topology, may enhance their capability to
understand their networks’ state in ways that 2D displays cannot afford. These 3D visualizations may also provide
a path for users who are trained and comfortable with textual and 2D representations of data to assess visualization
methods that may be suitably aligned to implicit knowledge of their networks. Thus, the premise of custom data-
visualizations forms the foundation for this study. Herein, we report on findings from a comparative, qualitative,
within-subjects usability analysis between 2D and 3D representations of the same network traffic dataset. Study
participants (analysts) provided information on: 1.) ability to create an initial understanding of the network, 2.) ease
of finding task-relevant information in the representation, and 3.) overall usability. Results indicated that
interviewees indicated a preference for 3D visualizations over the 2D alternatives and we discuss possible
explanations for this preference.
Keywords: visualization, cybersecurity, decision-making, data visualization, virtual reality.
1. Introduction
Cyber is the newest domain of war (Lynn, 2010), endowed with unique sensory characteristics that
differentiate this warfare environment from kinetic-physical warfare (Gonzalez, Ben-Asher, Oltramari, & Lebiere,
2014). Cyber security analysts who are responsible for ensuring the security of networks and other assets, utilize a
wide array of computer network defense (CND) tools, such as Security Information & Event Management (SIEM)
that allow data from various sources to be processed and alerted on. CND tools allow analysts to monitor, detect,
investigate and report incidents that occur in the network, as well as provide an overview of the network state. To
provide analysts with such capabilities, CND tools depend on the ability to query, process, summarize and display
large quantities of diverse data which have fast and unexpected dynamics (Ben-Asher & Gonzalez, 2015).
Shneiderman (Shneiderman, 1996) provided a taxonomy depicting 7 human-data interaction task levels: 1.) gaining
Overview of the entire dataset, 2.) Zoom on an item or subsets of items, 3.) Filter non relevant items, 4.) get Details-
on-Demand for an item or subset of items, 5.) Relate between items or subset of items, 6.) keep History of actions
and 7.) allow Extraction of subsets of items and query parameters. Traditionally, cyber defenders have used
command line tools and alphanumeric data displays to execute these seven tasks. With the need for faster and more
accurate situational awareness of increasing data volume, many CND products have integrated graphical user
interface (GUI) and 2-dimensional (2D) data visualizations to expedite human information acquisition.
Visualizing multidimensional data on 2D screens bears several limitations. First, the resulting visualization
after the dimensionality reduction methods have been applied will likely differ from the mental representation that
the analyst had acquired upon reviewing the same data in numerical and textual data. Contextual information will
likely be removed in the reduction process that could be crucial to the understanding of the situation and to identify
relevant clues (Rajivan, Konstantinidis, Ben-Asher, & Gonzalez, 2016). Also, three dimensional visualizations may
address some of the inherent limitations of 2D displays by aligning with the analysts’ internal representation of
their datasets, if the analyst naturally thinks about data in three dimensions. Users’ interactions in VR (Virtual
Reality) and XR (Mixed Reality) may also be more intuitive if users are expected to interact with the visualization
in ways that humans manipulate objects in the physical world. This way we could harness the dexterity of human
hand movement for interactions (Gershon, Klatzky, & Lee, 2014) if the tools used are capable enough, using haptic
feedback to further advance users interaction efficiency (Gershon, Klatzky, Palani, & Giudice, 2016).
2
However, the scale, heterogeneity, and complexity of cybersecurity datasets continue to pose challenges for
visualization and interaction designers (Reda, et al., 2013) despite the constant increase in computers’ abilities to
process and display more data on 2D displays. This could be because visualization designers lack an understanding
of cybersecurity operations and network infrastructure to create an effective visualization for cybersecurity analysts.
Yet, cybersecurity analysts who are familiar with the technical aspect of network monitoring do not have expertise
in data visualization and human perception. The visualization designer might need to have dual expertise.
By providing a data visualization environment where minimalistic visual, audio and haptic cues are informing
the user of what is happening in that environment, we allow the user to focus on the task. Furthermore
environmental cues should be perceptible and clear to avoid user confusion. Visualization should be functional
and available utilities should accurately convey their functions. Controls, navigation, interface, and all other
interface conventions should be consistent. Users cognitive and physical workload must be minimized. Human
errors should be anticipated and prevented if possible. The environment should be flexible to allow customization
for personal preferences, cultural differences, color vision deficiency etc. (Hodent, 2018). We hypothesize that the
analyst may find it intuitive to use a 3D representation of cybersecurity network data that is aligned with the above
guidelines as well as the analysts internalized understanding of the data. Intuitive interfaces may enable the analysts
to explore and understand their environment more efficiently.
2. Visualization for Cyber Defense
Cybersecurity visualizations provide analysts with visual representation of alphanumeric data that would
otherwise be difficult to comprehend due to its large volume. Such visualizations aim to effectively support
analyst’s tasks including detecting, monitoring and mitigating cyber attacks in a timely and efficient manner (Sethi
& Wills, 2017). Cybersecurity specific visualizations can be broadly classified into three main categories: 1.)
network analysis, 2.) malware and 3.) threat analysis, and situational awareness (Sethi & Wills, 2017). Timely and
efficient execution of tasks in each of these categories may require different types of visualizations addressed by a
growing number of cybersecurity specific visualization tools (Marty, 2008) as well as universal software with
visualization capabilities like Tableau, MS Excel, R, Python, and D3 libraries (d3.js) among others. These tools
could be used to visualize data in myriad of ways (Munzner, 2014) so that analysts could explore their datasets
visually and interactively (Ward, Grinstein, & Keim, 2015). Graphistry is one recent example of a 2D force-directed
graph visualization (Meyerovich & Tomasello, 2016) whose interface is easy to manage and visualization and is
responsive to queries on massive datasets. These are crucial qualities for cybersecurity analysts, with emphasis on
the importance of the low-latency between analyst’s request for a change in visualization (change in filter, time
window or other query parameters) and rendering of the visualized response from the system (Wu, Xu, Chang,
Hellerstein, & Wu, 2018).
The usability of data visualizations for CND operations that have not been evaluated, may lead to low adoption
rates by practitioners (Best, Endert, & Kidwell, 2014). The challenge in creating useful visualization for
cybersecurity practitioners is in aligning data visualization experts’ knowledge with cybersecurity analysts’ needs
and knowledge so, that the resulting visualizations would be useful for work tasking. A recent survey showed that
46% of 130 tools did not have any user-involvement in the evaluation phase (Sethi & Wills, 2017).
To achieve higher visualization adoption rates, analysts should have the ability to intuitively and iteratively
adjust the visualizations to suit with their changing needs (Kirk, 2016). Datasets used in cybersecurity operations
are often multi-dimensional and analysts would either have to scale down the number of dimensions viewed at one
time to be able to use 2D & 3D visualizations, or combine multiple 2D visualizations displaying different
dimensions of the same dataset in a single dashboard. This requires the designer to properly encode variables
(dimensions) into shapes, colors, sizes among others. The viewer has to translate that shape into spatial perception
and compare it to her internal understanding of the data, to decode the meaning of the visualizations; a task that
may be non-trivial (Ehrenstein, Spillmann, & Sarris, 2003). There have been numerous attempts to employ 3D
visualizations for cybersecurity data that are displayed on 2D computer screens with varying degrees of success.
Such visualizations sometimes use monocular depth cues (Lebreton, Raake, Barkowsky, & Le Callet, 2012) and
object movement to convey the 3D shape of the visualization; advantages and disadvantages of which were
thoroughly discussed in our previous paper (Kullman, Cowley, & Ben-Asher, 2018). VIDS (Shearer & Edwards,
2018) provides an interactive 3D environment for visualizing network and alert (or other) data in 3D shapes,
3
whereby users can seamlessly switch styles and layouts to dynamically shape their data and easily adjust their
viewpoint (Gaw, 2014). Real-time 3D visualization engine DAEDALUS-VIZ allows operators to grasp visually
and in real-time an overview of alert circumstances, while providing highly flexible and tangible interactivity
(Inoue, Suzuki, Suzuki, Eto, & Nakao, 2012). InetVis (van Riel & Irwin, 2006) allows the user to allocate source
and destination IPv4 addresses to X and Z axis, while destination ports are being allocated to the Y axis on a 3D
cube. To understand the shape of the cube and detect the positions on Z axis, user must manually change the
viewpoint with mouse. Shoki (Berry, n.d.) allows the user to define what values are plotted on which axis, while
the screen is divided to four squares, three of them showing each axis in 2D, while the fourth square displays the
cube as a 3D object.
Due to the emergence of commodity VR devices, multiple data visualization tools have implemented support
for VR headsets, that are capable of 6 degrees of freedom (6DOF) movement of the user’s viewpoint, allowing the
observer to perceive the depth of the visualization stereoscopically, avoiding the mental work needed to convert
2D images to 3D. OpenGraphiti (Reuille, Hawthorne, Hay, Matsusaki, & Ye, 2015) enables provides customizable
graphs, along with querying and filtering capabilities. However, OpenGraphiti does not provide 3D VR interaction
capacities. However, V-Arc (Maddix, 2015) enables the positioning of data in a predetermined layout, data selection
and color-coding amongst other capabilities. Virtual Data Explorer (VDE) is a VR tool that allows users to
collaborate while investigating 3D data visualizations, to find anomalies in a variety of cybersecurity-related
datasets (U.S. ARL, 2018). For our research herein, we used VDE (see 3.4), because it enables the user to perceive
the spatial layout of the topology based on observed network traffic, while the resulting visualization can be
augmented with additional data, like TCP/UDP session counts between network nodes (Kullman, Cowley, & Ben-
Asher, 2018). Due to the 6DOF of Oculus Rift VR headset (OVR) used for this study, VDE also allows us to test
the usefulness of stereoscopically perceived depth-cues (contrary to monocular depth-cues on flat screens) for
encoding data.
3. Method
To understand whether stereoscopically-perceivable 3D data-shapes representing a complex computer
network’s topology is usable, we conducted semi-structured interviews with 10 subject matter experts working as
cybersecurity analysts (as suggested in (Ward, Grinstein, & Keim, 2015)). Included in the usability assessment, we
asked analysts about whether network behavior is understandable, helpful and useful for cybersecurity analysts’
tasks.
3.1 Participants
Ten cybersecurity analysts (Mean age = 36.5yr., 20% females) were interviewed in a semi-structured format.
All participants work as cyber security practitioners, having 2 months to 10 years of experience in the field (Mean
= 4.5 years). Participation was voluntary and these volunteers were not compensated for their time.
3.2 Materials
The network traffic data used during the interviews was part of the NATO CCDCOE CDX Locked Shields
2018 (LS18) “Partner Run” (LS18PR) dataset. This dataset includes 23 defensive teams’ (Blue Teams, BT)
networks, an offensive (Red Team, RT), infrastructure support (Green Team), situational awareness (Yellow Team)
and the managing team (White Team) nodes and traffic. During LS18 exercise the network included more than
4000 virtual machines, with about 2500 attacks executed by the Red Team against all Blue Teams combined.
LS18PR function was to test Read Team and infrastructure readiness. Distinct of the main event, LS18 that ran
for 2 days (2x7 hours), LS18PR ran for 7 hours, during which only few Blue Team networks were attacked and
defended, while the rest of the networks were running as usual. Hence LS18PR dataset provides the ability to
observe networks in their “normal” as well as “under attack” and “compromised” states during the same time.
The time-window for the network sessions used in the visualizations shown to the participants was set to 40-
minute periods. For data preparation, Moloch (https://molo.ch/) was used to process the LS18PR packet data
(PCAP). The resulting data and metadata was stored in an Elasticsearch server to allow dynamic querying. Kibana
(https://www.elastic.co/products/kibana) was used to generate dynamic and interactive 2D visualizations based on
the data stored in Elasticsearch server. VDE queried the Elasticsearch server for session counts between entities
4
and presented the information using Oculus Rift (https://www.oculus.com/rift/) VR headset (OVR) while an Oculus
Touch controller (OTC) was used to interact with the VDE. The controller allowed users to move around the virtual
space (by changing their viewpoint), select different groups of objects (e.g. connections from/to a Blue Team), grab
a network node to alter its position (and better perceive the destinations of the connections that this node had) and
query additional information about the node (e.g. it’s IP addresses).
3.3 Procedure
Upon arrival to the interview, participants were asked about their cybersecurity expertise, experience with 2D
and 3D visualizations, as well as gaming preferences. Gaming preferences were discussed to build the rapport and
understand interviewee’s level of experience in cybersecurity. Below is a list of the basic questions asked in this
introduction portion of the procedure. Participants provided open ended answers that were documented by the
experimenter.
1. What are your favorite console / computer games?
2. Have you used Moloch and / or Kibana before?
3. Have you experienced Virtual, Augmented or Mixed Reality before?
4. Do you have formal education on IT and / or cybersecurity?
5. What area do you specialize in cybersecurity?
6. How long have you been working on your current specialty; on cybersecurity?
Then, participants received a short briefing about the purpose of the study and an overview of the dataset. They
were shown a printed diagram (see Figure 1) illustrating the network topology of a single blue team network. Based
on the diagram, participants were asked to consider, what (textual and visual) tools they would prefer to use to learn
that network’s topology to acquire situational awareness.
Figure 1. Simplified network topology diagram for traffic used in this study.
5
Following, the experimenter presented a 2D visualization of session counts between the different entities in the
Blue Team’s network as radial figures (Figure 2 using Kibana). As seen in Figure 2a and Figure 2b there is a
difference of an actively attacked and maintained network (on the left) compared to another one with exactly the
same topology and active services, while unmaintained and not attacked network (on the right).
Size of a sector on Figure 2 represents the count of observed sessions. The radial diagram also indicated how
many connections were initiated between the source and the inner ring, how many of those connections were
targeted at the one represented in the outer ring, etc. The color of the session-count-block is randomly allocated to
each node in that Blue Team network and does not have any relation to other networks. However, the color allocated
to each node does allow the observer to find that same node in that same radial. The other sectors belonging to that
same node are also highlighted by a mouse-over, which displays a popup detailing the IP addresses of source and
its destination nodes and their session counts (see Figure 3).
Figure 3. Mouse-over example for selected data-point displayed in
a popup window, with source IP address and observed session count
(srcIp) and destination IP address with its respective session count
(destIp).
Participants were also shown a 2D visualization of a force directed graph (using Moloch) to provide another
example of graphical representation of relations between networked entities (Figure 4).
(b)
(c)
Figure 4. Network connections observed in the two unused BT (a) and (b) networks compared to a BT (c) networks
that were actively engaged during LS18PR. Traffic observed during a set period of time is applied as pull strength
of the edges between nodes, while nodes represent the hosts initiating and/or receiving connections, visualized on
a 2D graph.
(a)
(b)
Figure 2. Radial 2D diagrams
visualizing activeness of Blue
Teams’ internal nodes. Inner ring
contains source addresses of the
entities present in that Blue Team’s
network while the outer ring
contains destination addresses of
that same Blue Team’s entities, with
network connection sessions to that
source address during a time-
window.
6
Following the review of various 2D visualizations, participants were introduced to the VDE and the reasoning
behind the spatial positioning of network elements in a 3D space. During this step, LS18PR networks were first
shown as 3D shapes on a 2D display and once participants felt comfortable with their understanding of the 3D
visualization as shown in VDE, they were fitted with an Oculus Rift VR Headset and Touch Controller. Participants
were encouraged to explore the 3D display by changing their viewpoint while using VDE so that they could observe
Blue Team networks from close distance (Figure 5) and would be able to read explanatory texts, reach out to grab
the nodes, move those around, highlight nodes’ features, select edges’ groups and so on. After participants had
familiarized themselves with the VDE environment and it’s 6DOF “rudder head movement” (Pruett, 2017), they
were asked to evaluate if and how such network topology visualization and its augmentation with additional data
(e.g. network session counts) would relate to the 2D visualizations (Figures 2, 3, 4) and the print-out topology
(Figure 1) shown to them before. Once the participant gained an understanding of a blue team’s network, she/he
was guided to adjust the viewpoint in VR so that all the LS18PR network components would be in the field of view.
Then, the participant was asked to provide feedback and critique regarding the usability of VDE, subjective ease of
stereoscopical perception of the 3D data shapes and her/his ability to acquire situation awareness with such tool.
3.4 Virtual Data Explorer
Virtual Data Explorer (VDE) was developed with the Unity 3D game engine to present users with
stereoscopically perceivable data visualizations in VR and XR.
Figure 5. VDE 3D display of network topology and traffic focusing on a single Blue Team network topology
(“Zoom” on “Details-on-Demand, per (Shneiderman, 1996)). Additional videos of VDE can be found:
https://coda.ee/vde
The VDE uses a predefined topology description (configuration) for the visualized network. Data-shapes were
spatially positioned into a meta-shape (viewed from different angles as shown in Figure 6) to allow the user to take
advantage of stereoscopic viewing that VR provides. Multiple layouts were considered to minimize possible edge
clutter and enable convenient distinguishability of intra- and extra-network connections and nodes’ relations. These
3D shapes are easily understandable in stereoscopically-perceivable VR headsets, while often cluttered and
unusable on a 2D screen or on a printed paper.
7
(a)
(b)
Figure 6. 3D display of LS18PR network topology and network traffic using VDE: (a) overall view of the meta-
shape (“Overview); (b), RT group (“Zoom”) with some added connections and selected (“Filtered”) edges
colored red (“Relations”, per (Shneiderman, 1996)).
VDE allows the spatial topology of the network to be augmented with additional data. For this study, the
visualization was enriched with network session counts so, that the most popular connection (represented as a green
line (edge) between the nodes that were observed connecting) was fully visible, while the least popular connection
was almost transparent. User could add additional sessions using VDE menu system in VR, in which case the added
edges were colored red until a next set of edges was added. Additional information about VDE design decisions
can be found in our previous paper (Kullman, Cowley, & Ben-Asher, 2018).
4. Results
Participants were asked for impressions on the technology, ideas or suggestions for modifications, should such
a tool be made available for their tasking. All participants had used 2D visualizations, 6 had used Kibana, and 4 out
had experienced VR prior to this study. Seven interviewees self-rated themselves as experienced in playing
computer games. All participants used command line interfaces or GUIs for tasks like querying NetFlow data,
inspect captured packets, review incidents in SIEM, explore various configuration files of the routing and perimeter
devices etc. Seven used Kibana and / or Excel for simple 2D visualization. All participants agreed that although
printed or electronically provided 2D diagrams of the network topology would be helpful, topologies provided are
usually outdated. Overall, there emerged a common process that analysts would have used to build their
understanding of a network’s topology. First, analysts would build an understanding of what is “normal” in that
specific environment and then find the behaviors that would need further investigation. In order to execute that
process, the analysts begins by building filters on available data to exclude the findings that are deemed “normal”
and continue fine-tuning that filter, while the analyst learns that network’s behavior and builds her/his internalized
understanding of that datasets expected demeanor.
To the questions How would you build the understanding of a network? How would you map the network
topology? participants responded with: “Textual logs from SIEM [or similar]”;”Textual tools”; “flow in Kibana,
xflow, tcpdump. Whatever tools that are available. Textual mostly, if some visualization is available, then that
too." To the question How would you establish expectations regarding normal behaviors of the different entities
in the network?participants responded with: “I just read the logs [..]. Look in packets.”; Check Ingress/egress
points, what services are allowed through firewall, what VLANs&VPNs are in use.”; filter logs, what type of
systems are in these networks from SIEM. Use different filters and queries in Kibana.”; “Work through
documentation, dive in, see configuration.”; “flow data, whos talking to each other, as opposed to heading outside;
I’ve used to bar graphs [..while trying to identify..] what IP’s are being hit, what hosts are endangered a lot in traffic.
Usually I would likely see just textual lists of IP addresses, ports etc."; “Look at netflows, do a graph chart. Use
excel, build bar graphs [and so on]"; “Use a sensor that sees all the traffic in this network, build a BPF to exclude
things (ICMP, TCP, UDP) [from tcpdump] to see what’s left, what falls out, what ports and protocols, [..], to find
what stands out, what weird ports are in use."
Once in the VDE VR environment, users had some trouble getting comfortable with the “rudder head
movement” to roam around in the VR environment. Few participants found the rudder head movement intuitive to
8
use. Learning to use this tool usually took between one to two minutes. Most problematic was the vertical
movement. To execute vertical movement, users had to look straight up or straight down and move backwards or
forwards to change their vertical viewpoint. However, one participant commented, “[moving around in this
environment] is very natural to me. Only two participants reported feeling dizzy or having any simulator
sickness/motion sickness symptoms during or after the session. The precision of depth perception or the predicting
the physical distance of each virtual object from the self was highly variable. Most of the participants managed to
grab and hold visualized entities easily, while others struggled. Further tuning of haptic and visual feedback is
needed to improve interaction with 3D data representations.
Once the participants were comfortable with adjusting their position and viewpoint in the VR environment, all
were able to observe the data-shapes and understand its relation to the network’s textual and 2D visualizations they
had seen moments before. Most participants stated that once the logical structure of the shapes positioned in VR
had been understood, the topology of these networks became much clearer. Participants also understood the
underling relations between 3D visualization and the textual / 2D representation they had seen previously:
[P1]: “Does this [3D visualization] map back to the topology you saw on the paper before? I think so, I
mean I'd have to figure out where things are, but... [yes]”;
[P6]: “It is not how I thought about, but... [...] I think it would definitely be helpful, but it would take some
re-learning, [e.g.] how to think visually. When you learn networking, then you’re kind of trying to build
this in your brain already. But then getting used to seeing this and not having to build your own picture.
Like, training brain to think visually not only textually.”;
[P4]: “So this is what you showed [me] before on the network diagram [on paper]? [...] Yes, I like this. This
is different than looking graphs on your computer screen. [...] I take it you’d prefer this to the regular
[tools]? Yes, definitely.”;
Participants could “see” where “things” are in the network, helping them spatially perceive and understand the
structure and topology of this computer network and networked entities’ (nodes) positions and logical grouping
inside that network (“Overview”, as per (Shneiderman, 1996) taxonomy):
[P1]: “This agrees with me particular. I like visual. I always kind of visualize things, in a way like this. And
this particularly agrees with me. That's very easy for me to understand.”;
[P6]: “If you now look at this network diagram [printed on paper], does this looks familiar? It is familiar,
but very flat. Would you prefer 3d? Well, of course. This 2D looks like... why are we still using this.. it
seems so.. like.. limited.”;
[P2]: “For our team this is a great representation of what we could use. This is a good representation of a
network that would work for us. A way to visualize this and interact with.;
Participants admitted that they perceived the traffic “going” between nodes, referring to the edges representing
the count of sessions observed between these two nodes (“Relations of groups, subgroups and nodes, as per the
taxonomy):
[P1]: “They are clearly grouped, and I can see exactly where everything is going [which node has been
connecting to what other nodes].
[P4]: “This would help a lot. You can see the traffic leaving networks and so...
[P5]: “This makes a lot of sense to me. I really like the visualization. I can see.. there’s the first firewall,
DMZ.. I can kind of understand how the network is built...
[P2]: This is useful... and this seems very utilizable. Useful in terms of what’s reaching out to what.
There’s definitely usefulness in this for what we do here. [..] This makes much sense for what we do here
in terms of usefulness and utility...
Participants recognized the advantages of using such visualizations could provide to transfer knowledge about
specific networks from senior analysts to trainees:
[P6]: “Since I’ve been here for 4 years, I’ve trained about 80 people. I think if we’d have something like
that from the start, it would change their whole perception of how to [think of networks] and jump start
[their ability to work the networks]. [] I think a lot of analysts would have different views, that would
depend on their knowledge base and artistic side also. [..] When you’re using tool like this, when you build
your network diagrams, you would like to have same setup, that way [when] you’re looking at them on a
pdf, you’d have the same layout.
9
Participants suggested capabilities (see correlation with taxonomy, described in Introduction and
(Shneiderman, 1996)), that they would like to have at their disposal:
[P9]: “This is awesome! [..] As an analyst I would want to see [in addition to the visualization] what's
happening, the [textual] details. [..] It is cool; you could definitely do a lot with it. [..] This is one of the
coolest things I’ve ever seen. But I do need [additional, textual] information. As an analyst, I could
definitely use this. [..] I could probably play with this all day.”;
[P1]: “[It] would be nice to control how far apart they [nodes, groups] are. Would be easier to navigate
between them. [..] [option to] change the icon of the node to something that would indicate the function of
the entity. I would prefer to use colors for grouping the entities.”;
[P3]: Color-code the layers in the network. Any host that is associated [has had sessions] with those
should also be color accordingly. 6DOF movement should be available.”;
[P6]: “Lines should have arrows showing the direction of the sessions.”;
5. Conclusion
This study captured cybersecurity analysts’ impressions of a network topology presented as a stereoscopically-
perceivable 3D structure. Overall, the impressions towards stereoscopically-perceivable 3D data visualizations
were highly favorable. Multiple participants acknowledged that such 3D visualizations of network topology could
assist in their understanding of the networks they use daily. Participants expressed a wish to integrate such
visualization capabilities in their workflow. Prior experience with 3D displays had no influence on user preferences,
while participants with prior gaming experience adjusted quickly to the Oculus Touch motion controllers,
suggesting that the relevant dexterity and muscle memory for gaming console controller usage helps users adjusting
from those controllers to handling input devices for VR experiences. Further research is needed to understand what
specific 3D data shapes would be useful and for which datasets (e.g. computer network topology) to create
additional 3D visualization suitable for analysts’ preferences and test the usefulness of those visualizations. Follow-
up studies should evaluate operator performance in 3D environments.
6. Acknowledgements
For all the hints, ideas and mentoring, authors thank Jennifer A. Cowley, Alexander Kott, Lee C. Trossbach,
Jaan Priisalu, Olaf Manuel Maennel. This research was partly supported by the Army Research Laboratory under
Cooperative Agreement Number W911NF-13-2-0045 (ARL Cyber Security CRA) and under Cooperative
Agreement Number W911NF-16-2-0113 and W911NF-17-2-0083. The views and conclusions contained in this
document are those of the authors and should not be interpreted as representing the official policies, either expressed
or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to
reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
7. References
Arthur, K. W., Booth, K. S., & Ware, C. (1995, 7). Evaluating 3D Task Performance for Fish Tank Virtual Worlds. ACM
Transactions on Information Systems, 11(3), 239-265. doi:10.1145/159161.155359
Aukstakalnis, S. (2017). Practical Augmented Reality. Addison-Wesley.
Ben-Asher, N., & Gonzalez, C. (2015). Effects of cyber security knowledge on attack detection. Computers in Human Behavior,
48, 51-61.
Berry, S. P. (n.d.). The Shoki Packet Hustler. Retrieved from http://shoki.sourceforge.net/
Best, D. M., Endert, A., & Kidwell, D. (2014). 7 Key Challenges for Visualization in Cyber Network Defens. In Proceedings of
the Eleventh Workshop on Visualization for Cyber Security (pp. 33-40). ACM.
Burnett, M. S., & Barfield, W. (1991). Perspective versus plan view air traffic control (ATC) displays - Survey and empirical
results. International Symposium on Aviation Psychology, 6th. Columbus. Retrieved from
http://adsabs.harvard.edu/abs/1991STIA...9244967B
Dennehy, M. T., Nesbitt, D. W., & Sumey, R. A. (1994). Real-Time Three-Dimensional Graphics Display for Antiair Warfare
Command and Control. Johns Hopkins APL Technical Digest, 15(2), 110-119.
Ehrenstein, W. H., Spillmann, L., & Sarris, V. (2003). Gestalt Issues in Modern Neuroscience. In Axiomathes (pp. 433-458).
Springer.
Gaw, T. J. (2014, 4). 3D Information Visualization of Network Security Event. Munice, Indiana, USA: Ball State University.
Retrieved from https://pdfs.semanticscholar.org/f3fa/c8a059369b96202a70ceb19828c07444dc42.pdf
Gazzaniga, M. S., Ivry, R. B., & Mangun, G. R. (2013). Cognitive Neuroscience: The Biology of the Mind, 4th Edition. W. W.
Norton & Company.
10
Gershon, P., Klatzky, R. L., & Lee, R. (2014). Handedness in a virtual haptic environment: Assessments from kinematic
behavior and modeling. Acta Psychologica, 37-42.
Gershon, P., Klatzky, R. L., Palani, H., & Giudice, N. A. (2016). Visual, Tangible, and Touch-Screen: Comparison of Platforms
for Displaying Simple Graphics. Assistive technology: the official journal of RESNA, 28(1), 1-6.
doi:10.1080/10400435.2015.1054566
Gonzalez, C., Ben-Asher, N., Oltramari, A., & Lebiere, C. (2014). Cognition and technology. In Cyber defense and situational
awareness, 93-117.
Hodent, C. (2018). The Gamer's Brain; How Neuroscience and UX Can Impact Video Game Design. CRC Press.
Hurter, C. (2016). Image-Based Visualization: Interactive Multidimensional Data Exploration. (N. Elmqvist, & D. Ebert, Eds.)
Morgan & Claypool.
Inoue, D., Suzuki, K., Suzuki, M., Eto, M., & Nakao, K. (2012). DAEDALUS-VIZ: Novel Real-time 3D Visualization for Darknet
Monitoring-based Alert System. VizSec (pp. 72-79). ACM.
Kirk, A. (2016). Data Visualisation, A Handbook for Data Driven Design. Sage.
Kullman, K., Cowley, J. A., & Ben-Asher, N. (2018). Enhancing Cyber Defense Situational Awareness Using 3D Visualizations.
Proceedings of the 13th International Conference on Cyber Warfare and Security ICCWS 2018: National Defense
University, Washington DC, USA 8-9 March 2018 (p. 369−378). Washington DC: Academic Conferences and
Publishing International Limited.
Lebreton, P., Raake, A., Barkowsky, M., & Le Callet, P. (2012). Evaluating Depth Perception of 3D Stereoscopic Videos. IEEE
Journal of Selected Topics in Signal Processing, 6(6). Retrieved from
http://ieeexplore.ieee.org/document/6269042/
Lynn, W. J. (2010). Defending a New Domain: The Pentagon's Cyberstrategy. Foreign Affairs, 89(5), 97-108. Retrieved from
https://city.rl.talis.com/items/71ACA705-9A0A-8C2B-EAD3-5FF314AAC847.html
Maddix, K. (2015). Big Data VR Challenge Winners! Retrieved from Masters of Pie: http://www.mastersofpie.com/big-data-
vr-challenge-winners/
Marty, R. (2008). Applied Security Visualization.
Meyerovich, L., & Tomasello, P. (2016). Display Relationships Between Data. IQT Quarterly, 7(4).
Munzner, T. (2014). Visualization Analysis & Design. A K Peters/CRC Press.
Payer, G., & Trossbach, L. (2015). The Application of Virtual Reality for Cyber Information Visualization and Investigation. In
M. Blowers, Evolution of Cyber Technologies and Operations to 2035 (Vol. 63, pp. 71-90). Springer.
doi:10.1007/978-3-319-23585-1_6
Pruett, C. (2017, 05 17). Vision 2017 - Lessons from Oculus: Overcoming VR Roadblocks. Retrieved from
https://youtu.be/swA8cm8r4iw?t=9m42s
Rajivan, P., Konstantinidis, E., Ben-Asher, N., & Gonzalez, C. (2016). Categorization of Events in Security Scenarios: The Role
of Context and Heuristics. Human Factors and Ergonomics Society Annual Meeting, 60(1), 274-278.
Reda, K., Febretti, A., Knoll, A., Aurisano, J., Leigh, J., Johnson, A., . . . Hereld, M. (2013). Visualizing large, heterogeneous data
in hybrid-reality environments. IEEE Computer Graphics and Applications, 33(4), 38-48.
Reuille, T., Hawthorne, S., Hay, A., Matsusaki, S., & Ye, C. (2015). OpenDNS Data Visualization Framework. Retrieved from
OpenGraphiti: http://www.opengraphiti.com/
Schneider, W., Dumais, S. T., & Shiffrin, R. N. (1982). Automatic and Control Processing and Attention. Illinois: University of
Illinois. Retrieved from http://www.dtic.mil/dtic/tr/fulltext/u2/a115078.pdf
Sethi, A., & Wills, G. (2017). Expert-interviews led analysis of EEVi A model for effective visualization in cyber-security.
IEEE Symposium on Visualization for Cyber Security (pp. 1-8). Phoenix, AZ, USA: IEEE.
Shearer, G., & Edwards, J. (2018). Vids: Version 2.0 Alpha Visualization Engine. Adelphi: US Army Research Laboratory.
Shneiderman, B. (1996). The eyes have it: a task by data type taxonomy for information visualizations. Proceedings 1996 IEEE
Symposium on Visual Languages. Boulder, CO, USA, USA: IEEE. doi:10.1109/VL.1996.545307
Smallman, H. S., St. John, M., Oonk, H. M., & Cowen, M. B. (2001). Information availability in 2D and 3D displays. IEEE
Computer Graphics and Applications, 21(5), 51-57. Retrieved from
http://journals.sagepub.com/doi/pdf/10.1518/001872001775992534
St. John, M., Cowen, M. B., Smallman, H. S., & Oonk, H. M. (2001). The Use of 2D and 3D Displays for Shape-Understanding
versus Relative-Position Tasks. Human Factors, Spring, 79-98. Retrieved from
http://journals.sagepub.com/doi/pdf/10.1518/001872001775992534
U.S. ARL. (2018, 02 12). SEEING THE CYBERTHREAT. Aberdeen Proving Ground, Maryland, USA: U.S. Army Research
Laboratory. Retrieved from https://dodstem.us/sites/default/files/lab-narratives/Seeing-the-Cyberthreat_0.pdf
van Riel, J.-P., & Irwin, B. (2006). InetVis, a Visual Tool for Network Telescope Traffic Analysis. AFRIGRAPH 2006. Cape Town:
Association for Computing Machinery, Inc.
Ward, M. O., Grinstein, G., & Keim, D. (2015). Interactive Data Visualization: Foundations, Techniques, and Applications,
Second Edition. A K Peters/CRC Press .
Ware, C., & Franck, G. (1996, 4). Evaluating Stereo and Motion Cues for Visualizing Information Nets in Three Dimensions.
ACMTransactions on Graphics, 15(2), 121-140. doi:10.1145/234972.234975
Wu, Y., Xu, L., Chang, R., Hellerstein, J. M., & Wu, E. (2018). Making Sense of Asynchrony in Interactive Data. JOURNAL OF
LATEX CLASS FILES, 14(8).
... One application was created as VR but also was designed to be easily changed to be MR, which hints to the direction of crossreality applications in the future in this domain. 5,23 Reports assessed for eligibility. ...
... Their qualitative study with 10 cybersecurity analysts revealed that this VR visualization was better than cluttered 2D screen representations or printed paper-based representations of the topologies, also analysts showed interest in incorporating this visualization in their workflow. 23 Additionally, in another study Ask et al. 22 showed that cadet teams that were using the VDE as 3D mixed reality applications had more situational awareness and better communication than teams using 2D visualization. Kabil et al. used 3D CyberCOP, a VR application that models cybersecurity activities to facilitate cybersecurity training and collaborative efforts, by providing visualization solutions. ...
... Other limitations were related to HoloLens device size, weight, field of view, gesture recognition customization, and overheating, which suggest the need for improvement of the device for optimal use. 22,35 Some studies reported that participants suffered from motion sickness or discomfort, 18,19,23 which might be the result of less-than optimal design of the VR environments, movement, and interactions. Others reported that XR technologies might need training to use them so this training should be provided before the study for a better experience for the participants. ...
... The effectiveness of information sharing platforms may depend on whether users know how to use them correctly (Brilingaitė et al., 2022). Other communication tools that help facilitate shared mental models of events in cyber space could be indicating how an attacker got access to a protected network on 2D schematics/diagrams of the network topology (Kullman et al., 2019b). Such solutions, however, do not scale well with increased complexity and their static nature does not allow for exploration of interactions between nodes in the network. ...
... Such solutions, however, do not scale well with increased complexity and their static nature does not allow for exploration of interactions between nodes in the network. Additional communication tools that allow for exploration and increased complexity could for example be graph representations of the network traffic in packet capture (pcap) software or radial diagrams (Kullman et al., 2019b). software rely solely on the visual sensory system to encode information when orienting in the network. ...
... To investigate the perceived usefulness of VDE for cyber analysts, Kullman and colleagues (2019b) conducted semi-structured interviews with ten subject matter experts working as analysts (mean experience = 4.5 years) to sample their impressions. They used network traffic data from the 2018 NATO CCDCOE CDX Locked Shields to visualize forty minutes of network traffic in dynamic and interactive 2D and 3D representations (Kullman et al., 2019b). 2D visualizations were presented as radial diagrams of source addresses and destination addresses using Kibana [https://www.elastic.co/products/kibana] ...
... It has been shown that first-time users tend to intuitively reach out to the data representations, as if to verify the existence of these artificial objects hanging in front of them [27]. Haptic feedback (using controllers), auditory cues, and realistic shaders further enhance users' immersion in an ISPMDV environment. ...
... To create and customize ISPMDVs for CSMEs with CSMEs, Virtual Data Explorer (VDE) software was created with US Army Research Lab support [27]. VDE has three components: ...
... Feedback from analysts on the ISPMDV shown in Figures 2-4 is covered in [27]. Overall, the impressions of stereoscopically perceivable 3D data visualizations were highly favorable, with multiple participants acknowledging that such 3D visualizations of network topology could assist in their understanding of the networks they use daily. ...
Article
Full-text available
Interactive Data Visualizations (IDV) can be useful for cybersecurity subject matter experts (CSMEs) while they are exploring new data or investigating familiar datasets for anomalies, correlating events, etc. For an IDV to be useful to a CSME, interaction with that visualization should be simple and intuitive (free of additional mental tasks) and the visualization’s layout must map to a CSME’s understanding. While CSMEs may learn to interpret visualizations created by others, they should be encouraged to visualize their datasets in ways that best reflect their own ways of thinking. Developing their own visual schemes makes optimal use of both the data analysis tools and human visual cognition. In this article, we focus on a currently available interactive stereoscopically perceivable multidimensional data visualization solution, as such tools could provide CSMEs with better perception of their data compared to interpreting IDV on flat media (whether visualized as 2D or 3D structures).
... Encoding information into depth cues while visualizing data has been avoided in the past for a good reason: on a flat screen, it's not helpful [19]. Nevertheless, recent studies have confirmed [23] that with equipment that provides the user with stereoscopic perception and parallax, three-dimensional shapes can be useful in providing users with insight into the visualized dataset [12]. Additionally, researchers have found that test subjects managed to gather data and to understand the cyber situation presented to them only after few sessions with great performance scores, even if the task seemed difficult to them on the first try [8]. ...
... These lessons were learned while developing VDE and adjusted later, as others reported success with the same or similar mitigation efforts [20]. Most importantly, if an immersed user can only move the viewpoint (e.g., its avatar) either forwards or backwards in the direction of user's gaze (or head-direction), the effects of simulator sickness can be minimized or avoided altogether [12]. This form of constrained navigation in VR is known as "the rudder movement" [20]. ...
... The ability to interact with the visualization, namely, to query information about a visual representation of a datapoint (ex: semi-transparent cube for a node or line for a relation between two nodes) using input devices (ex: hand-and fingertracking, input controllers) is imperative. While gathering feedback from SMEs [12], this querying capability was found to be crucial for the users' immersion in the VR data visualization to allow them to explore and to build their understanding of the visualized data. ...
Chapter
Full-text available
Cybersecurity practitioners face the challenge of monitoring complex and large datasets. These could be visualized as time-varying node-link graphs, but would still have complex topologies and very high rates of change in the attributes of their links (representing network activity). It is natural, then, that the needs of the cybersecurity domain have driven many innovations in 2D visualization and related computerassisted decision making. Here, we discuss the lessons learned while implementing user interactions for Virtual Data Explorer (VDE), a novel system for immersive visualization (both in Mixed and Virtual Reality) of complex time-varying graphs. VDE can be used with any dataset to render its topological layout and overlay that with time-varying graph; VDE was inspired by the needs of cybersecurity professionals engaged in computer network defense (CND). Immersive data visualization using VDE enables intuitive semantic zooming, where the semantic zoom levels are determined by the spatial position of the headset, the spatial position of handheld controllers, and user interactions (UIa) with those controllers. This spatially driven semantic zooming is quite different from most other network visualizations which have been attempted with time-varying graphs of the sort needed for CND, presenting a broad design space to be evaluated for overall user experience (UX) optimization. In this paper, we discuss these design choices, as informed by CND experts, with a particular focus on network topology abstraction with graph visualization, semantic zooming on increasing levels of network detail, and semantic zooming to show increasing levels of detail with textual labels.
... The example of research by Yang [201], discussed in Chapter 2, is representative of most efforts in the field of immersive 3D data analysis. Research by Kullman et al. [202] also involved experimentation with an immersive data visualisation tool for cybersecurity with 10 participants. Kullman's research is presented as a usability assessment for 3D ...
Thesis
Full-text available
The intent of this research is to develop and assess the application of 3D data visualisation to the field of computer security. The growth of available data relating to computer networks necessitates a more efficient and effective way of presenting information to analysts in support of decision making and situational awareness. Advances in computer hardware and display software have made more complex and interactive presentation of data in 3D possible. While many attempts at creation of data-rich 3D displays have been made in the field of computer security, they have not become the tool of choice in the industry. There is also a limited amount of published research in the assessment of these tools in comparison to 2D graphical and tabular approaches to displaying the same data. This research was conducted through creation of a novel abstraction framework for visualisation of computer network data, the Visual Interactive Network Analysis Framework (VINAF). This framework was implemented in software and the software prototype was assessed using both a procedural approach applied to a published forensics challenge and also through a human participant based experiment. The key contributions to the fields of computer security and data visualisation made by this research include the creation of a novel abstraction framework for computer network traffic which features several new visualisation approaches. An implementation of this software was developed for the specific cybersecurity related task of computer network traffic analysis and published under an open source license to the cybersecurity community. The research contributes a novel approach to human-based experimentation developed during the COVID-19 pandemic and also implemented a novel procedure-based testing approach to the assessment of the prototype data visualisation tool. Results of the research showed, through procedural experimentation, that the abstraction framework is effective for network forensics tasks and exhibited several advantages when compared to alternate approaches. The user participation experiment indicated that most of the participants deemed the abstraction framework to be effective in several task related to computer network traffic analysis. There was not a strong indication that it would be preferred over existing approaches utilised by the participants, however, it would likely be used to augment existing methods.
... In addition, there are geo-referenced visualization charts for assets [41,43,44], risks [45][46][47] and threats [41,44]. Furthermore, there are also immersive visualization techniques using 3D models instead of 2D models which have been designed for optimum visualization with an ultra-wide high-definition screen, wrap-around screen or three-dimensional Virtual-Reality (VR) goggles, which allows the user to look around 360 degrees while moving [42,44,[48][49][50]. ...
Article
Full-text available
The number and the diversity in nature of daily cyber-attacks have increased in the last few years, and trends show that both will grow exponentially in the near future. Critical Infrastructures (CI) operators are not excluded from these issues; therefore, CIs’ Security Departments must have their own group of IT specialists to prevent and respond to cyber-attacks. To introduce more challenges in the existing cyber security landscape, many attacks are unknown until they spawn, even a long time after their initial actions, posing increasing difficulties on their detection and remediation. To be reactive against those cyber-attacks, usually defined as zero-day attacks, organizations must have Threat Hunters at their security departments that must be aware of unusual behaviors and Modus Operandi. Threat Hunters must face vast amounts of data (mainly benign and repetitive, and following predictable patterns) in short periods to detect any anomaly, with the associated cognitive overwhelming. The application of Artificial Intelligence, specifically Machine Learning (ML) techniques, can remarkably impact the real-time analysis of those data. Not only that, but providing the specialists with useful visualizations can significantly increase the Threat Hunters’ understanding of the issues that they are facing. Both of these can help to discriminate between harmless data and malicious data, alleviating analysts from the above-mentioned overload and providing means to enhance their Cyber Situational Awareness (CSA). This work aims to design a system architecture that helps Threat Hunters, using a Machine Learning approach and applying state-of-the-art visualization techniques in order to protect Critical Infrastructures based on a distributed, scalable and online configurable framework of interconnected modular components.
Article
Full-text available
Cyber security analysts use data visualizations to speed up ingestion of security data. These visualizations typically take the form of 2D graphics displayed on computer monitors. Virtual reality has the potential to improve these visualizations with immersive 3D environments and unique interaction mechanics. However, research into this newly synergised area lacks evaluation, leading to unfounded claims of effectiveness. A potential cause for these missing evaluations was identified as a lack of guidance detailing how evaluations should be conducted in this area. Additionally, the small amount of research that does include evaluation incorrectly relies on subjective participant opinions to objectively measure system effectiveness. An example of this misuse is asking participants which system they thought was quicker, rather than timing them. The objective of this paper was to propose a solution to these issues in the form of a surveyed, categorised, and analysed set of evaluation metrics. A total of 49 metrics were identified from 41 papers. The presented metrics detail which dependent variables should be considered when evaluating works in the combined fields of cyber security, data visualization, and virtual reality. These metrics can be used to produce more accurate evaluations in future works in this area.
Article
Full-text available
Background Cyber defense decision-making during cyber threat situations is based on human-to-human communication aiming to establish a shared cyber situational awareness. Previous studies suggested that communication inefficiencies were among the biggest problems facing security operation center teams. There is a need for tools that allow for more efficient communication of cyber threat information between individuals both in education and during cyber threat situations. Methods In the present study, we compared how the visual representation of network topology and traffic in 3D mixed reality vs. 2D affected team performance in a sample of cyber cadets ( N = 22) cooperating in dyads. Performance outcomes included network topology recognition, cyber situational awareness, confidence in judgements, experienced communication demands, observed verbal communication, and forced choice decision-making. The study utilized network data from the NATO CCDCOE 2022 Locked Shields cyber defense exercise. Results We found that participants using the 3D mixed reality visualization had better cyber situational awareness than participants in the 2D group. The 3D mixed reality group was generally more confident in their judgments except when performing worse than the 2D group on the topology recognition task (which favored the 2D condition). Participants in the 3D mixed reality group experienced less communication demands, and performed more verbal communication aimed at establishing a shared mental model and less communications discussing task resolution. Better communication was associated with better cyber situational awareness. There were no differences in decision-making between the groups. This could be due to cohort effects such as formal training or the modest sample size. Conclusion This is the first study comparing the effect of 3D mixed reality and 2D visualizations of network topology on dyadic cyber team communication and cyber situational awareness. Using 3D mixed reality visualizations resulted in better cyber situational awareness and team communication. The experiment should be repeated in a larger and more diverse sample to determine its potential effect on decision-making.
Book
The main goal of the field of augmented cognition is to research and develop adaptive systems capable of extending the information management capacity of individuals through computing technologies. Augmented cognition research and development is therefore focused on accelerating the production of novel concepts in human-system integration and includes the study of methods for addressing cognitive bottlenecks (e.g., limitations in attention, memory, learning, comprehension, visualization abilities, and decision making) via technologies that assess the user’s cognitive status in real time. A computational interaction employing such novel system concepts monitors the state of the user, through behavioral, psychophysiological, and neurophysiological data acquired from the user in real time, and then adapts or augments the computational interface to significantly improve their performance on the task at hand. The International Conference on Augmented Cognition (AC), an affiliated conference of the HCI International (HCII) conference, arrived at its 16th edition and encouraged papers from academics, researchers, industry, and professionals, on a broad range of theoretical and applied issues related to augmented cognition and its applications. The field of augmented cognition has matured over the years to solve enduring issues such as portable, wearable neurosensing technologies and data fusion strategies in operational environments. These innovations coupled with better understanding of brain and behavior, improved measures of brain state change, and improved artificial intelligence algorithms have helped expand the augmented cognition focus areas to rehabilitation, brain-computer interfaces, and training and education. The burgeoning field of human-machine interfaces such as drones and autonomous agents are also benefitting from augmented cognition research. This volume of the HCII 2022 proceedings is dedicated to this year’s edition of the AC conference and focuses on topics related to understanding human cognition and behavior, brain activity measurement and electroencephalography, human and machine learning, and augmented cognition in extended reality. Papers of this one volume are included for publication after a minimum of two single-blind reviews from the members of the AC Program Board or, in some cases, from members of the Program Boards of other affiliated conferences. We would like to thank all of them for their invaluable contribution, support, and efforts.
Technical Report
Full-text available
Information superiority is one of the primary enablers for military dominance; the exploitation of all relevant information from multiple sources is a key factor for NATO’s information superiority. Visualization and visual analytics research are essential to address the needs of the 2015 NATO targets of emphasis in Information Analysis (IA) & Decision Support (DS): IA&DS-1 on Decision Support and IA&DS-2 on Big Data & Long Data Processing & Analysis. Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces. The Group investigated, researched and fostered collaborations in knowledge extraction and data analysis for timely situation awareness to support effective decision making. The IST-141 group researched, developed and applied exploratory visual analytics techniques: (1) to exploit and make sense of large and complex data sets, i.e. Big Data; (2) to help make tacit knowledge explicit; (3) to provide acute situation awareness, and; (4) to support informed decision making across a wide range of defence and security application domains including cyber, maritime, genomics and social media domains, as well as post analysis and in situ visualization for simulation data.
Conference Paper
Full-text available
The human visual system is generally more adept at inferring meaning from graphical objects and natural scene elements than reading alphanumeric characters. Graphical objects like charts and graphs in cybersecurity dashboards often lack the requisite numbers of features to depict behaviors of complex network data. For example, bar charts afford few features to encode a panoply of parameters in network data. Furthermore, dashboard visualizations seldom support the transition of human work from situation awareness building to requisite responses during intrusion detection events. This research effort aims to identify how graphical objects (also referred as data-shapes) depicted in Virtual Reality tools, developed in accordance with an analyst’s mental model of an intrusion detection event, can enhance analyst’s situation awareness. We demonstrate the proposed approach using Locked Shields 16 CDX network traffic. Implications of this study and future case study are discussed.
Conference Paper
Full-text available
The area of visualization in cyber-security is advancing at a fast pace. However, there is a lack of standardized guidelines for designing and evaluating the resulting visualizations. Furthermore, limited end-user involvement in the design process leads to visualizations that are generic and often ineffective for cyber-security analysts. Thus, the adoption of the resultant cyber-security visualizations is low and this highlights a major research gap. This paper presents expert-interview based validation of EEVi - a model developed to aid in the design and evaluation process of cyber-security visualizations, with a view to make them more effective for cyber-security analysts. A visualization is considered effective if the characteristics of the visualization are essential for an analyst to competently perform a certain task. Thirteen experts were interviewed (six visualization designers and seven cyber-security analysts) and their feedback guided revisions to the model. The responses were subsequently transposed from qualitative data to quantitive data in order to perform statistical analyses on the overall data. This demonstrated that the perspectives of visualization designers and cyber-security analysts generally agreed in their views of effective characteristics for cyber- security visualization, however there was no statistically significant correlation in their responses.
Article
Full-text available
Four different platforms were compared in a task of exploring an angular stimulus and reporting its value. The angle was explored visually, tangibly as raised fine-grit sandpaper, or on a touch-screen with a frictional or vibratory signal. All platforms produced highly accurate angle judgments. Differences were found, however, in exploration time, with vision fastest as expected, followed by tangible, vibration, and friction. Relative to the tangible display, touch-screens evidenced greater noise in the perceived angular value, with a particular disadvantage for friction. The latter must be interpreted in the context of a first-generation display and a rapidly advancing technology. On the whole, the results point both to promise and barriers in the use of refreshable graphical displays for blind users.
Article
Control of air defense operations - antiair warfare - requires commanders to interpret and act on computer-generated graphical representations of aircraft traffic throughout a defended area. A watch officer's ability to interpret this virtual environment rapidly and correctly determines operational success. As part of a continuing effort to improve coordination of antiair warfare, the Applied Physics Laboratory is building a prototype system having as one of its functions a three-dimensional perspective display that can improve the commander's effectiveness in assessing a tactical situation. A distributed, object-oriented computer program has been developed to provide the performance and adaptability required for this approach to graphical display.
Chapter
Performing the analysis of security data in the prevention of cyber-attacks on an organization’s information systems requires human analysts to make sense of ever-expanding amounts of information. In many security operation centers (SOCs), human analysts are presented with information through the use of multiple monitors. Information is processed using a number of commercial off-the-shelf and custom tools in order to carve information into sets of alerts that analysts can investigate. The amount and complexity of the data being presented to the analyst can significantly overwhelm a single or multiple displays. This avalanche of display information is alongside the additional research an analyst must perform in order to provide proper context to the alerts analysts may be investigating. Analyst investigations can include a number of competing interfaces. A non-exhaustive list includes web browsers with numerous tabs, documents, collaboration software, and both Graphical User Interface (GUI) and Command Line Interface (CLI) based command and control software. Adding additional monitors can lead to a diminishing rate of return in information processing as analysts now must physically observe multiple panels in fixed positions. With a virtual reality (VR) head-mounted (VRH) display, the display space for visualizing different information and data pertaining to cyber events becomes almost limitless. The information being displayed is no longer specifically restricted to a few small rectangular displays but is perceived as nearly infinite space. The OR can open the door to developing significantly more advanced VR experiences. Using more advanced VR technology not only can more information be displayed, but VR displays open the door to new and innovative visualization techniques, which enables us to model security information in new ways and which allows for the efficient identification of malicious behavior within information systems.
Book
Our society has entered a data-driven era, one in which not only are enormous amounts of data being generated daily but there are also growing expectations placed on the analysis of this data. Some data have become simply too large to be displayed and some have too short a lifespan to be handled properly with classical visualization or analysis methods. In order to address these issues, this book explores the potential solutions where we not only visualize data, but also allow users to be able to interact with it. Therefore, this book will focus on two main topics: large dataset visualization and interaction.Graphic cards and their image processing power can leverage large data visualization but they can also be of great interest to support interaction. Therefore, this book will show how to take advantage of graphic card computation power with techniques called GPGPUs (general-purpose computing on graphics processing units). As specific examples, this book details GPGPU usages to produce fast enough visualization to be interactive with improved brushing techniques, fast animations between different data representations, and view simplifications (i.e. static and dynamic bundling techniques).Since data storage and memory limitation is less and less of an issue, we will also present techniques to reduce computation time by using memory as a new tool to solve computationally challenging problems. We will investigate innovative data processing techniques: while classical algorithms are expressed in data space (e.g. computation on geographic locations), we will express them in graphic space (e.g., raster map like a screen composed of pixels). This consists of two steps: (1) a data representation is built using straightforward visualization techniques; and (2) the resulting image undergoes purely graphical transformations using image processing techniques. This type of technique is called image-based visualization.The goal of this book is to explore new computing techniques using image-based techniques to provide efficient visualizations and user interfaces for the exploration of large datasets. This book concentrates on the areas of information visualization, visual analytics, computer graphics, and human-computer interaction. This book opens up a whole field of study, including the scientific validation of these techniques, their limitations, and their generalizations to different types of datasets.
Article
As the previous chapters emphasized, the human cognition—and the technology necessary to support it—are central to Cyber Situational Awareness. Therefore, this chapter focuses on challenges and approaches to integration of information technology and computational representations of human situation awareness. To illustrate these aspects of CSA, the chapter uses the process of intrusion detection as a key example. We argue that effective development of technologies and processes that produce CAS in a way properly aligned with human cognition calls for cognitive models—dynamic and adaptable computational representations of the cognitive structures and mechanisms involved in developing SA and processing information for decision making. While visualization and machine learning are often seen among the key approaches to enhancing CSA, we point out a number of limitations in their current state of development and applications to CSA. The current knowledge gaps in our understanding of cognitive demands in CSA include the lack of a theoretical model of cyber SA within a cognitive architecture; the decision gap, representing learning, experience and dynamic decision making in the cyberspace; and the semantic gap, addressing the construction of a common language and a set of basic concepts about which the security community can develop a shared understanding.