ChapterPDF Available

HackIT: A Human-in-the-Loop Simulation Tool for Realistic Cyber Deception Experiments

Authors:

Abstract and Figures

Deception, an art of making someone believe in something that is not true, may provide a promising real-time solution against cyber-attacks. Honeypots, fake servers that pretend to be real, are one of the effective deception tools in the network defense to lure adversaries. In this paper, we propose a human-in-the-loop real-world simulation tool called HackIT, which could be config-ured to create different cyber-security scenarios involving deception. HackIT allows cyber-security researchers to create various deception situations in the laboratory involving honeypots and it helps evaluate the decision processes of human adversaries, who try to attack a simulated computer network. We discuss how researchers can use HackIT to create networks of different sizes; use deception and configure different web servers as honeypots; and, create any number of fictitious ports, services, fake operating systems, and fake files on honeypots. Next, we report a case-study involving HackIT where adversaries were tasked with stealing information from a simulated network over multiple rounds. In one condition in HackIT, deception occurred early; and, in the other condition, it occurred late. Results revealed that participants used different attack strategies across the two conditions. Overall, the case study was a step towards showing how adversarial decisions could be studied in HackIT in the presence of deception. We discuss the potential of using HackIT in helping cyber-security teams understand adversarial cognition in the laboratory.
Content may be subject to copyright.
HackIT: A Human-in-the-loop Simulation Tool for
Realistic Cyber Deception Experiments
Palvi Aggarwal1, Aksh Gautam2, Vaibhav Aggarwal2,
Cleotilde Gonzalez1 and Varun Dutt2
1 Dynamic Decision Making Lab, Carnegie Mellon University, Pittsburgh, USA
2 Applied Cognitive Science Laboratory, Indian Institute of Technology Mandi, India
palvia@andrew.cmu.edu, aksh@student.iitmandi .ac.in, vaibhav@student.iitmandi.ac.in,
coty@cmu.edu, varun@iitmandi.ac.in
Abstract. Deception, an art of making someone believe in something that is not
true, may provide a promising real-time solution against cyber-attacks. In this
paper, we propose a human-in-the-loop real-world simulation tool called
HackIT, which could be configured to create different cyber-security scenarios
involving deception. We discuss how researchers can use HackIT to create net-
works of different sizes; use deception and configure different webservers as
honeypots; and, create any number of fictitious ports, services, fake operating
systems, and fake files on honeypots. Next, we report a case-study involving
HackIT where adversaries were tasked with stealing information from a simu-
lated network over multiple rounds. In one condition in HackIT, deception oc-
curred early; and, in the other condition, it occurred late. Results revealed that
participants used different attack strategies across the two conditions. We dis-
cuss the potential of using HackIT in helping cyber-security teams understand
adversarial cognition in the laboratory.
Keywords: Cybersecurity · Simulation tools · Learning · Attack · Hackers · De-
fenders · Honeypots
1 Introduction
Deception is an art of making someone believe in something that is not true, may
provide a promising real-time solution against cyber-attacks [1]. Deception involves
interaction between two parties, a target and a deceiver, in which the deceiver effec-
tively causes the target to believe in a false description of reality [1]. The objective is
to cause the target to work in such a way that is beneficial to the deceiver. Deception
has been used as an offensive and defensive tool in cyber world by hackers and de-
fenders. Hackers used deception for exploiting cyber infrastructure, stealing infor-
mation, making money and defaming people. The deception techniques used by hack-
ers may involve malware signature, conceal code and logic, encrypted exploits, spoof-
ing, phishing, and social engineering (e.g., by deceiving help desk employees to in-
stall malicious code or obtain credentials). However, defenders used deception for
securing network infrastructure, luring hackers to fake information and understanding
Deleted: Honeypots, fake servers that pretend to be real, are
one of the effective deception tools in the network defense to
lure hackers. In this paper, we propose a h
Deleted: HackIT allows cyber-security researchers to create
various deception situations in the laboratory involving honey-
pots and it helps evaluate the decision processes of human adver-
saries, who try to attack a simulated computer network. We
Deleted: Overall, the case study was a step towards showing
how adversarial decisions could be studied in HackIT in the
presence of deception. We discuss the potential of using HackIT
hacker’s motives and strategies using tools such as honeypots. When used for defense,
cyber defenders may use feints and deceits to thwart hackers’ cognitive processes,
delay attack activities, and disrupt the breach process. When used for defense, decep-
tion may be achieved through miss-directions, fake responses, and obfuscations [2].
These techniques rely upon hacker’s trust in response from network, data, and appli-
cations during an actual attack. To create deception for defense, security experts have
been using honeypots, fake servers that pretend to be real, for gathering intelligence
about hackers. Honeypots are one of the effective deception tools in the network de-
fense to lure hackers.
Decisions-making process of hackers and defenders in cyber world is complex
task. Understanding their decision process in such complex environment is challeng-
ing. Simulation has been used as an effective way of understanding the hackers’ and
defenders’ decisions, testing new solutions for security, and training the models and
people in such complex task scenarios [3-10] have used behavioral game theoretic
approaches to understand the decisions of hackers and defenders in the abstract cyber-
security games. In such game theoretic approaches, the network structure and the set
of actions of hackers and defenders were abstracted to attack/not-attack and de-
fend/not-defend. Furthermore, the information provided to the participants was also
abstracted. The task simulated using game-theoretic approaches was less cognitively
challenging compared to the real cyber-security tasks. Thus, the conclusions made
based on behavioral game-theoretic approaches may or may not address the cyberse-
curity problems. Another approach used to study hackers and defenders behaviour
involve complex real time tools such as NeSSi, Network Simulator-2/3, Cymulate etc
[11-12]. However, using deception in uncontrolled environments makes it difficult to
answer cause-effect questions.
Aggarwal et al. [6] proposed HackIT tool to bridge the gap between behavioral
game-theoretic approaches and real-world cybersecurity tasks. HackIT tool provided
features to create more specific set of actions and information needed for cyber-
security tasks for both hackers and defenders. The HackIT tool was used to replicate
the results of a laboratory experiment using a deception game [3-4]. Results revealed
that the average proportion of attacks was lower and not-attack actions were higher
when deception occurred late in the game rather than earlier; and when the amount or
deception was high compared to low [6]. This result found in an abstract simplified
scenario was replicated in a real-world simulation tool called the HackIT.
In [6], the HackIT tool was available for creating deception with limited number
of systems only and for single player games only. In this paper, we define the en-
hanced capabilities of HackIT tool. Specifically, we detail how the HackIT tool is
capable of running experiments with different sized networks, different network con-
figurations, different deception strategies, and single player and multiplayer games.
Thus, the enhanced capabilities of HackIT tool can help us answer several questions
such as the effect of different network sizes and honeypot allocations on hacker’s
performance and the most effective way to present the “clues” of deception in net-
work nodes.
In what follows, we first discuss the functioning and different features in the of the
HackIT tool. Next, we detail an experiment involving the HackIT tool to showcase its
capabilities. Furthermore, we detail the results from the experiment and discuss the
Deleted: cyber
Deleted:
Deleted: challenged
Deleted:
Deleted: game
Deleted: cyber security
Deleted:
Deleted: cyber security
Deleted: cyber
Deleted: Aggarwal et al (2018) proposed HackIT tool for two
webservers only. The HackIt
Deleted: HackIt
Deleted: designed to investigate the role of deception (i.e., amount
and timing) on a hacker’s decisions
Formatted: Font: Not Italic
Deleted: [
Deleted:
Deleted: the
Deleted: ,
Deleted: the
Deleted: /distribution
Deleted: ,
Deleted: identifying
Deleted: of the HackIt tool
Deleted:
Deleted: explain the methods of
Deleted: run
Deleted: using
Deleted: HackIt
Deleted: from HackIt tool
implications of our results for investigating the decision-making of hackers in the real
world.
2 HackIT
HackIT is a generic framework for cybersecurity tasks to study human learning and
decision-making of hackers and analysts. It represents a simplified framework con-
taining the most essential elements for creating cybersecurity scenarios: network
nodes, which represents the characteristics of real nodes; strategies, which can be
configured for creating deception; and commands, which are used for communication
with the network. The analyst’s goal in the HackIT task is to evaluate different decep-
tion strategies and the hacker’s goal to identify the real network nodes and exploit
them. Hackers communicate with the network in HackIT using different commands
and gain information about different configurations. However, hackers are not aware
of the strategies used by analysts inside the HackIT scenario. Hackers basically learn
about these strategies overtime by playing different rounds. Thus, HackIT is a control-
lable and flexible simulation tool with the capability of creating various network sce-
narios and experiment with different techniques to lure hackers. Figure 1 shows the
typical flow of HackIT tool which uses the concept of stackelberg security games
[13], where first defenders create a network and use their defense techniques and next,
hackers try to exploit the security of the network.
Fig. 1. HackIT tool
HackIT is an experimental cybersecurity tool that allow analysts to simulate cyber
infrastructure during configuration phase and define the deception algorithm in decep-
tion phase. Next, these scenarios are presented to the hackers for exploitation. Hack-
ers use this tool in two phases: probe phase and attack phase. The probe phase in-
volves the process of reconnaissance where hackers gather information about open
port, services, operating systems, and available vulnerabilities using tools such as
nmap; whereas, the attack phase involves gaining access to different computers and
stealing information or compromising computer systems.
Despite the simplicity of HackIT, the tool has the potential to simulate many real-
world dynamic situations in the laboratory: testing different proportion of honeypots
in the network, testing optimal placement of honeypots, different configurations of
honeypots, such as, easy to exploit ports on honeypots, and availability of fake files
Deleted: attackers
Deleted:
Deleted: cyber security
Deleted: HackIt
Deleted: The h
Deleted: s
Deleted: the
Deleted: is
Deleted: defender
Deleted: s
Deleted: -
Deleted: defenders
on honeypots, availability of unpatched ports on honeypot etc. Specific characteristics
of HackIT tool are explained as followed:
2.1 Network Size
The HackIT tool is flexible to create any number of computers in the network. The
configuration of these systems will also be dynamically created. Thus, this tool can
allow researchers to easily run experiments with small, medium and large scaled net-
works as shown in Figure 2.
Fig. 2. Different Network Sizes: A) Small, B) Medium, and C) Large
2.2 Allocation of honeypots and regular computers
The HackIT tool is capable of creating any proportion of network computers as
honeypots, where honeypots are fake computer pretending to be real. This tools also
provides a functionality to define the features of honeypot that make them pretend as
real systems. These features include the ability to configure vulnerable ports, vulnera-
ble operating systems, and proportion of fake files on the honeypots. Thus, setting
different proportion of honeypots and defining deception via honeypots is relatively
easy in HackIT.
2.3 Configuration of Computers
The configuration of honeypots and regular computers is automatically generated
by a script in HackIT that would produce the configuration for a fixed number of
systems with a given proportion of honeypots and real systems. This script generates
the configuration consisting of real systems, honeypots, and files for each game round
in HackIT. By using this script, one could generate data onetime and encode it in the
experiment so that it would present all participants with same configuration. For ex-
ample, the regular systems could be configured as patched and difficult to exploit and
honeypot systems could be configured as easy to exploit. This configuration mapping
is shown in Table 1.
Table 1. Configuration of Honeypot and Regular systems
Easy to Attack
Difficult to Attack
Operating systems:
Windows Server 2003
Windows XP
HP-UX 11i
Solaris
Operating systems:
OpenBSD
Linux
Mac OS X
Windows 8
A
B
C
Deleted: a
Deleted: b
Deleted: c
Deleted: systems
Deleted: such as
Deleted: configuring
Deleted: etc
Deleted: easier
Deleted: of
Deleted: round
Deleted: i.e. the real systems, honeypots, and files
Deleted: we
Deleted: the
Deleted: the users
Formatted: Left
Formatted: Font color: Background 1
Formatted: Font color: Background 1
Formatted: Left
Formatted: Left
Formatted: Font color: Background 1
Services and ports:
21/tcp ftp
25/tcp smtp
80/tcp http
111/tcp rpcbind
135/tcp msrpc
Services and ports:
22/tcp-ssh
53/tcp-domain
110/tcp-pop3
139/tcp-netbios
443/tcp-https
445/tcp-microsoft-ds
3306/tcp-mysql
5900/tcp-vncc http
6112/tcp-dtspc
8080/tcp-apache
2.4 Content of Honeypots
HackIT tool provide a facility to create fictitious content on honeypots using fake
files. The proportion of fake files and useful files on a honeypot can be configured in
HackIT. Figure 3 shows the output of the ls command in the directory where only
pin.txt is a real file and rest are fake files. In HackIT tool, the number of files on each
server can be dynamically created. We tested our platform with the different number
of files ranging from 50-200.
Fig. 3. List of fake files on Honeypot Webserver
2.5 Single-player and Multi-player platform
HackIT tool provides a functionality to run single-player and multi-player experi-
ments. In single-player experiment settings, players cannot communicate with each
other. However, in multi-player experiment setup, players are provided with a chat
functionality to share their knowledge. Hackers usually penetrate into the network by
organizing themselves into a group. Hackers in a group rely on information gained
from fellow hackers who have already penetrated into the network or are trying to
penetrate it.
Deleted: factitious
Formatted: Font: Italic
Fig. 4 Chat functionality for multiplayer experiments in HackIT
2.6 Commands
The HackIT tool can run various network commands that include: nmap,
use_exploit, ls, and scp. Nmap is a network utility that shows the open ports, operating
system type, and services running on the specified webserver. The nmap utility also
provides the list of vulnerabilities on the corresponding webservers. The use_exploit
command exploits vulnerabilities of a system and helps attacker to gain access to a
particular webserver. Next, the ls command lists the files currently on the file system of
the machine. The scp command transfers files to the remote machine.
2.7 Measures for human-in-the-loop experiments
For understanding the decision-making process of hackers, it is important to ana-
lyze how do they gather information, their preferences while searching for infor-
mation, kind of exploits used, and most importantly how to they react to different
defense strategies. HackIT tool provides various measures to analyze the human per-
formance in cybersecurity experiments.
- Probing behavior: HackIT tool provides capability to record the probing behav-
ior of hacker participants. Probing is an important phase where hackers collect in-
formation before launching a real attack. Using deception in probing responses
and analyzing hackersreaction towards deceptive responses is an important
measure provided in HackIT tool.
- Attack behavior: HackIT measures the attack behavior of hackers by recording
their attack actions, exploit information, their specific selection of targets or con-
figuration. HackIT tool also records the vulnerabilities exploited, and exploits
used by hackers. Defenders can analyze this data to study their attack patterns.
Deleted: HackIt
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Deleted: Human
Deleted: T
Deleted: he
Deleted: Loop
Deleted: cyber security
Deleted: u
Deleted: their
Deleted: u
Deleted: provides
- Learning: HackIT tool can record overtime behavior of participants. Thus, it
provides data to analyze learning capabilities of hackers against different decep-
tion strategies.
- Communication History: HackIT tool provides functionality to analyze the chat-
ting history of hacker to investigate that how hackers make decisions or formu-
late strategies in team-based tasks.
- Time: HackIT tool also records time taken by hackers to exploit a system. Timing
is another measure to evaluate the success of deception.
3 CASE STUDY USING HACKIT TOOL
3.1 Experiment Design
In this paper, we demonstrated the ability of HackIT tool as a single-player plat-
form for conducting experiments. We analyzed the effect of timing of deception on
hacker’s actions in a study. The timing of deception was manipulated across two be-
tween-subject conditions: early deception (N =8) and late deception (N = 8). The total
number of servers were 40, where 20 webservers were assigned as honeypot webserv-
ers during deception rounds. In both conditions, participants playing as hackers were
given 6 game rounds in a sequence (end point unknown to participants), where 2-
game rounds possessed deception. In this experiment, if the timing of deception was
early, then deception was present on the second and third rounds in the sequence.
However, if the timing of deception was late, then deception was present in the fourth
and fifth rounds in the sequence (see Figure 5). The honeypots were easy to exploit
via common ports and vulnerabilities in the deception rounds compared to the non-
deception rounds, where there were no honeypots (more details ahead in this paper).
However, participants were not told that honeypots will involve easy to attack config-
urations in deception rounds. Also, participants were not disclosed the rounds on
which deception was involved. To analyze human data, we looked at the proportion of
honeypot attacks and proportion of regular attacks at the attack stage by the hacker
across six-rounds in each condition. These proportions were calculated by average the
attack decision over all the trials and participants. We also calculated frequency of
each exploits used on regular and honeypots in deception and no-deception trials.
Fig. 5 Experiment design using deception game with six rounds and two conditions, i.e. early decep-
tion and late deception. D: Deception Present -: Deception Not Present [1,2].
Deleted: u
Deleted: In this paper, we demonstrated the ability of HackIT tool
as a single-player platform for conducting experiments. The experi-
ment design was borrowed from [1] and scaled from 2 webservers to
40 webservers. We analyzed the effect of timing of deception on
hacker’s actions in a pilot study. The timing of deception was manip-
ulated across two between-subject conditions: early deception (N =8)
and late deception (N = 8). The total number of servers were 40
where 20 webservers were assigned as honeypot webservers during
deception rounds. In both conditions, participants playing as hackers
were given 6 game rounds in a sequence (end point unknown to
participants), where 2-game rounds possessed deception. Figure 5
shows the experimental design of deception game implemented in
HackIT tool. In this experiment, if the timing of deception was early,
then deception was present on the second and third rounds in the
sequence. However, if the timing of deception was late, then decep-
tion was present in the fourth and fifth rounds in the sequence. Pres-
ence of deception meant that the honeypots were easy to exploit via
popular ports and vulnerability in the deception rounds compared to
the non-deception rounds (more details ahead in this paper). Howev-
er, participants were not told that honeypots will involve easy to
attack configurations in deception rounds. Also, participants were not
disclosed the rounds on which deception was involved. To analyze
human data, we looked at the proportion of honeypot attacks and
proportion of regular attacks at the attack stage by the hacker across
six-rounds in each condition.
3.2 HackIT Task
The objective of attacker in HackIT was to steal real credit-card information locat-
ed on one of the webservers. As shown in Figure 1, defender first configure the net-
work with 40 webservers where 20 webservers acted as honeypots. Defender also set
up a strategy where honeypots were easier to attack during the deception rounds.
Based on these strategies, in this experiment, defender uses different configurations
shown in Table 2. For example, a system with Windows XP operating system, port
80/tcp, and service http was easily exploitable. Such a configuration was mapped to a
honeypot. However, a system with Linux operating system, port 22/tcp and service
ssh was difficult to attack. Such a configuration was mapped to a regular webserver.
Participants were informed about the easy to exploit and the difficult to exploit con-
figurations in Table 2 as part of the instructions.
Table 2. Configuration of Honeypot and Regular systems
Operating System
Ports
Exploits
Windows Server
2003
Windows XP
HP-UX 11i
Solaris
21/tcp ftp
25/tcp smtp
80/tcp http
111/tcp rpcbind
135/tcp msrpc
brute_force
directory_harvest
sql_injection
DDoS_attack
DoS_attack
OpenBSD
Linux
Mac OS X
Windows 8
22/tcp-ssh
53/tcp-domain
110/tcp-pop3
139/tcp-netbios
443/tcp-https
445/tcp-microsoft-
ds
3306/tcp-mysql
5900/tcp-vncc http
6112/tcp-dtspc
8080/tcp-apache
user_auth
DNS_zone_transfer
pop3_version
DCOM_buffer_overrun
drown_attack
windows_null_session
remove_auth
remote_auth
remote_exploit_buffer
url_decoder
First, the attacker probed the network using nmap command to gain information
about different webservers. Probing different webservers gave the information about
the operating system, open ports, services, and vulnerabilities. The information pro-
vided to the attacker as a result of probing systems gave him an idea about the type of
configuration on the probed system. Once the attacker collects information about open
ports and services, he could attack a webserver by using the “use_exploit” command.
The use_exploit command exploited vulnerabilities on a system and helped the at-
tacker to gain access to that webserver. Next, the attacker could list different files on
the exploited webserver by using the “ls” command. Next, the attacker could transfer
required files containing credit card information (e.g., “pin.txt”) using the “scp”
command. After attackers copied the file from the exploited system, he was informed
Deleted: HackIt
Deleted: In this paper, we simulated a network of forty webservers
in HackIT where 50% webservers were honeypots and others were
regular webservers. The objective of attacker was to steal real credit
card information located on one of these webservers. As shown in
Figure 1, defender first configure the network with 40 webservers
where 20 of the webservers acts as a honeypot webserver and another
one act as a real webserver. Next, defender defines the deception
strategy as honeypots are easy to attack and regular systems are
difficult to attack. Defender also sets up a strategy that honeypots will
only become easy to attack during the deception rounds shown in
figure 5. Based on these strategies, in this experiment, defender uses
the configuration shown in table 2. For example, a system with Win-
dows XP operating system, port 80/tcp, and service http will be easily
exploitable. However, a system with Linux operating system, port
22/tcp and service ssh will be difficult to attack. In deception rounds,
honeypot webservers were configured in such a way that attackers
could easily attack them. However, the regular webserver were diffi-
cult to attack in these deception rounds. In the experiment, partici-
pants were informed about the easy to exploit and the difficult to
exploit configurations in Table 2.
Formatted: Indent: First line: 0 cm
Deleted:
Strategy
... [1]
whether he was successful or not in stealing a real credit-card file from the computer
via a text-based feedback.
3.3 Participants
Participation was voluntary and a total of 16 male participants participated in the
study that was advertised via an email advertisement. Out of the 16 people, 62.5%
people had taken a course in computer networks/security. The age of participants
ranged from 18-22 years. About 56% were 3rd year and 44% were final year under-
graduate students from Indian Institute of Technology Mandi, India. All the partici-
pants were remunerated at a fixed rate of INR50 for their participation in the study.
3.4 Procedure
Participants were given instructions about their objective in the HackIT task, and
they were informed about their own action’s payoffs. Specifically, human hackers
were asked to maximize their payoff by stealing the real credit-card file from the net-
work over several rounds of play (participants were not aware of the endpoint of the
game). Each round had two stages: Probe stage and Attack stage. Hacker could probe
the network using “nmap” utility in first stage. After probing the webservers, he re-
ceived information about open ports, operating systems, services, and vulnerabilities
associated with different webserver. Next, the hacker had to choose one webserver to
exploit and exploit webservers using “use_exploit” command during attack stage.
Once the webserver was exploited, hackers transferred the credit-card file to their
remote computer.
3.5 Results
Figure 6 shows the proportion of attacks on honeypot and regular webservers.
There was no difference in the proportion of attacks in late and early deception condi-
tions.
Fig. 6 Proportion of attack actions on regular and honeypot webservers across rounds and participants.
0.56 0.56
0.44 0.44
0.00
0.20
0.40
0.60
Early
Deception
Late
Deception
Proportions of Attack
Actions
Conditions
Timings of Deception
Honeypot
Regular
Deleted: Next, the attacker probes the network using nmap com-
mand to gain information about both the webservers. For example, in
step 2 and step 3, attacker probes webserver 1 and webserver 2 re-
spectively (Figure 6). Probing both the webservers gave the infor-
mation about the operating system, open ports, services, and vulnera-
bilities. For example, probing webserver 1 gave the information to
attacker that webserver 1 is running on Solaris operating system. The
open port on webserver 1 are 80, 135, 21, and 111, where sql injec-
tion, DOS attack, brute force attack and DDOS attack are possible.
The information provided to the attacker as a result of probing the
systems gave him an idea of the possible success of an attack on that
system. Once the attacker collects information about open ports and
services, he could attack a webserver by using the “use_exploit”
command. The use_exploit command exploits vulnerabilities of a
system and helps attacker to gain access to that webserver. For exam-
ple, in step 4, the attacker used the use_exploit command to gain
access of webserver 1 using DoS_attack vulnerability. Once attacker
exploited a system and gained access, he lists all the files by using the
“ls” command. After this command, the attacker transfers required
file, i.e., “pin.txt” using the “scp” command (step 5, Figure 2). Once
the attacker copies the file from the exploited system, he is informed
whether he was successful or not in stealing a real credit-card file
from the computer. Once the task is complete, the attacker is given
the textual feedback about the success or failure of his action to copy
the real credit-card file (step 6, Figure 6).…
Deleted: A
Deleted: total of 16 male participants participated in this experi-
ment. Out of the 16 people, 62.5% people had taken a course in
computer networks/security. The age of participants ranged from 18-
22. About 56% were 3rd year and 44% were final year under gradua-
tion students from Indian Institute of Technology, Mandi, India. All
the participants were remunerated accordingly for their participation
in the study.
Deleted: Participants were given instructions about their objective
in the HackIT task, and they were informed about their own action’s
payoffs. Specifically, human hackers were asked to maximize their
payoff by stealing the real credit-card file from the network over
several rounds of play (participants were not aware of the endpoint of
the game). Each round had two stages: Probe stage and Attack stage.
Hacker could probe two webservers in the network using “nmap”
utility. After probing the webservers, he received information about
open ports, operating systems, services, and vulnerabilities associated
with each webserver. Next, the hacker had to choose one webserver
to exploit and exploit webservers using “use_exploit” command
during attack stage. Once the webserver was exploited, hackers
transferred the credit-card file to their remote computer.
Next, we analyzed the exploits used during deception rounds and non-deception
rounds by participants. When regular (honeypot) systems are attacked during decep-
tion rounds, that is called as deception failure (success). Figure 7a and 7b show the
number of regular attacks and honeypot attacks against different exploits in deception
failure and success, respectively. During deception failure, the remote_auth vulnera-
bility was more exploited in early condition compared to late condition and the
pop3_version vulnerability was exploited more in the late condition compared to early
condition. During deception success, the brute_force vulnerability was more exploited
more in early condition compared to late condition and the DOS_attack and
sql_injection vulnerabilities were exploited more in the late condition compared to
early condition.
Fig. 7 Vulnerability exploited on regular systems and honeypot in deception rounds
Figure 8 shows the number of attacks on regular webservers using different vul-
nerabilities. We found that during early deception conditions, mostly hackers used
remote_auth and drown_attack vulnerabilities. Furthermore, during late deception
condition, hackers used pop3_version and remote_auth vulnerabilities.
Fig. 8 Vulnerability exploited in non-deception rounds
Deleted: In this paper, we analyzed the exploits used during de-
ception rounds and non-deception rounds for exploiting regular or
honeypot webservers. First, we analyzed the vulnerabilities exploit-
ed on the regular systems during deception rounds. When regular
systems are attacked during deception rounds, that is called as de-
ception failure. The aim of using deception is increase honeypot
attacks. Figure 7 shows the number of regular attacks against differ-
ent exploits. During early deception, hackers exploited remote_auth
vulnerability more compare to other vulnerabilities.
Fig. 7 Vulnerability exploited on regular systems in deception rounds
Next, we analyzed the vulnerabilities exploited on honeypots dur-
ing deception rounds in Figure 8. The results depicted that the pro-
portion of successful honeypots attacks on deception rounds in case
of early deception was 56.25%, that is in 16 deception rounds for the
early deception participants 9 were successful. Similarly, for late
deception out of 16 deception rounds for the 8 participants, 9 were
successful i.e. Hacker was caught in 56.25% of the rounds.
Fig. 8 Vulnerability exploited on honeypot systems in deception
rounds
Next, we analyzed the number of attacks on regular webservers for
different vulnerabilities in Figure 9. We found that during early
deception conditions, hackers used remote_auth and drown_attack
vulnerabilities. Furthermore, during late deception condition, hackers
used pop3_version and remote_auth vulnerabilities.
... [2]
4 Discussion and Conclusions
In this paper, we discussed HackIT, a HITL simulation tool with a potential to help
cyber-security researchers to investigate the decision-making of attackers and defend-
ers in real-world cyber-security scenarios. In this paper, we showed different features
of HackIT tool and different ways to conduct multiplayer experiments. We also
showed a concrete example of using HackIT to investigate the effects of timing of
deception on hacker’s decisions. We believe that HackIT tool would be helpful in
creating other cyber-security scenarios involving dynamic network sizes, dynamic
network configurations, and various deception strategies.
First, we simulated an information stealing scenario in HackIT. We found that the
attacks on regular and honeypots were no different in early and late deception condi-
tion. One likely reason for this result could be that participants perceived easy to at-
tack and difficult to attack vulnerabilities similarly. In fact, hacker participants ex-
ploited remote_auth vulnerability to attack on regular machines and brute_force and
sql_injection vulnerabilities to attack honeypot systems. Furthermore, we found par-
ticipants attacked more number honeypot systems compared to regular systems.
Cybersecurity faces different open challenges while implementing the deception.
These challenges may involve the following questions: what an effective deception
strategy should be? when should the deception be used? how hackers can be deceived
during the probe phase? what are their probing patterns? and, how to make deception
less detectable? HackIT tool could provide a framework to investigate these ques-
tions. One way to make deception less detectable is to have effective configuration
and content on deceptive nodes. HackIT tool could be used to identify effective con-
figurations and contents on honeypots to make them less detectable.
In future, we plan to perform a series of experiments involving participants perform-
ing as attackers in other simulated network scenarios in HackIT. Here, we wish to
extend the HackIT tool to investigate the optimal proportion of honeypots and effec-
tiveness of deception in networks of different sizes. For example, a network could be
classified as small, medium, or large sized based on number of webservers present.
Furthermore, installing and maintaining honeypots is a costly affair for analysts in
terms of money, time, and manpower. Thus, identify the optimal proportion of honey-
pots required in the network and testing their effectiveness in HackIT could reduce
the overall cost. HackIT tool also provides the flexibility to configure any number of
webservers as honeypots and regular webservers. Thus, HackIT tool could be useful
in the investigation of optimal proportion of honeypots in the network. One could also
investigate the probing patterns of hackers in HackIT. For example, one could inves-
tigate the probing patterns [14] such as local preference scanning, preference sequen-
tial scanning, non-preference sequential scanning, and parallel scanning in the human
hackers data in HackIT to understand different strategies for collecting information.
Furthermore, the HackIT tool could be useful to understand the effectiveness of de-
ception against different cyber-attacks such as SQL injection, Denial of Service
(DoS), and zero-day attacks. Overall, the HackIT tool provides a powerful platform to
investigate various factors that could help analysts plan better cyber defense against
hackers.
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Acknowledgments. Research was partially sponsored by the Army Research Labora-
tory and was accomplished under Cooperative Agreement Number W911NF-13-2-
0045 (ARL Cyber Security CRA). The views and conclusions contained in this doc-
ument are those of the authors and should not be interpreted as representing the offi-
cial policies, either expressed or implied, of the Army Research Laboratory or the
U.S. Government. The U.S. Government is authorized to reproduce and distribute
reprints for Government purposes notwithstanding any copyright notation here on.
This research was also supported by the ICPS DST grant (T-533) from the Indian
Government to Dr. Varun Dutt.
References
1. Whaley, B. (1982). Toward a general theory of deception. Journal of Strategic Studies,
5(1), pp.178-192.
2. Rowe, Neil C., and E. John Custy: "Deception in cyber-attacks." Cyber warfare and cyber
terrorism (2008).
3. Aggarwal, P., Gonzalez, C., & Dutt, V. (2016). Cyber-security: role of deception in cyber-
attack detection. In Advances in Human Factors in Cybersecurity (pp. 85-96). Springer,
Cham.
4. Aggarwal, P., Gonzalez, C., & Dutt, V. (2016, June). Looking from the hacker's perspec-
tive: Role of deceptive strategies in cyber security. In Cyber Situational Awareness, Data
Analytics And Assessment (CyberSA), 2016 International Conference On (pp. 1-6). IEEE.
5. Aggarwal, P., Gonzalez, C., & Dutt, V. (2017, June). Modeling the effects of amount and
timing of deception in simulated network scenarios. In Cyber Situational Awareness, Data
Analytics And Assessment (Cyber SA), 2017 International Conference On (pp. 1-7).
IEEE.
6. P. Aggarwal, C. Gonzalez, and V. Dutt, “Hackit: A real-time simulation tool for studying
real-world cyber-attacks in the laboratory. tbd.,” 2018.
7. Bye, R., Schmidt, S., Luther, K., & Albayrak, S. (2008, March). Application-level simula-
tion for network security. In Proceedings of the 1st international conference on Simulation
tools and techniques for communications, networks and systems & workshops (p. 33).
ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engi-
neering).
8. Maqbool, Z., Makhijani, N., Pammi, V. C., & Dutt, V. (2017). Effects of motivation:
rewarding hackers for undetected attacks cause analysts to perform poorly. Human fac-
tors, 59(3), 420-431.
9. Dutt, V., Ahn, Y. S., & Gonzalez, C. (2013). Cyber situation awareness: modeling detec-
tion of cyber attacks with instance-based learning theory. Human Factors, 55(3), 605-618.
10. Aggarwal, P., Maqbool, Z., Grover, A., Pammi, V. C., Singh, S., & Dutt, V. (2015, June).
Cyber security: A game-theoretic analysis of defender and attacker strategies in defacing-
website games. In Cyber Situational Awareness, Data Analytics and Assessment (Cyber-
SA), 2015 International Conference on (pp. 1-8). IEEE.
11. https://cymulate.com/
12. Issariyakul, T., & Hossain, E. (2012). Introduction to Network Simulator 2 (NS2).
In Introduction to Network Simulator NS2(pp. 21-40). Springer, Boston, MA.
13. Tambe, M. 2011. Security and Game Theory: Algorithms, Deployed Systems, Lessons
Learned. Cambridge University Press.
14. Achleitner, S., La Porta, T. F., McDaniel, P., Sugrim, S., Krishnamurthy, S. V., & Chadha,
R. (2017). Deceiving Network Reconnaissance Using SDN-Based Virtual Topolo-
gies. IEEE Transactions on Network and Service Management, 14(4), 1098-1112.
Page 8: [1] Deleted Palvi 2/17/19 6:33:00 PM
Page 10: [2] Deleted Palvi 2/17/19 6:36:00 PM
Formatted
... [1]
Formatted
... [2]
Preprint
Full-text available
The performance of artificial intelligence (AI) algorithms in practice depends on the realism and correctness of the data, models, and feedback (labels or rewards) provided to the algorithm. This paper discusses methods for improving the realism and ecological validity of AI used for autonomous cyber defense by exploring the potential to use Inverse Reinforcement Learning (IRL) to gain insight into attacker actions, utilities of those actions, and ultimately decision points which cyber deception could thwart. The Tularosa study, as one example, provides experimental data of real-world techniques and tools commonly used by attackers, from which core data vectors can be leveraged to inform an autonomous cyber defense system.
Conference Paper
Full-text available
The performance of artificial intelligence (AI) algorithms in practice depends on the realism and correctness of the data, models, and feedback (labels or rewards) provided to the algorithm. This paper discusses methods for improving the realism and ecological validity of AI used for autonomous cyber defense by exploring the potential to use Inverse Reinforcement Learning (IRL) to gain insight into attacker actions, utilities of those actions, and ultimately decision points which cyber deception could thwart. The Tularosa study, as one example , provides experimental data of real-world techniques and tools commonly used by attackers, from which core data vectors can be leveraged to inform an autonomous cyber defense system.
Article
Full-text available
Deception via honeypots, computers that pretend to be real, may provide effective ways of countering cyber-attacks in computer networks. Although prior research has investigated the effectiveness of timing and amount of deception via deception-based games, it is unclear as to how the size of the network (i.e., number of computer systems in the network) influences adversarial decisions. In this research, using a deception game, we evaluate the influence of network size on adversary’s cyber-attack decisions. The deception game has two sequential stages, probe and attack, and it is defined as DG (n, k, γ), where n is the number of servers, k is the number of honeypots, and γ is the number of probes that adversary makes before attacking the network. In the probe stage, participants may probe a few web servers or may not probe the network. In attack the stage, participants may attack any one of the web servers or decide not to attack the network. In a laboratory experiment, participants were randomly assigned to a repeated deception game across three different between-subject conditions: small (20 participants), medium (20 participants), and large (20 participants). The small, medium, and large conditions used DG (2, 1, 1), DG (6, 3, 3), and DG (12, 6, 6) games, respectively (thus, the proportion of honeypots was kept constant at 50% in all three conditions). Results revealed that in the small network, the proportion of honeypot and no-attack actions were 0.20 and 0.52; whereas, in the medium (large) network, the proportion of honeypot and no-attack actions were 0.50 (0.50) and 0.06 (0.03), respectively. There was also an effect of probing actions on attack actions across all three network sizes. We highlight the implications of our results for networks of different sizes involving deception via honeypots.
Thesis
The threat of cyber attacks is a growing concern across the world, leading to an increasing need for sophisticated cyber defense techniques. The Tularosa Study, was designed and conducted to understand how defensive deception, both cyber and psychological, affects cyber attackers, Ferguson-Walter et al. [2019c]. More specifically, for this empirical study, cyber deception refers to a decoy system and psychological deception refers to false information of the presence of defensive deception techniques on the network. Over 130 red teamers participated in a network penetration test over two days in which we controlled both the presence of and explicit mention of deceptive defensive techniques. To our knowledge, this represents the largest study of its kind ever conducted on a skilled red team population. In addition to the abundant host and network data collected, we conducted a battery of questionnaires, e.g., experience, personality; and cognitive tasks, e.g., fluid intelligence, working memory; as well as physiological measures, e.g., galvanic skin response (GSR), heart rate, to be correlated with the cyber events at a later date. The design and execution of this study and the lessons learned are a major contribution of this thesis. I investigate the �effectiveness of decoy systems for cyber defense by comparing performance across all experimental conditions. Results support a new finding that the combination of the presence of deception and the true information that deception is present has the greatest effect on cyber attackers, when compared to a control condition in which no deception was used. Evidence of cognitive biases in the red teamers' behavior is then detailed and explained, to further support our theory of oppositional human factors (OHF). The final chapter discusses how elements of the experimental design contribute to the validity of assessing the �effectiveness of cyber deception and reviews trade-off�s and lessons learned.
Chapter
Full-text available
This chapter deals with the development of a real-world cyber-security tool, HackIt, to study the decision-making of adversaries in computer networks.
Conference Paper
Full-text available
With the growth of digital infrastructure, cyber-attacks are increasing in the real-world. Cyber-attacks are deliberate exploitation of computer systems, technology-dependent enterprises, and networks. Deception, i.e., the act of making someone believe in something that is not true, could be a way of countering cyber-attacks. In this paper, we use an existing real-time simulation environment (" Deception Game ") to model the decision making of hackers in the presence of deception. We use human data from a published experiment involving the use of Deception Game (N = 100 participants) to evaluate how a cognitive model based upon Instance-Based Learning Theory (IBLT) could account for attack decisions in the presence of different amounts of deception and different timings of deception. Results from IBL model revealed that using late and high deception caused a reduction in attacks on regular webserver compared to early and low deception. Furthermore, the parameters obtained from the IBL model helped provide reasons for the experimental results. We highlight implications of our results on computational modeling of decisions in the cyber world.
Chapter
Full-text available
Cyberspace, computers, and networks are now potential terrain of warfare. We describe some effective forms of deception in cyberspace and discuss how these deceptions are used in attacks. After a general assessment of deception opportunities in cyberspace, we consider various forms of identity deceptions, denial-of-service attacks, Trojan horses, and several other forms of deception. We then speculate on the directions in which cyber attacks may evolve in the future.
Article
Full-text available
Objective: To determine how monetary motivations influence decision-making of humans performing as security analysts and hackers in a cybersecurity game. Background: Cyber-attacks are increasing at an alarming rate. As cyber-attacks often cause damage to existing cyber infrastructures, it is important to understand how monetary rewards may influence decision-making of hackers and analysts in the cyber world. Currently, only limited attention has been given to this area. Method: In an experiment, participants were randomly assigned to three between-subjects conditions (N=26 for each condition): Equal-Payoff, where the magnitude of monetary rewards for hackers and defenders was the same; Rewarding-Hacker, where the magnitude of monetary reward for hacker’s successful attack was 10-times the reward for analyst’s successful defense; and, Rewarding-Analyst (RH), where the magnitude of monetary reward for analyst’s successful defense was 10-times the reward for hacker’s successful attack. In all conditions, half of the participants were human hackers playing against Nash analysts and half were human analysts playing against Nash hackers. Results: Results revealed that monetary rewards for human hackers and analysts caused a decrease in attack and defend actions compared to the baseline. Furthermore, rewarding human hackers for undetected attacks made analysts deviate significantly from their optimal behavior. Conclusions: If hackers are rewarded for their undetected attack actions, then this causes analysts to deviate from optimal defend proportions. Thus, analysts need to be trained not become over-enthusiastic in defending networks. Application: Applications of our results are to networks where the influence of monetary rewards may cause information theft and system damage.
Conference Paper
Full-text available
Cyber-attacks are increasing in the real-world and cause widespread damage to cyber-infrastructure and loss of information. Deception, i.e., actions to promote the beliefs of things that are not true, could be a way of countering cyber-attacks.. In this paper, we propose a deception game, which we use to evaluate the decision making of a hacker in the presence of deception. In an experiment , using the deception game, we analyzed the effect of two between-subjects factors in Hacker's decisions to attack a computer network (N = 100 participants): amount of deception used and the timing of deception. The amount of deception used was manipulated at 2-levels: low and high. The timing of deception use was manipulated at 2-levels: early and late. Results revealed that using late and high deception condition, proportion of not attack actions by hackers are higher. Our results suggest that deception acts as a deter-rence strategy for hacker.
Article
Full-text available
To determine the effects of an adversary's behavior on the defender's accurate and timely detection of network threats. Cyber attacks cause major work disruption. It is important to understand how a defender's behavior (experience and tolerance to threats), as well as adversarial behavior (attack strategy), might impact the detection of threats. In this article, we use cognitive modeling to make predictions regarding these factors. Different model types representing a defender, based on Instance-Based Learning Theory (IBLT), faced different adversarial behaviors. A defender's model was defined by experience of threats: threat-prone (90% threats and 10% nonthreats) and nonthreat-prone (10% threats and 90% nonthreats); and different tolerance levels to threats: risk-averse (model declares a cyber attack after perceiving one threat out of eight total) and risk-seeking (model declares a cyber attack after perceiving seven threats out of eight total). Adversarial behavior is simulated by considering different attack strategies: patient (threats occur late) and impatient (threats occur early). For an impatient strategy, risk-averse models with threat-prone experiences show improved detection compared with risk-seeking models with nonthreat-prone experiences; however, the same is not true for a patient strategy. Based upon model predictions, a defender's prior threat experiences and his or her tolerance to threats are likely to predict detection accuracy; but considering the nature of adversarial behavior is also important. Decision-support tools that consider the role of a defender's experience and tolerance to threats along with the nature of adversarial behavior are likely to improve a defender's overall threat detection.
Article
Advanced targeted cyber attacks often rely on reconnaissance missions to gather information about potential targets, their characteristics and location to identify vulnerabilities in a networked environment. Advanced network scanning techniques are often used for this purpose and are automatically executed by malware infected hosts. In this paper we formally define network deception to defend reconnaissance and develop a Reconnaissance Deception System (RDS), which is based on Software Defined Networking (SDN), to achieve deception by simulating virtual topologies. Our system thwarts network reconnaissance by delaying the scanning techniques of adversaries and invalidating their collected information, while limiting the performance impact on benign network traffic. By simulating the topological as well as physical characteristics of networks, we introduce a system which deceives malicious network discovery and reconnaissance techniques with virtual information, while limiting the information an attacker is able to harvest from the true underlying system. This approach shows a novel defense technique against adversarial reconnaissance missions which are required for targeted cyber attacks such as Advanced Persistent Threats (APT) in highly connected environments. The defense steps of our system aim to invalidate an attackers information, delay the process of finding vulnerable hosts and identify the source of adversarial reconnaissance within a network.
Conference Paper
Cyber-attacks are increasing in the real-world and the y cause widespread damage to cyber-infrastructure and loss of information. Deception, i.e., the act of mak ing someone believe some thing that is not true, could be a way of countering cyber-attacks. In this paper, we propose a deception game, which we use d to evaluate the decision-making of a hacker in the presence of de ce ption. In an experiment, using the dece ption game, we analyz ed the effect of two between-subjects factors (N = 100 participants): Amount of deception (high and low) and the timing of de ce ption (early and late). Results revealed that use of early de ce ption made hackers trust the system's respons e and get de ce ived. However, the amount of deception did not influence hacke r's trust on the system's response. In addition, use of a de ce ptive strategy, i.e., when hackers moved from deception rounds to non-deception rounds, caused hackers to get deceived and not attack the system
Chapter
Introduction to Network Simulator NS2 is a primer providing materials for NS2 beginners, whether students, professors, or researchers for understanding the architecture of Network Simulator 2 (NS2) and for incorporating simulation modules into NS2. The authors discuss the simulation architecture and the key components of NS2 including simulation-related objects, network objects, packet-related objects, and helper objects. The NS2 modules included within are nodes, links, SimpleLink objects, packets, agents, and applications. Further, the book covers three helper modules: timers, random number generators, and error models. Also included are chapters on summary of debugging, variable and packet tracing, result compilation, and examples for extending NS2. Two appendices provide the details of scripting language Tcl, OTcl and AWK, as well object oriented programming used extensively in NS2. © 2012 Springer Science+Business Media, LLC. All rights reserved.