HackIT: A Human-in-the-Loop Simulation Tool for Realistic Cyber Deception Experiments

Abstract

Deception, an art of making someone believe in something that is not true, may provide a promising real-time solution against cyber-attacks. Honeypots, fake servers that pretend to be real, are one of the effective deception tools in the network defense to lure adversaries. In this paper, we propose a human-in-the-loop real-world simulation tool called HackIT, which could be config-ured to create different cyber-security scenarios involving deception. HackIT allows cyber-security researchers to create various deception situations in the laboratory involving honeypots and it helps evaluate the decision processes of human adversaries, who try to attack a simulated computer network. We discuss how researchers can use HackIT to create networks of different sizes; use deception and configure different web servers as honeypots; and, create any number of fictitious ports, services, fake operating systems, and fake files on honeypots. Next, we report a case-study involving HackIT where adversaries were tasked with stealing information from a simulated network over multiple rounds. In one condition in HackIT, deception occurred early; and, in the other condition, it occurred late. Results revealed that participants used different attack strategies across the two conditions. Overall, the case study was a step towards showing how adversarial decisions could be studied in HackIT in the presence of deception. We discuss the potential of using HackIT in helping cyber-security teams understand adversarial cognition in the laboratory.

Full text

HackIT: A Human-in-the-loop Simulation Tool for
Realistic Cyber Deception Experiments
Palvi Aggarwal1, Aksh Gautam2, Vaibhav Aggarwal2,
Cleotilde Gonzalez1 and Varun Dutt2
1 Dynamic Decision Making Lab, Carnegie Mellon University, Pittsburgh, USA
2 Applied Cognitive Science Laboratory, Indian Institute of Technology Mandi, India
palvia@andrew.cmu.edu, aksh@student.iitmandi .ac.in, vaibhav@student.iitmandi.ac.in,
coty@cmu.edu, varun@iitmandi.ac.in
Abstract. Deception, an art of making someone believe in something that is not
true, may provide a promising real-time solution against cyber-attacks. In this
paper, we propose a human-in-the-loop real-world simulation tool called
HackIT, which could be configured to create different cyber-security scenarios
involving deception. We discuss how researchers can use HackIT to create net-
works of different sizes; use deception and configure different webservers as
honeypots; and, create any number of fictitious ports, services, fake operating
systems, and fake files on honeypots. Next, we report a case-study involving
HackIT where adversaries were tasked with stealing information from a simu-
lated network over multiple rounds. In one condition in HackIT, deception oc-
curred early; and, in the other condition, it occurred late. Results revealed that
participants used different attack strategies across the two conditions. We dis-
cuss the potential of using HackIT in helping cyber-security teams understand
adversarial cognition in the laboratory.
Keywords: Cybersecurity · Simulation tools · Learning · Attack · Hackers · De-
fenders · Honeypots
1 Introduction
Deception is an art of making someone believe in something that is not true, may
provide a promising real-time solution against cyber-attacks [1]. Deception involves
interaction between two parties, a target and a deceiver, in which the deceiver effec-
tively causes the target to believe in a false description of reality [1]. The objective is
to cause the target to work in such a way that is beneficial to the deceiver. Deception
has been used as an offensive and defensive tool in cyber world by hackers and de-
fenders. Hackers used deception for exploiting cyber infrastructure, stealing infor-
mation, making money and defaming people. The deception techniques used by hack-
ers may involve malware signature, conceal code and logic, encrypted exploits, spoof-
ing, phishing, and social engineering (e.g., by deceiving help desk employees to in-
stall malicious code or obtain credentials). However, defenders used deception for
securing network infrastructure, luring hackers to fake information and understanding
Deleted: Honeypots, fake servers that pretend to be real, are
one of the effective deception tools in the network defense to
lure hackers. In this paper, we propose a h
Deleted: HackIT allows cyber-security researchers to create
various deception situations in the laboratory involving honey-
pots and it helps evaluate the decision processes of human adver-
saries, who try to attack a simulated computer network. We
Deleted: Overall, the case study was a step towards showing
how adversarial decisions could be studied in HackIT in the
presence of deception. We discuss the potential of using HackIT

hacker’s motives and strategies using tools such as honeypots. When used for defense,
cyber defenders may use feints and deceits to thwart hackers’ cognitive processes,
delay attack activities, and disrupt the breach process. When used for defense, decep-
tion may be achieved through miss-directions, fake responses, and obfuscations [2].
These techniques rely upon hacker’s trust in response from network, data, and appli-
cations during an actual attack. To create deception for defense, security experts have
been using honeypots, fake servers that pretend to be real, for gathering intelligence
about hackers. Honeypots are one of the effective deception tools in the network de-
fense to lure hackers.
Decisions-making process of hackers and defenders in cyber world is complex
task. Understanding their decision process in such complex environment is challeng-
ing. Simulation has been used as an effective way of understanding the hackers’ and
defenders’ decisions, testing new solutions for security, and training the models and
people in such complex task scenarios [3-10] have used behavioral game theoretic
approaches to understand the decisions of hackers and defenders in the abstract cyber-
security games. In such game theoretic approaches, the network structure and the set
of actions of hackers and defenders were abstracted to attack/not-attack and de-
fend/not-defend. Furthermore, the information provided to the participants was also
abstracted. The task simulated using game-theoretic approaches was less cognitively
challenging compared to the real cyber-security tasks. Thus, the conclusions made
based on behavioral game-theoretic approaches may or may not address the cyberse-
curity problems. Another approach used to study hackers and defenders behaviour
involve complex real time tools such as NeSSi, Network Simulator-2/3, Cymulate etc
[11-12]. However, using deception in uncontrolled environments makes it difficult to
answer cause-effect questions.
Aggarwal et al. [6] proposed HackIT tool to bridge the gap between behavioral
game-theoretic approaches and real-world cybersecurity tasks. HackIT tool provided
features to create more specific set of actions and information needed for cyber-
security tasks for both hackers and defenders. The HackIT tool was used to replicate
the results of a laboratory experiment using a deception game [3-4]. Results revealed
that the average proportion of attacks was lower and not-attack actions were higher
when deception occurred late in the game rather than earlier; and when the amount or
deception was high compared to low [6]. This result found in an abstract simplified
scenario was replicated in a real-world simulation tool called the HackIT.
In [6], the HackIT tool was available for creating deception with limited number
of systems only and for single player games only. In this paper, we define the en-
hanced capabilities of HackIT tool. Specifically, we detail how the HackIT tool is
capable of running experiments with different sized networks, different network con-
figurations, different deception strategies, and single player and multiplayer games.
Thus, the enhanced capabilities of HackIT tool can help us answer several questions
such as the effect of different network sizes and honeypot allocations on hacker’s
performance and the most effective way to present the “clues” of deception in net-
work nodes.
In what follows, we first discuss the functioning and different features in the of the
HackIT tool. Next, we detail an experiment involving the HackIT tool to showcase its
capabilities. Furthermore, we detail the results from the experiment and discuss the
Deleted: cyber
Deleted:
Deleted: challenged
Deleted:
Deleted: game
Deleted: cyber security
Deleted:
Deleted: cyber security
Deleted: cyber
Deleted: Aggarwal et al (2018) proposed HackIT tool for two
webservers only. The HackIt
Deleted: HackIt
Deleted: designed to investigate the role of deception (i.e., amount
and timing) on a hacker’s decisions…
Formatted: Font: Not Italic
Deleted: [
Deleted: ¶
Deleted: the
Deleted: ,
Deleted: the
Deleted: /distribution
Deleted: ,
Deleted: identifying
Deleted: of the HackIt tool
Deleted:
Deleted: explain the methods of
Deleted: run
Deleted: using
Deleted: HackIt
Deleted: from HackIt tool

implications of our results for investigating the decision-making of hackers in the real
world.
2 HackIT
HackIT is a generic framework for cybersecurity tasks to study human learning and
decision-making of hackers and analysts. It represents a simplified framework con-
taining the most essential elements for creating cybersecurity scenarios: network
nodes, which represents the characteristics of real nodes; strategies, which can be
configured for creating deception; and commands, which are used for communication
with the network. The analyst’s goal in the HackIT task is to evaluate different decep-
tion strategies and the hacker’s goal to identify the real network nodes and exploit
them. Hackers communicate with the network in HackIT using different commands
and gain information about different configurations. However, hackers are not aware
of the strategies used by analysts inside the HackIT scenario. Hackers basically learn
about these strategies overtime by playing different rounds. Thus, HackIT is a control-
lable and flexible simulation tool with the capability of creating various network sce-
narios and experiment with different techniques to lure hackers. Figure 1 shows the
typical flow of HackIT tool which uses the concept of stackelberg security games
[13], where first defenders create a network and use their defense techniques and next,
hackers try to exploit the security of the network.
Fig. 1. HackIT tool
HackIT is an experimental cybersecurity tool that allow analysts to simulate cyber
infrastructure during configuration phase and define the deception algorithm in decep-
tion phase. Next, these scenarios are presented to the hackers for exploitation. Hack-
ers use this tool in two phases: probe phase and attack phase. The probe phase in-
volves the process of reconnaissance where hackers gather information about open
port, services, operating systems, and available vulnerabilities using tools such as
nmap; whereas, the attack phase involves gaining access to different computers and
stealing information or compromising computer systems.
Despite the simplicity of HackIT, the tool has the potential to simulate many real-
world dynamic situations in the laboratory: testing different proportion of honeypots
in the network, testing optimal placement of honeypots, different configurations of
honeypots, such as, easy to exploit ports on honeypots, and availability of fake files
Deleted: attackers
Deleted:
Deleted: cyber security
Deleted: HackIt
Deleted: The h
Deleted: s
Deleted: the
Deleted: is
Deleted: defender
Deleted: s
Deleted: -
Deleted: defenders

on honeypots, availability of unpatched ports on honeypot etc. Specific characteristics
of HackIT tool are explained as followed:
2.1 Network Size
The HackIT tool is flexible to create any number of computers in the network. The
configuration of these systems will also be dynamically created. Thus, this tool can
allow researchers to easily run experiments with small, medium and large scaled net-
works as shown in Figure 2.
Fig. 2. Different Network Sizes: A) Small, B) Medium, and C) Large
2.2 Allocation of honeypots and regular computers
The HackIT tool is capable of creating any proportion of network computers as
honeypots, where honeypots are fake computer pretending to be real. This tools also
provides a functionality to define the features of honeypot that make them pretend as
real systems. These features include the ability to configure vulnerable ports, vulnera-
ble operating systems, and proportion of fake files on the honeypots. Thus, setting
different proportion of honeypots and defining deception via honeypots is relatively
easy in HackIT.
2.3 Configuration of Computers
The configuration of honeypots and regular computers is automatically generated
by a script in HackIT that would produce the configuration for a fixed number of
systems with a given proportion of honeypots and real systems. This script generates
the configuration consisting of real systems, honeypots, and files for each game round
in HackIT. By using this script, one could generate data onetime and encode it in the
experiment so that it would present all participants with same configuration. For ex-
ample, the regular systems could be configured as patched and difficult to exploit and
honeypot systems could be configured as easy to exploit. This configuration mapping
is shown in Table 1.
Table 1. Configuration of Honeypot and Regular systems
Easy to Attack
Difficult to Attack
Operating systems:
• Windows Server 2003
• Windows XP
• HP-UX 11i
• Solaris
Operating systems:
• OpenBSD
• Linux
• Mac OS X
• Windows 8
A
B
C
Deleted: a
Deleted: b
Deleted: c
Deleted: systems
Deleted: such as
Deleted: configuring
Deleted: etc
Deleted: easier
Deleted: of
Deleted: round
Deleted: i.e. the real systems, honeypots, and files
Deleted: we
Deleted: the
Deleted: the users
Formatted: Left
Formatted: Font color: Background 1
Formatted: Font color: Background 1
Formatted: Left
Formatted: Left
Formatted: Font color: Background 1

Services and ports:
• 21/tcp – ftp
• 25/tcp – smtp
• 80/tcp – http
• 111/tcp – rpcbind
• 135/tcp – msrpc
Services and ports:
• 22/tcp-ssh
• 53/tcp-domain
• 110/tcp-pop3
• 139/tcp-netbios
• 443/tcp-https
• 445/tcp-microsoft-ds
• 3306/tcp-mysql
• 5900/tcp-vncc http
• 6112/tcp-dtspc
• 8080/tcp-apache
2.4 Content of Honeypots
HackIT tool provide a facility to create fictitious content on honeypots using fake
files. The proportion of fake files and useful files on a honeypot can be configured in
HackIT. Figure 3 shows the output of the ls command in the directory where only
pin.txt is a real file and rest are fake files. In HackIT tool, the number of files on each
server can be dynamically created. We tested our platform with the different number
of files ranging from 50-200.
Fig. 3. List of fake files on Honeypot Webserver
2.5 Single-player and Multi-player platform
HackIT tool provides a functionality to run single-player and multi-player experi-
ments. In single-player experiment settings, players cannot communicate with each
other. However, in multi-player experiment setup, players are provided with a chat
functionality to share their knowledge. Hackers usually penetrate into the network by
organizing themselves into a group. Hackers in a group rely on information gained
from fellow hackers who have already penetrated into the network or are trying to
penetrate it.
Deleted: factitious
Formatted: Font: Italic

Fig. 4 Chat functionality for multiplayer experiments in HackIT
2.6 Commands
The HackIT tool can run various network commands that include: nmap,
use_exploit, ls, and scp. Nmap is a network utility that shows the open ports, operating
system type, and services running on the specified webserver. The nmap utility also
provides the list of vulnerabilities on the corresponding webservers. The use_exploit
command exploits vulnerabilities of a system and helps attacker to gain access to a
particular webserver. Next, the ls command lists the files currently on the file system of
the machine. The scp command transfers files to the remote machine.
2.7 Measures for human-in-the-loop experiments
For understanding the decision-making process of hackers, it is important to ana-
lyze how do they gather information, their preferences while searching for infor-
mation, kind of exploits used, and most importantly how to they react to different
defense strategies. HackIT tool provides various measures to analyze the human per-
formance in cybersecurity experiments.
- Probing behavior: HackIT tool provides capability to record the probing behav-
ior of hacker participants. Probing is an important phase where hackers collect in-
formation before launching a real attack. Using deception in probing responses
and analyzing hackers’ reaction towards deceptive responses is an important
measure provided in HackIT tool.
- Attack behavior: HackIT measures the attack behavior of hackers by recording
their attack actions, exploit information, their specific selection of targets or con-
figuration. HackIT tool also records the vulnerabilities exploited, and exploits
used by hackers. Defenders can analyze this data to study their attack patterns.
Deleted: HackIt
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Formatted: Font: Italic
Deleted: Human
Deleted: T
Deleted: he
Deleted: Loop
Deleted: cyber security
Deleted: u
Deleted: their
Deleted: u
Deleted: provides

- Learning: HackIT tool can record overtime behavior of participants. Thus, it
provides data to analyze learning capabilities of hackers against different decep-
tion strategies.
- Communication History: HackIT tool provides functionality to analyze the chat-
ting history of hacker to investigate that how hackers make decisions or formu-
late strategies in team-based tasks.
- Time: HackIT tool also records time taken by hackers to exploit a system. Timing
is another measure to evaluate the success of deception.
3 CASE STUDY USING HACKIT TOOL
3.1 Experiment Design
In this paper, we demonstrated the ability of HackIT tool as a single-player plat-
form for conducting experiments. We analyzed the effect of timing of deception on
hacker’s actions in a study. The timing of deception was manipulated across two be-
tween-subject conditions: early deception (N =8) and late deception (N = 8). The total
number of servers were 40, where 20 webservers were assigned as honeypot webserv-
ers during deception rounds. In both conditions, participants playing as hackers were
given 6 game rounds in a sequence (end point unknown to participants), where 2-
game rounds possessed deception. In this experiment, if the timing of deception was
early, then deception was present on the second and third rounds in the sequence.
However, if the timing of deception was late, then deception was present in the fourth
and fifth rounds in the sequence (see Figure 5). The honeypots were easy to exploit
via common ports and vulnerabilities in the deception rounds compared to the non-
deception rounds, where there were no honeypots (more details ahead in this paper).
However, participants were not told that honeypots will involve easy to attack config-
urations in deception rounds. Also, participants were not disclosed the rounds on
which deception was involved. To analyze human data, we looked at the proportion of
honeypot attacks and proportion of regular attacks at the attack stage by the hacker
across six-rounds in each condition. These proportions were calculated by average the
attack decision over all the trials and participants. We also calculated frequency of
each exploits used on regular and honeypots in deception and no-deception trials.
Fig. 5 Experiment design using deception game with six rounds and two conditions, i.e. early decep-
tion and late deception. D: Deception Present -: Deception Not Present [1,2].
Deleted: u
Deleted: In this paper, we demonstrated the ability of HackIT tool
as a single-player platform for conducting experiments. The experi-
ment design was borrowed from [1] and scaled from 2 webservers to
40 webservers. We analyzed the effect of timing of deception on
hacker’s actions in a pilot study. The timing of deception was manip-
ulated across two between-subject conditions: early deception (N =8)
and late deception (N = 8). The total number of servers were 40
where 20 webservers were assigned as honeypot webservers during
deception rounds. In both conditions, participants playing as hackers
were given 6 game rounds in a sequence (end point unknown to
participants), where 2-game rounds possessed deception. Figure 5
shows the experimental design of deception game implemented in
HackIT tool. In this experiment, if the timing of deception was early,
then deception was present on the second and third rounds in the
sequence. However, if the timing of deception was late, then decep-
tion was present in the fourth and fifth rounds in the sequence. Pres-
ence of deception meant that the honeypots were easy to exploit via
popular ports and vulnerability in the deception rounds compared to
the non-deception rounds (more details ahead in this paper). Howev-
er, participants were not told that honeypots will involve easy to
attack configurations in deception rounds. Also, participants were not
disclosed the rounds on which deception was involved. To analyze
human data, we looked at the proportion of honeypot attacks and
proportion of regular attacks at the attack stage by the hacker across
six-rounds in each condition.¶

3.2 HackIT Task
The objective of attacker in HackIT was to steal real credit-card information locat-
ed on one of the webservers. As shown in Figure 1, defender first configure the net-
work with 40 webservers where 20 webservers acted as honeypots. Defender also set
up a strategy where honeypots were easier to attack during the deception rounds.
Based on these strategies, in this experiment, defender uses different configurations
shown in Table 2. For example, a system with Windows XP operating system, port
80/tcp, and service http was easily exploitable. Such a configuration was mapped to a
honeypot. However, a system with Linux operating system, port 22/tcp and service
ssh was difficult to attack. Such a configuration was mapped to a regular webserver.
Participants were informed about the easy to exploit and the difficult to exploit con-
figurations in Table 2 as part of the instructions.
Table 2. Configuration of Honeypot and Regular systems
Strategy
Operating System
Ports
Exploits
Honeypots:
Easy to attack
Windows Server
2003
Windows XP
HP-UX 11i
Solaris
21/tcp – ftp
25/tcp – smtp
80/tcp – http
111/tcp – rpcbind
135/tcp – msrpc
brute_force
directory_harvest
sql_injection
DDoS_attack
DoS_attack
Regular:
Difficult to attack
OpenBSD
Linux
Mac OS X
Windows 8
22/tcp-ssh
53/tcp-domain
110/tcp-pop3
139/tcp-netbios
443/tcp-https
445/tcp-microsoft-
ds
3306/tcp-mysql
5900/tcp-vncc http
6112/tcp-dtspc
8080/tcp-apache
user_auth
DNS_zone_transfer
pop3_version
DCOM_buffer_overrun
drown_attack
windows_null_session
remove_auth
remote_auth
remote_exploit_buffer
url_decoder
First, the attacker probed the network using nmap command to gain information
about different webservers. Probing different webservers gave the information about
the operating system, open ports, services, and vulnerabilities. The information pro-
vided to the attacker as a result of probing systems gave him an idea about the type of
configuration on the probed system. Once the attacker collects information about open
ports and services, he could attack a webserver by using the “use_exploit” command.
The use_exploit command exploited vulnerabilities on a system and helped the at-
tacker to gain access to that webserver. Next, the attacker could list different files on
the exploited webserver by using the “ls” command. Next, the attacker could transfer
required files containing credit card information (e.g., “pin.txt”) using the “scp”
command. After attackers copied the file from the exploited system, he was informed
Deleted: HackIt
Deleted: In this paper, we simulated a network of forty webservers
in HackIT where 50% webservers were honeypots and others were
regular webservers. The objective of attacker was to steal real credit
card information located on one of these webservers. As shown in
Figure 1, defender first configure the network with 40 webservers
where 20 of the webservers acts as a honeypot webserver and another
one act as a real webserver. Next, defender defines the deception
strategy as honeypots are easy to attack and regular systems are
difficult to attack. Defender also sets up a strategy that honeypots will
only become easy to attack during the deception rounds shown in
figure 5. Based on these strategies, in this experiment, defender uses
the configuration shown in table 2. For example, a system with Win-
dows XP operating system, port 80/tcp, and service http will be easily
exploitable. However, a system with Linux operating system, port
22/tcp and service ssh will be difficult to attack. In deception rounds,
honeypot webservers were configured in such a way that attackers
could easily attack them. However, the regular webserver were diffi-
cult to attack in these deception rounds. In the experiment, partici-
pants were informed about the easy to exploit and the difficult to
exploit configurations in Table 2.
Formatted: Indent: First line: 0 cm
Deleted: ¶
Strategy
... [1]

whether he was successful or not in stealing a real credit-card file from the computer
via a text-based feedback.
3.3 Participants
Participation was voluntary and a total of 16 male participants participated in the
study that was advertised via an email advertisement. Out of the 16 people, 62.5%
people had taken a course in computer networks/security. The age of participants
ranged from 18-22 years. About 56% were 3rd year and 44% were final year under-
graduate students from Indian Institute of Technology Mandi, India. All the partici-
pants were remunerated at a fixed rate of INR50 for their participation in the study.
3.4 Procedure
Participants were given instructions about their objective in the HackIT task, and
they were informed about their own action’s payoffs. Specifically, human hackers
were asked to maximize their payoff by stealing the real credit-card file from the net-
work over several rounds of play (participants were not aware of the endpoint of the
game). Each round had two stages: Probe stage and Attack stage. Hacker could probe
the network using “nmap” utility in first stage. After probing the webservers, he re-
ceived information about open ports, operating systems, services, and vulnerabilities
associated with different webserver. Next, the hacker had to choose one webserver to
exploit and exploit webservers using “use_exploit” command during attack stage.
Once the webserver was exploited, hackers transferred the credit-card file to their
remote computer.
3.5 Results
Figure 6 shows the proportion of attacks on honeypot and regular webservers.
There was no difference in the proportion of attacks in late and early deception condi-
tions.
Fig. 6 Proportion of attack actions on regular and honeypot webservers across rounds and participants.
0.56 0.56
0.44 0.44
0.00
0.20
0.40
0.60
Early
Deception
Late
Deception
Proportions of Attack
Actions
Conditions
Timings of Deception
Honeypot
Regular
Deleted: Next, the attacker probes the network using nmap com-
mand to gain information about both the webservers. For example, in
step 2 and step 3, attacker probes webserver 1 and webserver 2 re-
spectively (Figure 6). Probing both the webservers gave the infor-
mation about the operating system, open ports, services, and vulnera-
bilities. For example, probing webserver 1 gave the information to
attacker that webserver 1 is running on Solaris operating system. The
open port on webserver 1 are 80, 135, 21, and 111, where sql injec-
tion, DOS attack, brute force attack and DDOS attack are possible.
The information provided to the attacker as a result of probing the
systems gave him an idea of the possible success of an attack on that
system. Once the attacker collects information about open ports and
services, he could attack a webserver by using the “use_exploit”
command. The use_exploit command exploits vulnerabilities of a
system and helps attacker to gain access to that webserver. For exam-
ple, in step 4, the attacker used the use_exploit command to gain
access of webserver 1 using DoS_attack vulnerability. Once attacker
exploited a system and gained access, he lists all the files by using the
“ls” command. After this command, the attacker transfers required
file, i.e., “pin.txt” using the “scp” command (step 5, Figure 2). Once
the attacker copies the file from the exploited system, he is informed
whether he was successful or not in stealing a real credit-card file
from the computer. Once the task is complete, the attacker is given
the textual feedback about the success or failure of his action to copy
the real credit-card file (step 6, Figure 6).…
Deleted: A
Deleted: total of 16 male participants participated in this experi-
ment. Out of the 16 people, 62.5% people had taken a course in
computer networks/security. The age of participants ranged from 18-
22. About 56% were 3rd year and 44% were final year under gradua-
tion students from Indian Institute of Technology, Mandi, India. All
the participants were remunerated accordingly for their participation
in the study.¶
Deleted: Participants were given instructions about their objective
in the HackIT task, and they were informed about their own action’s
payoffs. Specifically, human hackers were asked to maximize their
payoff by stealing the real credit-card file from the network over
several rounds of play (participants were not aware of the endpoint of
the game). Each round had two stages: Probe stage and Attack stage.
Hacker could probe two webservers in the network using “nmap”
utility. After probing the webservers, he received information about
open ports, operating systems, services, and vulnerabilities associated
with each webserver. Next, the hacker had to choose one webserver
to exploit and exploit webservers using “use_exploit” command
during attack stage. Once the webserver was exploited, hackers
transferred the credit-card file to their remote computer.

Next, we analyzed the exploits used during deception rounds and non-deception
rounds by participants. When regular (honeypot) systems are attacked during decep-
tion rounds, that is called as deception failure (success). Figure 7a and 7b show the
number of regular attacks and honeypot attacks against different exploits in deception
failure and success, respectively. During deception failure, the remote_auth vulnera-
bility was more exploited in early condition compared to late condition and the
pop3_version vulnerability was exploited more in the late condition compared to early
condition. During deception success, the brute_force vulnerability was more exploited
more in early condition compared to late condition and the DOS_attack and
sql_injection vulnerabilities were exploited more in the late condition compared to
early condition.
Fig. 7 Vulnerability exploited on regular systems and honeypot in deception rounds
Figure 8 shows the number of attacks on regular webservers using different vul-
nerabilities. We found that during early deception conditions, mostly hackers used
remote_auth and drown_attack vulnerabilities. Furthermore, during late deception
condition, hackers used pop3_version and remote_auth vulnerabilities.
Fig. 8 Vulnerability exploited in non-deception rounds
Deleted: In this paper, we analyzed the exploits used during de-
ception rounds and non-deception rounds for exploiting regular or
honeypot webservers. First, we analyzed the vulnerabilities exploit-
ed on the regular systems during deception rounds. When regular
systems are attacked during deception rounds, that is called as de-
ception failure. The aim of using deception is increase honeypot
attacks. Figure 7 shows the number of regular attacks against differ-
ent exploits. During early deception, hackers exploited remote_auth
vulnerability more compare to other vulnerabilities.¶
¶
Fig. 7 Vulnerability exploited on regular systems in deception rounds¶
➝Next, we analyzed the vulnerabilities exploited on honeypots dur-
ing deception rounds in Figure 8. The results depicted that the pro-
portion of successful honeypots attacks on deception rounds in case
of early deception was 56.25%, that is in 16 deception rounds for the
early deception participants 9 were successful. Similarly, for late
deception out of 16 deception rounds for the 8 participants, 9 were
successful i.e. Hacker was caught in 56.25% of the rounds. ¶
¶
Fig. 8 Vulnerability exploited on honeypot systems in deception
rounds¶
➝Next, we analyzed the number of attacks on regular webservers for
different vulnerabilities in Figure 9. We found that during early
deception conditions, hackers used remote_auth and drown_attack
vulnerabilities. Furthermore, during late deception condition, hackers
used pop3_version and remote_auth vulnerabilities. ¶
... [2]

4 Discussion and Conclusions
In this paper, we discussed HackIT, a HITL simulation tool with a potential to help
cyber-security researchers to investigate the decision-making of attackers and defend-
ers in real-world cyber-security scenarios. In this paper, we showed different features
of HackIT tool and different ways to conduct multiplayer experiments. We also
showed a concrete example of using HackIT to investigate the effects of timing of
deception on hacker’s decisions. We believe that HackIT tool would be helpful in
creating other cyber-security scenarios involving dynamic network sizes, dynamic
network configurations, and various deception strategies.
First, we simulated an information stealing scenario in HackIT. We found that the
attacks on regular and honeypots were no different in early and late deception condi-
tion. One likely reason for this result could be that participants perceived easy to at-
tack and difficult to attack vulnerabilities similarly. In fact, hacker participants ex-
ploited remote_auth vulnerability to attack on regular machines and brute_force and
sql_injection vulnerabilities to attack honeypot systems. Furthermore, we found par-
ticipants attacked more number honeypot systems compared to regular systems.
Cybersecurity faces different open challenges while implementing the deception.
These challenges may involve the following questions: what an effective deception
strategy should be? when should the deception be used? how hackers can be deceived
during the probe phase? what are their probing patterns? and, how to make deception
less detectable? HackIT tool could provide a framework to investigate these ques-
tions. One way to make deception less detectable is to have effective configuration
and content on deceptive nodes. HackIT tool could be used to identify effective con-
figurations and contents on honeypots to make them less detectable.
In future, we plan to perform a series of experiments involving participants perform-
ing as attackers in other simulated network scenarios in HackIT. Here, we wish to
extend the HackIT tool to investigate the optimal proportion of honeypots and effec-
tiveness of deception in networks of different sizes. For example, a network could be
classified as small, medium, or large sized based on number of webservers present.
Furthermore, installing and maintaining honeypots is a costly affair for analysts in
terms of money, time, and manpower. Thus, identify the optimal proportion of honey-
pots required in the network and testing their effectiveness in HackIT could reduce
the overall cost. HackIT tool also provides the flexibility to configure any number of
webservers as honeypots and regular webservers. Thus, HackIT tool could be useful
in the investigation of optimal proportion of honeypots in the network. One could also
investigate the probing patterns of hackers in HackIT. For example, one could inves-
tigate the probing patterns [14] such as local preference scanning, preference sequen-
tial scanning, non-preference sequential scanning, and parallel scanning in the human
hacker’s data in HackIT to understand different strategies for collecting information.
Furthermore, the HackIT tool could be useful to understand the effectiveness of de-
ception against different cyber-attacks such as SQL injection, Denial of Service
(DoS), and zero-day attacks. Overall, the HackIT tool provides a powerful platform to
investigate various factors that could help analysts plan better cyber defense against
hackers.
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt
Deleted: HackIt

Acknowledgments. Research was partially sponsored by the Army Research Labora-
tory and was accomplished under Cooperative Agreement Number W911NF-13-2-
0045 (ARL Cyber Security CRA). The views and conclusions contained in this doc-
ument are those of the authors and should not be interpreted as representing the offi-
cial policies, either expressed or implied, of the Army Research Laboratory or the
U.S. Government. The U.S. Government is authorized to reproduce and distribute
reprints for Government purposes notwithstanding any copyright notation here on.
This research was also supported by the ICPS DST grant (T-533) from the Indian
Government to Dr. Varun Dutt.
References
1. Whaley, B. (1982). Toward a general theory of deception. Journal of Strategic Studies,
5(1), pp.178-192.
2. Rowe, Neil C., and E. John Custy: "Deception in cyber-attacks." Cyber warfare and cyber
terrorism (2008).
3. Aggarwal, P., Gonzalez, C., & Dutt, V. (2016). Cyber-security: role of deception in cyber-
attack detection. In Advances in Human Factors in Cybersecurity (pp. 85-96). Springer,
Cham.
4. Aggarwal, P., Gonzalez, C., & Dutt, V. (2016, June). Looking from the hacker's perspec-
tive: Role of deceptive strategies in cyber security. In Cyber Situational Awareness, Data
Analytics And Assessment (CyberSA), 2016 International Conference On (pp. 1-6). IEEE.
5. Aggarwal, P., Gonzalez, C., & Dutt, V. (2017, June). Modeling the effects of amount and
timing of deception in simulated network scenarios. In Cyber Situational Awareness, Data
Analytics And Assessment (Cyber SA), 2017 International Conference On (pp. 1-7).
IEEE.
6. P. Aggarwal, C. Gonzalez, and V. Dutt, “Hackit: A real-time simulation tool for studying
real-world cyber-attacks in the laboratory. tbd.,” 2018.
7. Bye, R., Schmidt, S., Luther, K., & Albayrak, S. (2008, March). Application-level simula-
tion for network security. In Proceedings of the 1st international conference on Simulation
tools and techniques for communications, networks and systems & workshops (p. 33).
ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engi-
neering).
8. Maqbool, Z., Makhijani, N., Pammi, V. C., & Dutt, V. (2017). Effects of motivation:
rewarding hackers for undetected attacks cause analysts to perform poorly. Human fac-
tors, 59(3), 420-431.
9. Dutt, V., Ahn, Y. S., & Gonzalez, C. (2013). Cyber situation awareness: modeling detec-
tion of cyber attacks with instance-based learning theory. Human Factors, 55(3), 605-618.
10. Aggarwal, P., Maqbool, Z., Grover, A., Pammi, V. C., Singh, S., & Dutt, V. (2015, June).
Cyber security: A game-theoretic analysis of defender and attacker strategies in defacing-
website games. In Cyber Situational Awareness, Data Analytics and Assessment (Cyber-
SA), 2015 International Conference on (pp. 1-8). IEEE.
11. https://cymulate.com/
12. Issariyakul, T., & Hossain, E. (2012). Introduction to Network Simulator 2 (NS2).
In Introduction to Network Simulator NS2(pp. 21-40). Springer, Boston, MA.
13. Tambe, M. 2011. Security and Game Theory: Algorithms, Deployed Systems, Lessons
Learned. Cambridge University Press.
14. Achleitner, S., La Porta, T. F., McDaniel, P., Sugrim, S., Krishnamurthy, S. V., & Chadha,
R. (2017). Deceiving Network Reconnaissance Using SDN-Based Virtual Topolo-
gies. IEEE Transactions on Network and Service Management, 14(4), 1098-1112.

Page 8: [1] Deleted Palvi 2/17/19 6:33:00 PM
Page 10: [2] Deleted Palvi 2/17/19 6:36:00 PM
Formatted
... [1]
Formatted
... [2]