Content uploaded by Rebecca Long
Author content
All content in this area was uploaded by Rebecca Long on Mar 27, 2016
Content may be subject to copyright.
`
USING PHISHING TO TEST SOCIAL ENGINEERING
AWARENESS OF FINANCIAL EMPLOYEES
A Thesis
Presented To
Eastern Washington University
Cheney, Washington
In Partial Fulfillment of the Requirements
for the Degree
Master of Science
By
Rebecca M. Long
Winter 2013
ii
Thesis of Rebecca M. Long approved by
Date
Dr. Carol Taylor, Co-Advisor and Graduate Study Committee
Date
Dr. Paul Schimpf, Co-Advisor and Graduate Study Committee
iii
MASTER’S THESIS
In presenting this thesis in partial fulfillment of the requirements for a master’s degree
at Eastern Washington University, I agree that the JFK Library shall make copies freely
available for inspection. I further agree that copying of this project in whole or in part is
allowable only for scholarly purposes. It is understood, however, that any copying or
publication of this thesis for commercial purposes, or for financial gain, shall not be
allowed without my written permission.
Signature
Date
iv
ABSTRACT
Social engineering is the biggest security threat to financial institutions because
it exploits the weakest link in any security system: the human element. It is proposed
here that combining specialized training on social engineering followed by repeated audit
tests will be more effective at lowering employee vulnerability than standard security
training alone. This research developed a training module specializing in social
engineering with an extra emphasis on phishing, then used phishing trials on financial
employees to audit their awareness and knowledge of social engineering to determine if
it lowers the vulnerability level to phishing attacks.
v
ACKNOWLEDGEMENTS
My thanks to my advisor Dr. Carol Taylor, computer science professor at Eastern
Washington University (EWU), for all her help with this project. Thanks to Dr. Paul
Schimpf, the chair of the Computer Science Department at EWU, for his help and
support. Thanks to Dr. Elizabeth Tipton, the associate dean of the Department of
Accounting and Information Systems at EWU, for her guidance into the world of stats
and research projects.
Thanks to Stu Steiner, computer science senior lecturer at EWU, for his creative
motivational tactics to help ensure that this project really did finish. Thanks to Tom
Capaul, computer science senior lecturer at EWU, for his ongoing moral support and
guiding hand. Thanks to Margo Stanzak, computer science operations manager at EWU,
for the academic career long support to me and the whole department.
Thanks to the bank who partnered with us to allow this research to happen. Also, thanks
to Intrinium Security for all their hard work and help with this research process. This
project would not have happened without your help and support!
Thanks also to my parents, Mary and Lawrence Long, for giving me their unbiased love
and support, and always helping proof read my papers. Thanks to my wonderful
husband, Jay Logsdon, for his love and support in finishing out this massive project.
vi
TABLE OF CONTENTS
Abstract .....................................................................................................................iv
Acknowledgements ..................................................................................................... v
Table of Contents ........................................................................................................vi
List of Tables .............................................................................................................. x
List of Figures ............................................................................................................ xi
1. Introduction ........................................................................................................... 1
1.1 Social Engineering ............................................................................................. 1
1.1.1 Definition .................................................................................................... 1
1.1.2 Goals and Motivation of a Social Engineer .................................................... 2
1.1.3 Aspects of a Social Engineering Attack ......................................................... 3
1.1.4 Anatomy of a Social Engineering Attack ........................................................ 4
1.1.5 “The Cycle” .................................................................................................. 6
1.2 Phishing ............................................................................................................ 9
1.2.1 Definition .................................................................................................... 9
1.2.2 Types of Phishing ......................................................................................... 9
1.2.3 Anatomy of a Phishing Attack ..................................................................... 10
1.3 Combat Techniques ......................................................................................... 11
1.3.1 Usable Security .......................................................................................... 12
1.3.2 Security Policy ........................................................................................... 13
1.3.3 Education and Training ............................................................................. 14
1.3.4 Culture ...................................................................................................... 15
1.3.5 Auditing .................................................................................................... 15
2. Background .......................................................................................................... 16
2.1 Social Engineering Research ............................................................................ 16
2.2 Phishing Research ........................................................................................... 16
vii
2.2.1 Designing Ethical Phishing Experiments .................................................... 16
2.2.2 Understanding Why Phishing Works .......................................................... 17
2.2.3 Methods to Defend Against Phishing .......................................................... 17
2.3 Social Engineering, Phishing, and Financial Institutions ................................... 18
3. Scope of Study ...................................................................................................... 19
3.1 Research Problem ............................................................................................ 19
3.2 Research Questions ......................................................................................... 19
3.3 Hypothesis ....................................................................................................... 19
3.4 Objectives ........................................................................................................ 20
4. Approach .............................................................................................................. 21
4.1 Investigated Parameter Space ........................................................................... 21
4.1.1 The Bank ................................................................................................... 21
4.1.2 Subject Population ..................................................................................... 21
4.1.3 Control Group............................................................................................ 21
4.2 Experimental Design ........................................................................................ 22
4.2.1 Original Experimental Design ..................................................................... 22
4.2.2 Modified Experimental Design .................................................................... 23
4.2.3 Experimental Variables .............................................................................. 24
4.3 Evaluation Metrics ........................................................................................... 25
4.3.1 Individual Phishing Experiments ................................................................ 25
4.3.2 Research Project as a Whole ....................................................................... 26
4.4 Privacy and Security ........................................................................................ 26
4.4.1 Intrinium Security ..................................................................................... 26
4.4.2 Institutional Review Board ......................................................................... 27
5. Research .............................................................................................................. 28
5.1 Overview .......................................................................................................... 28
viii
5.2 Social Engineering and Information Security Training ....................................... 29
5.3 First Phishing Trial .......................................................................................... 30
5.3.1 The Lure .................................................................................................... 30
5.3.2 The Hook ................................................................................................... 30
5.3.3 Subject Selection ....................................................................................... 31
5.3.4 Results ...................................................................................................... 31
5.4 Second Phishing Trial ...................................................................................... 37
5.4.1 The Lure .................................................................................................... 37
5.4.2 The Hook ................................................................................................... 37
5.4.3 Subject Selection ....................................................................................... 39
5.4.4 Results ...................................................................................................... 39
5.5 Third Phishing Trial ......................................................................................... 44
5.5.1 The Lure .................................................................................................... 44
5.5.2 The Hook ................................................................................................... 45
5.5.3 Subject Selection ....................................................................................... 45
5.5.4 Results ...................................................................................................... 45
6. Conclusion ........................................................................................................... 50
6.1 Results Summary ............................................................................................ 50
6.2 Future Work .................................................................................................... 51
6.2.1 Research Setup and Preparation ................................................................ 51
6.2.2 Overview of Proposed Research Design ....................................................... 52
6.2.3 Hypothesis ................................................................................................. 54
6.2.4 Investigated Parameter Space ..................................................................... 54
6.2.5 Methodology .............................................................................................. 57
6.3 Future Questions ............................................................................................. 61
References ................................................................................................................ 63
ix
Appendix A: Training Materials ................................................................................. 68
Social Engineering and Information Security Training Slides .................................. 69
Training Questions ................................................................................................ 77
Appendix B: Phishing Emails .................................................................................... 80
First Phishing Trial: Sample Phishing Email ........................................................... 81
Second Phishing Trial: Sample Phishing Email ....................................................... 82
Third Phishing Trial: Sample Phishing Email .......................................................... 83
Appendix C: Phishing Website User Interfaces ........................................................... 84
First Phishing Trial: User Interface ......................................................................... 85
Second Phishing Trial: User Interface ..................................................................... 87
Third Phishing Trial: User Interface ........................................................................ 88
VITA ......................................................................................................................... 90
x
LIST OF TABLES
Table 1: Information Gathering Techniques ................................................................. 7
Table 2: Post-test Only Control group Experimental Design ....................................... 22
Table 3: Results Summary of First Phishing Experiment ............................................ 32
Table 4: First Phishing Experiment Age and Gender Results ...................................... 33
Table 5: Length of Employment Results from First Phishing Experiment .................... 34
Table 6: Results Summary of Second Phishing Experiment ........................................ 40
Table 7: Second Phishing Experiment Age and Gender Results .................................. 41
Table 8: Length of Employment from Second Phishing Experiment ............................. 42
Table 9: Results Summary of Third Phishing ............................................................. 46
Table 10: Third Phishing Experiment Age and Gender Results ................................... 47
Table 11: Length of Employment from Third Phishing Experiment ............................. 48
xi
LIST OF FIGURES
Figure 1: Original Experimental Design ..................................................................... 23
Figure 2: Modified Experimental Design .................................................................... 24
Figure 3: Information Phished from First Phishing Experiment .................................. 36
Figure 4: Information Phished from Second Phishing experiment ............................... 44
Figure 5: Original Research Design ........................................................................... 53
Figure 6: 2-Sample Z-Test Proposed Research Design ................................................ 54
Figure 7: Future Training Design .............................................................................. 58
Figure 8: Future Phishing Experimental Design ......................................................... 59
Figure 9: First Phishing Trial Home Page ................................................................... 85
Figure 10: First Phishing Trial Hook .......................................................................... 85
Figure 12: First Phishing Trial Account Creation Page................................................ 86
Figure 11: First Phishing Trial Logon Page ................................................................. 86
Figure 13: Second Phishing Trial Hook ...................................................................... 87
Figure 14: Second Phishing Trial Sign Up Page .......................................................... 87
Figure 16: Third Phishing Trial Error Message ........................................................... 88
Figure 15: Third Phishing Trial Hook ......................................................................... 88
Figure 17: Third Phishing Trial Thank You ................................................................ 89
Figure 18: Third Phishing Trial Warning .................................................................... 89
1
1. INTRODUCTION
Computer security is a vast field consisting of many different methodologies and
technologies. It defends against a variety of attack vectors, both internal and external.
Traditionally computer science thinks of “computer security” only in the technological
sense. The technological aspect of computer security includes things such as
cryptography, encryption, firewalls, access control, and passwords. While this side of
computer security is extremely important, it is critical to remember the less technical side
of computer security. This aspect of computer security focuses on the design,
development, implementation, and execution of security technology. This is the human
aspect. Nothing is secure that does not take both aspects into consideration.
The greatest challenge to the human aspect of computer security is social
engineering. The human element of any system is considered to be the hardest to secure
and control. Regardless of the security a system uses, there will always be people involved
with the operations. If the people are not secured, the entire system is vulnerable to
attack through social engineering.
Social engineering is a very real and dangerous threat to the reputation and wealth
of individuals and organizations of all types. Famous social engineers, such as Kevin
Mitnick, and top security professionals have published writings on social engineering but
there has been little academic research done on the actual phenomenon itself. This
chapter will review the current academic and non-academic works on social engineering.
1.1 Social Engineering
Most of the academic research on social engineering as a whole has not been in
the form of original research or even scientifically tested theories. Instead, most research
consists of industry report analysis and anecdotes given by self-proclaimed social
engineers. This could be due partly to the risky nature of research in this area. A number
of ethical hurdles present themselves when attempting experiments involving deception.
A social engineering research experiment that is simulating an attack is difficult to setup
properly due to the precautionary measures needed to ensure the safety and security of
research participants.
1.1.1 Definition
Social engineering has no universally agreed upon definition. Examples include:
2
“…the exploitation of psychological triggers and cognitive biases as a means to gain
unauthorized access to information or information systems” [1].
Another definition is:
“… the art and science of getting people to comply with your wishes” [2] [3].
Mitnick uses a more detailed definition:
“Social engineering uses influence and persuasion to deceive people by convincing
them that the social engineer is someone he is not, or by manipulation. As a result, the
social engineer is able to take advantage of people to obtain information with or without
the use of technology” [4].
The general theme among these and other definitions of social engineering is the
focus on the weakest link in any security system: the human element. It is a well-known
fact among security professionals (and hackers) that people are the weakest part of any
security system, rather than technology [3] [5]. Social engineers exploit the human
element of systems to meet their goals. Sarah Granger, an award-winning innovator and
thought leader who writes, speaks, advocates and advises on how information technology
and new media are changing society, writes that everyone seems to agree that “social
engineering is generally a hacker’s clever manipulation of the natural human tendency
to trust” [6].
Brad Sagarin, PhD, a social psychologist is quoted in Mitnick’s book describing
the social engineer’s use of psychology and persuasion as follows [7]:
“There’s nothing magic about social engineering. The social engineer employs the same
persuasive techniques the rest of us use every day. We take on roles. We try to build
credibility. We call in reciprocal obligations. But unlike most of us, the social engineer
applies these techniques in a manipulative deceptive, highly unethical manner, often
to devastating effect.”
A social engineer carefully uses psychology to manipulate unsuspecting people
into doing something the social engineer wants them to do, which under normal
circumstances they would not do. This could be convincing a person to provide
information or duping that person into performing some action on behalf of the social
engineer. Therefore, it is human behavior norms that provide social engineers with the
ability to exploit the trust of legitimate users so as to circumvent standard security
measures, in other words, it is an organization’s own employees that pose the most
dangerous threat to its own security [8].
1.1.2 Goals and Motivation of a Social Engineer
The social engineer is a hacker who utilizes social engineering techniques to aid
in his or her hack. The goals of a social engineer are thus generally the same as a hacker:
3
“to gain unauthorized access to systems or information in order to commit fraud, network
intrusion, industrial espionage, identity theft, or simply to disrupt the system or network”
[9].
A variety of reasons exist to motivate a person to venture into hacking and social
engineering. Sarah Granger writes that “since social engineering involved the human
element of any attack, it’s important to get into the head of the hacker and understand
her motivation … By knowing why we are at risk, we can better protect ourselves from
the foolish things we do, thereby allowing social engineers to exploit us” [5].
Granger writes that “historically, the motivation has been intellectual challenge,
bragging rights, access to sensitive information, simple curiosity, or our biggest fear –
malicious intent” [5]. Other motivating factors include but are not limited to [2]:
Financial Gain
An individual might feel entitled to more money than what he or she makes or
perhaps they have some expensive habit (e.g. gambling or drugs) that needs
satisfying.
Self-Interest
An individual might wish to gain access to information for self-serving reasons or
perhaps they wish to change information that is associated with themselves, a
friend or a family member.
Revenge
An individual may wish to target a friend, colleague, organization or even a total
stranger to satisfy an emotional need for vengeance.
External Pressure
An individual may be under external pressure from friends, family members, or
an organized crime syndicate for the reasons listed above: financial gain, self-
interest, or revenge.
1.1.3 Aspects of a Social Engineering Attack
There are two main aspects to a social engineering attack: the physical and the
psychological. The physical aspect is essentially “the location of the attack, such as in
the workplace, over the phone, dumpster diving, on-line” [9]. The psychological aspect
“refers to the manner in which the attack is carried out, such as persuasion,
impersonation, ingratiation, conformity, and friendliness” [9]. Charles Lively breaks the
psychological aspect down into four distinct attack vectors [10]:
4
Careless Attack Vector
This attack vector is “made exploitable due to the indifference of implementing,
using or enforcing proper defensive countermeasures. It is often the first phase
of a more complex overall attack.”
Comfort Zone Attack Vector
This attack vector is exploited because “the user is in an environment they feel
comfortable in, therefore, their level of threat perception is lower.”
Helpful Attack Vector
This attack vector “is used on the premise that people generally will try to be
helpful, even if they do not know whom they are helping.”
Fear Attack Vector
This attack vector is “often the most aggressive type of psychological attack … Its
foundation is based on attacking the user in such a way that the user provides
the attacker with the information or access needed due to putting the user in a
state of anxiety, pressure, stress and fear.”
1.1.4 Anatomy of a Social Engineering Attack
A social engineering attack can be broken up into different steps or phases.
Malcolm Allen developed one generally accepted description of a social engineering attack
called “The Cycle” [2]. The Cycle breaks down a social engineering attack into four phases
which can be repeated as necessary until the goal is fully accomplished:
1. Information Gathering
This step includes “a variety of techniques … used by an aggressor to gather
information about the target(s).”
2. Developing Relationships
This step is where “an aggressor may freely exploit the willingness of a target to
be trusting in order to develop rapport with them.”
3. Exploitation
This step is where “the target may then be manipulated by the ‘trusted’ aggressor
to reveal information (e.g. passwords) or perform an action (e.g. creating an
account or reversing telephone charges) that would not normally occur … This
action could be the end of the attack or the beginning of the next stage.”
5
4. Execution
This is the final step in the cycle “once the target has completed the task requested
by the aggressor.”
Jason Baker and Belinda Lee from Florida Atlantic University break down the
attack into nine steps which also can repeat sections as necessary in order to reach the
end goal [8]:
1. Footprinting
This step includes “information gathering, and other data gathering queries.”
2. Scanning
This step includes “identifying specific areas for security intrusion.”
3. Enumeration
This step includes “intrusive probing and usernames, password and other
vulnerabilities.”
4. Gaining Access
This step includes “viruses, worms, Trojans, Spyware, brute-force attacks and
software vulnerabilities, etc.”
5. Privilege Escalation
This step includes “exploiting system bugs to gain complete control.”
6. Pilfering
This step includes “gaining immediate access to trusted hosts and the removal of
valuable information.”
7. Covering Tracks
This step includes “editing or removing logs.”
8. Creating Backdoors
This step includes “creating other vulnerabilities to ensure further access in the
future.”
9. Denial of Service
This step is “designed to stop computer systems from working.”
Baker and Lee’s steps overlap Allen’s Cycle but focus on a specific technical type
of social engineering attack. Baker and Lee assume the end goal of the attack is a Denial
of Service (DoS) on the target’s computer systems. Naturally, a social engineering attack
can have many different end goals, not just a DoS. Social engineering attacks can also
be much simpler than what Baker and Lee imply with their attack anatomy. Allen’s four
6
step approach to a social engineering attack more accurately reflects all cases of social
engineering.
1.1.5 “The Cycle”
This section will describe Allen’s Cycle in more detail: (1) information gathering,
(2) developing relationships, (3) exploitation, and (4) execution.
1.1. 5.1 Inf orm ation Gathering
In order for an attacker to appear legitimate when duping his or her target,
background research on the target is necessary. This phase is referred to as
“Footprinting” by Granger (as well as Baker and Lee [8]) which she defines as “the art of
gathering information … It’s commonly done to research a predetermined target and
determine the best opportunities for exploitation” [5]. This allows the social engineer to
get the lingo of the company, names and positions of employees, and any other
information that a legitimate user would likely know. Having this knowledge in the
attacker’s tool-belt allows him or her to sound more believable, thereby resulting in fewer
questions and greater trust from the target.
Information collected on the target may or may not seem sensitive to the average
person but could prove invaluable to a social engineer. Such information could include
phone lists (current or outdated), birthdates, and an organization’s organizational chart
[2]. Given the common conception of such information as non-sensitive it often gets
discarded without a second thought and without taking any security precautions (i.e.
shredding of documents). The availability of this type of information simplifies the
attacker’s job.
Methods of gathering this information can vary. It can be as simple as doing a
Web search or looking through someone’s trash, to a more complicated forensic analysis
on discarded hardware. Information gathering techniques used by social engineers can
include: pretexting, phishing, cold calling, asking for favors, shoulder surfing, and
impersonation. In Table 1: Information Gathering Techniques is a list of many of the
possible information gathering techniques used by social engineers [1].
As an example, a social engineer interested in Financial Institution A would do
his or her research on the business before moving on to the next step in the cycle.
Information can be easily gathered by looking on the company’s website and doing a
Google Web search for blogs, news articles, videos, or any other related information
7
regarding Financial Institution A. This public information can provide enough clues to
dig them deeper into the institution to learn less-publicly available information.
Asking for Favors
Phishing
Cold Calling
Pretexting
Contriving Situations
Reverse Social Engineering
Dumpster Diving
Reconnaissance
Forensic Analysis
Simple Requests
Giving out Free Software
Shoulder-Surfing
Impersonation
Surveys
Mail-Outs
Tailgating
Photography
Theft
Pharming
Trojan Horses
TABLE 1: INFORMATION GATHERING TECHNIQUES
Digging deeper can include using the surface information gathered from public
locations to make phone calls to the help desk or customer support to learn names and
phone numbers of managers and other employees or learn what support software is being
used. This is where the information gathering stage begins to phase into the next stage:
Developing Relationships.
1.1. 5.2 Devel oping Relatio nshi ps
With the information gathered, the attacker will now begin to use it to develop
relationships with employees within the target organization. This can be accomplished
by making phone calls to employees within the target. The attacker might pretend to
need help from the employee or to be an employee themselves at another location (e.g.
another office campus or off-campus working from home). Another tactic could be what’s
known as “reverse social engineering” where the social engineer creates a made up
problem where a real employee is impacted and needs to contact the attacker to resolve
it. These attacks are generally more complicated but can be very effective. It could
8
include creating a real problem such as disabling the employee’s network access or just
pretending there is a problem such as an email security virus loose on the network. The
social engineer always plays the part of the person the employee (victim) needs to contact
in order to get the problem (real or pretend) resolved.
Multiple phone calls to the same employee within the target can build trust and
a relationship. Making small requests for help that seem within reason build that
relationship. A social engineer may also engage the target in small talk and chit-chat to
build trust. Over time, trust is gained with this employee making it more likely they will
help the attacker during the next stage of the cycle.
To continue with the example started in the previous section, the social engineer
calls a teller at Financial Institution A pretending to be a teller themselves at a different
financial institution that frequently works with Financial Institution A. The social
engineer pretends to be new and have a question about the application used by the tellers,
such as how to transfer money between the two institutions. After a few calls like this, a
relationship is created based on helpfulness. This is the perfect type of relationships for
a social engineer to start for the next stage: Exploitation.
1.1. 5.3 Exploitation
At this point, the social engineer has information on the target and a good rapport
with an employee within the target. This is the stage where the social engineer can make
a riskier request that seeks more confidential information or to convince the employee to
perform some action. The trust built with the employee is exploited by the social engineer,
yet the employee may never know they have been used and will most likely walk away
feeling good about the encounter. [2]
The social engineer in the continuing example requests information from their
helpful teller that gains them remote access to the computer system, such as dial-up
access or login information to an intranet. From this point the social engineer has all the
access they need to execute their plan in the next step: Execution.
1.1. 5.4 Execution
Having completed the previous stages of the cycle, the social engineer has all the
pieces he or she needs to finish their attack.
To finish the example, the social engineer may just make a major request to the
friendly teller such as to transfer a large sum of money from one of the accounts at
9
Financial Institution A to an account the attacker has access to. Or if the request in the
previous step was to gain remote access to the institution, the social engineer could use
that access to install a virus to gather financial information on clients.
1.2 Phishing
Phishing is a very common technique used by social engineers. It can be a means
to gather information for a larger attack (e.g. acquire logon information for an organization
in order to gain access to private systems) or the phish itself could be the whole attack
(e.g. to steal credit card numbers).
1.2.1 Definition
Phishing is defined by Dr. Markus Jakobsson, a security researcher and Principal
Scientist of Consumer Security at PayPal:
“A form of social engineering in which an attacker, also known as a phisher, attempts
to fraudulently retrieve legitimate users’ confidential or sensitive credentials by
mimicking electronic communications from a trustworthy or public organization in an
automated fashion. Such communications are most frequently done through emails
that direct users to fraudulent websites that in turn collect the credentials in question.
Examples of credentials frequently of interest to phishers are passwords, credit card
numbers, and national identification numbers.
The word phishing is an evolution of the word fishing by hackers who frequently
replace the letter ‘f’ with the letters ‘ph’ in a typed hacker dialect. The word arises
from the fact that users, or phish, are lured by the mimicked communication to a trap
or hook that retrieves their confidential information [11].”
1.2.2 Types of Phishing
There are many different forms of phishing utilized by social engineers. A few of
the more common types are described here.
1.2. 2.1 De ceptive Phishing
Deceptive phishing is an email sent out in mass that includes a “call to action”
demanding that the recipient clicks on a provided link [12]. A “call to action” example
would be telling the recipient that their account at some institution (e.g. PayPal, eBay or
Bank of America) has a problem that needs their immediate attention. Another “call to
action” example would be claiming that there is a “new service … being rolled out at a
financial institution, and offering the recipient, as a current member, a limited-time
opportunity to get the service for free” [12].
10
1.2. 2.2 Spe ar Phishi ng
Spear phishing attacks are similar to deceptive phishing except that they are
focused in on a single person, department or organization. With this form of phishing,
the phishing email appears to be addressed from a legitimate person often from within
the same company who holds a position of trust [13]. This phish could also come from a
seemingly trusted outside source that appears to be legitimate and specifically targets an
individual or department within the target organization.
1.2. 2.3 Conten t-Inje ction Phis hin g
Content-injection phishing is a form of phishing where the phisher inserts
malicious content into a legitimate website. This content could redirect users to another
website of the phisher’s choosing, install malware onto the user’s computer, or insert a
frame onto the legitimate website that will redirect data entered by the user back to the
phisher [12] [13]. Cross-site scripting and SQL injection techniques are used in this form
of phishing [12]. Cross-site scripting is a content-injection technique “… done by
inserting a script into an URL or a form that is later executed in the client browser” [14].
SQL injection is a means to execute database commands on a remote server that can
cause information leakage [12]. Both cross-site scripting and SQL injection
vulnerabilities are a result of improper filtering.
1.2.3 Anatomy of a Phishing Attack
Phishing attacks all share three common components: (1) the lure, (2) the hook,
and (3) the catch [11].
1.2. 3.1 The Lure
The phisher sends out a mass email which uses a convincing story to persuade
the user to follow a URL hyperlink inside the email to a website controlled by the phisher.
The social engineering aspect of a phishing attack comes out in “the lure” as it tries to
make the story sound legitimate enough to get the user to hand over confidential
information (e.g. username, password, credit card numbers). Often the story is that the
social engineer is a legitimate, well-known organization whom the user may have an
association with (e.g. Bank of America, PayPal, eBay). The story often includes an urgent
message to the user that they need to update their account information. Different story
scenarios the phisher might use include [11]:
11
Security Upgrade
The user is told that there is an important security update that they need to install
or that there is a new service being provided to increase security and protect them
from fraud and they need to enroll in it.
Incomplete Account Information
The user is told that their account information is out of date or has missing
information that requires them to log in to update or complete.
Financial Incentive
The user is enticed to follow a hyperlink and provide information with some
financial incentive such as a coupon, discount or a chance to win some sort of
prize.
False Account Updates
The user is thanked for updating their account information and also told that if
they have received this message in error to follow the URL hyperlink provided, log
in and report the incident.
1.2. 3.2 The Hook
The website that the phisher sends the user to is considered “the hook”. The
website typically completely mimics the appearance of the real website belonging to the
organization the phisher is spoofing. The phisher wants to make the website as
indistinguishable as possible from the real website to get the user to believe in its
authenticity so they will hand over their confidential information.
1.2. 3.3 The Catch
This is considered the final piece of the phishing attack and is sometimes called
“the kill.” In this step the phisher uses the information collected from the user for his or
her advantage (e.g. identify theft, fraud).
1.3 Combat Techniques
Combatting social engineering requires solid standard security to already be in
place. Technological security should be utilized as much as possible to protect the
hardware, software, and networks. This can include cryptography, secure protocols,
firewalls, antivirus software, etc.
Social engineering counter-measures are additional security measures to
computer security. These counter-measures must secure the human-element of the
12
system. Even with the best computer security in place, as computer security specialist
Bruce Schneier points out, the “computer system … is going to have to interact with users
in some way, at some time, for some reason. And this interaction is the biggest security
risk of them all. People often represent the weakest link in the security chain and are
chronically responsible for the failure of security systems” [15].
Special counter-measures for social engineering focus on the vulnerability of the
users and their natural tendency to trust and be helpful. A social engineer “bypasses
cryptography, computer security, network security, and everything else technological. It
goes straight to the weakest link in any security system: the poor human being trying to
get his job done, and wanting to help out if he can” [15].
Since social engineering attacks have two different aspects the “combat strategies
… require action on both the physical and psychological levels” [9]. Douglas P. Twitchell,
an assistant professor at Illinois State University and information assurance and security
researcher, lists the “three ways that are commonly suggested for defending against social
engineering attacks: (1) education, training, and awareness (ETA) and (2) policy backed
up with (3) auditing” [16]. Social engineer, Kevin Mitnick recommends the following series
of countermeasures [7]:
Develop clear, concise security protocols that are enforced consistently
Develop security awareness training
Develop simple rules defining what information is considered sensitive
Develop a rule to require verifying the identity of any requestor asking for
restricted information or for a restricted action
Develop a data classification policy
Train employees to resist social engineering attacks
Test employee susceptibility to social engineering attacks via conducting a
security assessment
Other means to combat social engineering that are often suggested are changing
the organization’s security culture and applying human-computer interaction (HCI)
principles to make usable security [17] [18]. Difficult to use security makes users more
likely to try to by-pass it or help someone else by-pass it [15].
1.3.1 Usable Security
The greatest security in the world is useless if it is not user friendly. If it is too
difficult to use or is irritating to users, the security system simply will not be used [15].
13
Security expert Bruce Schneier writes that “a smart security designer knows that users
find security measures intrusive, that they will work around them whenever possible,
that they will screw with the system at every turn” [15]. Social engineers know this and
will take advantage of it. Schneier continues by pointing out that “when a deadline
approaches and you have to get the job done, people don’t even think twice about
bypassing security. They’ll prop the fire door open so that someone can get into the
building more easily, and they’ll give out their password or take down a firewall because
work has to get done” [15]. Sasse, Brostoff, and Weirich suggest applying existing
knowledge of HCI to usability issues within security to help create usable security [18].
1.3.2 Security Policy
Clear and concise security policies must be developed and implemented which
specifically address social engineering. Schneier explains that a “security policy provides
a framework for selecting and implementing countermeasures against threats” [15]. This
framework is crucial for users to know how to respond in the case of a social engineering
threat. A “policy should outline who is responsible for what (implementation,
enforcement, audit, review), what the basic network security policies are, and why they
are the way they are … A clear concise, coherent, and consistent policy is more likely to
be followed” [15].
Granger recommends creating strong policies which are somewhere between
general and specific to allow for flexibility in the future development of procedures [9].
The policies “should address information access controls, setting up accounts, access
approval, and password changes. Modems should never be permitted on the company
intranet. Locks, IDs, and shredding should be required. Violations should be posted and
enforced” [9]. Granger also suggests implementing a strict policy “that passwords …
never be disclosed over the phone or by e-mail; rather, they should only be disclosed in
person to trusted, authorized personnel” [9]. Arthurs lays out some specific policies to
help defend against social engineering in her SANS whitepaper such as how and when
information can be released, how system access is granted and revoked, password
requirements, no modems under any circumstances on the intranet, when the help desk
can give out passwords, employee and visitor identification, document shredding, general
physical security requirements, how violations are reported, and how data and hardware
are stored and ultimately destroyed [19]. Almost all the policies suggested by security
professionals and researchers may be summarized as locking down a system as much as
14
possible, being able to verify the identity of legitimate users, and protecting all potentially
confidential information.
All policies should be maintained and updated as regularly as possible to ensure
they are still relevant and effective. Arthurs recommends that “policies … be reviewed,
on a rotational basis, at least every five years, with 20% of the policies under review each
year … that way, old policies can be updated, obsolete policies can be cleared out and
new requirements blended into a living document” [19]. The latest version of each policy
should be posted for users to see, such as on the organization’s intranet.
Security policies may be perceived as expressions of the organization’s distrust
for its employees. It is important to get employees to understand that security policies
are not personal. They are in place to protect organizational assets, which ultimately
includes the employees. Flechais, Riegelsberger, and Sasse suggest “if company
employees understand that … [the] polic[ies] … [are] necessary to comply with external
regulations, or to protect the reputation of the organization, it de-personalizes the fact
that employees are not trusted … making it clear that the lack of trust is ‘business not
personal’” [20]. They also suggest that a “good strategy for increasing dependability of
employees in the face of … [social engineering] attacks is to institute simple, reliable rules
for mutual authentication, and a supportive point of contact for no-fault reporting and
clarifying rules” [20].
1.3.3 Education and Training
Users should be properly educated and trained on social engineering. The
training program should cover what social engineering is, techniques used by a social
engineer, how it impacts the organization as well as the user, what to do in the case of a
suspected social engineering attack, and who to report incidents to. The training should
explain to users the purpose behind the security policies in place, “to sensitize them to
risks and potential losses, and to train them to recognize social engineering techniques”
[19]. Users need to understand and also appreciate the reasons behind the rules [19].
Granger suggests that “one of the best methods for educating employees to these risks is
to take social engineering stories from current events and post them on an internal web
site, or use email for safety tips and informational stories” [9]. She continues by writing
that “telling authentic stories of what happened to the ‘other poor guy’ increases
resistance to these exploits in a non-threatening way, inoculating the employee against a
vulnerability to social engineering” [9].
15
This training should be ongoing, not just limited to new users: “security
awareness and training [needs] to be given continuously to all employees, as opposed to
just giving it to newcomers” [20]. The ongoing nature of the training helps to keep users
security conscious and up to date on any new attack trends or security policy updates.
Granger writes that “continued awareness throughout the organization is the key to
ongoing protection” [9].
The training program should be multifaceted including “some combination of the
following: videos, newsletters, brochures, booklets, signs, posters, coffee mugs, pens and
pencils, printed computer mouse pads, screensaver, logon banners, note pads, desktop
artifacts, tee shirts and stickers” [19]. These items should be changed and updated
frequently to keep them effective [19].
1.3.4 Culture
Additionally, users must believe that the security is necessary and that they are
playing an important role in enforcing it. Sasse, Brostoff, and Weirich insist that “security
design has to integrate all aspects of security, from the technical to the user interface and
user training, with the organization’s work practices and overall culture” [18]. They also
believe that “for effective security, organizations must develop culture in which … security
is adopted as a shared concern by all employees” [18]. Users are more likely to comply
with the security measures put in place if they believe in them.
Once security measures and policies are in place, it is important to note that
“imposing sanction on some members of an organization, but not on others, prevents the
development of a shared set of values that could foster a better security culture, and thus
increase dependability” [20]. If the culture of the organization includes a shared value of
everyone being responsible for security it will increase the chance that users will be
mindful of security policies and procedures and be on the lookout for potential social
engineering attacks. Granger writes that “in order to be successful, organizations must
make computer security part of all jobs, regardless of whether the employees use
computers. Everyone in the organization needs to understand exactly why it is so crucial
for the confidential information to be designated as such, therefore it benefits
organizations to give them a sense of responsibility for the security of the network” [9].
1.3.5 Auditing
After the security measures have been implemented and users have been trained,
there are a couple recommended means for maintaining a state of preparedness. The
16
first is to conduct regular reviews of the security controls implemented to ensure they are
of an acceptable standard [2]. The second method is to conduct an actual simulated
attack [2].
2. BACKGROUND
2.1 Social Engineering Research
There has been little academic research into social engineering. The majority of
the research on social engineering is on the social engineering technique of phishing.
Lena Laribee developed a social engineering taxonomy where she analyzed Kevin
Mitnick’s stories of social engineering [21]. Studying industry trends or stories from self-
proclaimed social engineers seems to be a common trend among academics writing about
social engineering. Nathaniel Joseph Evans developed an academic definition for social
engineering and analyzed the human vulnerability in security which is exploited by social
engineers [22]. Unfortunately, Evans only explained why people are vulnerable and did
not give any suggestions or ideas as to how to help counter this innate human
vulnerability.
2.2 Phishing Research
Phishing is a social engineering technique that is well recognized as a serious
security problem needing to be addressed. Academics in computer security all over the
world are working on understanding why phishing works and finding methods to defend
against it.
2.2.1 Designing Ethical Phishing Experiments
All researchers conducting phishing experiments must first get their project
reviewed and ultimately approved by the Institutional Review Board (IRB). The IRB
oversee all academic research conducted involving human subjects, mandate that the
research is done in compliance with federal regulations and ensure that it is done in a
manner consistent with the three ethical principles outlined in the Belmont Report [23]
[24]. These principles are: 1) respect for persons, 2) beneficence, and 3) justice [23] [24].
The details of how this process works will not be covered here as it is a topic all its own
and is well covered by Markus Jakobsson et al. in [23] and [25]. Given that most phishing
experiments will require human subjects, it is important to understand how to properly
setup the experiment in an ethical manner so that the IRB will approve the research.
17
2.2.2 Understanding Why Phishing Works
In order to be able to defend against phishing one must first understand why
phishing works. Rachna Dhamija et al. analyzed malicious strategies used in phishing
attacks by conducting a usability study with twenty participants who were aware they
were being evaluated [26]. From this study it was discovered that twenty-three percent
of their participants did not pay attention to browser-based cues which could indicate a
phishing website [26]. Browser-based cues include the address bar, the status bar, and
security cues. This lack of attention to detail by users, led to users incorrectly
determining if a website was legitimate or phishing forty percent of the time [26].
Jakobsson found that the majority of people actually do notice browser cues and
signs of phishing within the content itself [27]. This corresponds with the findings of
Dhamija et al., however, Jakobsson found that some of these stimuli can backfire when
overused [26] [27]. The results also showed, that while users pay attention to these
details they are often misinterpreting what it means often mistaking legitimate emails
and websites for phishing and vice versa [27].
2.2.3 Methods to Defend Against Phishing
Education is naturally considered one of the main methods to defend against
phishing. Users must be educated on what phishing is, how to recognize a phishing
email or website or phone call, and what to do when they encounter phishing. The trouble
is finding effective education techniques. Jakobsson points out that there are “inherent
limitations in what can efficiently be communicated, given the complexity of the problem
and the relative lack of interest in active involvement on behalf of typical users” [27].
There are many traditional methods, such as books and articles, and non-traditional
methods, such as computer games and comics, being used to try and reach users on this
topic [27].
A suggested method of defense for organizations against phishing is to register
cousin-name domains [27]. Cousin-name domain names are similar enough to the
organization’s real domain that a user could easily be fooled if used by a phisher. If the
real organization owns their domains and all cousin-name domains, phishers will not be
able to use them to try and trick users.
Some researchers are working on developing various technological techniques to
try and combat phishing attacks. Engin Kirda and Christopher Kruegel developed a
browser extension called AntiPhish that tries to protect users from spoofed websites by
18
tracking the sensitive information of users and presents a warning to them when they
attempt to give that information away to an untrusted website [28]. Mohamad Badra et
al. developed the TLS-SRP (Transport Layer Security Secure Remote Password) and TLS-
PSK (Transport Layer Security Pre Shared Key) protocols to help reduce the threat of
phishing [13].
2.3 Social Engineering, Phishing, and Financial Institutions
Financial institutions have been the number one target of phishing attacks.
According to the Anti-Phishing Working Group, the second quarter of 2012 had thirty-
four percent of all phishing attacks targeted at financial services [29]. The second largest
group to be targeted by phishers, according to this same report, was payment services at
thirty-two percent.
There are many reports of big banks being the target of successful phishing
attacks. Chase bank’s members have fallen victim to a number of phishing emails which
they now list on their website to increase awareness of online fraud [30]. Wachovia has
been a similar victim of phishing and has also posted sample phishing emails on their
website for their customers to see and become aware of [31].
The majority of the attacks sent to these big-name banks are targeted spear
phishing and malware attacks [32]. Michael Murray, managing partner of MAD Security
told TechNewsWorld recently [32]:
"The majority of the attacks right now involve targeted phishing and malware attacks
-- where the most common attack vector a few years ago was Web applications, the
most common attack vector today comes through our people… Spear phishing through
email, social media and even IM has been used to cause a large number of breaches
in the last two years."
There is no real academic research done on phishing and financial institutions
specifically. The SANS Institute published a report on phishing and banks but it only
covered why it is important to take phishing attacks into consideration for security
measures and offers up some general ideas for combatting it [33]. It is critical to fill the
research gap on social engineering’s, especially with phishing, impact on financial
institutions and how to best protect them from this attack vector.
19
3. SCOPE OF STUDY
3.1 Research Problem
People are the most vulnerable component of any system. They are particularly
susceptible to social engineering attacks where they are tricked and deceived by
attackers. In order to properly secure a computer system, the people need to be secured
against this type of attack vector. Social engineering is now widely accepted to be the
greatest threat to the security of any financial institution [34]. As security technology
improves it becomes increasingly harder for cybercriminals to break into a bank via
traditional hacking methods. Hacking the person who can grant you access to the bank’s
network is becoming a far more appealing approach, given its pure simplicity. This makes
it all the more important to start research into this problem with a focus on financial
institutions.
This project is focused on financial institutions because they have a great deal of
confidential personal and financial information that must be protected from all attack
vectors. This project seeks to discover how effective some social engineering counter-
measures are in the financial institution setting. While the objectives remain the same,
because of circumstances related to the actual experimental conditions, the research
questions needed revision which will be described in Section 4.
3.2 Research Questions
The questions being asked are:
Is standard training sufficient to increase awareness of social engineering and
phishing and also decrease the vulnerability of employees?
Is ongoing training targeted specifically at social engineering combined with
security audit testing better than standard training at decreasing the vulnerability
of employees?
Are employees equally vulnerable to all forms of phishing and if not, to which
forms are they more susceptible (such as spear phishing)?
3.3 Hypothesis
This project hypothesized that there is a link between repeated specialized training
with testing and a person’s vulnerability to social engineering attacks. Training is often
considered unhelpful in the fight against social engineering due to the unpredictability of
people. However, it is possible that specialized training which drives home the danger of
20
social engineering and how it impacts that person both at work and at home might leave
a deeper impact thus making the training more effective. In addition to specializing the
training itself, the training should be repeated regularly (e.g. yearly) to remind those being
trained about social engineering. Training should be updated regularly to keep those
being trained up to date on any new threat vectors or attack methods. In between
training, there should be testing done randomly to test trainees on their training retention
as well as provide gentle reminders of their training in the case they fail the test. The
hypothesis for this research was that specialized training is more effective in reducing the
vulnerability level of financial employees to social engineering tactics than standard
training alone. It should help ensure that those being trained are properly prepared for
a real social engineering attack.
3.4 Objectives
The main objective of this research is to determine if there is a potential link
between repeated specialized training with testing and social engineering vulnerability.
This research project does not anticipate being able to prove the stated hypothesis, but
merely to determine if it is a worthwhile area for further study. Proving the hypothesis
will require a much deeper level of research which cannot be accomplished during this
master’s level project.
21
4. APPROACH
4.1 Investigated Parameter Space
4.1.1 The Bank
For this research project, a bank from the western United States was an integral
partner. For confidentiality reasons to protect them and their employees, the bank does
not wish to be identified and will be referred to only as the “Bank” for the remainder of
this paper.
4.1.2 Subject Population
The population for this research project was the employees from the Bank. The
exact size of this population during each experiment was accidentally lost but it is known
that it was always under 3000 people. The Bank wishes that its employees remain
anonymous. They were only interested in learning how effective their security training is
with their employees as a whole and whether or not it needs to focus more energy toward
defending against social engineering attacks.
4.1.3 Control Group
A control group was a desired goal for this project, but it was one that was not
met due to uncontrollable circumstances. Ideally there would have been a group of
subjects who had no specialized training so as to have data with which one could compare
and contrast the results from the experimental group. With the schedule placed on the
project, there would have only been a single month worth of potential control group
subjects available. This was due to the Bank’s requirement for all employees to complete
their annual training at a specific time of the year. This would have made the control
group available for just the first experiment at most. Because of this no full control group
was utilized.
The experiments did divide the subjects into two groups. The first group of
subjects was given the phishing test without informing them it was a test. The second
group of subjects was given the phishing test and if they followed the phishing lure the
entire way through they were told at the end that it was a phishing test. From this
division it could be seen if a person who is told they just fell for a phishing attack is less
susceptible to further phishing attacks.
22
Because of the lack of a control population, the revised research question is as
follows: Is there a difference in susceptibility to phishing for someone who is informed
they have been a victim of phishing as compared to someone who has not been so
informed?
The new hypothesis is that informing people that they have fell victim to a
phishing attack will make them more vigilant and less susceptible to further phishing
attempts.
4.2 Experimental Design
4.2.1 Original Experimental Design
The original experimental design planned for this project was a modified version
of the posttest only control group design [35]. Where represents the random selection
of individuals selected for the experiment, represents the specialized training,
represents the exposure to the phishing email, and represents the observation of the
subjects with the phish.
TABLE 2: POST-TEST ONLY CONTROL GROUP EXPERIMENTAL DESIGN
The experiments were originally planned to be conducted using a control group.
The experimental group would receive the specialized training. Each group would
participate in the phishing experiments.
23
4.2.2 Modified Experimental Design
The modified experimental design did not include a control group. Due to the
financial institutions training schedule, it was not possible to give the specialized security
training to only one group of subjects. The entire population ended up receiving the
specialized security training along with their regular annual training courses. This
ultimately made the training portion of this research moot since there was no way to
determine its real impact on the subjects given that there was no group to compare with
who had not received training.
The phishing portion of this research ended up being slightly modified with each
round of phishing, however, the basic foundation of the design remained the same. Each
FIGURE 1: ORIGINAL EXPERIMENTAL DESIGN
24
phishing email included a link to the phishing website. The website would toggle between
showing a phishing warning to the subjects who were successfully phished and not giving
a warning. The warning consisted of internal and external links to phishing resources
(e.g. how to spot a phishing email tutorial).
Unfortunately, the subjects for each experiment were freshly selected with no
guarantee of being carried through to the next experiment. As such, it was not possible
to determine if the phishing warning made any difference to the subjects’ vulnerability.
4.2.3 Experimental Variables
This research has the following independent variables: type of security training
program, and form of phishing attack. The following are dependent variables: security
FIGURE 2: MODIFIED EXPERIMENTAL DESIGN
25
awareness level, and vulnerability level. Other covariates are: sex, age, length of
employment, job position, organizational level, branch location, and employment status.
The following intervening variables also need to be taken into consideration: spam filter,
workload, motivation, and out of office.
4.3 Evaluation Metrics
4.3.1 Individual Phishing Experiments
The first two phishing experiments attempted to gather the following data from
the subjects:
Username
Password
Security Question
o City of Birth
o Mother’s Maiden Name
o Name of First Pet
Security Answer
Birthdate
Contact Information
o Address
o Phone Number
o Email Address
Each experiment varied slightly as to what data was being requested from the
subject. The goal was to acquire their username and password. It is common for users
to reuse both their username and password for multiple logins to different systems [36]
[37]. It is hard to remember a unique username and password set for each system
especially if the password required is a secure password. This, unfortunately for users
but fortunately for social engineers, is a security vulnerability which is not hard to
compromise.
The individual phish was considered a success if the username and password was
submitted by even just one subject. However, the more subjects who submitted their
username and password the greater probability of getting a match with their Bank
domain credentials.
The third phishing experiment modified the phishing tactic used. Instead of
attempting to acquire information from the subjects, the goal was to get the subjects to
26
download a file onto their computer. This experiment was considered a success if the
link to download the desired file was clicked by the subject. Details of the phishing tests
are provided in Section 5.
4.3.2 Research Project as a Whole
Half of the subjects were told at the end of the phishing lure that they just fell for
a phishing test. That same half were also provided with resources to learn more about
phishing, how to recognize a phishing email or phishing website, and what to do better
in the future.
The overall research project is considered a success if:
The subjects who received training at the beginning of the study never fell for any
of the phishing experiments.
The subjects who received training at the beginning of the study, but failed one of
the phishing experiments, saw the warning message, and did not fail any further
phishing experiments.
4.4 Privacy and Security
All possible means to protect the subjects were taken into consideration and
implemented for this research project.
4.4.1 Intrinium Security
A local security company, Intrinium Security, partnered with this project to act as
the secure middle-man. Intrinium is in the business of performing social engineering
and penetration testing within financial institutions. Intrinium Security provided time
and resources to sanitize the datasets, host the phishing websites and databases, and to
release the phishing email to the subjects. This allowed the Bank to send the subject
information for each experiment directly to Intrinium Security to remove any identifying
information such as names or employee identification numbers. Intrinium Security
performed the random sampling to select the subjects for each experiment to insure that
the Bank did not know which employees were being tested and which were not.
All phishing websites, including their databases, were hosted on their secure
servers to guarantee maximum protection of subject data. Emails were sent from their
networks and any bounce-backs or replies were received on their secure network as well.
Any data collected from the subjects during the experiments would be safely and securely
27
stored with Intrinium Security. Only sanitized data was given to the researchers and the
Bank.
4.4.2 Institutional Review Board
This research project was reviewed and approved by the IRB of Eastern
Washington University. This was an important step in being allowed to proceed with the
research after partnering with the Bank and Intrinium Security. The IRB put some extra
constraints on what the research team was allowed to do and how the research team was
allowed to do it in order to best protect the subjects.
None of the subjects were directly informed that they were being volunteered for
this project by their employer. However, it is known that they are regularly trained and
tested for their security awareness. And to reiterate, all possible precautions were taken
to protect the subjects themselves and their confidential information.
28
5. RESEARCH
5.1 Overview
The results from the actual phishing experiments were mixed. As previously
mentioned, the original experimental design was modified because of the nature of the
bank’s training program. Thus, we could not directly answer the originally pose d
research questions. As the experiments progressed, results from the experiments led to
further modification of the ensuing phishing tests. Working with live subjects in an
uncontrolled experimental setting is inherently more risky than conducting experiments
in a more controlled environment. However, the nature of the research, social
engineering, does not lend itself to controlled experiments.
Along with the inherent risks from an uncontrolled experimental setting, several
errors in data management led to problems in analyzing the results. These problems are
noted in this section. Section 6, Future Work presents an experimental design that avoids
many of the problems experienced during this project.
The original plan to test the hypothesis was to develop specialized training on
social engineering and to then conduct three phishing experiments. All subjects were to
be given the social engineering training first. Afterward, each phishing experiment would
be carried out individually with at least one month in between to space them apart.
The first phishing experiment would contain the full subject sample set from the
population of Bank employees. The second experiment would only have the subjects from
the first experiment who at least clicked on the URL from the first phishing email. The
third experiment would only have the subjects from the second experiment who at least
clicked on the URL from the second phishing email. This would have eliminated the
subjects who did not fall for the phishing experiments either due to prior self-knowledge
or the specialized training provide and focused the research on the subjects who needed
more than just training. The subjects who were kept throughout each experiment would
have been tracked to determine if they were in the group of subjects shown the phishing
warning at the end of the experiment. It could then have been determined whether after
seeing this message they fell for one of the following phishing experiments or not.
The population set used for this research was the employees of the Bank. The
population for each experiment was based on the current employee set at the time of each
29
experiment. The exact population size of each experiment was accidentally lost. However,
it is known that the population size was always under 3000 people.
Due to the limited population and small sample size used for this project, the
subjects were not carried through between experiments for privacy reasons. It would
have been too easy to personally identify individuals when the subjects needed to remain
anonymous to protect their privacy and eliminate any chance of possible negative
repercussions based on the research results. This ultimately impacted the overall plan
for each experiment causing reevaluation midstream of the subject selection. In the end,
each experiment had a fresh sample set chosen to accommodate for the privacy concerns.
This decision, while necessary, changed the outcome of the research by making the
results not statistically significant. Each experiment will be explained in this chapter and
evaluated individually.
5.2 Social Engineering and Information Security Training
Every year the Bank requires all employees to complete a variety of trainings.
Besides standard compliance trainings, employees are also subjected to rotating security
trainings. Specific training on social engineering was not part of their training set. This
training was developed and administered to the subjects before the phishing experiments
began.
Included in the training was (see Appendix A: Training Materials):
A broad overview of what social engineering is
Why social engineering is a risk
How it impacts the organization as well as the employees personally
Examples of a social engineering attack
What to do if a social engineering attack is suspected
An overview of information security
Different forms of security (e.g. physical security and security technology)
Security guidelines to personally follow at work and at home
Security policies specific to the Bank
The training was developed originally in Microsoft PowerPoint then handed off to
The Bank to convert to their training course system. The system allowed for interactive
slides with a question set at the end of the course which the subjects had to pass in order
to complete the training.
30
All Bank employees were subjected to this training during their fall annual
required training set. They are given one full month from the time the training becomes
available to complete it. All employees had completed this specialized social engineering
and phishing training by the time the first phishing experiment began.
5.3 First Phishing Trial
5.3.1 The Lure
The lure used for the first phishing experiment was the guise of a national bank
association which was conducting its own research on banks of similar size to the Bank.
The fake association sent out an email to the subjects asking them to click a hyperlink
provided in the email linking to their website and take their research survey. The lure
claimed that the association was researching how to improve the culture of small-medium
sized banks by conducting a survey of employees at banks of that size. In addition to
trying to tap into the desire to improve the subject’s own work environment, prizes were
offered to five lucky participants of this survey. The prize offered was a Samsung Galaxy
tablet.
The URL hyperlink provided in the phishing email linked to the phishing website
but also included a generic integer employee tracker.
In order to get entered into the drawing and take the survey, the subjects would
need to create an account; thus allowing the association to contact them in case they win
a tablet.
5.3.2 The Hook
National Bank Association was picked as the name of the fake association for the
lure. The domain name of nationalbankassociation.org was chosen as the fake
association’s website and is where the phishing emails were sent from.
The phishing website was designed and developed to look similar to a real bank
association’s website. The design was intended to ensure that the site fit the look of a
real financial institution’s website rather than to pretend to be the real association upon
which it was based. Names with associated emails were chosen and listed on the contact
page to increase the feel of legitimacy of the website. Two levels of the website were
created to further the illusion of it being a real website for a real association.
A survey was created and presented to subjects to potentially gather some helpful
information for the bank association. This was also used to further the illusion of a
31
legitimate website for a bank association. The survey questions were based from a Google
search for bank culture survey question examples.
The phishing website was developed in ASP.NET with C# (see Appendix B: ) and a
MS SQL backend database. The database was used to store the account information, the
survey answers, and the employee generic tracking number.
When a subject created an account, the site would show the subject one of two
pages. The first page was the banking survey originally promised as part of the guise.
The second page showed a warning that the subject just fell for a phishing attack and
included a list of internal and external resources regarding phishing. The website would
toggle who got to see which page at the end.
5.3.3 Subject Selection
The sample size for this experiment was 600 randomly selected employees.
Subject selection was divided up among two groups of employees at the Bank: one third
of the subjects were randomly selected out of a newly added division at the Bank and the
other two thirds were randomly selected from the main employee pool. The new Bank
division had not received any of the Bank’s yearly required training until this research
began.
The random selection process was performed by Intrinium Security. Intrinium
Security was provided with the current full population of employees from the Bank.
Subjects were picked using a variant of the simple random sampling method by Intrinium
Security. Each subject was selected out of the population dataset spreadsheet by hand
at random. The specific demographics for this sample set were accidentally lost.
5.3.4 Results
Out of the original 600 subjects used in this experiment, six percent of them
responded to the phishing email with an automatic reply for being out of the office and
thus were not available to participate. The results have been adjusted to accommodate
for this; based on the actual sample set of 564 subjects.
This first experiment was able to successfully phish approximately two percent of
the subjects getting them to provide sensitive information to the research team. A total
of approximately ten percent of the subjects clicked the URL hyperlink provided in the
phishing email that navigated them to the research team’s phishing website.
32
Since ten percent of the subjects followed the provided link to the phishing website
but only two percent completed the phishing exercise by providing sensitive information,
seven percent of the subjects who followed the link did nothing after landing on the site.
It could be that the subjects got to the website and saw that they needed to create the
account but did not have time to do so because the phishing email went out during the
workday so they abandoned the exercise. It could also be that the subjects did not find
the website legitimate enough to convince them to provide any further information.
Actual
Adjusted
Total Subjects:
600
100%
564
100%
Out of Office Replies:
36
6%
0
0%
Clicked Phishing Link:
54
9%
54
10%
Provided Sensitive
Information:
14
2%
14
2%
TABLE 3: RESULTS SUMMARY OF FIRST PHISHING EXPERIMENT
The majority of subjects (ninety percent) did not click on the URL hyperlink
provided in the phishing email at all. This could have been due to them recognizing the
email for being a phishing email. However, it is highly likely that these subjects did not
look at the email or if they did, they saw that they needed to perform an action (click on
the URL in the email) and decided they did not have time to put toward this endeavor.
Each subject who received this phishing email was in the middle of their workday, busy
with tasks, not necessarily having time to pay attention to emails which are not directly
related to their job. It is also true that any given day a person receives a great number of
phishing emails. Most people have learned that clicking on a link in an email is not
recommended and that if an email asks you to do so it is probably phishing related. Thus,
it could be that the subjects who did not follow the phishing hyperlink viewed the email,
determined that it was likely a phishing email and deleted it immediately. Unfortunately,
since no follow up survey was given to subjects to learn their thought process regarding
the email, one can only speculate at the reason the email was unsuccessful.
33
5.3. 4.1 Phishing Responden t Demographics
The demographics of the subjects who did ultimately get phished are presented
in Table 4. Two of the subjects who were recorded as falling for the phish somehow did
not have any tracking data logged on them so their demographics are unknown. The
results below are adjusted to account for this missing data.
Gender-wise the respondents were predominantly female. Only twenty-five
percent of the phished subjects were male. Again, it is unknown if the whole sample set
was predominantly female or not. Regardless of the overall sample set gender
distribution, it is still interesting that three times as many women as men bit the phishing
hook and provided sensitive information. This raises some questions around the
vulnerability of women compared to men; are women more susceptible to phishing
because they are often more trusting? The gender breakdown of phishing and social
engineering vulnerability as a whole requires further study.
Age Group
Female
Male
Total
Adjusted Total
< 20
1
0
1
1
8%
20-29
3
1
4
4
33%
30-39
3
1
4
4
33%
40-49
1
1
2
2
17%
50 <=
1
0
1
1
8%
Unknown
-
-
2
0
0%
Total
9
3
14
12
100%
TABLE 4: FIRST PHISHING EXPERIMENT AGE AND GENDER RESULTS
The age range of subjects who fell for the phish was between 18 and 60 years old.
The majority of the respondents of the phishing experiment were in their twenties or
thirties. Both the 20-29 and 30-39 age groups took up thirty-three percent of the
responding subjects for a total of 66% of the total respondents. The second largest age
group for respondents was 40 and over at 25%. Only 8% of the respondents were in the
34
below 20 age group. It had been expected that the majority of those falling prey to the
phishing hook would have been in the 40 and over age group.
Looking at the demographics related to the subjects’ employment at the Bank it
can be seen that the majority of subjects who fell for the phish had only been employed
for less than one year. A full 83% of the subjects who fell for the phish fall into this
category. Only 8% of the responding subjects had been employed between two and five
years; 17% of the responding subjects had been employed between 10 and 20 years.
Length of Employment
Total
Adjusted Total
< 1 year
10
10
83%
1 – 2 years
0
0
0%
2 – 5 years
1
1
8%
5 – 10 years
0
0
0%
10 – 20 years
2
2
17%
20 years <
0
0
0%
Unknown
2
0
0%
Total
14
12
100%
TABLE 5: LENGTH OF EMPLOYMENT RESULTS FROM FIRST PHISHING
EXPERIMENT
This is interesting because even though the Bank requires mandatory annual
security training it is only given once a year. It is highly likely that these subjects had
not yet participated in a security training session, making them more vulnerable to
phishing and social engineering attacks. This seems to make a good case for including
security training in the new-hire training. These results could also imply that repeated
yearly training lowers a person’s vulnerability to this type of attack.
The spectrum of subjects who fell for the phish spanned six different departments.
The majority of the respondents were from the Retail Production department at 42%. It
is unknown if this department was the majority in this experiment due to having the most
35
subjects in the sample set or if for some other reason. The second major department was
Corporate Technology Service and Support at 25%. The remaining departments had an
equal number of subjects responding – Credit, Mortgage Investment, Portfolio, and
Commercial Production – each at 8%.
Further, there were eight different job positions among the respondents. Both the
Compliance Specialist and Customer Service Representative positions had the most
respondents at twenty-five percent each. Other positions included Commercial Bank
Team Leader, Credit Analyst, Loan Analyst, Personal Banker, Private Banking
Representative, and REO Property Manager, each making up eight percent of the
respondents.
5.3. 4.2 Inf orm ation Phi she d
The hook for this experiment asked the subjects to create an account and provide
specific information in order for them to be eligible for the prize drawing. The account
setup required that subjects provide a username, password, security question and
answer, first and last name, birthdate, and email address. Optional information gathered
from subjects via this process included: address, city, state, ZIP code, and phone number.
All of the subjects who fell for the phish had to provide the basic required
information. For some reason part of the data is missing for who provided a username,
password, security question and answer. With this first experiment, the usernames and
passwords were not cross-referenced with the Bank’s network login.
36
The security question field of the account creation form on the phishing site gave
the subjects the option of picking their mother’s maiden name, city of birth or their first
pet’s name. The majority of subjects opted to use their mother’s maiden name. Only one
subject chose to use their city of birth and none of the subjects picked their first pet’s
name. Mother’s maiden name is a standard security question for logins or other identity
verification processes even though this is public information which any social engineer
could easily access without needing to phish for it.
A small percentage of the respondents provided some of the optional account
information asked for. Twenty-nine percent of the subjects provided their address, city,
state, and ZIP code. Twenty-one percent of the subjects also provided their phone
number. Handing this information over to the phishing website could be due to habit of
just blindly filling out paperwork without asking questions. It could also be from the
subjects wanting to be sure they could be reached in the case they won the prize drawing.
In addition to asking the subjects to create an account to be eligible for the prize
drawing, they were also shown the promised bank culture survey which was part of the
lure. Thirty-six percent of the subjects who created an account on the phishing site also
FIGURE 3: INFORMATION PHISHED FROM FIRST PHISHING EXPERIMENT
37
completed the survey. Interestingly enough, 80% of those who completed the survey were
women. One could speculate that this is due to the natural tendency of women to care
about and want to help others; the lure had told subjects that the survey was to learn
how to improve the culture and ultimately the work environment of fellow bank
employees.
5.4 Second Phishing Trial
5.4.1 The Lure
In the second phishing experiment, the lure attempted to use fear to convince the
subjects to hand over their personal information. The guise was a fake online reputation
monitoring service provider called Grapevine Watchdog who alleged to have just
partnered with the Bank. This organization emailed the subjects to inform them of a
special offer for Bank employees to monitor their information and protect their reputation
online.
Around the time of this experiment, the Bank opened up social networking
websites on their internal network. Bank employees had just been granted the ability to
look at Facebook, Twitter, MySpace, and LinkedIn while at work. Like a social engineer
having done surface research, the project used this information to cater to the concern
of employees of what they had made publicly available on their online profiles which
coworkers and employers could now potentially see. The guise offered to help employees
have a “work friendly” online reputation. It promised to ensure that there was no
information online which could get them in trouble with their employer.
The phishing email sent to subjects used this lure to convince them to click on
the URL hyperlink provided. The hyperlink would navigate the subject to the phishing
website created to support the lure.
5.4.2 The Hook
The domain name of grapevinewatchdog.com was used to host the hook of this
phishing experiment. This domain is also where the phishing emails were sent from. The
domain was hosted by Intrinium Security on their secure Web servers.
The website was developed using the same code base as the first phishing
experiment. A site hit tracker was implemented with this experiment to log every time a
subject visited the phishing website. Again the site used a MS SQL database backend
and the code was written in ASP.NET with C#.
38
The design of the website was created to look professional and trustworthy. A
special page was created for the offer sent out to the Bank subjects that included the
Bank’s logo. The footer of the website had a copyright and claimed that Grapevine
Watchdog was a Better Business Bureau™ accredited business. The hope was these
would further the sense of legitimacy of the lure.
Once the subjects reached the website through the provided URL hyperlink in the
phishing email, they were shown the special Bank page explaining the lure’s offer. This
page also included a promo code for them to use and a button for them to sign up for this
special deal. The text content on this page said the following (with the Bank’s name
removed for privacy):
“Grapevine Watchdog and the Bank have partnered together to offer Bank employees
a special monitoring package. Now that the Bank has opened up Facebook to
employees in the office, it is good to be extra careful and aware of what is posted to
Facebook and other social networking sites about you and what you post about the
Bank.
As a special offer to you, a Bank employee, Grapevine Watchdog will actively monitor
social networking sites as well as what is reported on top search engine sites such as
Google free for 1 year.”
After the subject clicked the “Sign Up Now!” button, the enrollment page was
displayed with a sign-up form. This form was similar to the account creation form from
the first phishing experiment. The pretense on this page was that the subject was
supposed to fill out the form including all contact information so that a Grapevine
Watchdog representative would be able to reach them to get the process started. The
form asked the subject to provide the following required information:
Username/password
Security question/answer
First/last name
Birthdate
Email
It also tried to gather the following optional information:
Company name
Job title
Address
City
State
39
ZIP code
Phone number
What social networking sites the subject used (Facebook, Twitter, MySpace,
LinkedIn, something else)
A field for the subject to enter their special Bank promo code was provided at the bottom
of the form as well.
After submitting the form, the site would again show the subject one of two pages.
The first page was a standard “Thank you, a representative will contact you soon” page.
The second page showed the same warning from the first experiment telling the subject
they just fell for a phishing attack and included a list of internal and external resources
regarding phishing. The website would toggle who got to see which page at the end.
5.4.3 Subject Selection
Subject selection for this experiment was the same as the first experiment. As
stated at the beginning of this chapter, originally this second experiment would have only
contained the subjects who were successfully phished in the first experiment. Since the
population size from the first experiment was fairly small, the following experiments could
not contain only the subjects successfully phished from the previous experiment due to
privacy concerns.
As a result, 600 new randomly selected subjects were chosen for this experiment.
The same ratio of subjects was used with this selection. The subject sampling was again
divided amongst the main employee population and the employees at the newly acquired
division: one-third randomly selected from the new division and two-thirds of the subjects
randomly selected from the main pool.
5.4.4 Results
From the 600 subjects in this experiment’s sample set, three percent of them were
successfully phished. This was a slightly higher rate of success than the first experiment.
It could be due to the fear motivating factor used in this experiment compared to the
passive approach at offering the chance for a prize in the first experiment. In the second
experiment, both the name and logo of the Bank were used to help further the legitimacy
of the phishing lure which could have helped trick a few extra subjects.
40
Total
Number of Subjects
600
100%
Personal Emails Regarding Phish
11
2%
Unique Phishing Site Visits
157
26%
Bank Credentials Provided
0
0%
Provided Sensitive Information
16
3%
TABLE 6: RESULTS SUMMARY OF SECOND PHISHING EXPERIMENT
It is unknown how many replied with “Out of Office” replies in this experiment.
However, there were help tickets opened by ten percent of the subjects referencing
phishing in the subject line. The research team’s Bank member in charge of security
personally received emails from 11 different employees (approximately two percent)
regarding phishing.
The site tracker did log that each subject phished visited the website multiple
times. The phishing website was visited by 157 different subjects a total of 261 unique
times with an average of two visits per subject. The average number of visits per phished
subject was four. The least number of visits was two times and the greatest number of
visits was eight by one of the subjects. This could imply that the subjects either went to
the site initially saw they had to sign-up but had to come back to complete the process
due to lack of time. Perhaps they went back to re-read the site’s content or show it to a
coworker or employer. Since no follow-up survey was given to subjects immediately
following the experiment it is only speculation as to why each subject visited the phishing
site so many times.
After this experiment completed Intrinium Security performed a test to check the
usernames and password combinations provided during the phish against the Bank’s
network. None of the subjects who were phished provided their Bank domain logon
credentials.
The Bank’s anti-phishing vendor did notify them about the phishing experiment,
which they thought was a real phishing attack. The vendor checked to see if they should
initiate site take-down actions to shut down the phishing “attack.”
41
5.4. 4.1 Phishing Respondent De mograp hics
The demographics for the entire sample set used in this experiment were lost.
However, the demographics on the subjects who were successfully phished was preserved
and described here. There was one subject phished whose demographic information was
lost and so the data in this section is adjusted to reflect that.
Once again, there were a higher percentage of females who were successfully
phished in this experiment. Sixty-three percent of those successfully phished were female
compared to the 38% male subjects successfully phished.
Age Group
Female
Male
Total
Adjusted Total
< 20
0
0
0
0
0%
20-29
2
2
4
4
25%
30-39
3
1
4
4
25%
40-49
2
1
3
3
19%
50 <=
3
2
5
5
31%
Unknown
-
-
1
0
0%
Total
10
6
17
16
100%
TABLE 7: SECOND PHISHING EXPERIMENT AGE AND GENDER RESULTS
The age range of subjects who fell for the phish was from 24 to 63 years old. The
subgroups of age ranges were pretty evenly distributed. The 20-29, 30-39, and 50-59
age ranges all had 25 of the successfully phished subjects. The 40-49 age range had the
next largest group of phished subjects at nineteen percent. Then 6% of the subjects who
were phished were above 60 years old.
Similar to the first experiment, the majority of the subjects who fell for the
phishing lure had not been employed with the Bank for very long. Thirty-one percent of
the subjects had been working at the bank for less than one year. Half of the subjects
who were phished this time had been employed for between one and two years. After that
the subjects are evenly distributed at 6% across the following employment ranges of
between 2 and 5 years, 5 and 10 years, and between 10 and 20 years. There does not
appear to be any direct correlation between length of employment and age so as to
42
indicate why so many subjects employed for less than two years would fall for this lure
over the last one. It is likely that new employees are vulnerable due to having had the
least amount of security training.
Length of Employment
Total
Adjusted Total
< 1 year
5
5
31%
1 – 2 years
8
8
50%
2 – 5 years
1
1
6%
5 – 10 years
1
1
6%
10 – 20 years
1
1
6%
20 years <
0
0
0%
Unknown
1
0
0%
Total
17
16
100%
TABLE 8: LENGTH OF EMPLOYMENT FROM SECOND PHISHING EXPERIMENT
All subjects phished were full-time employees. No part-time employees were
hooked in this experiment.
This experiment hooked subjects from eight different departments. The
department who had the most subjects fall for the phish was the Home Loan Division.
The second highest rating department was Retail Production at 25, which had the
majority in the first experiment. The rest of the subjects were evenly distributed amongst
the following departments each at 6%: Audit, Credit, Deposit Management, Human
Resources, Mortgage Investment, and Corporate Technical Service and Support.
The respondent group consisted of nine different job titles. Given that the main
department phished was the Home Loan Division, it is not surprising that the main job
title held of those hooked was Loan Officer at 31%. The next largest groups were
Customer Service Representative and Manager both at 13%. The remaining departments
each held 6% of the subjects successfully phished: Administrative Assistant, Analyst,
43
Business Systems Analyst, Personal Banker, Private Banking Team Leader, and Special
Assets Administrator.
5.4. 4.2 Inf orm ation Phished
All subjects phished provided their first/last name, birthdate and email address.
For some reason, not all usernames/passwords and security question/answers were
recorded so the official results show only 75 of the phished subjects providing this
information. Address information including street address, city, state, and ZIP code were
provided by 63% of the phished subjects. More than half of the subjects (56%) provided
their phone number as well.
The form asked subjects to checkmark the social networks they currently use. It
was found that 19% of the subjects have a Facebook profile, 44% use Twitter, 13% still
have a MySpace account, 6% use LinkedIn, and 38% of the subjects have some other
social networking account.
44
5.5 Third Phishing Trial
5.5.1 The Lure
The lure for the third and final phishing experiment was under the guise of a
security update. This time the phish did not come from a third party unassociated with
the Bank (first phishing experiment) or a third party who has partnered with the Bank
(second phishing experiment) but instead it came from the Bank itself. The guise here
was to have an email sent out from the IT Security Department requiring that Bank
employees download a critical security patch for Microsoft Outlook.
The email was straight forward and simple. It said:
“URGENT: The Microsoft Outlook Security April 2012 Patch is required to be performed
no later than April 27, 2012 by 5pm or all of your email could be lost due to possible
virus attacks from this security vulnerability.”
A URL hyperlink was provided in the email for the subjects linking them to the
page where the “security update” could be downloaded.
FIGURE 4: INFORMATION PHISHED FROM SECOND PHISHING EXPERIMENT
45
To increase the believability of the email, it was signed by a fictitious person
claiming to be in the Bank’s IT department. Had this been a real phishing attack, the
social engineer would have used the name of a real employee working in that department.
It was decided to use a fake name to prevent a bombardment of emails and phone calls
to any real employee in regard to why they signed an email such as this.
5.5.2 The Hook
The phishing website for this experiment was made to look like it came from the
Bank itself. The method used in this case was what is called “typejacking” where one
character of the Bank’s real domain name was switched out with a number or vice versa
[26] [38]. This makes it easier to trick people into believing it is the real URL. A domain
name was purchased through GoDaddy.com that matched the Bank’s main domain name
“typejacked.”
A single web page was created to explain the security update that subjects were
being asked to download. It also had a link to the download the “security update”. In
essence, just one subject needed to click the link to download the “security update” to
compromise the whole network. Unfortunately, the only thing tracked in this experiment
was the number of visits per subject to the website not how many times the link was
clicked to download the file. No real file was downloaded to the subject’s computer.
As in the previous experiments, the “you were phished” warning message was
displayed to half of the subjects who went to the phishing site and clicked the link to
download the security patch. The other half of the subjects were shown a “Thank you”
message. The website would toggle the message back and forth with each click.
5.5.3 Subject Selection
As in the previous experiments, due to the small population and smaller sample
sizes for each experiment, 600 new randomly chosen subjects were picked for this last
experiment. Again, one-third of the subjects were randomly selected out of newly added
subset of employees at the Bank and the other two thirds were randomly selected from
the main employee pool.
5.5.4 Results
This experiment resulted in far more excitement than its predecessors. Shortly
after the phishing emails began to be sent out to the 600 subjects, the Bank’s own IT
Security team (who knew nothing of the research project) learned of the phishing attack
46
in progress and leapt into action. GoDaddy.com was notified and called the research
team to explain that the domain name had been used in a phishing attack and was being
shut down. Thus, this experiment lasted the least amount of time and was taken down
by the Bank’s official anti-phishing procedures within a matter of hours.
The raw results were somewhat skewed due to the efforts of the IT Security team
at the Bank. Three visits were from members of the IT Security team testing the phishing
site during their attempts to trace it to its origins and shut it down. One other result is
questionable as to whether it was legitimately a visit or another Bank member trying to
test the site. This data has been removed from the final data set.
A total of three percent of the 600 subjects included in this experiment fell for the
phish. This was the same percentage as the second experiment. Only one of the subjects
visited the phishing website more than once. It is unknown how many “Out of Office”
replies were received back. Fifteen percent of the subjects who received the email did
click the link provided and visit the phishing website.
Total
Number of Subjects
600
100%
Unique Phishing Site Visits
88
15%
Provided Sensitive Information
20
3%
TABLE 9: RESULTS SUMMARY OF THIRD PHISHING
5.5. 4.1 Phishing Responden t Demographics
Again, the overall demographics of the sample set were lost. Only the
demographics of the subjects successfully phished were preserved and described here.
The great majority of the subjects who fell for the phish, same as before, were
women. Eighty-two percent of the subjects successfully phished were women compared
to the 18% of men.
47
Age Group
Female
Male
Total
Adjusted Total
< 20
0
0
0
0
0%
20-29
4
0
4
4
24%
30-39
4
1
5
5
29%
40-49
3
1
4
4
24%
50 <=
3
1
4
4
24%
IT Security
0
3
3
0
0%
Total
14
6
20
17
100%
TABLE 10: THIRD PHISHING EXPERIMENT AGE AND GENDER RESULTS
The age of the phished subjects ranges from 21 to 66 years old. The main age
group was 30-39 years old at 29%. The second largest age range was tied between the
20-29 years old and 40-49 years old at 24%. The 50-59 age range had 18% of the
successfully phished subjects. The greater than 60 age range group only had 6% of the
phished subjects.
Just as in the previous two experiments, the majority of the subjects phished were
employed with the Bank for under two years. Thirty-five percent of the subjects were
employed at the Bank for under one year and another 41% between one and two years
totaling 76% of the subjects working for the Bank less than two years. There were 12%
of the subjects phished in the two to five years of employment group. Another 24% were
in the 10-20 years of employment group. Six percent of the subjects successfully phished
had been employed for greater than 20 years.
The majority of the subjects successfully phished were full-time employees of the
Bank. Eighty-eight percent were full-time and 12% were part-time employees.
48
Length of Employment
Total
Adjusted Total
< 1 year
6
6
1%
1 – 2 years
7
7
1%
2 – 5 years
2
2
0.3%
5 – 10 years
0
0
0%
10 – 20 years
1
1
0.2%
20 years <
1
1
0.2%
IT Security
3
0
0%
Total
20
17
100%
TABLE 11: LENGTH OF EMPLOYMENT FROM THIRD PHISHING EXPERIMENT
Seven departments had subjects who were successfully phished. As in the second
experiment, the department with the most successfully phished subjects was once again
the Home Loan Division at 41% and the department with the second highest percentage
of subjects successfully phished was the Retail Production Department at 29%. The third
most successfully phished department was the Corporate Technology Service and
Support Department at 24%. The remaining departments each had 6% of the
successfully phished subjects in them: Executive Administration, Financial Services,
Portfolio, and Product and Department Strategy.
There were 15 different job titles that the subjects phished held. The top three
job titles with 12% of the phished subject were: Assistant Branch Manager, Customer
Service Specialist, and Mortgage Loan Officer. The other job titles had an equal
distribution of successfully phished subjects at 6% each: Credit Administrative Executive,
Credit Card Specialist, Customer Care Representative, Customer Service Representative,
Deposit Support Manager, Fixed Income Investment Manager, Home Loan Division
Branch Manager, Legal Assistant, Loan Processor, Q/A Representative, Shipping
Specialist, Underwriting Coordinator.
49
5.5. 4.2 Inf orm ation Phi she d
No real information was phished or requested of the subjects with this experiment.
The purpose was solely to get the subjects to visit the phishing website and click the link
to download the patch. In doing so they compromised the integrity of the Bank’s network
and a social engineer performing this attack would now have full access to their system.
50
6. CONCLUSION
6.1 Results Summary
The experiments conducted in this research study cannot be compared directly
due to the fact that each was significantly different from the others. Each experiment
contained a different set of randomly selected subjects with no guarantee that subjects
were the same in each experiment. The information showing which participants carried
through between experiments was accidentally lost.
Each experiment used a different phishing lure to hook the subjects. There is no
way of knowing if that was the first experiment they participated in, or if a subject who
fell for one of the lures was more susceptible to that specific lure, or if they were more
susceptible to phishing in general. Additionally, the distribution of subjects across each
experiment and throughout the project as a whole was lost. This made it impossible to
know if there was a proper randomization of gender, age, department, job position and
length of employment across all sample sets.
While the experiments may not be directly comparable in order to draw any
statistically significant results, they can be summarized here. In each experiment, the
number of subjects successfully phished ranged from two to three percent. This number
is within the expected range of successful responses to phishing: “Data suggests that
some phishing attacks have convinced up to 5% of their recipients to provide sensitive
information to spoofed websites” [26].
Each experiment had more women than men successfully phished. This could be
due to a sample set that had a higher percentage of women than men. This is a likely
scenario given that there are more women working in the financial sector than men [39].
Regardless of the gender distribution in each sample sets used here, these results do fall
in alignment with what one research group from Carnegie Mellon University and
Indraprastha Institute of Information Technology found with women being more likely
than men to fall for phishing [40]. Since this CMU study was carried out on college
students, however, the external validity is questionable because “the student population
is likely to be atypical” [35].
The experiments each had a different age group that was more successfully
phished than the other age groups. The 30-39 age range was the most successfully
phished group overall. This group makes up the largest age range of overall employees
51
working within the financial sector [39]. This makes it impossible to know if this group
is actually more vulnerable or if it just happened to be more heavily represented amongst
the subjects being phished. Interestingly, though, the CMU/IIIT study found that 18-25
year olds were the most vulnerable to phishing attacks [40]. Again, the CMU/IIIT study
was carried out with college students which may or may not be extended to the general
public or to financial employees specifically. This may mean that age plays little or no
role at all, as apparently whatever age is in the majority always is more successfully
phished. Further well-designed experiments on age and phishing would be very
illuminating.
Out of the subjects successfully phished, the majority were employed with the
Bank for less than one year. This finding seems rather obvious and points to the
importance of implementing stronger new-hire security training. It also points to the
important of annual retraining for employees.
The Home Loan Division and Retail Production departments were the departments
with the most successfully phished subjects. The Customer Service Representative
position had the largest number of successfully phished subjects in two of the three
experiments. Given that the demographics of the sample set as a whole is unknown,
these findings do not imply that these departments or this position are more vulnerable
than any other.
6.2 Future Work
This research was not executed in an ideal manner due in part to uncontrollable
circumstances and in part to the research team not fully being aware of all the
considerations that were necessary to properly carry out an experiment. Such research
requires basic knowledge of statistical testing methods, experimental design and how to
manage the required restrictions put in place on this type of research. Many lessons were
learned while conducting this research project. Based on these lessons, suggestions for
how to better setup and design a future study on this subject are described in this section.
6.2.1 Research Setup and Preparation
All parties involved in the research project must fully understand and agree to the
required restrictions and precautionary measures before beginning any of the
experiments. While this may seem obvious, it is important to reiterate. Clear
communication of the project plan and rules must be maintained throughout the entirety
of the project.
52
A codebook should be used to record all project related information throughout
the course of the project. This would include who the researchers are, what role they
play, and what they are responsible for; project details, such as original purpose and
timeline; major decisions made before, during, and following the project; data collection
and data entry procedures; and data analysis procedures such as how variables are coded
and how missing data is handled [41].
6.2.2 Overview of Proposed Research Design
The recommended research design is a much simplified version of the original
experimental design of this thesis project. The original design, done correctly, proved to
be too complicated for a master’s level research project and even at a higher research
level it proved likely to be impossible due to the necessary initial population size to achieve
significant results while maintaining subject anonymity. The details of this problem and
the proposed new design are explained here.
6.2. 2.1 Original Resear ch Des ign Fla ws
The original design of this study involved the performance of multiple rounds of
phishing trials on the same control and treatment groups. This was to test the hypothesis
that follow-up training reminders following a subject failing a test, combined with the
initial training would significantly lower the vulnerability of subjects. The design
eventually consisted of a mixed method of repeated measures which would have required
the use of an ordinary logit mixed model on the data results during the analysis phase of
the project [42].
The population size of the original research was limited to the number of
employees working at the Bank who partnered with the research team. The sample size
was further constrained to 600 initial employees who were then to be shrunk with each
trial to only include those who were successfully phished in the previous trial. This
became problematic due to the fact only approximately 3% of the subjects were
successfully phished on average with each trial.
A proper population size proved to be essentially unobtainable. The original
design would shrink the control group and treatment group with each trial to include
only those who were successfully phished in the previous trial. This is problematic due
to the fact it has been shown that only about 5% of people who receive a phishing email
respond to it and are successfully phished as a result [26]. Assuming it was desired to
have at least 50 people successfully phished in each group at the end of the third trial in
53
the mixed model repeated measures study, 1000 people would be needed to have been
successfully phished in each group at the end of the second trial and 20,000 people at
the end of the first trial. As a result, the required population to start a study like this
would be 800,000 assuming a 5% response rate. Thus, a simplified study has been
developed and proposed in section 6.2.2.2.
6.2. 2.2 Ne w Rese arc h Design
The new research design is a two-sample z-test comparing the proportions of
successfully phished subjects between the control and treatment groups. Only one trial
is performed in this design so the initial size of the population is not as much of a concern.
FIGURE 5: ORIGINAL RESEARCH DESIGN
54
Population details are discussed in section 6.2.4.1. The focus of the new study would be
on two specific variables: age and length of employment (confounded with gender). This
will help to focus the study in on a specific area which a bank who would partner for this
might be most interested in.
6.2.3 Hypothesis
The hypothesis was not properly validated from the research described in this
thesis paper. Future research needs to be done around the same hypothesis in order to
do so. The hypothesis is that targeted specialized security training will reduce the
vulnerability of financial employees to social engineering attacks. Using
to represent
the proportion of the group who received the specialized training that was successfully
phished and
to represent the proportion of the group who does not receive any
specialized training that was successfully phished, the null and alternative hypotheses
are stated as:
6.2.4 Investigated Parameter Space
6.2. 4.1 Populatio n
A financial institution’s employee base needs to continue to be the population for
a future study. With the new research design only requiring one trial, the population size
can be the same as what was used in the study described in this thesis paper – the
number of employees that a typical small to midsized financial institution has employed.
FIGURE 6: 2-SAMPLE Z-TEST PROPOSED RESEARCH DESIGN
55
Assuming a 3% response rate, to guarantee at least 50 successfully phished subjects in
each group a sample size of at least 3333 subjects. As a result population size would
need to be at least this large. Although, a larger sized financial institution would also
work and give a greater subject pool.
The primary researcher needs to be in charge of handling the population
information. To meet the needs of the IRB and to protect the privacy of the subjects this
information will need to be sanitized before the primary researcher begins managing it.
All demographic data related to the population should be included in this dataset the
primary researcher holds. However, any and all information that could lead to subject
identification needs to be removed: name, employee identification number, address,
phone number, email address. The name and employee identification number should be
replaced with a subject tracking number to allow the researchers to track the subject
between trials with the secure third party being the only entity with access to the true
identity of the subject. Age, gender and length of employment should all be tracked in
the data file for all subjects.
6.2. 4.2 Variables
The variables for this proposed future research are the same variables used in the
research described in this thesis. The proposed future research will have the following
independent variables: security training program, and phishing attack. The following are
dependent variables: security awareness level, and vulnerability level. Other extraneous
variables are: sex, age, length of employment. The following variables are unmeasurable
and may influence individual behavior: spam filter, workload, motivation, and out of
office.
6.2. 4.3 Sub ject Selecti on
If the financial institution partnered with allows it, ideally the sample set should
be the entire population. This would allow for the maximum coverage of the financial
institution’s demographic coverage. However, it is understood that any institution
participating in a study such as this would opt out of including top executive positions
from the population base. In the case that the entire employee set cannot be utilized for
both the population and sample set, the employees chosen to participate in the
population should be chosen using a stratified sampling method to ensure proper
balanced coverage of the variables being tracked. Stratified sampling is “a probability
sampling procedure in which simple random subsamples that are more or less equal on
56
some characteristic are drawn from within each stratum of the population” [35]. Equal
representation (as best as possible) of age, gender and length of employment should be
included in the subject sample set as they are the main variables being studied. Other
covariates such as position, department, and location are not the focus of this particular
study being proposed but could prove interesting and could be considered in future
related studies.
The sample set should be managed and owned by the primary researcher. This
subject dataset should be sanitized, same as the population set, and include all necessary
demographics and tracked variables.
6.2. 4.4 Co ntrols
This type of study is best done in real-life environments rather than a simulated
laboratory environment. However, in a real-life environment it can be difficult to control
all the variables. A control group should be used to help control the independent and
dependent variables. Half of the sample set should be used for the control group. Equal
representation (as best as possible) of age and length of employment should be balanced
across the treatment group and the control group. It needs to be properly recorded which
subjects are in this group and which are not.
The control group will not receive any specialized training on social engineering.
This includes the initial pre-trial training course as well as the on-going training
reminders given via warning messages following a successful phishing attack trial.
6.2. 4.5 Sub ject P rotection
Since subjects are being selected from a financial institution’s employee pool, the
financial institution itself will ultimately know who the potential subjects are. The
employees will need to be protected against any possible negative repercussions of
participating in the study. To do this, the financial institution should not know which
employees are ultimately picked from the population to participate in the experiments.
It is highly recommended to have the primary researcher partner with a secure
third party who is very familiar with security protocols and is setup to protect sensitive
and confidential information. In the study described in this thesis, that entity was
Intrinium Security. This secure third party allows safe passage of the full sensitive (pre-
sanitized) population data to be transferred out of the financial institution. This third
57
party is responsible for scrubbing the data and then passing it along, in full, to the
primary researcher to manage.
Sanitizing the population dataset must be a well tracked process. Employees will
either have an employee number of their own or their social security number (SSN)
available as the tracker on the pre-sanitized dataset. The secure third party entity will
be responsible for creating a tracking system to track the employee listed between the
pre-sanitized population and the sanitized population. Names, any employee number,
SSN, email address, phone number, and physical address all need to be removed from
the sanitized population set. The sanitized population set should only contain the
tracking number created by the secure third party and the demographical information
being tracked; no identifying information should be present.
All studies involving human subjects require being reviewed and approved by the
IRB; this and all future related studies are no exception. The complete research project
plan to guarantee subject anonymity along with written agreements from all parties
involved will need to be presented to the IRB for approval before any experiments begin.
All parties will need to understand, and fully agree to everything laid out in the final
project plan that is approved.
6.2.5 Methodology
6.2. 5.1 Experimen tal Design
First, there should be upfront specialized security training given to subjects who
are in the treatment group. Before any training is given, however, a security awareness
survey should be given to all subjects to assess their initial security awareness and
vulnerability level. The subjects who receive the training should be given a post-training
follow-up survey to discover how much subjects’ security awareness level increased and
potential vulnerability levels decreased.
The means for distributing the training to subjects need to be handled carefully
so that the financial institution does not know who the subjects are or which group they
are in. This could be done using the secure third-party. All results need to be accurately
recorded, tied to the individual subject, sanitized and given to the primary researcher for
managing.
58
Following a reasonable delay from when training ends, the phishing experiment
phase will begin. The secure third-party entity will be responsible for sending the
phishing emails since they will be the only party who knows which employees are
participating in the research, which group they are in, and what their email is. The
phishing experiment phase can be repeated as many times as needed or allowed within
the constraints of time and resources available. Each experiment can involve a different
type of phishing attack. If the attack has a link to a phishing website, the subjects who
click the link and visit the website must be tracked. If the attack involves submitting
confidential information, the subjects who submitted the information must be tracked.
If the attack involves downloading a file, the subjects who download the file must be
tracked.
FIGURE 7: FUTURE TRAINING DESIGN
59
All of the subjects should see a warning message upon being successfully phished.
This project will focus on only splitting out if the initial specialized training makes a
significant impact on the subjects’ vulnerability level compared with only being showed
the warning message after a successful phishing attack. This message will be a
continuation of their training. Just like with the research in this thesis, it will provide a
message to let them know they were just phished along with resources on phishing
attacks and how to not be a victim. It can also provide specific phishing information
provided by the financial institution such as security protocols employees are required to
follow.
FIGURE 8: FUTURE PHISHING EXPERIMENTAL DESIGN
60
Additionally, statistics from the financial institution and the secure third-party
should be gathered where possible on the number of help desk calls and reports of
phishing. This information can be helpful in knowing who saw the phishing emails but
followed procedures and reported the incident rather than falling for it. Post-phishing
experiment, any usernames and passwords gathered need to be cross-checked with the
financial institution’s network domain logins to see if anyone provided the same login
information.
6.2. 5.2 Data A nalysis
Once the data from this research is all gathered properly and stored in a data file,
the analysis phase can begin. Given that this is a two-sample z-test study, the analysis
phase should be relatively simple.
The z-test can be used to calculate the difference between the proportions
successfully phished in the control group and those successfully phished in the treatment
group using the following formula:
Combined with:
Where the following variables are used:
61
This equation can then be applied to test the null hypothesis. Using the standard
level of significance of , the critical values would be -1.96 and +1.96, and the
decision rule would be:
otherwise do not reject .
Additionally, the confidence interval estimate of the average difference can be
calculated using the following formula:
6.3 Future Questions
The purpose of this proposed study is to focus on whether the treatment impacts
the vulnerability to being successfully phished. Age and length of employment
(confounded with gender) are taken into consideration as potentially impacting the
likelihood of being successfully phished. However, there are still more interesting
questions to raise surrounding this topic which can be addressed in other future studies.
These include:
Are women in the financial sector more susceptible to phishing than men?
Are financial employees in the age range of 30-39 more vulnerable to phishing
than other ages?
Which departments are truly most vulnerable to phishing and why? How can
these departments be better secured against this attack vector?
Which positions are most vulnerable to phishing and why? How can these
positions be better secured against this attack vector?
Does length of employment really impact the level of vulnerability an employee
has to phishing and social engineering? Is this related to the number of times an
employee has taken the security training course?
There are many more questions related to this topic and all deserve investigation.
This paper explained the importance of research into social engineering, phishing, and
effective defense techniques. Humans have been shown to be the weakest link in need of
understanding, education, training, and securing. The research done here should benefit
62
future research into this realm of security and help future researchers in the design
process of phishing experiments.
63
REFERENCES
[1]
D. P. Twitchell, "Social Engineering and its Countermeasures," in Handbook of
Research on Social and Organizational Liabilities in Information Security, Idea Group
Inc, 2008, pp. 228-242.
[2]
M. Allen, "Social Engineering: A Means to Violate a Computer System," SANS
Institute, 2007.
[3]
Harl, "The Psychology of Social Engineering," in Access All Areas III, 1997.
[4]
K. Mitnick and W. L. Simon, The Art of Deception: Controlling the Human Element
of Security, Indianapolis, Indiana: Wiley Publishing Inc., 2002.
[5]
S. Granger, "Social Engineering Reloaded," 2 November 2010. [Online]. Available:
http://www.symantec.com/connect/articles/social-engineering-reloaded.
[6]
S. Granger, "Social Engineering Fundamentals, Part I: Hacker Tactics," 18
December 2001. [Online]. Available: http://www.securityfocus.com/infocus/1527.
[7]
K. D. Mitnick and W. L. Simon, The Art of Intrusion: The Real Stories Behind the
Exploits of Hackers, Intruders & Deceivers, Indianapolis, IN: Wiley Publishing, Inc.,
2006.
[8]
J. Baker and B. Lee, "The Impact of Social Engineering Attacks on Organizations:
A Differentiated Study," Florida Atlantic University, Boca Raton, FL, 2005.
[9]
S. Granger, "Social Engineering Fundamentals, Part II: Combat Strategies," 9
January 2002. [Online]. Available: http://www.securityfocus.com/infocus/1533.
[10]
C. E. Lively, Jr., "Psychological Based Social Engineering," SANS Institute, 2003.
[11]
M. Jackobsson and S. Myers, Phishing and Countermeasures: Understanding the
Increasing Problem of Electronic Identity Theft, Hoboken, New Jersey: John Wiley
& Sons, Inc, 2007.
64
[12]
A. Emigh, "Online Identity Theft: Phishing Technology, Chokepoints and
Countermeasures," Infosec Technology Transition Council, 2005.
[13]
M. Badra, S. El-Sawda and I. Hajjeh, "Phishing Attacks and Solutions," in
MobiMedia '07, Nafpaktos, Greece, 2007.
[14]
L. James, Phishing Exposed, Rockland, MA: Syngress Publishing, Inc., 2005.
[15]
B. Schneier, Secrets & Lies: Digitial Security in a Networked World, Indianapolis,
Indiana: Wiley Publishing, Inc., 2004.
[16]
D. P. Twitchell, "Social Engineering in Information Assurance Curricula," in
InfoSecCD Conference 06, Kennesaw, Georgia, 2006.
[17]
J. Hiner, "Change your company's culture to combat social engineering attacks,"
30 May 2002. [Online]. Available: http://www.techrepublic.com/article/change-
your-companys-culture-to-combat-social-engineering-attacks/1047991.
[18]
M. A. Sasse, S. Brostoff and D. Weirich, "Transforming the 'weakest link' - a
human/computer interaction approach to usable and effective security," BT
Technol, vol. 19, no. 3, July 2001.
[19]
W. Arthurs, "A Proactive Defence to Social Engineering," SANS Institute, 2001.
[20]
I. Flechais, J. Riegelsberger and M. A. Sasse, "Divide and Conquer: The role of trust
and assurance in the design of secure socio-technical systems," in New Security
Paradigms Workshop '05, Lake Arrowhead, CA, 2005.
[21]
L. Laribee, "Development of Methodical Social Engineering Taxonomy Project,"
Naval Postgraduate School, Monterey, California, 2006.
[22]
N. J. Evans, "Information technology social engineering: an academic definition and
study of social," Iowa State University, Ames, Iowa, 2009.
65
[23]
P. Finn and M. Jakobsson, "Designing and Conducting Phishing Experiments,"
IEEE Technology and Society Magazine, Special Issue on Usability and Security,
2007.
[24]
The National Commission for the Protection of Human Subjects of Biomedical and
Behavioral Research, "The Belmont Report: Ethical Principles and Guidelines for
the Protection of Human Subjects of Research," U.S. Department of Health,
Education, and Welfare.
[25]
M. Jakobsson and J. Ratkiewicz, "Designing Ethical Phishing Experiments: A study
of (ROT13) rOnl query features," in International World Wide Web (WWW)
Conference, Edinburgh, Scotland, 2006.
[26]
R. Dhamija, J. D. Tygar and M. Hearst, "Why Phishing Works," in CHI-2006:
Conference on Human Factors in Computing Systems, Montreal, Canada, 2006.
[27]
M. Jackobsson, "The Human Factor in Phishing," Indiana University at
Bloomington, Bloomington, Indiana, 2007.
[28]
E. Kirda and C. Kruegel, "Protecting Users Against Phishing Attacks with
AntiPhish," Technical University of Vienna, Vienna, Austria.
[29]
Anti-Phishing Working Group, "Phishing Activity Trends Report 2nd Quarter 2012,"
Anti-Phishing Working Group, 2012.
[30]
JPMorgan Chase & Co, "Fraudulent E-mail Examples," 2009. [Online]. Available:
http://www.social-engineer.org/wiki/archives/Phishing/Phishing-Chase.html.
[31]
Wachovia Corporation, "Phishing Examples," 2009. [Online]. Available:
http://www.social-engineer.org/wiki/archives/Phishing/Phishing-
Wachovia.html.
[32]
R. Dragani, "Big US Banks Under Active Attack, Napolitano Warns," 1 November
2012. [Online]. Available: http://www.ecommercetimes.com/story/76533.html.
[Accessed 4 November 2012].
66
[33]
T. UcedaVelez, "Phishing for Banks: A Timely Analysis on Identity Theft & Fraud in
the Financial Sector," SANS Institute, Atlanta, Georgia, 2004.
[34]
T. Armerding, "Cybercriminals shift focus to bank employees," 20 September 2012.
[Online]. Available: http://www.csoonline.com/article/716685/cybercriminals-
shift-focus-to-bank-employees. [Accessed 15 October 2012].
[35]
W. G. Zikmund, B. J. Babin, J. C. Carr and M. Griffin, Business Research Methods,
8th ed., Mason, Ohio: South-Western Cengage Learning, 2010.
[36]
J. E. Dunn, "Too Many People Reuse Logins, Study Finds," PC World, 7 February
2010. [Online]. Available: http://www.pcworld.com/article/188763/
too_many_people_reuse_logins_study_finds.html. [Accessed 28 October 2012].
[37]
T. Samson, "Study finds high rate of password reuse among users," InfoWorld, 10
February 2011. [Online]. Available: http://www.infoworld.com/t/data-
security/study-finds-high-rate-password-reuse-among-users-188. [Accessed 28
October 2012].
[38]
C. Williams, N. Campbell and G. Chan, "Phishing – Methods," 17 April 2010.
[Online]. Available: http://phishyfraud.wordpress.com/2010/04/17/phishing-
methods/. [Accessed 24 11 2012].
[39]
H. Metcalf and H. Rolfe, "Employment and earnings in the finance sector: A gender
analysis," National Institute of Economic and Social Research.
[40]
S. Sheng, M. Holbrook, P. Kumaraguru, L. Cranor and J. Downs, "Who Falls for
Phish? A Demographic Analysis of Phishing Susceptability and Effectiveness of
Interventions," in CHI 2010, Atlanta, Georgia, 2010.
[41]
S. Boslaugh and D. P. A. Watters, Statistics in a Nutshell, O'Reilly Media, 2012.
[42]
T. F. Jaeger, "Categorical Data Anlysis: Away from ANOVAs (transformation or not)
and towards Logit Mixed Models," Journal of Memory and Language, vol. 59, no. 4,
pp. 434-446, 2008.
67
[43]
Institutional Review Board, "Eastern Washington University Policy and Procedures
for the Protection of Human Subjects in Research".
[44]
I. Mann, Hacking the Human, Gower, 2008.
[45]
A. Dolan, "Social Engineering," SANS Institute, 2004.
[46]
D. B. Bailey, "Age Classifications: When Considering the Age of Users, How Old is
"Old"?," August 2002. [Online]. Available:
http://webusability.com/article_age_classifications_8_2002.htm. [Accessed 8 July
2009].
[47]
K. Poulsen, "Mitnick to Lawmakers: People, Phones are the Weakest Links," 2 March
2000. [Online]. Available: http://www.politechbot.com/p-00969.html. [Accessed
11 July 2009].
[48]
D. A. Dillman, J. D. Smyth and L. M. Christian, Internet, Mail, and Mixed-Mode
Surveys: The Tailored Design Method, 3rd ed., Hoboken, New Jersey: John Wiley &
Sons, Inc., 2009.
[49]
S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin and E.
Singer, Methods for Testing and Evaluating Survey Questionnaires, Hoboken, New
Jersey: John Wiley & Sons, Inc., 2004.
[50]
D. T. Campbell and J. C. Stanley, Experimental and Quasi-Experimental Designs
for Research, Wadsworth Publishing, 1963.
[51]
T. Turner, "Social Engineering - Can Organizations Win the Battle?," East Carolina
University, Greenville, NC, 2005.
68
APPENDIX A: TRAINING MATERIALS
69
Social Engineering and Information Security Training Slides
70
71
72
73
74
75
76
77
Training Questions
Mult iple choic e a nd true/f alse qu estion s. Answers are hi ghlighted in red .
1. What is considered by some to be the greatest security risk to any company?
a. Viruses
b. Social Engineering
c. Spam
2. Social engineering bypasses all security efforts put forth by the IT and security
departments via _____________.
a. Breaking encryption
b. Cracking through the network firewalls
c. The employees
3. What can be at risk if social engineer gains access to your company’s network or
to your specific computer?
a. Work documents stored on your work computer or on the network
b. Personal documents stored on your work computer
c. Personal email access via your work computer
d. All of the above
4. What do you call an email that tries to trick you into scams by pretending to be
a legitimate business that you may have an account with?
a. Phishing
b. Cracking
c. Virus
d. Encryption
5. Social engineering affects you outside of work.
a. True
b. False
6. The size of a company matters when a social engineer chooses a target.
a. True
b. False
7. Only obvious criminals could be social engineers, not someone I know and work
with on a regular basis.
a. True
b. False
8. An ex-employee could potentially turn into a social engineer.
a. True
b. False
9. What is the weakest link in security?
a. Physical security (e.g. cameras, locks, guards)
b. Technological security (e.g. firewalls, encryption)
c. Employees (e.g. the people who use the systems)
78
10. What do you call the social engineering technique where the attacker pretends
to be someone they are not or invents a scenario to engage the target victim in?
a. Phishing
b. Pretexting
c. Dumpster diving
d. Sniffing
11. What should you do if you receive a phishing text message (SMiShing)?
a. Call the number provided in the message
b. Reply to the message via text message
c. Notify the helpdesk immediately and do not reply to the message
12. When typing in your password on a keyboard, you should make sure no one
sees what you type; even if that means covering your hand as you type or asking
someone to look away.
a. True
b. False
13. Information can come in the following forms:
a. Print
b. Hand-written
c. Digital
d. Spoken
e. All of the above
14. ___________________ are rules for the organization to help secure it, its
employees, and its assets.
a. Security policies
b. Rules of engagement
c. Protective measures
15. You should always be aware of your surroundings, who is near by that could see
what you are working on or hear what you are talking about.
a. True
b. False
16. When a visitor comes to your branch, you should:
a. Welcome them openly and let them in wherever they say they need to go
b. Have them sign-in, and then show them where they say they need to go
c. Verify their identity and that they are expected, have them sign-in,
then allow them to go where they need to
17. Is there ever a reason to not follow the Branch Visitation Procedures laid out in
the Branch Security Manual when a visitor comes to your branch?
a. Yes
b. No
18. All passwords must follow Sterling’s strong password policy.
a. True
b. False
19. If you were going on vacation and your boss asked you for your password while
you were gone, just in case… should you give it to them?
a. Yes
b. No
79
20. You should lock your computer if you are going to walk away from your desk for
_________.
a. Any amount of time, even for just a few minutes
b. More than 5 minutes
c. More than 15 minutes
d. You do not need to lock your computer if you walk away from your desk
21. After finishing a transaction