Content uploaded by Hennie A Kruger
Author content
All content in this area was uploaded by Hennie A Kruger on May 06, 2019
Content may be subject to copyright.
A prototype for assessing information security
awareness
H.A. Kruger
a,
*, W.D. Kearney
b,1
a
School of Computer-, Statistical- and Mathematical Sciences, North-West University (Potchefstroom Campus),
Private Bag X6001, Potchefstroom 2520, South Africa
b
AngloGold Ashanti, Level 13, St Martins Tower, 44, St Georges Terrace, Perth WA 6000, Australia
article info
Article history:
Received 19 January 2005
Revised 22 December 2005
Accepted 6 February 2006
Keywords:
Information security awareness
Quantitative modelling
Knowledge
Attitude
Behaviour
abstract
Due to the intensified need for improved information security, many organisations have
established information security awareness programs to ensure that their employees are
informed and aware of security risks, thereby protecting themselves and their profitability.
In order for a security awareness program to add value to an organisation and at the same
time make a contribution to the field of information security, it is necessary to have a set of
methods to study and measure its effect. The objective of this paper is to report on the
development of a prototype model for measuring information security awareness in an
international mining company. Following a description of the model, a brief discussion
of the application results is presented.
ª2006 Elsevier Ltd. All rights reserved.
1. Introduction
Security risks associated with information technology are
a topic that has become increasingly significant. As corpora-
tions rely ever more on technology to run their businesses,
security is becoming a major concern rather than an after-
thought. The CERT Co-ordination Center at Carnegie Mellon
University has reported that security incidents, reported secu-
rity attacks that may involve one site or thousands of sites,
have increased by 68% from 2003 to 2004 (CERT/CC, 2004).
Whilst information security generally focuses on protect-
ing the confidentiality, integrity and availability of informa-
tion, information security awareness deals with the use of
security awareness programs to create and maintain secu-
rity-positive behaviour as a critical element in an effective in-
formation security environment. According to Hansche (2001:
p. 14) the goal of a security awareness program is to heighten
the importance of information systems security and the
possible negative effects of a security breach or failure. The In-
formation Security Forum (ISF, 2003) defines information
security awareness as the degree or extent to which every
member of staff understands the importance of information
security, the levels of information security appropriate to
the organisation, their individual security responsibilities,
and acts accordingly.
The effective management of information security re-
quires a combination of technical and procedural controls to
manage information risk. The value of controls usually de-
pends on the people implementing and using them and in
information security that is no different. Controls can be
circumvented or abused by employees who ignore security
policies and procedures. The implementation of effective se-
curity controls is thus dependent upon the creation of a secu-
rity positive environment, where everyone understands and
engages in the behaviours that are expected of them. See for
example Von Solms and Von Solms (2004) who offered
*Corresponding author. Tel.: þ27 18 2992539; fax: þ27 18 2992570.
E-mail addresses: rkwhak@puknet.puk.ac.za (H.A. Kruger), wkearney@anglogoldashanti.com (W.D. Kearney).
1
Tel.: þ61 8 94254621; fax: þ61 8 94254650.
available at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/cose
0167-4048/$ – see front matter ª2006 Elsevier Ltd. All rights reserved.
doi:10.1016/j.cose.2006.02.008
computers & security 25 (2006) 289–296
guidelines on how to move from information security policies
to a positive information security culture. The change to a se-
curity positive environment or culture is, however, not always
easy and straightforward. The ISF’s Information Security Sta-
tus Survey (ISF, 2002) indicated that most members believe
that the effectiveness of their security awareness initiatives
does not rate especially highly, and that more than four out
of five feel they do not commit sufficient time and resources
to their awareness activities. In a similar vein, a recent com-
puter crime and security survey (CSI, 2004) found that the
vast majority of the organisations in the review view informa-
tion security awareness training as important, though (on
average) respondents from all sectors do not believe their
organisations invest enough in this area.
There appears to be sufficient material to help organisa-
tions with delivering a proper security awareness program
and what to do to influence employees positively, e.g. Infor-
mation Technology – Code of Practice for Information Security
Management (ISO 17799, 2000), The Standard of Good Practice
for Information Security (ISF, 2003), Leach (2003), Furnell et al.
(2002), Hansche (2001) and Spurling (1995). There is, however,
a lot less available in the literature on how to measure the ef-
fectiveness of these programs. Pentasafe Security Technolo-
gies (Pentasafe, 2002) produced a comprehensive security
awareness report that was based on responses from 1348
workers and 583 organisations worldwide. It represents a
major effort to measure how organisations improve security
awareness and how well employees understand and act
upon information security policies, threats and issues in their
respective organisations. Examples of other information secu-
rity measurement aspects can be found in Martins and Eloff
(2001) who suggested that the measurement of information
security management should be performed on two levels,
viz. a business and management process level and a technical
level; Stanton et al. (2005) presented work on the systematical
classification of information security end user behaviours that
could be used when analysing (measure) security behaviour.
Information security awareness is a dynamic process,
made even more difficult in that risks continuously change.
As a result, any awareness program needs to be continually
measured and managed to keep abreast of changes in risk pro-
files. To keep the users current and their memories refreshed,
any awareness program must be ongoing and be an integral
part of the very culture of the enterprise. The key to success
in awareness is keeping the messages relevant and consistent,
while varying the delivery mechanisms, to keep everyone in-
terested. Both the delivery mechanism and the risk areas
could change as the information risk profile changes.
Schlienger and Teufel (2003) discuss this where awareness
and training programs lead from ‘‘become aware’’ to ‘‘stay
aware’’ and ends up in ‘‘be aware’’, which changes a security
culture definitively. To address the issue of continuous change
and measurement of information security awareness, a pro-
ject was initiated at an international gold mining company
to investigate the feasibility of developing a measurement
model. The goal of the model, which forms part of an ongoing
research project, was to monitor change in security behaviour
and as a result revise or repeat security awareness campaigns.
The remainder of this paper is organised as follows. Sec-
tion 2briefly introduces the international gold mining
company where the study was performed. Section 3describes
the methodology used while Section 4deals with an applica-
tion of the model, the modelling results and recommendations.
Section 5concludes the paper with some general comments.
2. Background
An international gold mining company, which recently imple-
mented a security awareness program worldwide at all their
operations agreed to assist with the research project. The
company is a global African gold producer with 25 operations
in 11 countries and has gold production of over 6 million oun-
ces annually. The company was formed in 2004 out of the
merging of two existing gold mining companies. The merged
company has one of the world’s largest reserves and resource
bases and focused exploration activities around the globe.
Employing more than 62,500 people around the world, the
company is listed on the following exchanges: JSE Securities
Exchange, NYSE, ASX, LSE, GSE and the Euronext Paris and
Euronext Brussels.
In an organisation of this size and diverse locations of op-
erations, it was clear that there is no silver bullet or clean and
simple answer to create an effective and secure information
environment. The management of the risk in this fluid and dy-
namic environment involves significant expenditure together
with an ongoing business and information technology part-
nership. The company, like every other organisation using in-
formation technology, faces a real internal and external threat
in terms of information risk. Information security apathy and
ignorance are some of the biggest threats to computer sys-
tems and a significant and lasting improvement in informa-
tion security will not be achieved by throwing more
technical solutions and sophisticated processes at the prob-
lem – it is by raising the general level of information security
awareness and educating all computer users in the basics of
information security. One of the first steps in this challenge
was then to create an awareness of the risks and then to en-
sure the risk is managed. The initial aim or objective was to
ensure that computer users are aware of the risks associated
with using information technology as well as understanding
and abiding by the policies and procedures that are in place.
To achieve this, an information security awareness pro-
gram was initiated. Briefly the program involved the follow-
ing. A comprehensive toolkit was purchased from a vendor
and detailed development of the program started in mid-
2003. The first priority was to narrow the focus of the program
into a manageable size. After careful deliberation and follow-
ing a risk elimination process, the program was focused on six
critical risk areas or ‘Golden Rules’, the first five being:
Always adhere to company policies
Keep passwords and personal identification numbers (PINs)
secret
Use e-mail and the Internet with care
Be careful when using mobile equipment
Report incidents like viruses, thefts and losses
The last is the heart of the program, namely Be aware, all ac-
tions carry consequences – once again, back to the people issue.
computers & security 25 (2006) 289–296290
The toolkit purchased, consisted of a complete awareness
solution, and it was decided to use only those portions rele-
vant to the specific needs of the company and to customise
other areas to suit specific needs. The program was rolled
out to all computer users, not all employees.
The program was developed as follows:
Basic presentation to all computer users, including a video,
not longer than an hour
Brochures to all participants
Different posters put up in all regional offices and Business
Units
Website with all details, including ‘‘Ask a question’’ option
available on the global Intranet
Articles in the company’s in-house magazine
Presentations were geared towards different audiences,
with a similar core message but delivered to suit the audience.
The video was customised with the company’s logo and
name, as well as digital images of Business Units, staff mem-
bers and corporate office. The video has been translated into
French, Spanish and Brazilian Portuguese, together with UK,
Australian, South African and American English versions.
While it may seem excessive, it was very important to get
the local buy-in and identification with the program. Presenta-
tions, posters and brochures were in English, Spanish, French
and Portuguese.
Following the implementation of the program there was
a need to evaluate and measure the success and effectiveness
of it. The purchased toolkit contains a basic measuring tool
based on multiple-choice questions that a respondent has to
answer. The number of right answers is then used as an indi-
cation of the awareness in a certain region. There was, how-
ever, a need for a more comprehensive measuring tool that
can be applied globally and that will address the company’s
unique requirements at the different operations.
3. Methodology
The methodology used to develop the measuring tool was
based on techniques borrowed from the field of social psy-
chology that proposes that learned predispositions to respond
in a favourable or unfavourable manner to a particular object
have three components: affect, behaviour and cognition. The
affect component encompasses one’s positive and negative
emotions about something, the behaviour component con-
sists of an intention to act in a particular manner while the
cognition component refers to the beliefs and thoughts one
holds about an object (Feldman, 1999; Michener and Delam-
ater, 1994). These three components were used as a basis
and the model was developed on three equivalent dimensions
namely what does a person know (knowledge); how do they
feel about the topic (attitude); and what do they do (behav-
iour). This approach is not completely new and other re-
searchers have already performed work where the social
sciences were related to the field of information security
awareness. Thomson and Von Solms (1998) have shown how
social psychological principles could be utilised to improve
the effectiveness of an information security awareness
program while Schlienger and Teufel (2003) made use of so-
cial–cultural measures to define a model for analysing infor-
mation security culture in organisations.
To develop a measuring tool and perform the actual mea-
surements, the researcher or decision maker is confronted
with two distinctive challenges: what to measure and how
to measure it. In this study, requirements such as sustainabil-
ity, ease of use, the use of scientific methods and complying
with the organisation’s unique requirements, all add to the
challenge of finding a suitable methodology to create the mea-
suring tool with.
3.1. What to measure
A global information security awareness level for the organi-
sation was the main measurement required. To achieve this,
it would be necessary to measure awareness levels at each re-
gion and then in a meaningful way combine those regional
levels into an overall measurement.
It was agreed that one ‘‘set of aspects’’ would be measured
at all the regions although they might not be of equal impor-
tance in all the regions – importance was handled through
a weighting system which is discussed later on. This approach
called for the identification of key factors that would form the
basis of the evaluation. To assist in the problem structuring
process a hierarchy of criteria was identified using a tree
structure. This process is often referred to as a value tree,
which is a simple representation, capturing the essence of
a problem, extracted from a complex problem description
and can be constructed by using either a top-down or bot-
tom-up approach. The top-down approach was used, as it is
objective led, beginning with a general statement of the over-
all objective and expanding the initial values into more de-
tailed concepts, which help to explain or clarify the former.
A complete discussion of value trees, how they are con-
structed and used, can be found in Belton and Stewart (2002).
As a first classification of what to measure, it was decided
to measure the three dimensions’ knowledge (what you
know), attitude (what you think) and behaviour (what you
do). Each one of these dimensions was then subdivided into
the six focus areas as discussed in Section 2and on which
the awareness program was based. Where appropriate and
through consensus the six focus areas were further subdi-
vided into specific factors, for example, the focus area Pass-
words was broken down into two subcategories Purpose of
passwords and Confidentiality of passwords. Confidentiality of
passwords was then further broken down into Writing down
of passwords and Giving passwords to others.
It is significant to note that the construction of the tree of
aspects that could be measured is directly linked to the overall
complexity of the model i.e. data gathering, importance
weights for different factors, use and interpretation of results,
justification for results, etc. Keeping it simple but meaningful
was one of the major challenges in the design. An illustration
of the tree structure developed is shown in Fig. 1.
Once the factors to be measured were identified, it was
clear that they would not contribute in equal proportions to
the final awareness level measurement. Therefore, another
issue that needed to be measured was the importance of con-
tributing factors. This was achieved through a measurement
computers & security 25 (2006) 289–296 291
process where importance weights were allocated to all fac-
tors in a specific branch of the tree of factors. For example,
the different regions will have different weights as they
have different influences on the overall awareness levels;
the three dimensions, knowledge, attitude and behaviour
will have different importance levels; and the six focus areas
will have different importance weights if management de-
cides that it is more important to measure specific focus areas
than others.
3.2. How to measure
The use of a value tree suggests solving the tree in a backward
manner i.e. the tree is solved from the lowest level working
upwards through the different levels. This was done using
a simple scorecard approach defined as VðaÞ¼Pn
i¼1viðaÞwi
where V(a) is the overall value of alternative a,v
i
(a) is the value
score reflecting alternative a’s performance on criterion iand
w
i
, the weight assigned to reflect the importance of criterion i.
This additive model is one of the most widely used forms of
a value function and is described in detail in Belton and
Stewart (2002).
Scoring models are well known management science tech-
niques and a complete description of the technique can be
found in many textbooks. See for example Taylor (2002).
The performance, v
i
(a), was determined using a question-
naire. Thirty-five questions were designed to test the knowl-
edge, attitude and behaviour of respondents pertaining to
the six main focus areas and their factors and sub-factors.
Some of the questions were answered on a 3-point scale –
true, don’t know and false, while others only needed a true
or false response. This way of measuring how respondents
may act is in line with methods suggested in social psychology
(Michener and Delamater, 1994) and agrees with methods
used and proposed by other researchers and practitioners in
the field of information security awareness e.g. Pentasafe’s
security awareness report (Pentasafe, 2002), Schlienger and
Teufel (2003), Teare and Da Veiga (2003) and Martins and Eloff
(2001).Fig. 2 shows an example of a question in each of the
three dimensions.
It is important to note that actual behaviour may not be
measured accurately by a questionnaire alone as respondents
do not necessarily tell the truth when asked about their
behaviour. It should also be accepted that not all respondents
will lie about their behaviour and that the use of a question-
naire will, in general, give at least an indication of the level
of security behaviour. The measuring process can be sup-
ported by physical tests to verify actual behaviour and Internal
Audit departments may be a valuable source of help in this
regard. The incorporation of physical tests and other mea-
sures, to confirm actual security behaviour, forms part of an
ongoing research process and is currently being investigated.
The importance weights, w
i
, was determined using the an-
alytic hierarchy process (AHP). The AHP approach makes use
of pairwise comparisons to provide a subjective evaluation
of factors based on management’s professional judgment
and opinion. The comparisons are made using a preference
scale, which assigns numerical values to different levels of
preference. A square matrix is then derived from the pairwise
comparisons and a scale is extracted based on the matrix’s ei-
genvector associated with the largest eigenvalue. When this
vector is normalised to sum to one, the solution is unique
and represents a numerical measure of the decision maker’s
perceptions of the relative importance of criteria. A consis-
tency index can then be computed to measure the degree of
inconsistency in the pairwise comparisons. Saaty developed
the AHP and a good description of the technical details and ap-
plication possibilities can be found in Saaty (1980) and Vargas
and Dougherty (1982). A comprehensive literature review of
AHP applications in different fields and areas can also be
found in Vaida and Kumar (2006).
The methodologyoutlined above complies withthe organisa-
tion’s specific requirements. It is specific to the mining company
in the sense that it is based on thesix focus areas as approved by
senior management and it can be applied to each one of their
regions to providea final global awareness level. The method is
sustainable as it can be applied over and over. It is fairly easy to
use and output is given in a quantitative manner that is easy to
understand. The techniques and principles used, such as value
trees, scorecards and the analytic hierarchy process, are all
acceptedscientific methods thathave been used numerous times
before inresearch projects. In general the methodology provides
a number of opportunities to benefit from.
Not only will the model provide an overall global awareness
level, but awareness levels are also measured (and reported
Overall
Awareness
level
Region 1 Attitude
Policies
Confiden-
tiality
Write
Region n
Region 2
Behavior
Knowledge
Password Give
Purpose
Regions Dimensions 6 focus
areas Factors
Sub-
Factors
Fig. 1 – Tree structure of problem.
computers & security 25 (2006) 289–296292
on) at intermediate levels i.e. per region, dimension, focus
area and factor per focus area. If needed, low-level informa-
tion from the questionnaires and importance weights can fi-
nally be used to explain specific performances.
The data and the tree structure can be used to prepare a drill
down system. Such a drill down facility would enable man-
agement to easily view the awareness at different levels of
detail and plan accordingly what actions to take and where
to focus these actions.
By applying the model at regular intervals, the change in
awareness levels can be measured and an index of aware-
ness can be constructed. This will assist management to
measure the change towards, or away from, security-
positive behaviour over time, and to take corrective action
if necessary. The index figure of awareness levels may also
act as an important indicator of when to review, and possibly
adjust, importance weights of those aspects being measured.
Sensitivity analysis can be performed e.g. if management
wants to change the importance weights of the different
branches in the tree or when they want to study the effect
of adding or deleting factors from the tree.
A possible negative aspect is the time it takes to perform
the pairwise comparisons necessary to calculate the impor-
tance weights. This can take long depending on the number
of factors identified in the tree structure as well as the number
of managers involved in the process. Simplifying the process
with user-friendly graphic interfaces or the use of an alterna-
tive weighting process is currently considered as part of the
ongoing research process.
4. Application
The prototype tool was applied to the Australian regional office
of the company discussed in Section 2. The choice of region was
based on a management request as well as the fact that the en-
vironment (staff, infrastructure, etc.) was reasonably stable.
The staff complement was small enough to easily obtain the re-
quired feedback and input, and all of them have already been
exposed to the information security awareness program.
The first task was to determine what to measure. To this
end, a value tree, similar to the one in Fig. 2, was constructed.
From the tree, 44 aspects were identified that could be
measured to cover the knowledge, attitude and behaviour di-
mensions with the associated six focus areas in each dimen-
sion. Next, a simple questionnaire, containing 35 questions
(some questions were used to measure more than one aspect),
to capture the information required was developed and tested
at the region’s head office as well as at one of the operational
sites in the region. Different tests were performed and include
tests using open-ended questions, multiple-choice questions,
one-on-one contact with respondents and the use of e-mail
facilities. These tests have provided valuable input and
helped to refine the questionnaire. The refinement process
took some time and involved a number of iterations with
samples from staff to ensure that a model was developed
which complies with the principles of sustainability, ease of
use and scientifically sound. The final questionnaire was
distributed and a response rate of almost 51% was recorded.
Finally, the importance weights were determined using the
AHP. The Information Security Manager, responsible for the
organization’s global information security, provided the pair-
wise comparisons to calculate the importance weights. Natu-
rally importance weights will be based on input from all
relevant managers – this study, however, was more focused
on the feasibility and development of an acceptable method-
ology and therefore initial ratings from only one (and
probably the most appropriate) manager were accepted as
sufficient.
Questionnaire results and importance weights were pro-
cessed in a spreadsheet application and output was finally
presented in the form of graphs and awareness maps. Fig. 3
contains one example of a graph showing the overall aware-
ness level (as being average) as measured with the prototype
tool. Similar graphs were produced for each dimension as
well as for each focus area. The following awareness scale,
which was defined in accordance with management’s view
on awareness performance, was used to explain the level of
awareness:
Example question to test knowledge:
Internet access on the company’s systems is a corporate resource and should be used for
business purposes only 1. True 2. False 3. Do not know
Example question to test attitude:
Mobile equipment is usually covered with existing insurance cover and there is no special need
to include them in security policies 1. True 2. False 3. Do not know
Example question to test behaviour:
I am aware that you should never give your password to somebody else – however, my work is
of such a nature that I do give my password from time to time to a colleague (only to those that I
trust!) 1. True 2. False
Fig. 2 – Example questions.
Awareness Measurement (%)
Good 80–100
Average 60–79
Poor 59 and less
computers & security 25 (2006) 289–296 293
The overall awareness in the region reviewed was mea-
sured as 65% and according to the awareness scale considered
as ‘average’.
The output of the measurement was also used to construct
a colour coded ‘Regional Awareness Map’ – see Fig. 4(a). Using
this map it is easy to see immediately how the region is per-
forming in each dimension and each focus area. For example,
Fig. 4(a) shows that the overall awareness for the region is
65%. This is made up by the three dimensions, which mea-
sured as 77% awareness in terms of knowledge, 76% aware-
ness in terms of attitude, and 54% in terms of behaviour.
The colour codes immediately show that information security
behaviour in the region needs attention while knowledge and
attitude were measured as average. The map also details mea-
surements for each focus area. Consider for example the focus
area Adhere to policies – the total awareness for this area is 44%
and is made up of 18% awareness in terms of behaviour, 55%
for attitude and 81% for knowledge. The colour codes suggest
that the focus area needs attention and that the attention
should be directed towards behaviour and attitude with ac-
ceptable knowledge. In the same manner, management can
easily identify where to focus attention in each one of the
six focus areas, thereby addressing the complete security
awareness needs in the region.
Having produced a regional awareness map for each re-
gion, a final ‘Global Awareness Map’, consisting of the aware-
ness levels in each region, can be constructed to show the
global awareness level. Colour codes were again added to fa-
cilitate the direction of new or changed awareness campaigns
to those dimensions and/or focus areas that did not measure
satisfactorily. Fig. 4(b) shows the global awareness map – the
global awareness figure (86%) was inserted for illustrative
purposes.
4.1. Recommendations
The prototype tool, as applied in practice, was in line with ini-
tial management requirements and was regarded as success-
ful with significant results. This section will highlight some of
the issues that were identified during the development and
verification process that need more attention to ensure ongo-
ing and effective use. The recommendations form part of an
ongoing research and development process and some of
them are currently being addressed.
A comprehensive and complete bank of questions should be devel-
oped. It is recommended that some quality time be spent to
research this aspect of the model more in depth. The model
can only be successful if the ‘‘right’’ questions are asked to
obtain correct data as input to the model. Cognizance
should be taken from current best practices as described
by various resources/organizations e.g. the Information Se-
curity Forum (ISF) and ISO 17799.
A comprehensive set of questions is necessary to ensure,
firstly that a different set of randomly selected questions is
used every time the model is applied (if respondents have to
answer the same questions every time, they might ‘‘learn’’
what the expected responses are) and secondly, to present
different questions, randomly selected, to respondents in
Awareness
65%
Non-
awareness
35%
Fig. 3 – Overall awareness level.
(a) (b)
00E-mail and Internet
00
00
00
00
Adhere to policies
Keep passwords secret
Mobile equipment
Report security incident
00
00
00
00 00 00 65 00
Behavior
Attitude
Knowledge
Overall Awareness
GLOBAL AWARENESS
86
South Africa
Corporate Office
East and West Africa
Australia
North America
South America
Regions
Dimensions
Focus Areas
00 00
00
00
77 00
00 76 00
0000
00
00
54 00 00
00 00 44 0000
00 00 83 00 00
00 00 81 00 00
00
00 00
00
64
74
00 00
00
00
Actions 00 00 51 00 0000
Unsatisfactory –action required
Monitor –action potentially require
Adhere to policies
Focus Areas
Behavior (50)
Attitude (20)
Knowledge (30)
Total awareness/dimension
Total awareness/focus area
Dimensions (weight)
65
TOTAL REGIONAL AWARENESS
Keep passwords secret
E-mail and internet
Mobile equipment
Report security incidents
Actions consequences
Satisfactory –no need for action
18 93
54 55
84
81
76
44
81
76
83
83
87
74
64
77
50
77
78
87 51
67
91
74
59
32
77
consequences
Fig. 4 – (a) Regional awareness map of Australia; (b) global awareness map.
computers & security 25 (2006) 289–296294
the same office or region (to prevent respondents discussing
questions and come up with consensus answers).
Importance weightings should be obtained from relevant
managers. The effectiveness of measurements, produced
by the model, is dependent on proper evaluation (impor-
tance weights) of factors. These importance evaluations of
factors are based on management’s professional judgment
and opinion and it is therefore imperative that the right
level of management be identified and that sufficient time
is allowed for gathering their ratings and then to convert it
into input to the model.
The use of practical system data obtained from, for example,
a system administrator should be considered. Due to time
constraints this was not fully explored during the test of
the prototype. Practical data from a system could (should)
be used as additional input to the model to test behaviour
factors. Such data would be more reliable (not subjective
or human dependent) and easy to get without making use
of staff’s working time to complete (longer) questionnaires.
Examples may include number of virus infections, requests
to visit unauthorized websites, number of IT security inci-
dents, etc. To further enhance the quality of behaviour
data, the Internal Audit department might be considered
as an aid to assist with compliance tests.
The tool should be automated. The information gathering pro-
cess and the importance weight allocation process should
be developed into a web-based tool that is controlled from
a central point and then be made available to regions. The
tool should:
- Randomly generate a new set of questions every time it is
used and then present it to the respondents
- Facilitate the allocation of importance weights
- Automatically feed the responses (questionnaires and im-
portance weights) into the model
- Solve the model and perform reporting activities (graphs,
awareness maps and drill down facilities)
- Keep track of responses from regions (database)
- Keep track of awareness levels each time the model is
applied
- Automatically calculate, update and report on changes
from one model application to the next (index figures)
5. Conclusion
There are numerous reasons why organisations have to spend
effort and resources on the evaluation or measurement of in-
formation security awareness successes. Posthumus and Von
Solms (2004) motivated the need to integrate information se-
curity into corporate governance and proposed a framework
to aid organisations in their integration efforts. The impor-
tance of an information security awareness-measuring tool
can therefore – apart from reasons such as return on invest-
ment, re-directing of security campaigns, etc. – also be linked
to the highest management level in an organisation. Informa-
tion security has much to do with management and aspects,
such as directing and controlling, are important. These as-
pects are functions of the Board of directors of a company
and for them to fulfill their role and have a proper corporate
and information security governance framework in place;
they need feedback on what is happening in the company in
terms of information security. The awareness measurement
tool, developed in this study, may assist a great deal in provid-
ing feedback to the Board of directors on the success of an in-
formation security awareness program, and will assist them
in their function of controlling and directing strategic objec-
tives set for information security.
Having implemented an information security awareness
program does not automatically guarantee that all employees
understand their role in ensuring the security and safeguard-
ing of information and information assets. In order for secu-
rity awareness programs to add value to an organisation and
at the same time make a contribution to the field of informa-
tion security it is necessary to follow a structured approach to
study and measure its effect.
This paper described the development of a prototype to
measure information security awareness at an international
gold mining company. The model makes use of a simple
data gathering process and weighting system and, combined
with certain multi-criteria problem solution techniques,
provides a quantitative measurement of security awareness
levels. It is based on the sound principles of sustainability,
sophistication and scientific validity and could be used as
a basis for a more comprehensive and sophisticated measur-
ing system. The model offers several opportunities for
enhancement and several aspects are currently considered
to improve the model, e.g. the use of a 5- or 7-point Likert-
type scale to evaluate questions, a more user-friendly system
to derive importance weights, etc. The tool will also be
applied in other regions and more data will increase insight
into the model and the framework and may lead to further
enhancements.
Acknowledgement
The authors would like to thank Prof Rossouw von Solms of
the Nelson Mandela Metropolitan University for the useful
comments and suggestions made. The authors alone are re-
sponsible for any errors and omissions.
references
Belton V, Stewart TJ. Multiple criteria decision analysis. An inte-
grated approach. Dordrecht: Kluwer Academic Publishers;
2002.
CERT/CC. Date revised. CERT/CC statistics 1988–2004. Web:
<http://www.cert.org/cert_stats.html>; 2004 [accessed
September 2004].
CSI. CSI/FBI computer crime and security survey. Computer
Security Institute; 2004.
Feldman RS. Understanding psychology. 5th ed. Boston, River
Ridge, IL: McGraw-Hill College; 1999.
Furnell SM, Gennatou M, Dowland PS. A prototype tool for infor-
mation security awareness and training. Logistics Information
Management 2002;15(5/6):352–7.
Hansche S. Designing a security awareness program: Part 1, in-
formation. Systems Security January/February 2001:14–22.
computers & security 25 (2006) 289–296 295
ISF. Effective security awareness – workshop report. Information
Security Forum; April 2002.
ISF. The standard of good practice for information security. Ver-
sion 4.0. Information Security Forum; 2003.
ISO 17799. Information technology, code of practice for informa-
tion security management. Geneva: International Standards
Organisation; 2000.
Leach J. Improving user security behaviour. Computers and Se-
curity 2003;22(8):685–92.
MartinsA, Eloff JHP.Measuringinformationsecurity,<http://philby.
ucsd.edu/wcse291_IDVA/papers/rating-position/Martins.pdf>;
2001 [accessed August 2004].
Michener HA, Delamater JD. Social psychology. 3rd ed. Orlando,
Florida: Harcourt Brace College Publishers; 1994.
Pentasafe. Security awareness index report: the state of security
awareness among organisations worldwide. Pentasafe Secu-
rity Technologies; 2002. p. 55.
Posthumus S, Von Solms R. A framework for the governance of
information security. Computers and Security 2004;23(8):
638–46.
Saaty TL. The analytic hierarchy process. McGraw-Hill; 1980.
Schlienger T, Teufel S. Information security culture – from
analysis to change. South African Computer Journal 2003;31:
46–52.
Spurling P. Promoting security awareness and commitment.
Information Management and Computer Security 1995;3(2):
20–6.
Stanton JM, Stam KR, Mastrangelo P, Jolton J. Analysis of end user
security behaviours. Computers and Security 2005;24(2):
124–33.
Taylor BW. Introduction to management science. 7th ed. Prentice
Hall; 2002.
Teare G, Da Veiga A. Information security culture and awareness.
Paper presented at the 2003 ISSA Conference, Sandton Con-
vention Centre, South Africa; 9–11 July 2003.
Thompson ME, Von Solms R. Information security awareness:
educating your users effectively. Information Management
and Computer Security 1998;6(4):167–73.
Vaida OS, Kumar S. Analytic hierarchy process: an overview of
applications. European Journal of Operational Research 2006;
169(1):1–29.
Vargas LG, Dougherty JJ. The analytic hierarchy process and
multicriterion decision making. American Journal of Mathe-
matical and Management Sciences 1982;19(1):59–92.
Von Solms R, Von Solms B. From policies to culture. Computers
and Security 2004;23(4):275–9.
Kruger HA is an Associate Professor in the School of Com-
puter-, Statistical- and Mathematical Sciences at the North-
West University (Potchefstroom Campus) in South Africa. He
previously worked for Anglo American Corporation as a senior
Computer Auditor and has more than 10 years experience in
Information Risk Management. He has a PhD in Computer Sci-
ence, a MCom (Information Systems) and an MSc (Mathemat-
ical Statistics). His current interests include decision modeling
and the use of linear programming models.
Kearney WD currently works as a Manager, IT Risk and Com-
pliance. He has over 15 years experience in Information Risk
Management in a number of positions in large international
companies, the last 5 years with AngloGold Ashanti. He has
an MSc degree, numerous diplomas, and earned a number of
certifications, including CISA and CIA. He was also successful
in passing the CISM exam and has applied for certification. He
is currently registered for a PhD (Information Security) and is
a member of ISACA (Perth Chapter) and the Computer Society
of South Africa.
computers & security 25 (2006) 289–296296
